Skip to content
PYD ENTERPRISES
Start free trial

Migration guide

COBOL batch to Kafka: control breaks, trailers and restart become events and contracts

Nightly batch assumes sorted input, a business date and a rerunnable job. Kafka gives you none of those for free. This guide shows how to extract what the batch relies on so the streaming design provides it deliberately.

Request a codebase scanExplore demo
COBOL batchKafka event streams
DIFFICULTY
Very high
OUTPUT
OpenAPI / YAML
SYNTAX TRAPS
5 covered
WORKFLOW
6 steps

By Pranav Kulkarni, Head of Language Engineering · Updated 15 Oct 2026

Technical syntax challenges

Where COBOL batch migrations go wrong.

01

Control breaks assume sorted input

IF TXN-ACCT-NO NOT = WS-PREV-ACCT totals transactions per account because a SORT step ordered them. In Kafka, order holds only within a partition, so the account number has to be the message key and the aggregation becomes keyed state, not a comparison with the previous record.

02

Header, detail and trailer records

Sequential files mix record types by a code in byte 1: a header with the business date, details, and a trailer with record count and hash total. The trailer is a reconciliation rule. Streams need an explicit completeness signal per business date to replace it.

03

Business date versus event time

Batch reads the business date from a control file or header, so a transaction posted at 00:10 can belong to yesterday. Stream processors default to event or processing time. The spec must state which date every rule uses.

04

Restart and idempotency

A failed job is rerun from a step with its input files intact, so each record is processed once per successful run. Kafka consumers see at-least-once delivery by default, which requires idempotency keys and deduplication rules the batch never needed.

05

Default branches in posting logic

IF TXN-TYPE = 'D' SUBTRACT ... ELSE ADD ... treats every code that is not a debit as a credit, including reversal codes added later. Moving to typed events forces a decision about every code, which is exactly the point.

Extraction workflow

6 steps from source to signed-off spec.

The sequence a PYD ENTERPRISES Discovery Scan runs for this stack. Every output links back to source file and line range.

  1. 1

    Ingest the batch suite end to end

    Provide the COBOL programs, copybooks, JCL including SORT steps, and the scheduler definitions for the batch window you want to move to streaming.

  2. 2

    Map record types and file contracts

    PYD ENTERPRISES resolves each file's record types from copybooks and the code that tests the type byte, producing a schema per record type with header and trailer semantics.

  3. 3

    Identify ordering and grouping assumptions

    Control-break comparisons, SORT keys and sequence checks are traced so each assumption about order becomes an explicit keying or ordering requirement.

  4. 4

    Extract posting and aggregation rules

    Calculations inside the read loop become DMN tables and aggregation rules, including what happens for each record type and code.

  5. 5

    Derive reconciliation and restart rules

    Trailer counts, hash totals and checkpoint logic become reconciliation and idempotency rules attached to the event contracts.

  6. 6

    Export AsyncAPI 2.6 and fixtures

    Channels, message schemas and Kafka key requirements are exported as AsyncAPI 2.6, with fixtures that replay a day's records and expected totals.

Legacy in, specification out

The source, and what PYD ENTERPRISES extracts from it.

Left: the COBOL batch construct as it sits in production. Right: the reviewed artifact your Kafka event streams engineers build against. PYD ENTERPRISES produces the spec and the tests; your team writes the service.

LEGACY SOURCE

TXNSUM01.cbl (lines 210 to 232)COBOL
1       2000-PROCESS-TXN.
2           READ TXN-FILE
3               AT END SET END-OF-FILE TO TRUE
4           END-READ
5           IF NOT END-OF-FILE
6              IF TXN-ACCT-NO NOT = WS-PREV-ACCT
7                 PERFORM 3000-WRITE-ACCT-SUMMARY
8                 MOVE TXN-ACCT-NO TO WS-PREV-ACCT
9                 MOVE ZERO TO WS-ACCT-TOTAL
10              END-IF
11              IF TXN-TYPE = 'D'
12                 SUBTRACT TXN-AMOUNT FROM WS-ACCT-TOTAL
13              ELSE
14                 ADD TXN-AMOUNT TO WS-ACCT-TOTAL
15              END-IF
16              ADD 1 TO WS-REC-COUNT
17              ADD TXN-AMOUNT TO WS-HASH-TOTAL
18           END-IF.

EXTRACTED ARTIFACT

ledger-events.asyncapi.yamlYAML
1asyncapi: 2.6.0
2info: { title: Posted transactions (from TXNSUM01), version: 1.0.0 }
3channels:
4  ledger.transactions.posted:
5    description: Replaces detail records of PRD.TXN.DAILY
6    subscribe:
7      operationId: onTransactionPosted
8      message:
9        name: TransactionPosted
10        bindings:
11          kafka:
12            key: { type: string, description: acctNo, keeps per-account order }
13        payload:
14          type: object
15          required: [txnId, acctNo, txnType, amount, businessDate]
16          properties:
17            txnId: { type: string, description: idempotency key }
18            acctNo: { type: string, maxLength: 12 }
19            txnType: { type: string, description: "D debit; any other code adds" }
20            amount: { type: string, pattern: "^\\d+\\.\\d{2}$" }
21            businessDate: { type: string, format: date }
22x-pyd-source: TXNSUM01.cbl:210-232
23x-pyd-rules:
24  - "TXN-SUM-01: delta = -amount if D, else +amount"
25  - "TXN-REC-02: trailer count and hash total per businessDate"

Architectural best practices

Build the Kafka event streams side to the spec, not to the old shape.

  • Key every event by the field the batch sorted and broke on, so per-key order survives partitioning.
  • Carry the business date in every event and make it the basis of daily aggregates, not the broker timestamp.
  • Replace trailer records with an explicit end-of-day or completeness event that carries counts and totals for reconciliation.
  • Design consumers to be idempotent from day one, keyed on a transaction identifier the batch already had or one you add at the source.
  • Replay the generated day-file fixtures through the stream and compare per-account totals with the batch output before cutover.

In depth

Field notes from real scans.

Streaming changes the guarantees, not just the transport

Teams often frame batch-to-streaming as a transport change: files become topics. The harder change is in guarantees. Batch gets total order from a SORT step, completeness from a trailer record, exactly-once processing from rerunnable steps and a stable business date from a control file. Each of those must be rebuilt explicitly in a streaming design, and you can only rebuild what you have identified.

PYD ENTERPRISES extracts those assumptions along with the business rules, and exports AsyncAPI 2.6 contracts, DMN tables and replay fixtures linked to source lines. Your platform and application teams build the producers, topics and stream processors; PYD ENTERPRISES does not.

What the read loop tells a reviewer

The loop posts debits as subtractions and everything else as additions, counts records and accumulates a hash total of amounts regardless of sign. Two review items follow. The ELSE branch means any new transaction code is treated as a credit, so the streaming design should reject unknown codes or route them for review rather than silently adding them. And the first record always triggers a summary write for the initial value of WS-PREV-ACCT; the analyzer checks whether 3000-WRITE-ACCT-SUMMARY guards against that blank key, and if it does not, today's output contains an empty summary row that downstream jobs may already skip or depend on.

FAQ

COBOL batch to Kafka event streams questions

Does PYD ENTERPRISES generate Kafka producers or stream processing code?

No. It exports AsyncAPI 2.6 contracts, JSON Schemas, rules and replay fixtures. Your teams build producers and processors with the stack they choose, such as Kafka Streams or Flink.

How are multi-record-type files modeled?

Each record type gets its own schema. Detail types usually map to event messages, while header and trailer semantics map to business date fields and completeness or reconciliation events.

Can we run batch and streaming in parallel?

That is the usual approach. The generated fixtures and per-key totals let you compare stream results with the batch output for the same business date until they match consistently.

Get the spec for your COBOL batch estate.

Request a codebase scan and review extracted rules, contracts and fixtures for your own COBOL batch estate, inside your VPC.

Request a codebase scanStart 14-day trial