Resolving COBOL REDEFINES in the AST: byte offsets, aliases and discriminators

Why REDEFINES breaks naive analysis

In COBOL, REDEFINES lets two data descriptions share the same storage. It was the standard way to save memory and to model records whose layout depends on a type code. Analysis tools that work on names rather than storage see two unrelated fields. A write to PAY-AMT and a read of REV-ORIG-ID look independent, even when they touch the same six bytes.

For rule extraction this matters in both directions. Miss the alias and you lose a dataflow edge, so a rule that depends on a value written through the other name disappears. Treat every overlay as always live and you invent dependencies between record variants that never coexist at runtime.

1       01  TXN-RECORD.
2           05  TXN-TYPE                PIC X(01).
3               88  TXN-PAYMENT         VALUE 'P'.
4               88  TXN-REVERSAL        VALUE 'R'.
5           05  TXN-BODY                PIC X(30).
6           05  TXN-PAY  REDEFINES TXN-BODY.
7               10  PAY-ACCT            PIC 9(10).
8               10  PAY-AMT             PIC S9(9)V99 COMP-3.
9               10  PAY-DATE            PIC 9(08).
10               10  FILLER              PIC X(06).
11           05  TXN-REV  REDEFINES TXN-BODY.
12               10  REV-ORIG-ID         PIC X(16).
13               10  REV-REASON          PIC X(02).
14               10  REV-AMT             PIC S9(7)V99 COMP-3.
15               10  FILLER              PIC X(07).

Step one: a byte-accurate storage layout

The front end computes an offset and length for every data item before any semantic pass runs. Lengths come from the PICTURE string and the USAGE. Display numerics take one byte per digit. Packed decimal (COMP-3) takes (digits + 1) / 2 bytes, rounded up, because each byte holds two digits except the last, which holds one digit and the sign nibble. Binary items take 2, 4 or 8 bytes depending on digit count; the TRUNC compiler option changes their value range, not their size.

For the copybook above, PAY-AMT is S9(9)V99: eleven digits, so six bytes. REV-AMT is S9(7)V99: nine digits, so five bytes. The V is an implied decimal point and occupies no storage. Both variants sum to exactly thirty bytes, matching TXN-BODY, and the record is thirty-one bytes long.

• PAY-ACCT offset 1, length 10
• PAY-AMT offset 11, length 6
• PAY-DATE offset 17, length 8
• REV-ORIG-ID offset 1, length 16
• REV-REASON offset 17, length 2
• REV-AMT offset 19, length 5

When a redefining item is shorter than the item it redefines, the layout records the uncovered tail. When the compiler would reject the layout, we report the diagnostic the same way the compiler would, because a copybook that does not compile cleanly is usually a sign that the extracted source set is incomplete.

Step two: alias sets instead of name equality

With offsets in hand, the symbol table stores each elementary item as an interval over its 01-level record. Two references alias when their intervals intersect. The dataflow engine then works on intervals, not names: a MOVE to PAY-AMT kills and defines bytes 11 to 16, and any later read of an interval that intersects them, through any name, picks up that definition.

Group moves are where this pays off. A MOVE SPACES TO TXN-BODY defines all thirty bytes. A MOVE WS-REV-AREA TO TXN-REV is an alphanumeric group move that copies bytes without conversion, which is exactly how a packed amount can end up holding spaces and raise a data exception (S0C7) three programs later. The engine records that edge and the type mismatch it implies.

Step three: finding the discriminator

An overlay is safe to model as a tagged union only if something in the code decides which variant is live. Most of the time that something is an 88-level condition on a type field, tested before the variant is touched. We look for it with a dominance check on the control-flow graph: if every read of a TXN-PAY field is dominated by a branch where TXN-PAYMENT is true, and every read of a TXN-REV field by a branch where TXN-REVERSAL is true, the type byte is the discriminator.

When the check holds, the record becomes a discriminated union in the exported data contract. When it does not hold, for example when a utility program reads REV-AMT without testing the type, the access is flagged for review with the exact path that reaches it. Those flags are often the most valuable findings in a scan, because they mark places where the code depends on an assumption nobody wrote down.

1{
2  "$schema": "https://json-schema.org/draft/2020-12/schema",
3  "title": "TxnRecord",
4  "oneOf": [
5    {
6      "type": "object",
7      "properties": {
8        "txnType": { "const": "P" },
9        "payAcct": { "type": "string", "pattern": "^[0-9]{10}$" },
10        "payAmt": { "type": "string", "pattern": "^-?[0-9]{1,9}\\.[0-9]{2}$" },
11        "payDate": { "type": "string", "format": "date" }
12      },
13      "required": ["txnType", "payAcct", "payAmt", "payDate"]
14    },
15    {
16      "type": "object",
17      "properties": {
18        "txnType": { "const": "R" },
19        "revOrigId": { "type": "string", "maxLength": 16 },
20        "revReason": { "type": "string", "maxLength": 2 },
21        "revAmt": { "type": "string", "pattern": "^-?[0-9]{1,7}\\.[0-9]{2}$" }
22      },
23      "required": ["txnType", "revOrigId", "revReason", "revAmt"]
24    }
25  ],
26  "x-pyd-source": { "file": "TXNREC.cpy", "lines": [1, 15] }
27}

Why the amounts are strings

Packed decimal amounts are exported as decimal strings with a fixed scale, not JSON numbers. JSON parsers in most languages read numbers into binary floating point, and an amount like 12345.67 does not survive a round trip through a double in every case. The pattern constrains digits before and after the point to the PICTURE clause, so a consumer that sends 1234567890.00 for PAY-AMT fails validation instead of being truncated on the mainframe side.

Dates follow the same principle. PAY-DATE is PIC 9(08), but the extractor only emits format: date after it finds the field being validated or used as CCYYMMDD in the procedure division. Otherwise it stays an eight-digit string and the rule inventory notes the ambiguity.

What this buys the modernization team

A storage model that is exact to the byte is not an academic nicety. It is the reason extracted rules can be trusted: every condition and every assignment in the rule inventory resolves to concrete storage, so the behavioral test generator can build fixtures that exercise each variant, and the data migration team gets a contract that says which bytes mean what under which type code. The REDEFINES entry in the patterns catalog covers the modern mappings in more depth.