Behavioral equivalence testing: proving a new service matches the old one

The question every cutover has to answer

Before a modernized service replaces a legacy program, someone has to sign a statement that says, in effect: for every input that matters, the new system produces the same result, or a different result we chose on purpose. Parallel runs answer part of that question with production data. They cannot answer it for inputs that did not occur during the run window, and the rare inputs are where the rules nobody remembered live.

Behavioral equivalence testing closes that gap with targeted fixtures: inputs chosen so that every extracted rule fires at least once, and every boundary in every condition is exercised from both sides.

Fixtures from branch predicates

Every rule in the PYD ENTERPRISES inventory carries the path condition that leads to it: the conjunction of branch predicates on the control-flow path from the decision's entry point. For the Plus tier fee rule, which fires when TIER-PLUS is true, WS-AVG-BAL < 5000.00 and WS-DIRECT-DEP-CNT = ZERO, the generator solves for inputs that satisfy the condition, then perturbs each atomic predicate to its boundary.

• WS-AVG-BAL = 4999.99 (inside the rule) and WS-AVG-BAL = 5000.00 (outside it)
• WS-ACCT-TIER = 'P' and each other value declared in the 88-level list
• WS-DIRECT-DEP-CNT of 0 and of 1

Boundaries respect the data division. A PIC S9(9)V99 balance has a smallest increment of 0.01, so the boundary pair is 4999.99 and 5000.00, not 4999.999. Fields with 88-levels get one fixture per declared value plus one undeclared value, because the WHEN OTHER branch is usually where the surprise lives.

1{
2  "decision": "MonthlyMaintenanceFee",
3  "source": "DDAFEE01.cbl",
4  "fixtures": [
5    { "id": "R4-in",  "rule": "R4", "given": { "tier": "P", "avgBalance": "4999.99", "directDeposits": 0, "age": 52 }, "expect": { "fee": "18.00" } },
6    { "id": "R4-out", "rule": "R5", "given": { "tier": "P", "avgBalance": "5000.00", "directDeposits": 0, "age": 52 }, "expect": { "fee": "0.00" } },
7    { "id": "R6-undeclared-tier", "rule": "R6", "given": { "tier": "Z", "avgBalance": "8200.00", "directDeposits": 0, "age": 52 }, "expect": { "fee": "30.00" } }
8  ]
9}

Where the expected values come from

A fixture is only as good as its expected output. The generator computes expected values from the extracted decision model, and each value carries the rule and source lines that produced it. For the first suite on a system we strongly recommend confirming expected values against the legacy program itself: run the fixtures through the original COBOL in a test region or through a batch driver, and diff. Where the extracted model and the legacy run disagree, either the extraction is wrong or the test environment differs from production in a parameter file, and both are worth knowing before the new service is built.

Once confirmed, the suite becomes the acceptance contract for the build team. It runs against the new service in CI on every change.

Comparing results exactly

Results are compared as decimal strings at the scale the legacy field defines. A new service returning 12 for a fee the legacy program returns as 12.00 passes after scale normalization; one returning 11.999999 because it used a double fails, and the report shows the rule, both values and the COBOL arithmetic that produced the expected one.

Text fields are compared after the trailing-space trimming the API adapter applies, and dates after normalization to ISO format. Every normalization is explicit in the suite configuration, so nobody can quietly widen a tolerance to get a build green.

1import suite from "./ddafee.fixtures.json";
2import { getFee } from "../src/fees";
3 
4for (const f of suite.fixtures) {
5  test(`${f.id} (${f.rule})`, async () => {
6    const out = await getFee(f.given);
7    expect(toScale(out.fee, 2)).toBe(f.expect.fee);
8  });
9}

Recording intentional differences

Not every difference is a defect. Modernization is often the moment a business fixes a rule it has disliked for years: a rounding step, a fee that applies to an invalid tier code, a cap that was never indexed. When a reviewer changes a rule in the sign-off view, the generator regenerates the affected fixtures with the new expected values and keeps the old ones in the suite, marked as superseded with the reviewer, the date and the reason.

That record matters at cutover. Operations teams reconciling the first month of production can tell immediately whether a difference they see was decided or is a regression, without reopening the discovery documents.

Coverage per rule, not per line

Line coverage of the new code says little about equivalence. The suite reports coverage against the rule inventory instead: how many extracted rules have at least one passing fixture, how many boundaries were exercised, and which rules were marked out of scope by a reviewer, with the reviewer's name. A rule the business decided to change appears as an intentional difference with its sign-off, not as a failure to suppress.

That report is the artifact risk committees ask for. It does not say the new system is correct. It says precisely which behaviors were proven equal, which were changed on purpose, and which were not tested, and it says so in terms of business rules a reviewer can read. See the behavioral test generator for formats and CI integration.