"It's backward compatible": an AI prompt that reads your API diff like a hostile consumer

Somewhere in a refinement near you, a contract change is being waved through with the sentence “it’s backward compatible.” Nobody in the room has read the diff from a consumer’s seat. The sentence is not a finding; it is a hope with good posture. The cost arrives three weeks later, in another team’s sprint, as a production incident with your endpoint’s name on it.

The annoying truth about backward compatibility is that it is not a property of your change. It is a property of your change multiplied by every consumer’s existing code — code you cannot see. A field you renamed is a null in someone’s invoice job. An enum value you added is an unhandled case in someone’s exhaustive switch. Your diff shows none of this, because diffs show shapes, not consequences.

You can, however, force the question to be asked properly — every time, in about a minute.

The strict consumer’s definition

First, the bar. A change is breaking if any existing valid request, or any reasonable handling of today’s responses, could fail after the change. Not “fails for a well-behaved consumer”. Not “fails for consumers who read the changelog”. Any reasonable existing consumer, including the one that deserializes strictly, switches exhaustively, and indexes arrays with the number you gave them.

That definition is deliberately strict, because the generous definitions are how the incident happens. Under the strict one, most “safe” diffs turn out to carry one or two changes that deserve a version bump — and the review’s job is to find out which ones, with reasons a dev can check.

The prompt

Compare the two contract versions below from a STRICT CONSUMER's point of view: if any existing
valid request, or any reasonable handling of today's responses, could fail after the change, it
is breaking. Output: (1) a table — change · type (added / removed / renamed / type-changed /
constraint-tightened / semantics-changed) · BREAKING or NON-BREAKING · one-line justification
written as the consumer's failure scenario; (2) a section for changes that LOOK safe but aren't:
tightened validation, new enum values consumers may switch on exhaustively, nullability changes,
changed defaults; (3) verdict — ship in place, new version, or staged rollout. If a versioning
policy is provided, judge the verdict against it; if not, infer the platform's implied practice
from the contracts themselves, tag it [INFERRED — confirm], and flag [NO WRITTEN POLICY] as a
finding in its own right. Semantic changes (same field, new meaning) cannot be seen in a diff:
end by asking me explicitly whether any field's MEANING changed.

OLD CONTRACT: [paste]
NEW CONTRACT: [paste]
VERSIONING POLICY (optional): [paste if written]

Three design choices in there earn their keep. The justification column must be written as the consumer’s failure scenario — “a caller sending "1263" got 201 yesterday and 400s tomorrow” — because a verdict you can picture failing is a verdict you can challenge. Anything the model infers gets tagged [INFERRED — confirm] instead of stated as fact — the same discipline as every prompt on this site: assumptions surface, they don’t ship silently. And the prompt ends by asking you a question, because there is a class of break it cannot see. More on that below, since it is the whole point.

The worked example

Parcelio — the fictional parcel-logistics platform in every example on this site — is shipping v2 of its quoting endpoint out of a cross-border VAT story. The diff, as the team summarized it: request gains an optional incoterms enum; unNumber gains the pattern ^UN[0-9]{4}$; responses gain options[].vatAmount; the serviceLevel enum gains a freight value; excludedCarriers[].reasonCode is renamed to reason; surcharges[].parcelIndex becomes nullable (“null = shipment-level surcharge”). No written versioning policy. The room’s read: mostly additive, ship it.

The strict consumer disagrees on four of six:

| Change                          | Type                  | Verdict     | Consumer's failure scenario |
|---------------------------------|-----------------------|-------------|-----------------------------|
| optional incoterms (request)    | added                 | NON-BREAKING| Existing requests omit it and still validate. |
| unNumber pattern ^UN[0-9]{4}$   | constraint-tightened  | BREAKING    | A caller sending "1263" got 201 yesterday; the same request 400s tomorrow. |
| options[].vatAmount (response)  | added                 | NON-BREAKING* | *Unless a deserializer rejects unknown fields — [NEEDS: consumer inventory]. |
| serviceLevel gains "freight"    | enum value added      | BREAKING (conditional) | An exhaustive switch hits an unhandled case on the first freight option. |
| reasonCode → reason             | renamed               | BREAKING    | Every reader of reasonCode gets nothing; the portal's exclusion hint quietly goes blank. |
| parcelIndex nullable            | nullability change    | BREAKING    | Consumers doing parcels[parcelIndex] throw on the first shipment-level surcharge. |

Then the section people actually need — the changes that look safe. The unNumber pattern is a data-quality fix dressed as a schema detail: it converts old lenience into hard 400s for exactly the callers who relied on it. How many send junk today is measurable — measure before shipping, don’t guess. And vatAmount arriving with the note “0 for domestic” raises a question the diff cannot answer: consumers who verify that total equals bases plus surcharges will mismatch on cross-border quotes if total is now VAT-inclusive.

Verdict: new version. Two response-shape breaks plus a conditional enum break do not ship in place. The paths carry /v1/, so URL versioning is the platform’s implied practice [INFERRED — confirm] — and [NO WRITTEN POLICY] is a finding in its own right, because without one, this entire judgment gets re-made from scratch on every diff, by whoever happens to be reviewing.

The break no diff can show

The prompt’s last instruction is the one that pays for the other three. It ends by asking, explicitly: did any surviving field change meaning?

On the Parcelio diff, the concrete form of that question was: does options[].total now include vatAmount? Same field name, same type, silently different number — a break that no diff, no schema validator, and no amount of model cleverness can detect, because the contract file is identical either way. In the worked example the team’s answer was yes: total had become VAT-inclusive on cross-border quotes. The closing question was the only place in the entire review that could have caught the worst break in the diff.

That is the honest division of labour. The model is faster and more complete than you at the mechanical sweep — six changes classified with failure scenarios in seconds, including the looks-safe traps. It is structurally incapable of the semantic check, so the prompt makes it hand that question back instead of pretending. A review loop where the machine does the enumeration and the human answers exactly one sharp question beats both the unaided human (who missed the rename) and the unsupervised machine (which would have blessed total).

One standing caution: the model treats enum-value additions generously — whether they break depends on whether consumers switch exhaustively, and it cannot know that. Treat every enum addition it waves through as unconfirmed until someone has looked at a real consumer. That is not a flaw to fix; it is a failure mode to know about. Every prompt worth running has one, and the ones sold as flawless have merely not documented theirs.

Get the system

The breaking-change analyzer is one station in a review chain — it feeds a change-spec prompt that turns the same diff into prose testers and POs will actually review, and both sit beside the endpoint drafter and error-case enumerator in the free slice: 10 documented AI prompts for POs, drafted from a production AI-assisted specification pipeline. Each ships with when to use it, what to paste in, a worked example on Parcelio, and its documented failure modes. Email-gated, free, no listicle.

Subscribe and get the free slice →

— Pierre K.