pa-02 — The Hitchhiker's Guide to API & Contract Design

Companion to CONCEPTS.md, with the runnable contract toolkit in src/go/apicontract/. The architect's job: design interfaces that evolve without breaking consumers you'll never meet.

bash scripts/verify.sh runs the demo:

contract compatibility check (old -> new):
  [BREAKING] tag 2: field "email" became required (old producers may omit it)
  [SAFE    ] tag 3: optional field "nickname" added
  => CI gate: breaking=true
idempotent retries (same key runs once): side effect executed 1 time(s)
opaque pagination cursor: tampered cursor rejected

Three small pieces, each a contract-design lesson.

1. Compatibility as a CI gate (compat.go)

Protobuf identifies fields by a stable tag number, which is exactly what lets schemas evolve. Check(old, new) encodes the rules the ecosystem enforces (the same ones buf breaking checks), and HasBreaking is the gate you wire into CI. The tests pin each rule: adding an optional field is SAFE, adding a required one is BREAKING, changing a field's type on a tag is BREAKING (the wire misreads it), a rename is a WARNING (wire-OK, source change), and tightening optional→required is BREAKING.

The architect's insight: these dangerous changes don't fail to compile — they fail in production, far from the change, as corrupted data or broken consumers. So you make them fail in CI instead, as a fitness function (pa-10). "Reserve the tag" on removal is the one counter-intuitive rule: a removed tag must never be reused, or old data on that tag is silently misinterpreted.

Java/Python note: the rules are identical; the tooling differs (buf, Confluent Schema Registry with Avro/Protobuf compat modes). The policy — compatible-by-default, breaking-changes-get-a-new-version — is the architecture decision; the checker enforces it.

2. Idempotency = safe retries (idempotency.go)

Networks retry, queues deliver at-least-once (pa-03/05), gateways retry (gw-06). Without protection, "create order" and "charge card" double-apply. IdempotencyStore.Do(key, fn) executes fn once per key and returns the cached result on retries (TestIdempotencyReplay: three attempts, one side effect). Two design details that matter:

  • Failures aren't cached (TestIdempotencyDoesNotCacheFailures) — a genuinely failed operation must remain retryable.
  • Keys expire (TTL) — you can't remember every key forever; the window must exceed the client's retry horizon.

Designing idempotency into the contract (the client sends an Idempotency-Key header / field) is what makes at-least-once delivery (pa-05) safe — "effectively-once" = at-least-once + idempotent consumer.

3. Opaque cursors = freedom to change (cursor.go)

EncodeCursor/DecodeCursor produce a base64, checksum-protected position token. The client treats it as opaque; TestCursorTamperRejected shows a mangled cursor is rejected (not silently mis-served), and TestCursorRoundTrip confirms clean round-trips. Why this is API design, not a utility: an offset in the contract freezes your storage model — you can never switch to keyset pagination without breaking clients, and offsets are wrong under concurrent inserts (items shift between pages). An opaque cursor hides the mechanism, so you keep that freedom. The general principle — expose intent, hide implementation — is the heart of durable API design.

4. The architect's decision: which style per edge

There's no single right answer; it's per interface:

gRPC   — typed internal RPC, low latency, strong schemas (gw-02).
REST   — broad/external reach, cacheable, simple.
events — decouple in time, fan-out, load-level, audit (pa-03).

A real platform uses all three. The architect's contribution is choosing per edge (by coupling + reach), and governing evolution centrally (schema registry + compat checks + deprecation policy) so 50 teams ship independently without a review bottleneck.

5. Hands-on

cd src/go
bash ../scripts/verify.sh
go run ./cmd/apidemo

6. Exercises

  1. Wire the gate into CI: make HasBreaking fail a test when a committed schema breaks the previous one (a fitness function, pa-10).
  2. Reserved tags: extend Check to track reserved (removed) tags and flag a new field that reuses one — the wire-corruption case a two-schema diff can't see without history.
  3. Consumer-driven contracts: model a consumer's expected fields and verify a producer change doesn't break that specific consumer.
  4. Keyset pagination: change the cursor to encode a (last_id, last_sort_key) keyset; show the contract (opaque cursor) is unchanged — the payoff of opacity.
  5. gRPC trailers: model gRPC status-in-trailers (gw-02) and show why an HTTP-200-but-grpc-error must be counted as a failure (gw-11).