pa-05 — Analysis

What the patterns must get right

  1. Atomic state+event (outbox): they can never diverge (StateCount == OutboxCount).
  2. At-least-once relay: a crash between publish and mark re-publishes; the idempotent consumer makes it effectively-once.
  3. Saga compensation in reverse for exactly the completed steps.
  4. Crash recovery: resume forward from persisted progress, or compensate everything completed on failure.

Design decisions

  • Event in the same transaction as state. The whole point: one commit, no dual write. The relay is decoupled and may be at-least-once.
  • Don't make publish+mark atomic. Forcing that would recreate a distributed transaction; instead embrace at-least-once + idempotency — the honest, scalable choice.
  • Compensation is semantic undo, recorded in reverse. The result tracks completed-forward and compensated lists so the outcome is auditable.
  • RunFrom for resumption. Progress is an index persisted via a hook; resume skips the completed prefix. This models a persisted orchestrator without a real datastore.

Tradeoffs worth flagging

  • Outbox latency vs CDC complexity. Polling adds latency + DB load; CDC removes the poll but couples to the DB log and adds a connector to operate. Same guarantee.
  • Saga = eventual consistency, not isolation. Partial effects are visible during the workflow; consumers must tolerate "reserved but not yet charged" states. That's the cost of avoiding 2PC.
  • Compensations can fail. No automatic recovery beyond retries; a stuck compensation is an operational/SLO event (pa-09). Design them idempotent and reliable.
  • Orchestration adds a component. The coordinator is a service to build, scale, and make HA; choreography avoids it but scatters the workflow.

What production adds beyond this lab

  • A real DB transaction + CDC (Debezium) or a robust polling publisher.
  • A durable, replicated orchestrator (state machine persisted per saga) with retries, timeouts, and alerting on stuck sagas.
  • Idempotent consumers + a schema registry (pa-02/03) on the published events; DLQ + replay for poison events (pa-03).
  • End-to-end tracing across the async hops (gw-11).