pa-08 — Analysis

What the reconciler must get right

  1. Sync create/update from git; self-heal revert manual drift (same mechanism, run continuously).
  2. Prune only when enabled (destructive, opt-in).
  3. Sync-wave ordering (low waves first).
  4. Idempotent converged passes; accurate Diff.
  5. SLO-gated promotion with instant rollback to the live version.

Design decisions

  • Pull/converge model. Reconcile takes desired (git) + live (cluster) and mutates live toward desired — the in-cluster agent model, not a push pipeline. This is what gives continuous self-heal.
  • Self-heal == sync. A drifted resource is just "live differs from desired," handled by the same apply path. The power is in running it on a loop, not in special-casing drift.
  • Prune is opt-in. "Delete what git doesn't declare" is powerful and dangerous; a flag (and, in production, owner-reference scoping) guards it.
  • Deterministic wave+name ordering. Reproducible apply order for tests and for declaration-before-use (CRDs first).

Tradeoffs worth flagging

  • Self-heal vs break-glass. Continuous reversion fights emergency manual fixes; production needs a pause switch and should treat recurring drift as a git bug.
  • Prune blast radius. A mis-scoped app or uncommitted resource can be deleted; scope with owner refs, review prune diffs.
  • One-repo blast radius. A bad commit can sync everywhere; structure by app/env + progressive delivery + staged rollout (gw-12).
  • Pull adds reconcile latency. Convergence is eventual (poll/notify interval); fine for config, but not a substitute for a real rollout gate on risky changes.

What production adds beyond this lab

  • Real source/kustomize/helm controllers, health assessment, app-of-apps, RBAC, drift notifications, and a pause/break-glass.
  • Progressive delivery (Argo Rollouts/Flagger) with SLO analysis (pa-09) and the full ladder (gw-12).
  • Secrets management and multi-cluster/multi-env promotion via git merges.