gw-10 — Analysis

The design review for the operator you build in steps/.

Required behaviors

  1. Level-triggered convergence. Reconcile drives actual → desired from current state every time; missing an event or restarting the controller never leaves the world permanently wrong.
  2. Idempotent reconcile. Reconciling the same state twice makes no additional change (a hash/version guard prevents spurious data-plane pushes — same idea as gw-08).
  3. Status reporting. Every reconciled CR gets status.conditions (Accepted, Programmed) so users/dashboards know if it took effect.
  4. Clean teardown. A finalizer deprograms the data plane before the CR is deleted; owner references cascade-delete created children.
  5. HA-safe. Leader election ensures one active reconciler; no double-writes.

Design decisions

  • controller-runtime over raw client-go. It provides caching informers (watch once, serve many reads), a rate-limited work queue, and leader election out of the box. The lab focuses on the reconcile logic, not the plumbing.

  • Reconcile = compute desired data-plane config, then push. The operator watches Gateway/HTTPRoute (+ EndpointSlices), builds the same kind of snapshot as gw-08, and pushes it (to an in-process xDS cache in the lab; to Envoy in production). This makes gw-08 and gw-10 literally the same control plane with a Kubernetes front end.

  • Content-hash versioning. The pushed config version is a hash of the desired state, so a reconcile that computes the same config is a no-op — debouncing under the heavy event churn of a busy cluster.

  • Status as the contract. The lab writes Programmed=true only after the data-plane push succeeds, so status reflects reality, not intent. A failed push sets Programmed=false with a reason — the CRD analog of an xDS NACK.

Tradeoffs worth flagging

  • Reconcile granularity vs churn. Reconciling the whole gateway on any change is simple but can thrash under churn; reconciling per-route is finer but more complex. Debounce + content-hash versioning is the pragmatic middle (gw-04/gw-08 echo).

  • Caching staleness vs API load. Informers serve cached reads (fast, low API load) but can be momentarily stale; level-triggered reconcile tolerates this (it'll re-run). Don't bypass the cache with live reads except where you truly need strong consistency.

  • Finalizer deadlocks. A finalizer that can't complete (e.g. the data plane is unreachable) blocks deletion forever. Bound the cleanup, and have a break-glass to remove a stuck finalizer — a real operational hazard to call out.

  • CRD evolution. Changing the CRD schema is itself a migration: additive optional fields are safe; renames/removals need a new version

    • conversion webhook + deprecation window. Plan it like any migration (gw-12).

What production adds beyond this lab

  • Full Gateway API conformance (all route types, ReferenceGrant, cross-namespace rules, policy attachment).
  • Admission/validation webhooks for richer validation than the OpenAPI schema allows, and defaulting webhooks.
  • Multi-tenancy + RBAC mapped to the Gateway API role split (infra/operator/app-dev).
  • Robust status + events + metrics (reconcile latency, queue depth, error rate, programmed-vs-desired drift) — gw-11.
  • Conversion webhooks and a tested upgrade path for CRD versions, plus leader election, graceful shutdown, and a tested HA failover.