gw-08 — Analysis

The design review for the xDS control plane you build in steps/.

Required behaviors

  1. Off the request path. The control plane computes and pushes config; Envoy serves from cached config. A control-plane outage must not drop traffic (last-known-good).
  2. Versioned, atomic snapshots. Each push is a consistent set at a version; Envoy never sees a half-applied snapshot.
  3. Ordering. Within a snapshot, clusters are consistent with the endpoints and routes that reference them (ADS guarantees this).
  4. ACK/NACK visibility. The control plane records which version each node applied and surfaces NACKs (rejected config) as alerts.
  5. Per-node config. Different Envoys (by node ID) can get different snapshots (canary vs baseline) for staged rollout.

Design decisions

  • SnapshotCache keyed by node ID. go-control-plane's cache holds one versioned snapshot per node group. Setting a new snapshot triggers the push; the library handles the stream, ACK/NACK, and nonce. The lab focuses on computing good snapshots.

  • ADS (single aggregated stream). The lab uses ADS so cluster/ endpoint/route ordering is correct by construction — demonstrating why it matters by first (optionally) showing the SotW race without it.

  • A tiny reconcile loop. A goroutine watches a mock "source of truth" (a file or an in-memory registry that you mutate) and rebuilds the snapshot on change, bumping the version. This previews the Kubernetes controller in gw-10 (same shape: watch → reconcile → desired state).

  • Consistent versioning. The version string is derived from a hash of the desired state, so identical desired state ⇒ identical version ⇒ no spurious pushes (debouncing). This mirrors the determinism discipline from the consensus phase.

Tradeoffs worth flagging

  • SotW simplicity vs Delta scale. State-of-the-world re-sends the full set per type; fine for small fleets/resource sets, costly when EDS has thousands of endpoints churning. Delta xDS is the scale answer; the lab notes where it would plug in. Netflix's on-demand discovery is the "don't push everything to everyone" extreme.

  • Push-all vs staged rollout. Bumping every node's snapshot at once maximizes blast radius. Per-node snapshots let you canary, at the cost of managing more snapshot state and a rollout controller (gw-12).

  • Reconcile debounce vs freshness. Rebuilding on every source change can thrash under rapid churn (pod flapping in gw-09); debouncing adds latency to legitimate changes. Tune the debounce window; it's the same trade-off as re-subsetting in gw-04.

  • Control-plane HA. The control plane should be replicated; but because it's off the request path, its consistency model can be looser than the data plane's availability requirement. A short control-plane outage is tolerable; a data-plane outage is not.

What production adds beyond this lab

  • Delta xDS and on-demand resource discovery (push only what a node needs) for fleet/endpoint scale.
  • A real source of truth: the Kubernetes API (gw-10) or a service registry, with watches and resync.
  • SDS for certs (gw-07) and runtime/RTDS for feature flags.
  • Rollout safety: per-node-group canaries, automated rollback on NACK or SLO breach, and a config-validation gate before any push (gw-12).
  • Deep observability: per-node applied version, NACK rate + reasons, push latency, snapshot size, ADS stream health (gw-11).