Phases & Labs
This curriculum has 7 phases and 46 labs. Phases 1–5 build a
database and the consensus protocols underneath it; Phase 6 turns to the
adjacent domain of cloud gateways and application networking (the
Netflix Cloud Gateway track); Phase 7 zooms out to platform &
distributed-systems architecture (the Apple Software Architect track) —
how you decompose, connect, partition, ship, and govern a platform.
Phases build on each other, but within Phase 4 (consensus) you can do
Raft → Paxos → ZAB in any order after the foundations in db-16, and
Phases 6 and 7 can each be read independently of Phases 2–5 once you have
the distributed-systems mindset from Phase 4.
Legend: ✅ complete · 🟡 scaffolded · ⬜ planned
Phase 1 — Storage Primitives & Foundations
Before you can build a database, you need to understand the medium it lives on.
| Lab | Title | Status | Key Concepts |
|---|---|---|---|
| db-01 | Storage Primitives | ✅ | Pages, byte order, mmap vs pread, alignment, HDD/SSD/NVMe latency |
| db-02 | Data Structures for Storage | 🟡 | Skip lists, hash tables, when in-memory vs on-disk structures differ |
| db-03 | Write-Ahead Log | 🟡 | WAL framing, CRC32, fsync semantics, group commit |
| db-04 | Bloom Filters & Hashing | 🟡 | FPR math, xxHash vs Murmur, cuckoo & xor filter alternatives |
Phase 2 — LevelDB / LSM-Tree
Build a production-shape LSM-tree key-value store, the way Google built LevelDB and Meta forked it into RocksDB.
| Lab | Title | Status | Key Concepts |
|---|---|---|---|
| db-05 | LSM MemTable | 🟡 | Skip-list MemTable, immutable MemTable, flush trigger |
| db-06 | SSTable Format | 🟡 | Data/index/filter blocks, restart points, footer |
| db-07 | LSM Compaction | 🟡 | Level vs size-tiered vs universal, write amplification |
| db-08 | Block Cache & Iterators | 🟡 | LRU, MergingIterator, snapshot via sequence numbers |
| db-09 | LevelDB Complete | 🟡 | Open/close, WriteBatch, recovery, YCSB benchmark |
Phase 3 — SQLite / B-Tree
Build a B+-tree storage engine, a pager, a SQL parser, a bytecode VM, and a transaction manager.
| Lab | Title | Status | Key Concepts |
|---|---|---|---|
| db-10 | B-Tree Fundamentals | 🟡 | B-Tree vs B+-Tree, page layout, splits & merges |
| db-11 | Pager System | 🟡 | Page cache, rollback journal vs WAL mode, checkpointing |
| db-12 | SQL Frontend | 🟡 | Tokenizer, parser, AST, VDBE bytecode VM |
| db-13 | Transactions & MVCC | 🟡 | ACID, isolation levels, SQLite locks, MVCC vs 2PL |
| db-14 | Indexes & Query Planning | 🟡 | Secondary indexes, cost-based planner, ART, BRIN |
| db-15 | SQLite Complete | 🟡 | JOINs, aggregation, TPC-H subset benchmark |
Phase 4 — Consensus Algorithms
The three canonical consensus families — implemented, tested, and compared.
| Lab | Title | Status | Key Concepts |
|---|---|---|---|
| db-16 | Distributed Fundamentals | 🟡 | CAP, FLP, linearizability, vector clocks, HLC |
| db-17 | Raft | 🟡 | Election, AppendEntries, snapshotting, ReadIndex |
| db-18 | Paxos | 🟡 | Single-decree, Multi-Paxos, Flexible Paxos |
| db-19 | ZAB | 🟡 | Epochs, zxids, primary-backup vs leader-based |
| db-20 | Distributed KV Store | 🟡 | Raft + LevelDB backend, linearizable reads, sharding |
Phase 5 — Advanced Storage & Capstone
| Lab | Title | Status | Key Concepts |
|---|---|---|---|
| db-21 | Advanced Storage | 🟡 | io_uring, O_DIRECT, columnar layout, WiscKey |
| db-22 | Performance & Benchmarking | 🟡 | YCSB A–F, flamegraphs, NUMA, perf counters |
| db-23 | Capstone Distributed DB | 🟡 | SQL → planner → LevelDB → Raft; 2PC over Raft groups |
Phase 6 — Cloud Gateway & Application Networking
The adjacent domain: the L4/L7 data plane, the API gateway, the WebSocket fleet, the control plane that programs them, and the Kubernetes substrate they run on. Built toward the Netflix Cloud Gateway role; each lab maps to a JD requirement and a Netflix talk. See gw-00 overview and the interview playbook.
| Lab | Title | Status | Key Concepts |
|---|---|---|---|
| gw-00 | Overview & Role Readiness | ✅ | JD↔lab map, talks decoded, interview/30-60-90 playbooks |
| gw-01 | The L4 Data Plane | ✅ | TCP/UDP, epoll/event loop, backpressure, PROXY protocol, drain |
| gw-02 | L7 Protocols | ✅ | HTTP/1.1, HTTP/2 frames/HPACK/flow control, gRPC, HTTP/3/QUIC |
| gw-03 | API Gateway (Zuul model) | ✅ | Filter chain, event-loop async, dynamic routing, hot-reload |
| gw-04 | Connection Management | ✅ | Pooling, per-event-loop pools, subsetting (Van der Corput), churn |
| gw-05 | WebSockets & Pushy | ✅ | RFC 6455, push registry, async delivery, density, reconnect storms |
| gw-06 | Resilience & Load Balancing | ✅ | P2C, outlier ejection, retry budgets, circuit breakers, adaptive concurrency |
| gw-07 | Edge Security (mTLS) | ✅ | TLS 1.3, mTLS, SPIFFE/SVID, cert rotation, authz, zero-trust |
| gw-08 | Envoy & xDS Control Plane | ✅ | Data/control plane split, xDS (LDS/RDS/CDS/EDS), ACK/NACK, ADS |
| gw-09 | Kubernetes Networking | ✅ | CNI, kube-proxy, Services/EndpointSlices, conntrack, drain ordering |
| gw-10 | Gateway API & Operators | ✅ | CRDs, reconcile loop, Gateway API, controller-runtime, finalizers |
| gw-11 | Data-Plane Observability | ✅ | RED/USE, histograms, trace-context propagation, SLOs, error budgets |
| gw-12 | Capstone: Gateway Migration | ✅ | Shadow→canary→ramp→soak, reversibility, stakeholder alignment, NRI/OCI |
Phase 6 note on shape: Phases 1–5 prove correctness with byte-identical cross-language dumps. Networking systems aren't byte-deterministic, so Phase 6 proves things the industry way: real, compilable,
go test -race-green mini-implementations (stdlib-only Go, offline) plus demo programs that print each lab's headline result. Each lab shipsCONCEPTS.md(the why), a maintainer- levelGUIDE.md(the deep, hands-on companion),references.md,docs/(analysis + execution + verification), code-richsteps/,scripts/verify.sh, and a workingsrc/go/you hack on. Run the whole phase withbash gw-00-cloud-gateway-overview/verify-all.sh. Start with the Hitchhiker's Guide.
Phase 7 — Platform & Distributed Systems Architecture
The architect's view: how you decompose a platform into services, connect them (sync + async), partition and keep data consistent, ship it (IaC, GitOps), keep it reliable (SLOs), and govern the decisions. Built toward the Apple Software Architect – Distributed Systems & Platform Engineering role; overlapping topics (operators/CRDs, service mesh, tracing, circuit breakers) are cross-referenced to Phase 6 / the consensus phase rather than rebuilt. See pa-00 overview, Hitchhiker's Guide, and the architect interview playbook.
| Lab | Title | Status | Key Concepts |
|---|---|---|---|
| pa-00 | Overview & Role Readiness | ✅ | JD↔lab map, architect interview playbook, 30-60-90, the architect's job |
| pa-01 | Decomposition & Contracts | ✅ | Bounded contexts, distributed-monolith detection, blast radius, layering rules |
| pa-02 | API Design (REST/gRPC/Events) | ✅ | Contract compatibility (CI gate), idempotency keys, opaque cursors, versioning |
| pa-03 | Event-Driven Architecture | ✅ | Pub/sub, at-least-once + idempotent consumers, DLQ, choreography vs orchestration |
| pa-04 | Partitioned Log (Kafka) | ✅ | Partitions, offsets, consumer groups, assignment, retention |
| pa-05 | Delivery Semantics & Sagas | ✅ | Transactional outbox, dual-write problem, sagas + compensation, crash recovery |
| pa-06 | Partitioning & Consistency | ✅ | Consistent hashing (vnodes), quorums (R+W>N), CAP/PACELC |
| pa-07 | Infrastructure as Code | ✅ | Declarative DAG, plan/diff, topological apply, state, drift detection |
| pa-08 | GitOps & Progressive Delivery | ✅ | Reconcile/sync/prune/self-heal, sync waves, SLO-gated promotion |
| pa-09 | Reliability: SLOs & Bulkheads | ✅ | Error budgets, multi-window burn-rate alerting, bulkheads |
| pa-10 | Architecture in Practice | ✅ | Fitness functions (architecture-as-tests), ADRs, design reviews, consensus |
Phase 7 shape: same as Phase 6 — real,
go test -race-green, stdlib-only Go (an event bus, a partitioned log, the outbox/saga, a consistent-hashing ring, an IaC engine, a GitOps reconciler, an SLO engine, fitness functions) + a maintainer-levelGUIDE.mdper lab. Run it all withbash pa-00-platform-architecture-overview/verify-all.sh.
Suggested Pace
- Full-time learner: ~2 labs per week ⇒ ~12 weeks end-to-end.
- Side-project learner: ~1 lab every 1–2 weeks ⇒ ~6 months.
- Reading-only path: skim
CONCEPTS.md+docs/analysis.mdper lab ⇒ ~1 week for the entire curriculum.
Recommended Progression
Phase 1 (must do all 4 in order)
│
├─→ Phase 2 (LevelDB) ──┐
│ │
└─→ Phase 3 (SQLite) ────┤
↓
Phase 4 (Consensus) ──→ Phase 6 (Cloud Gateway)
↓
Phase 5 (Capstone)
Phase 2 and Phase 3 are independent — pick the storage style that excites you first. Phase 4 only references Phase 1 fundamentals, so you can detour into consensus early if you want. Phase 5's capstone assumes all four prior phases.
Phase 6 is a standalone track. It targets the Netflix Cloud Gateway role and only assumes the distributed-systems mindset from Phase 4 (the data-plane/control-plane split echoes consensus; config propagation is a consistency problem). If you're prepping for an application-networking / gateway role, you can jump straight to gw-00 after Phase 4. Within Phase 6, do gw-01 → gw-03 in order; after that the branches are independent and gw-12 (capstone) assumes the rest.
Phase 7 is also a standalone track, aimed at the Apple Software Architect role. It assumes only the Phase 4 distributed-systems mindset and cross-references Phase 6 for the overlapping infrastructure topics (operators/CRDs, service mesh, tracing, circuit breakers). Within Phase 7, do pa-01 → pa-05 in order (the comms + data spine); after that pa-06…pa-09 are independent and pa-10 (architecture-in-practice) ties it together. Start at pa-00.