Introduction
Overview
1.
Phases & Labs
2.
Glossary
3.
Toolchain Setup
Phase 1 — Storage Primitives & Foundations
4.
Lab 01 — Storage Primitives
❱
4.1.
References
4.2.
Analysis
4.3.
Execution Guide
4.4.
Observation & Debugging
4.5.
Verification
4.6.
Broader Ideas
4.7.
Step 1 — Open and pwrite
4.8.
Step 2 — pread and hexdump
4.9.
Step 3 — Benchmark and page cache
5.
Lab 02 — Data Structures for Storage
❱
5.1.
References
5.2.
Analysis
5.3.
Execution Guide
5.4.
Observation & Debugging
5.5.
Verification
5.6.
Broader Ideas
5.7.
Step 1 — Skip list
5.8.
Step 2 — Hash table
5.9.
Step 3 — Bench compare
6.
Lab 03 — Write-Ahead Log
❱
6.1.
References
6.2.
Analysis
6.3.
Execution Guide
6.4.
Observation & Debugging
6.5.
Verification
6.6.
Broader Ideas
6.7.
Step 1 — Record framing
6.8.
Step 2 — Append, sync, iterate
6.9.
Step 3 — Group commit bench
7.
Lab 04 — Bloom Filters & Hashing
❱
7.1.
References
7.2.
Analysis
7.3.
Execution Guide
7.4.
Observation & Debugging
7.5.
Verification
7.6.
Broader Ideas
7.7.
Step 1 — Hash chain
7.8.
Step 2 — Bit vector add and contains
7.9.
Step 3 — Sizing, encode, FPR
Phase 2 — LevelDB / LSM-Tree
8.
Lab 05 — LSM MemTable
❱
8.1.
References
8.2.
Analysis
8.3.
Execution Guide
8.4.
Observation & Debugging
8.5.
Verification
8.6.
Broader Ideas
8.7.
Step 1 — Sorted map
8.8.
Step 2 — Encode and decode
8.9.
Step 3 — CLI cross-test
9.
Lab 06 — SSTable Format
❱
9.1.
References
9.2.
Analysis
9.3.
Execution Guide
9.4.
Observation & Debugging
9.5.
Verification
9.6.
Broader Ideas
9.7.
Step 1 — Data block
9.8.
Step 2 — Writer, index, footer
9.9.
Step 3 — CLI cross-test
10.
Lab 07 — LSM Compaction
❱
10.1.
References
10.2.
Analysis
10.3.
Execution Guide
10.4.
Observation & Debugging
10.5.
Verification
10.6.
Broader Ideas
10.7.
Step 1 — K-way merge
10.8.
Step 2 — Tombstone purging
10.9.
Step 3 — CLI cross-test
11.
Lab 08 — Block Cache & Iterators
❱
11.1.
References
11.2.
Analysis
11.3.
Execution Guide
11.4.
Observation & Debugging
11.5.
Verification
11.6.
Broader Ideas
11.7.
Step 1 — LRU block cache
11.8.
Step 2 — Merging iterator
11.9.
Step 3 — CLI cross-test
12.
Lab 09 — LevelDB Complete
❱
12.1.
References
12.2.
Analysis
12.3.
Execution Guide
12.4.
Observation & Debugging
12.5.
Verification
12.6.
Broader Ideas
12.7.
Step 1 — Write path
12.8.
Step 2 — Flush and recovery
12.9.
Step 3 — CLI cross-test
Phase 3 — SQLite / B-Tree
13.
Lab 10 — B-Tree Fundamentals
❱
13.1.
References
13.2.
Analysis
13.3.
Execution Guide
13.4.
Observation & Debugging
13.5.
Verification
13.6.
Broader Ideas
13.7.
Step 1 — Tree shape and get
13.8.
Step 2 — Insert and delete
13.9.
Step 3 — Serialize and cross-test
14.
Lab 11 — Pager System
❱
14.1.
References
14.2.
Analysis
14.3.
Execution Guide
14.4.
Observation & Debugging
14.5.
Verification
14.6.
Broader Ideas
14.7.
Step 1 — Page I/O and layout
14.8.
Step 2 — LRU cache
14.9.
Step 3 — Cross-language bytes
15.
Lab 12 — SQL Frontend
❱
15.1.
References
15.2.
Analysis
15.3.
Execution Guide
15.4.
Observation & Debugging
15.5.
Verification
15.6.
Broader Ideas
15.7.
Step 1 — Tokenizer
15.8.
Step 2 — Parser and AST
15.9.
Step 3 — Serializer and cross-test
16.
Lab 13 — Transactions & MVCC
❱
16.1.
References
16.2.
Analysis
16.3.
Execution Guide
16.4.
Observation & Debugging
16.5.
Verification
16.6.
Broader Ideas
16.7.
Step 1 — Version chain and snapshot read
16.8.
Step 2 — Write conflict detection
16.9.
Step 3 — Cross-language bytes
17.
Lab 14 — Indexes & Query Optimization
❱
17.1.
References
17.2.
Analysis
17.3.
Execution Guide
17.4.
Observation & Debugging
17.5.
Verification
17.6.
Broader Ideas
17.7.
Step 1 — Secondary index
17.8.
Step 2 — Rule-based planner
17.9.
Step 3 — Cross-language bytes
18.
Lab 15 — SQLite Complete
❱
18.1.
References
18.2.
Analysis
18.3.
Execution Guide
18.4.
Observation & Debugging
18.5.
Verification
18.6.
Broader Ideas
18.7.
Step 1 — Pager and rows
18.8.
Step 2 — SQL surface and MVCC
18.9.
Step 3 — Cross-language snapshot
Phase 4 — Consensus Algorithms
19.
Lab 16 — Distributed Fundamentals
❱
19.1.
References
19.2.
Analysis
19.3.
Execution Guide
19.4.
Observation & Debugging
19.5.
Verification
19.6.
Broader Ideas
19.7.
Step 1 — Logical clocks
19.8.
Step 2 — Deterministic simulator
19.9.
Step 3 — CLI cross-test
20.
Lab 17 — Raft
❱
20.1.
References
20.2.
Analysis
20.3.
Execution Guide
20.4.
Observation & Debugging
20.5.
Verification
20.6.
Broader Ideas
20.7.
Step 1 — Leader election
20.8.
Step 2 — Log replication
20.9.
Step 3 — Cross-test and partition
21.
Lab 18 — Paxos
❱
21.1.
References
21.2.
Analysis
21.3.
Execution Guide
21.4.
Observation & Debugging
21.5.
Verification
21.6.
Broader Ideas
21.7.
Step 1 — Single-decree Paxos
21.8.
Step 2 — Multi-Paxos log
21.9.
Step 3 — Cross-language determinism
22.
Lab 19 — ZAB
❱
22.1.
References
22.2.
Analysis
22.3.
Execution Guide
22.4.
Observation & Debugging
22.5.
Verification
22.6.
Broader Ideas
22.7.
Step 1 — Epoch and leader election
22.8.
Step 2 — Discovery, sync, and broadcast
22.9.
Step 3 — Cross-language determinism
23.
Lab 20 — Distributed KV Store
❱
23.1.
References
23.2.
Analysis
23.3.
Execution Guide
23.4.
Observation & Debugging
23.5.
Verification
23.6.
Broader Ideas
23.7.
Step 1 — Cluster and replica
23.8.
Step 2 — Quorum replication
23.9.
Step 3 — Partitions and catch-up
Phase 5 — Advanced Storage & Capstone
24.
Lab 21 — Advanced Storage Engine
❱
24.1.
References
24.2.
Analysis
24.3.
Execution Guide
24.4.
Observation & Debugging
24.5.
Verification
24.6.
Broader Ideas
24.7.
Step 1 — Range tombstones
24.8.
Step 2 — Tiered and universal compaction
24.9.
Step 3 — Cross-language bytes
25.
Lab 22 — Performance & Benchmarking
❱
25.1.
References
25.2.
Analysis
25.3.
Execution Guide
25.4.
Observation & Debugging
25.5.
Verification
25.6.
Broader Ideas
25.7.
Step 1 — Counter store
25.8.
Step 2 — Snapshot and workload
25.9.
Step 3 — Bench harness
26.
Lab 23 — Capstone: Distributed Database
❱
26.1.
References
26.2.
Analysis
26.3.
Execution Guide
26.4.
Observation & Debugging
26.5.
Verification
26.6.
Broader Ideas
26.7.
Step 1 — Cluster and log
26.8.
Step 2 — Replication and commit
26.9.
Step 3 — Fault injection and catch-up
Phase 6 — Cloud Gateway & Application Networking
27.
Phase 6 Overview & Role Readiness
❱
27.1.
Hitchhiker's Guide (warm-up)
27.2.
Interview Playbook
27.3.
References
28.
Lab GW-01 — The L4 Data Plane
❱
28.1.
Guide (deep dive)
28.2.
References
28.3.
Analysis
28.4.
Execution
28.5.
Verification
28.6.
Step 1 — TCP proxy & backpressure
28.7.
Step 2 — PROXY protocol & sockopts
28.8.
Step 3 — Graceful drain & load test
29.
Lab GW-02 — L7 Protocols (HTTP/2, gRPC, HTTP/3)
❱
29.1.
Guide (deep dive)
29.2.
References
29.3.
Analysis
29.4.
Execution
29.5.
Verification
29.6.
Step 1 — HTTP/2 frame parser
29.7.
Step 2 — HPACK & flow control
29.8.
Step 3 — gRPC & the LB trap
30.
Lab GW-03 — API Gateway (The Zuul Model)
❱
30.1.
Guide (deep dive)
30.2.
References
30.3.
Analysis
30.4.
Execution
30.5.
Verification
30.6.
Step 1 — Filter chain
30.7.
Step 2 — Routing & hot-reload
30.8.
Step 3 — Async endpoint & errors
31.
Lab GW-04 — Connection Management (Curbing Churn)
❱
31.1.
Guide (deep dive)
31.2.
References
31.3.
Analysis
31.4.
Execution
31.5.
Verification
31.6.
Step 1 — Pool & measure churn
31.7.
Step 2 — Per-event-loop pools
31.8.
Step 3 — Subsetting (Van der Corput)
32.
Lab GW-05 — WebSockets & Pushy
❱
32.1.
Guide (deep dive)
32.2.
References
32.3.
Analysis
32.4.
Execution
32.5.
Verification
32.6.
Step 1 — WebSocket server & registry
32.7.
Step 2 — Async delivery via queue
32.8.
Step 3 — Drain & reconnect storm
33.
Lab GW-06 — Resilience & Load Balancing
❱
33.1.
Guide (deep dive)
33.2.
References
33.3.
Analysis
33.4.
Execution
33.5.
Verification
33.6.
Step 1 — Power of two choices
33.7.
Step 2 — Adaptive concurrency
33.8.
Step 3 — Retry storms & budgets
34.
Lab GW-07 — Edge Security (mTLS & Zero-Trust)
❱
34.1.
Guide (deep dive)
34.2.
References
34.3.
Analysis
34.4.
Execution
34.5.
Verification
34.6.
Step 1 — mTLS & identity
35.
Lab GW-08 — Envoy & the xDS Control Plane
❱
35.1.
Guide (deep dive)
35.2.
References
35.3.
Analysis
35.4.
Execution
35.5.
Verification
35.6.
Step 1 — Minimal xDS server
35.7.
Step 2 — Reconcile & dynamic endpoints
36.
Lab GW-09 — Kubernetes Networking Internals
❱
36.1.
Guide (deep dive)
36.2.
References
36.3.
Analysis & Exercises
36.4.
Execution
36.5.
Verification
37.
Lab GW-10 — Gateway API, CRDs & Operators
❱
37.1.
Guide (deep dive)
37.2.
References
37.3.
Analysis
37.4.
Execution
37.5.
Verification
37.6.
Step 1 — CRD & reconcile loop
37.7.
Step 2 — Program data plane & finalizers
38.
Lab GW-11 — Data-Plane Observability
❱
38.1.
Guide (deep dive)
38.2.
References
38.3.
Analysis
38.4.
Execution
38.5.
Verification
38.6.
Step 1 — RED metrics & trace propagation
39.
Lab GW-12 — Capstone: Gateway Migration
❱
39.1.
Guide (deep dive)
39.2.
References
39.3.
Analysis & Capstone Exercise
39.4.
Execution
39.5.
Verification
Phase 7 — Platform & Distributed Systems Architecture
40.
Phase 7 Overview & Role Readiness
❱
40.1.
Hitchhiker's Guide (warm-up)
40.2.
Architect Interview Playbook
40.3.
References
41.
Lab PA-01 — Microservices Decomposition & Contracts
❱
41.1.
Guide (deep dive)
41.2.
References
41.3.
Analysis
41.4.
Execution
41.5.
Verification
42.
Lab PA-02 — API Design (REST, gRPC, Events)
❱
42.1.
Guide (deep dive)
42.2.
References
42.3.
Analysis
42.4.
Execution
42.5.
Verification
43.
Lab PA-03 — Event-Driven Architecture
❱
43.1.
Guide (deep dive)
43.2.
References
43.3.
Analysis
43.4.
Execution
43.5.
Verification
44.
Lab PA-04 — A Partitioned Log (Kafka in Miniature)
❱
44.1.
Guide (deep dive)
44.2.
References
44.3.
Analysis
44.4.
Execution
44.5.
Verification
45.
Lab PA-05 — Delivery Semantics: Outbox & Sagas
❱
45.1.
Guide (deep dive)
45.2.
References
45.3.
Analysis
45.4.
Execution
45.5.
Verification
46.
Lab PA-06 — Partitioning & Consistency Models
❱
46.1.
Guide (deep dive)
46.2.
References
46.3.
Analysis
46.4.
Execution
46.5.
Verification
47.
Lab PA-07 — Infrastructure as Code
❱
47.1.
Guide (deep dive)
47.2.
References
47.3.
Analysis
47.4.
Execution
47.5.
Verification
48.
Lab PA-08 — GitOps & Progressive Delivery
❱
48.1.
Guide (deep dive)
48.2.
References
48.3.
Analysis
48.4.
Execution
48.5.
Verification
49.
Lab PA-09 — Reliability: SLOs & Bulkheads
❱
49.1.
Guide (deep dive)
49.2.
References
49.3.
Analysis
49.4.
Execution
49.5.
Verification
50.
Lab PA-10 — Architecture in Practice
❱
50.1.
Guide (deep dive)
50.2.
References
50.3.
Analysis
50.4.
Execution
50.5.
Verification
50.6.
ADR Template
50.7.
Design-Review Checklist
Light
Rust
Coal
Navy
Ayu
Distributed Systems Engineer — Build Databases & Consensus From Scratch
pa-07 — References
IaC engines
Terraform
docs — resources, the dependency graph, plan/apply, state + locking,
import
, modules, providers.
https://developer.hashicorp.com/terraform/docs
Pulumi
docs — the same model with general-purpose languages instead of HCL; the state/engine concepts are identical.
Crossplane
— the reconcile loop
inside
Kubernetes (operators, gw-10) instead of an external CLI.
Terraform: Up & Running
(Brikman) — state, modules, gotchas.
Concepts
Kahn's algorithm / topological sort (apply ordering).
Declarative vs imperative; idempotency; convergence — the reconcile paradigm shared with Kubernetes (gw-10), GitOps (pa-08), xDS (gw-08).
Infrastructure as Code
(Kief Morris) — patterns and practices.
Cross-lab links
pa-08
(GitOps runs this loop continuously + self-heal),
gw-08/gw-10
(the same reconcile loop in a control plane / k8s),
pa-01
(dependency DAG + topo + cycle detection),
db-17
(desired-state convergence as the consensus instinct).