pa-04 — References
The log model
- Jay Kreps, The Log: What every software engineer should know about real-time data's unifying abstraction — the foundational essay. https://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying
- Apache Kafka docs — topics, partitions, offsets, consumer groups, rebalancing (range/round-robin/sticky/cooperative), retention, delivery semantics. https://kafka.apache.org/documentation/
- Designing Data-Intensive Applications (Kleppmann) — ch. 11 (stream processing) and "turning the database inside out."
- Kafka: The Definitive Guide (Narkhede et al.).
Contrasting brokers
- Apache Pulsar (segmented log + tiered storage), NATS JetStream (subjects + streams), RabbitMQ (queues/exchanges, per-message ack). Know the log-vs-queue distinction and when each fits.
Related concepts
- db-03 (write-ahead log) — the same append-only primitive for recovery.
- Event sourcing / CQRS (microservices.io) — the log as source of truth.
- Sticky/cooperative rebalancing (KIP-429) — minimal-movement assignment, the gw-04/pa-06 stability theme.
Cross-lab links
- pa-03 (the patterns that run on this log), pa-05 (outbox → log), pa-06 (partitioning + rebalancing for data), pa-09 (consumer lag as an SLI), gw-04 (minimal-movement reassignment).