db-09 — Execution

What we built, in the order we built it.

1. Rust (src/rust)

  • Cargo.toml declares crate leveldb09 (lib) and a binary dbctl.
  • path dependencies to db-03-write-ahead-log, db-05-lsm-memtable, db-06-sstable-format, and db-08-block-cache-and-iterators. No network-fetched deps.
  • src/lib.rs defines Db, WriteBatch, Op, OpType, and re-uses the upstream types directly (wal::Wal, memtable::{MemTable, EntryType}, sstable::{SstReader, SstWriter}, blockcache::{MergingIterator, SerializeStream, EntriesFromReader}).
  • 11 inline #[cfg(test)] tests covering: batch round-trip, batch trailing- byte rejection, memtable put/get, delete-shadows-value, flush+memtable cleared, flush+recovery, WAL replay, newest-SST-wins, scan dedupe and tombstone drop, deterministic serialize_view, recovery with both an SST and a non-empty WAL tail.
  • bin/dbctl.rs is a stdin-driven CLI used by the cross-language script.

2. Go (src/go)

  • go.mod module github.com/10xdev/dse/db09 with replace directives pointing at the sibling labs' Go modules.
  • db.go ports the Rust API one-for-one. The WriteBatch wire format is byte-for-byte identical (u32 LE count, then per op: type byte, u32 LE klen, key, optional u32 LE vlen + value).
  • db_test.go mirrors all 11 Rust tests.
  • cmd/dbctl/main.go is the matching CLI.

3. C++ (src/cpp)

  • CMakeLists.txt compiles upstream .cc files directly into local static libraries (wal_lib, memtable_lib, sstable_lib, blockcache_lib). We do not add_subdirectory(../../../db-NN) because that would leak the upstream lab's add_test calls into our ctest.
  • src/db.h and src/db.cc provide db09::Db, constructed via Db::Open(dir) -> std::unique_ptr<Db>. Db is non-copyable and non-movable (its dse::wal::Wal member is itself non-copyable, and exposing moves would require fiddly forwarding for little gain in a one-process toy).
  • WAL move-assignment (wal_ = dse::wal::Wal::Open(path)) is what makes the post-flush WAL reset work; this required confirming the upstream header declares Wal& operator=(Wal&&) noexcept.
  • src/dbctl.cc and tests/test_db09.cc mirror their Rust/Go siblings. The test file uses #undef NDEBUG before <cassert> to guarantee asserts fire under Release builds.

4. Scripts

  • scripts/verify.sh builds and runs each implementation's unit tests.
  • scripts/cross_test.sh:
    1. Builds Rust/Go/C++ dbctl binaries.
    2. Defines one canonical command script (run.script) covering multi- flush, overwrites that land in newer SSTables, tombstones, and a non-empty WAL tail.
    3. For each language: pipes run.script into dbctl --dir db-LANG (writes + close), then reopens the same dir and pipes DUMP and DUMP_WITH_TOMBS into separate files. Reopen forces a real WAL replay and SST reload path.
    4. Computes sha256 of DUMP and DUMP_WITH_TOMBS for each language and asserts all three match.
    5. Spot-checks the rust DUMP stream hex for the presence of the expected final key-value bytes (b=222, e=5) and the expected tombstone bytes (key a in DUMP_WITH_TOMBS).

What we deliberately didn't build

  • Compaction — db-21.
  • Block cache wiring inside Db — db-08 has the cache; db-09 doesn't need it because each SSTable reader already holds the file bytes in memory. We'll plug in the LRU during db-21 when SSTable I/O becomes cold.
  • Bloom-filter probing — db-04 has bloom; db-21 will skip SSTables whose Bloom rejects the key.