Warning

This is not authoritative documentation. It describes a plan of work that is not yet implemented and will change as it lands.

Task 1: slatedb-capi shim, container image, and spike

Repos: https://opendev.org/drizzle/drizzle (plugin/slatedb/capi/, container) and drizzle-test (fixtures). Goal: de-risk the FFI and the dependency before any engine code exists, and land the second source-built engine dependency under the composition decision recorded in the spec.

Part 1: the shim crate

plugin/slatedb/capi/: a Cargo crate, crate-type = ["cdylib"], pinning slatedb = "=0.14.1" (or current at execution — record the pin and why) with a committed Cargo.lock and rust-toolchain.toml (1.91.1, matching upstream’s own pin). The dependency enables the non-default compaction_filters cargo feature from this first commit — task 7 needs it, and a feature flip is not a thing to discover at the end of the series. One hand-written header, slatedb_capi.h, with a SLATEDB_CAPI_VERSION integer the plugin checks at init.

Surface (complete list — resist additions). Every function returns an integer status and takes a trailing slatedb_error_t **out_err; on any non-OK status the shim writes an owned error object there and the caller frees it with slatedb_error_free:

• slatedb_open(url, path, settings_toml_or_null, out_db, out_err) — builds the object store via Db::resolve_object_store, spawns the tokio runtime, opens the Db. slatedb_close(db, out_err).

• slatedb_flush(db, out_err).

• slatedb_txn_begin(db, out_txn, out_err) (Snapshot isolation), slatedb_txn_commit(txn, out_err), slatedb_txn_rollback(txn, out_err).

• slatedb_txn_get(txn, key, keylen, out_value, out_err) with slatedb_value_free; slatedb_txn_put(txn, k, klen, v, vlen, out_err); slatedb_txn_delete(txn, k, klen, out_err).

• slatedb_txn_scan(txn, start, startlen, end, endlen, descending, out_scan, out_err), slatedb_scan_next(scan, out_k, out_v, out_err) (NOT_FOUND = exhausted), slatedb_scan_seek, slatedb_scan_close.

• slatedb_error_code(err), slatedb_error_message(err) -> const char* (borrowed until the free), slatedb_error_free(err).

There is no ``slatedb_last_error``. Per-handle error strings are rejected in the spec’s FFI section, and the header’s comments must carry the reason because it is the thing an implementer would “simplify” back: DbTransaction::commit and rollback take self by value in 0.14.1 (db_transaction.rs), so a failing commit has no handle left to hold a message — and slatedb_scan_close has the same shape. Hoisting the string onto the Db handle would make it shared mutable state across every session thread. Owned errors, uniformly, no exemptions.

The lifetime contract, stated in the header and asserted by the spike:

• slatedb_txn_commit and slatedb_txn_rollback always consume the transaction handle, on success and on failure alike. On return the pointer is dangling.

• No separate free is required or permitted: there is no slatedb_txn_free, and a second call on the same pointer is a use-after-free, not a leak.

• The shim moves the DbTransaction out of its box and drops the box. It deliberately does not keep an Option<DbTransaction> to make double calls survivable — a shim that quietly tolerates use-after-free teaches the C++ side to write it.

• Consequently the C++ engine clears its session slot before it inspects the returned status; the error object outlives the handle by construction, which is the whole point of owning it.

• Handles with a plain destructor (slatedb_close, slatedb_scan_close, slatedb_value_free, slatedb_error_free) keep the ordinary paired registry pattern.

Status codes: OK, NOT_FOUND, CONFLICT, FENCED, INVALID, IO. Every Rust Error maps into these in one impl, which also builds the owned error object carrying code and message.

Panics never cross the boundary: every entry point is wrapped in catch_unwind returning IO.

Part 2: the container image

plugin/slatedb/container/Containerfile, same lineage as WiredTiger’s: FROM quay.io/drizzle/libdrizzle:trixie AS build installs the pinned Rust toolchain (rustup pinned by version and checksum, or the distro toolchain if trixie’s rustc satisfies the pin — decide once, record why), runs cargo build --release --locked, installs libslatedb_capi.so + slatedb_capi.h into /install/usr/local; final stage copies the payload onto libdrizzle. Comment discipline matching the WiredTiger Containerfile (every non-obvious choice explained in place).

Also in this task, per the composition decision in the spec: the drizzle builder image gains the COPY --from= composition of wiredtiger + slatedb-capi payloads and the payload-disjointness check (find /install -type f | sort manifests compared with uniq -d). This is the moment the one-off wiredtiger layering becomes the pattern.

Part 3: the spike program

A standalone C++ program (plain Makefile, explicit flags, inside the image) exercising, in order:

1. Open against file:///tmp/spike-db; put/get/delete round trips.

2. A transaction: begin, put N keys, scan-before-commit sees them (read-your-own-writes), commit, reopen, scan sees them.

3. Two transactions racing a write to one key: loser’s commit returns CONFLICT. The error path is under test, not incidental: the loser’s slatedb_error_t* is retrieved, its message asserted non-empty and logged, and freed — after the commit that produced it already consumed the transaction handle. Then the loser’s dangling handle is not touched again, and the spike asserts (under ASan, step 8) that no double-free or leak resulted.

4. Descending scan and seek behave as documented.

5. Kill -9 the process mid-write-load with await_durable on; reopen; verify every acknowledged commit survived.

6. Against MinIO (fixture below): repeat 1–2 and measure commit latency at default flush_interval; record the numbers in the README.

7. Fencing: open the same path from a second process; verify the first process’s next write fails FENCED, and that a commit from the fenced writer likewise fails with a retrievable owned error.

8. The whole spike rebuilt with the C++ side under -fsanitize=address -fno-omit-frame-pointer (ASAN_OPTIONS=detect_leaks=1) and run again, including the failing commit from step 3 and the fenced commit from step 7. The practical limit, stated rather than glossed: cargo -Zsanitizer=address is nightly-only and this crate is pinned to stable 1.91.1, so the Rust side is not instrumented. It is built with RUSTFLAGS="-Cforce-frame-pointers=yes -Cdebuginfo=2" so ASan can symbolize through it, and the instrumented C++ links against that ordinary release cdylib. What this catches is the class of bug that actually threatens this design — C++ use-after-free and double-free of consumed transaction handles, and leaked owned errors, values, and scans — not memory errors inside SlateDB. The README says exactly that, so a clean ASan run is never read as blessing the Rust.

Fixtures (drizzle-test): a MinIO image pinned by digest plus a kube yaml + Justfile in the established shape (just images, just up, just spike, just down). Local-filesystem mode means most CI needs no MinIO at all; the MinIO job exists to keep S3 semantics honest.

README deliverables: exact pins and why (including the compaction_filters feature); commit-latency numbers from steps 5–6; the ASan caveat from step 8; the cargo license transitive inventory with the GPLv2-compat note per the spec’s license posture; any API gaps found.

Gate: if steps 2, 3, or 5 fail against the pinned release, stop and report — the transaction and durability semantics are the engine’s foundation and the plan re-sequences around what the library actually does. Step 8 is equally blocking: an owned-error contract that leaks is strictly worse than the per-handle string it replaced.

Verification

• podman build green for the capi image and the composed builder.

• Spike green under both file:// and MinIO, run by a non-voting Zuul job in vouched using the identical just targets.

• Header + crate reviewed as the FFI contract; SLATEDB_CAPI_VERSION starts at 1. The consuming-call and owned-error rules are reviewed as the contract’s load-bearing clauses, not as comments.