.. warning:: This is not authoritative documentation. It describes a plan of work that is not yet implemented and will change as it lands. Task 1: slatedb-capi shim, container image, and spike ===================================================== Repos: https://opendev.org/drizzle/drizzle (``plugin/slatedb/capi/``, container) and drizzle-test (fixtures). Goal: de-risk the FFI and the dependency before any engine code exists, and land the second source-built engine dependency under the composition decision recorded in the spec. Part 1: the shim crate ---------------------- ``plugin/slatedb/capi/``: a Cargo crate, ``crate-type = ["cdylib"]``, pinning ``slatedb = "=0.14.1"`` (or current at execution — record the pin and why) with a committed ``Cargo.lock`` and ``rust-toolchain.toml`` (1.91.1, matching upstream's own pin). The dependency enables the **non-default** ``compaction_filters`` cargo feature from this first commit — task 7 needs it, and a feature flip is not a thing to discover at the end of the series. One hand-written header, ``slatedb_capi.h``, with a ``SLATEDB_CAPI_VERSION`` integer the plugin checks at init. Surface (complete list — resist additions). Every function returns an integer status and takes a trailing ``slatedb_error_t **out_err``; on any non-``OK`` status the shim writes an **owned** error object there and the caller frees it with ``slatedb_error_free``: - ``slatedb_open(url, path, settings_toml_or_null, out_db, out_err)`` — builds the object store via ``Db::resolve_object_store``, spawns the tokio runtime, opens the Db. ``slatedb_close(db, out_err)``. - ``slatedb_flush(db, out_err)``. - ``slatedb_txn_begin(db, out_txn, out_err)`` (Snapshot isolation), ``slatedb_txn_commit(txn, out_err)``, ``slatedb_txn_rollback(txn, out_err)``. - ``slatedb_txn_get(txn, key, keylen, out_value, out_err)`` with ``slatedb_value_free``; ``slatedb_txn_put(txn, k, klen, v, vlen, out_err)``; ``slatedb_txn_delete(txn, k, klen, out_err)``. - ``slatedb_txn_scan(txn, start, startlen, end, endlen, descending, out_scan, out_err)``, ``slatedb_scan_next(scan, out_k, out_v, out_err)`` (``NOT_FOUND`` = exhausted), ``slatedb_scan_seek``, ``slatedb_scan_close``. - ``slatedb_error_code(err)``, ``slatedb_error_message(err) -> const char*`` (borrowed until the free), ``slatedb_error_free(err)``. **There is no ``slatedb_last_error``.** Per-handle error strings are rejected in the spec's FFI section, and the header's comments must carry the reason because it is the thing an implementer would "simplify" back: ``DbTransaction::commit`` and ``rollback`` take ``self`` by value in 0.14.1 (``db_transaction.rs``), so a failing commit has no handle left to hold a message — and ``slatedb_scan_close`` has the same shape. Hoisting the string onto the ``Db`` handle would make it shared mutable state across every session thread. Owned errors, uniformly, no exemptions. The lifetime contract, stated in the header and asserted by the spike: - ``slatedb_txn_commit`` and ``slatedb_txn_rollback`` **always consume the transaction handle, on success and on failure alike.** On return the pointer is dangling. - **No separate free is required or permitted**: there is no ``slatedb_txn_free``, and a second call on the same pointer is a use-after-free, not a leak. - The shim moves the ``DbTransaction`` out of its box and drops the box. It deliberately does *not* keep an ``Option`` to make double calls survivable — a shim that quietly tolerates use-after-free teaches the C++ side to write it. - Consequently the C++ engine clears its session slot *before* it inspects the returned status; the error object outlives the handle by construction, which is the whole point of owning it. - Handles with a plain destructor (``slatedb_close``, ``slatedb_scan_close``, ``slatedb_value_free``, ``slatedb_error_free``) keep the ordinary paired registry pattern. Status codes: ``OK``, ``NOT_FOUND``, ``CONFLICT``, ``FENCED``, ``INVALID``, ``IO``. Every Rust ``Error`` maps into these in one ``impl``, which also builds the owned error object carrying code and message. Panics never cross the boundary: every entry point is wrapped in ``catch_unwind`` returning ``IO``. Part 2: the container image --------------------------- ``plugin/slatedb/container/Containerfile``, same lineage as WiredTiger's: ``FROM quay.io/drizzle/libdrizzle:trixie AS build`` installs the pinned Rust toolchain (rustup pinned by version and checksum, or the distro toolchain if trixie's rustc satisfies the pin — decide once, record why), runs ``cargo build --release --locked``, installs ``libslatedb_capi.so`` + ``slatedb_capi.h`` into ``/install/usr/local``; final stage copies the payload onto ``libdrizzle``. Comment discipline matching the WiredTiger Containerfile (every non-obvious choice explained in place). Also in this task, per the composition decision in the spec: the drizzle builder image gains the ``COPY --from=`` composition of wiredtiger + slatedb-capi payloads and the payload-disjointness check (``find /install -type f | sort`` manifests compared with ``uniq -d``). This is the moment the one-off wiredtiger layering becomes the pattern. Part 3: the spike program ------------------------- A standalone C++ program (plain Makefile, explicit flags, inside the image) exercising, in order: 1. Open against ``file:///tmp/spike-db``; put/get/delete round trips. 2. A transaction: begin, put N keys, scan-before-commit sees them (read-your-own-writes), commit, reopen, scan sees them. 3. Two transactions racing a write to one key: loser's commit returns ``CONFLICT``. **The error path is under test, not incidental**: the loser's ``slatedb_error_t*`` is retrieved, its message asserted non-empty and logged, and freed — after the commit that produced it already consumed the transaction handle. Then the loser's dangling handle is *not* touched again, and the spike asserts (under ASan, step 8) that no double-free or leak resulted. 4. Descending scan and seek behave as documented. 5. Kill -9 the process mid-write-load with ``await_durable`` on; reopen; verify every acknowledged commit survived. 6. Against MinIO (fixture below): repeat 1–2 and measure commit latency at default ``flush_interval``; record the numbers in the README. 7. Fencing: open the same path from a second process; verify the first process's next write fails ``FENCED``, and that a *commit* from the fenced writer likewise fails with a retrievable owned error. 8. The whole spike rebuilt with the C++ side under ``-fsanitize=address -fno-omit-frame-pointer`` (``ASAN_OPTIONS=detect_leaks=1``) and run again, including the failing commit from step 3 and the fenced commit from step 7. The practical limit, stated rather than glossed: ``cargo -Zsanitizer=address`` is nightly-only and this crate is pinned to stable 1.91.1, so **the Rust side is not instrumented**. It is built with ``RUSTFLAGS="-Cforce-frame-pointers=yes -Cdebuginfo=2"`` so ASan can symbolize through it, and the instrumented C++ links against that ordinary release ``cdylib``. What this catches is the class of bug that actually threatens this design — C++ use-after-free and double-free of consumed transaction handles, and leaked owned errors, values, and scans — not memory errors inside SlateDB. The README says exactly that, so a clean ASan run is never read as blessing the Rust. Fixtures (drizzle-test): a MinIO image pinned by digest plus a kube yaml + ``Justfile`` in the established shape (``just images``, ``just up``, ``just spike``, ``just down``). Local-filesystem mode means most CI needs no MinIO at all; the MinIO job exists to keep S3 semantics honest. README deliverables: exact pins and why (including the ``compaction_filters`` feature); commit-latency numbers from steps 5–6; the ASan caveat from step 8; the ``cargo license`` transitive inventory with the GPLv2-compat note per the spec's license posture; any API gaps found. **Gate**: if steps 2, 3, or 5 fail against the pinned release, stop and report — the transaction and durability semantics are the engine's foundation and the plan re-sequences around what the library actually does. Step 8 is equally blocking: an owned-error contract that leaks is strictly worse than the per-handle string it replaced. Verification ------------ - ``podman build`` green for the capi image and the composed builder. - Spike green under both ``file://`` and MinIO, run by a non-voting Zuul job in ``vouched`` using the identical ``just`` targets. - Header + crate reviewed as the FFI contract; ``SLATEDB_CAPI_VERSION`` starts at 1. The consuming-call and owned-error rules are reviewed as the contract's load-bearing clauses, not as comments.