Task 2: Plugin skeleton and build integration

Warning

This is not authoritative documentation. It describes a plan of work that is not yet implemented and will change as it lands. Once the work is complete this document should be removed; the finished system is described by the Iceberg storage engine spec, the plugin’s user documentation, and the repos themselves.

Repo: https://opendev.org/drizzle/drizzle (plus the drizzle-test builder-image change). Depends on task 1 (pins and linkage facts come from its README and its iceberg-deps image). One commit per repo. Goal: plugin/iceberg exists, builds when explicitly asked to, loads, registers an engine that owns no tables yet, and carries the server-level configuration. All later tasks land inside this shell.

Build integration — switch, not detection

Per the design doc’s container-first policy: no configure probes. The builder image is FROM iceberg-deps (task 1) and definitionally contains iceberg-cpp and Arrow at the pinned versions and known prefix.

  • An explicit --with-iceberg configure argument, default off. Off means the plugin directory is excluded from the build — a deliberate choice by whoever is building, not a fallback. On means the build requires the libraries at the known prefix and fails loudly if they are absent. There is no probe, no ac_cv_libiceberg, no graceful anything.

  • plugin/iceberg/plugin.ini: load_by_default=no; build_conditional keyed to the switch’s result variable (the mechanism is how the plugin build system includes/excludes directories — keep it, but its condition is the human’s switch, never a detection result); ldflags/cxxflags written out explicitly against the known prefix, the same explicit-flags discipline as the spike’s Makefile. Pin-drift shows up as a link error at build time, which is correct.

  • drizzle-test: the drizzle builder image gains the FROM iceberg-deps ancestry (or COPY --from= of the install prefix — whichever the task-1 image structure makes cleaner) and its build invocation passes --with-iceberg. The default/plain build job does not pass the switch and is unchanged in every way — that is the switch-off coverage.

  • C++ standard: the spike already compiled against the pinned headers with the tree-relevant toolchain (same image); if it surfaced a standard/flags gap, resolve it here explicitly and tree-visibly, not via per-plugin flag creep.

Engine skeleton

iceberg_engine.{h,cc}: a plugin::StorageEngine subclass (transactional conversion happens in task 5; do not pre-plumb it) registered via the module init(module::Context&) with context.add(new IcebergEngine(...)) per house pattern.

  • Flags: HTON_ALTER_NOT_SUPPORTED | HTON_TEMPORARY_NOT_SUPPORTED | HTON_SKIP_STORE_LOCK | HTON_HAS_SCHEMA_DICTIONARY (HTON_PARTIAL_COLUMN_READ arrives with the cursor in task 4).

  • Pure virtuals stubbed honestly: create() returns a cursor that refuses to open (unreachable until task 4 — assert-and-error, not silent); doCreateTable/doRenameTable/doDropTable return HA_ERR_WRONG_COMMAND-shaped errors; doGetTableIdentifiers returns nothing; doGetTableDefinition returns ENOENT; doCanCreateTable returns false (v1 is attach-only by design, not by omission — comment says so). bas_ext() returns an empty list; the engine owns no files.

Configuration

Module options via the existing plugin option machinery (note for the CLI11 migration: this plugin adds ~6 options to the boost::program_options surface; keep them boring):

  • iceberg.catalog-uri (required for the engine to activate; absent → module init logs and registers the engine in a disabled state that refuses all operations with a config error, rather than failing server start),

  • iceberg.warehouse,

  • iceberg.catalog-credential / iceberg.catalog-token (whichever auth shapes the spike validated; secrets via file-path variants, not bare CLI, matching how other plugins handle secrets),

  • iceberg.s3-endpoint / iceberg.s3-region overrides for the MinIO/CI case if the catalog does not vend FileIO config (spike README says which).

catalog_client.{h,cc}: owns the iceberg-cpp catalog handle, constructed from the options at module init; single instance owned by the engine (unique_ptr), handed around by reference. Connection validation is lazy (first use), not at init — server start must not depend on catalog availability.

Verification

  • Container build with --with-iceberg green; the untouched default build job green (switch-off coverage, zero diff to its behavior).

  • Deliberate-breakage check, once, by hand: build with the switch against an image lacking the deps → loud, immediate link/compile failure, no half-configured state.

  • Server starts with the plugin loaded and without; DATA_DICTIONARY.MODULES shows the module; the engine appears in the engine list.

  • With no catalog-uri: server starts, engine refuses operations with the config error.

  • CI: the task-1 Zuul job now builds drizzle with the switch inside the builder image and runs the (still placeholder) suite with the plugin loaded, via the same just targets a developer runs.