Task 8: Dictionary tables and time travel
=========================================

.. warning:: This is not authoritative documentation.  It describes a
   plan of work that is not yet implemented and will change as it lands.
   Once the work is complete this document should be removed; the
   finished system is described by the *Iceberg storage engine* spec,
   the plugin's user documentation, and the repos themselves.

Repo: https://opendev.org/drizzle/drizzle. Depends on task 4. Floats
relative to 5/6/7. Two commits. Goal: Iceberg's metadata becomes
queryable the Drizzle way, and the pinning machinery gains its
explicit-pin writer.

Commit 1: DATA_DICTIONARY tables
--------------------------------

``plugin::TableFunction`` implementations inside ``plugin/iceberg`` (the
``registry_dictionary`` pattern), registered only when the engine is
active:

-  ``DATA_DICTIONARY.ICEBERG_SNAPSHOTS(schema, table, snapshot_id, parent_id, committed_at, operation, summary_json)``
   — one row per snapshot per attached table. Enumerating every table's
   history on a full scan of this dictionary is catalog-request-heavy;
   that is inherent and acceptable for a dictionary table (document it)
   — but implement the standard WHERE-on-schema/table narrowing if the
   TableFunction interface's existing usage supports it (check how
   schema_dictionary handles it; mirror that, invent nothing).
-  ``DATA_DICTIONARY.ICEBERG_MANIFESTS(...)`` and
   ``DATA_DICTIONARY.ICEBERG_DATA_FILES(schema, table, snapshot_id, file_path, record_count, file_size, partition_json)``
   — straight from iceberg-cpp metadata for the *pinned* snapshot of the
   querying session (so a transaction inspecting a table sees the
   metadata of what it is reading — the pin map is already the source of
   truth; reuse it).

All values come from the library's metadata objects; no manifest parsing
of our own.

Commit 2: explicit pins (time travel)
-------------------------------------

A session variable, registered via the module's variable mechanism:

-  ``SET @@iceberg_as_of_snapshot = <snapshot-id>`` scoped by a
   companion ``@@iceberg_as_of_table = 'schema.table'`` — or, cleaner if
   the variable machinery allows structured values poorly, a single
   ``@@iceberg_as_of = 'schema.table@snapshot_id'`` string parsed by the
   engine. Pick whichever is least clever; the variable is an interim UX
   until/unless ``AS OF`` grammar happens, and grammar would be a third
   writer to the same map — say so in a comment, add nothing
   speculative.
-  Semantics: setting it writes an explicit ``PinEntry`` into the
   session pin map (task 4's structure, unchanged); the pin survives
   until the variable is cleared (``= ''``/``= 0``) or the session ends
   — explicit pins are *not* cleared at transaction end (that is their
   point); the pin-map clearing logic distinguishes implicit from
   explicit entries.
-  Validation at set time: table attached, snapshot exists (specific
   errors).
-  Interaction with writes: an INSERT into a table with an explicit
   historical pin is refused (appending "into the past" is incoherent;
   Iceberg would commit onto current anyway — refuse rather than
   surprise).

Tests
-----

-  Snapshots dictionary lists the fixture table's known history (seeded
   count); data-files dictionary row counts sum to the table count.
-  Time travel: pin snapshot N-1 on the multi-snapshot fixture, SELECT
   shows the old data; clear pin, SELECT shows current; pinned SELECT
   inside a transaction alongside unpinned tables behaves.
-  Dictionary metadata reflects the session's pin (query
   ICEBERG_DATA_FILES under a historical pin → old file list).
-  Refusals: bad snapshot id, unattached table, INSERT under historical
   pin.

Verification
------------

-  Build green in both switch states (``--with-iceberg`` on/off) per
   commit; suites green.
-  Docs: an "inspecting and time-traveling Iceberg tables" page with the
   dictionary schemas and the variable, including the explicit-pin
   lifetime rule.