# Architecture

The library owns the byte-source lifecycle, the postscript, the footer, and the public read API. Rust decodes one CASCADE encoding unit at a time and returns owned Elixir terms. Arrow buffers do not cross the NIF. Wasm runs only when a read passes `wasm: :sandbox`.

```mermaid
flowchart LR
  Read[F3.read or F3.stream]
  Meta[F3.Footer]
  Slice[pread one EncUnit]
  Nif[dirty CPU NIF]
  Batch[F3.Batch]
  Read --> Meta --> Slice --> Nif --> Batch
```

## Components

`F3` is the public module. `open/2` builds an `F3.File` around `F3.IO.Local`, `F3.IO.Memory`, or `F3.IO.HTTP`. The HTTP source is opt-in. It sends one `HEAD`, then one ranged `GET` per `pread/3`, and it does not follow redirects. `close/1` returns the updated handle and is idempotent on that handle. `schema/1`, `metadata/1`, and `inspect/2` decode the tail when the handle is open. `read/2` returns columnar batches. `stream/2` returns a stream and does not pread until the first pull.

`F3.Format` holds the pinned magic, the 32-byte postscript layout, and `compatible?/2`. The only accepted wire version is 0.1. `F3.Postscript` reads that layout, rejects a short tail, a bad magic, any other version, a compressed footer, and a claimed size past the file or past `max_metadata_bytes:`. The default cap is 16 MiB and is an argument, not a process global.

`F3.FlatBuffer` walks one buffer: root offset, vtable, scalars, strings, vectors, and unions. Offsets are checked against the buffer length. Depth and the number of visited tables are capped. `F3.Footer` turns the footer and the out-of-band column-metadata roots into `F3.Schema`, `F3.Metadata`, and `F3.RowGroup`. Arrow types the fixtures do not use become `{:unsupported, id}` and do not fail the file. Wasm section offsets are recorded and the bytes are not read. Checksums are recorded and not verified.

`F3.Batch` stores columns keyed by logical field name. `to_rows/1` zips them into maps and is the convenience path. `F3.Writer` checks arguments in Elixir, then streams batches to an out-of-process `f3_write` helper. The helper is `fff-poc`'s `FileWriter` on `nightly-2025-02-24`. `close/1` renames a temp file in the destination directory onto the path. That rename is local-filesystem behavior. Without `F3_BUILD_NATIVE=1`, a valid `open/2` returns `:not_implemented` and creates nothing.

`F3.Reader` selects columns and row groups, checks a chunk checksum when the footer stored one, and slices a decoded unit to `batch_size` (default 8192). `F3.Logical` stitches unnamed physical columns into the logical tree. `F3.Native` is compiled only when `F3_BUILD_NATIVE=1`. `decode_encunit/3` and `decode_wasm/5` run on the dirty CPU scheduler. The crate calls Vortex, Arrow, and Wasmtime 28. It does not link `fff-poc` and it does not install a global allocator. Without the NIF, a CASCADE read returns `:native_unavailable`. A `CUSTOM_WASM` read with `wasm: :off` returns `:unsupported_encoding` without the NIF. Metadata inspection still does not execute a module.

There is no `Application` callback, no supervision tree, and no process registry. See [decisions/0001-hybrid-elixir-rust.md](decisions/0001-hybrid-elixir-rust.md).

## What stays off

Wasm stays opt-in. A shared dictionary is still `:unsupported_encoding`. A local dictionary is gathered in Elixir after the NIF returns the values and the indexes. `lib_urls` stay unread. An HTTP source does not fetch them and does not follow redirects. The writer does not embed Wasm and does not write to a remote URL.

## Fixture path

`scripts/fixtures/generate.sh` clones the pinned commit into gitignored `tmp/upstream-f3`, copies `scripts/fixtures/generator/src/main.rs` to `fff-poc/src/bin/f3_fixture_gen.rs` in that checkout, and builds it with `cargo build --locked`. `scripts/fixtures/dump_metadata.sh` runs the pinned reader once and writes `test/fixtures/golden/*.json`. `mix test` checks the manifest and those JSON files. It does not compile upstream.