Changelog

Copy Markdown

This library's version is independent of the upstream F3 wire version. Nothing here has been published to Hex.

[0.1.0] - Unreleased

Experimental scaffold. Schema and footer metadata are decoded. CASCADE row reads and local writes need the optional native build. Wasm stays opt-in.

Added

  • Public contracts for F3.open/2, F3.close/1, F3.schema/1, F3.metadata/1, F3.read/2, F3.stream/2, and F3.inspect/2.
  • F3.Error, F3.File, F3.Schema, F3.Metadata, F3.RowGroup, and F3.Batch.
  • Local and in-memory byte sources with positioned reads.
  • Pinned compatibility with upstream F3 commit bd92506447dc1333820cfe9a34ebbf497fe0219e (wire version 0.1). Any other wire version is rejected by F3.Format.compatible?/2.
  • Optional f3_native Rustler scaffold, compiled only when F3_BUILD_NATIVE=1.
  • Fixtures generated by the pinned upstream writer, with a checksum manifest.
  • Draft tutorials 1 and 2, technically reviewed against the pinned schema and the SIGMOD 2025 paper.
  • F3.Postscript decodes the 32-byte tail, enforces a metadata size cap, and reports checksums without verifying them.
  • F3.schema/1, F3.metadata/1, and F3.inspect/2 decode the Arrow IPC schema, row groups, column chunks, and encoding units. mix f3.inspect prints that text.
  • Golden metadata JSON from the pinned upstream reader, under test/fixtures/golden/.
  • Draft tutorial 3. Tutorial 2 now includes the real inspect output of small.f3.
  • Credo, and mix ci for the required CI checks. The alias stops at the first failure and runs mix test --cover with a 90% threshold.
  • Mox doubles for F3.IO.Source, F3.IO.FileSystem, and F3.Decoder. The default suite covers projection, batching, and laziness without Rust.
  • F3.read/2, F3.stream/2, and F3.Batch.to_rows/1. One dirty-CPU NIF call decodes a CASCADE EncUnit. Elixir projects columns, selects row groups, stitches nested fields, gathers a local dictionary, and slices to batch_size (default 8192).
  • The NIF depends on Vortex c0d89a66cbaa542bd9aa277cfdf52f2728c4b37a and Arrow 53.3.0. It does not link fff-poc and it does not install a global allocator. The crate pins Rust nightly-2025-11-15.
  • Golden row JSON from one local run of the pinned reader, except dictionary.f3, which that reader rejects with Incorrect type of indices. The native test compares the other seven files to that JSON and all eight to the manifest.
  • Tutorials 3 and 4 are technically reviewed. Tutorial 4 covers streaming and projection. There is no benchmark table.
  • Opt-in Wasmtime 28 host for decode_general_ffi, major ABI 1 only, with module, memory, and fuel caps. wasm: :off does not read the WASMBinaries section. lib_urls are not fetched.
  • F3.Extension and F3.Registry. An extension match does not fall through into Wasm. Registries are not shared unless the caller passes the same pid.
  • test/fixtures/custom_wasm.f3 from the pinned writer after building fff-ude-example-fff for wasm32-wasip1. small.f3 stays on the Vortex NIF.
  • Tutorial 6 is technically reviewed. The default script path leaves Wasm off.
  • F3.Writer writes a local file through an out-of-process fff-poc helper on nightly-2025-02-24. The NIF does not link fff-poc. close/1 renames a temp file onto the destination. Without F3_BUILD_NATIVE=1, a valid open returns :not_implemented and creates nothing.
  • Tutorial 5 is technically reviewed. The default script path prints that the writer is not loaded.
  • F3.IO.HTTP reads an object with one HEAD and ranged GETs. Redirects are refused. lib_urls are not fetched. SigV4 is used only when the six signing options are set. The writer stays on a local temp file.
  • scripts/bench.exs records wall time for the committed fixtures. It does not download data and it is not a CI check.
  • Hex files includes the writer helper sources and the native lockfile. The library version is 0.1.0. Hex has not been published.
  • Release workflows build precompiled NIFs and f3_write for aarch64-apple-darwin and x86_64-unknown-linux-gnu, then publish to Hex when HEX_API_KEY is set. A git checkout has no checksum file, so the default build stays rust-free.
  • Tutorials 7–9 are technically reviewed. They are not ready to publish.