F3.Metadata (F3 v0.1.0)

Copy Markdown

File metadata from the postscript, the FlatBuffers footer, and the out-of-band column metadata those tables point at.

Checksums are copied from the postscript. They are not recomputed. Optional section offsets are recorded and not read. Nothing in this struct is a Wasm module or a column payload.

to_map/1 is the JSON shape compared with the upstream reader. It uses string keys and does not include column names, which exist only on row_groups after the schema is aligned.

Summary

Types

encoding_version()
logical_id()
logical_tree()
section()
shared_dictionary()
t()

Functions

to_map(meta)

Map compared with the committed upstream golden files.

Types

encoding_version()

@type encoding_version() :: %{
  encoding: F3.RowGroup.encoding(),
  version: %{
    major: non_neg_integer(),
    minor: non_neg_integer(),
    patch: non_neg_integer()
  }
}

logical_id()

@type logical_id() ::
  :flat | :list | :struct | :list_of_struct_of_primitive | non_neg_integer()

logical_tree()

@type logical_tree() :: %{id: logical_id(), children: [logical_tree()]}

section()

@type section() :: %{
  name: String.t(),
  offset: non_neg_integer(),
  size: non_neg_integer(),
  compression: F3.RowGroup.compression()
}

shared_dictionary()

@type shared_dictionary() :: %{
  chunks: [F3.RowGroup.chunk()],
  positions: [[non_neg_integer()]],
  schema: F3.Schema.t() | nil,
  schema_bytes: non_neg_integer()
}

t()

@type t() :: %F3.Metadata{
  checksum_type: F3.Postscript.checksum_type(),
  data_checksum: non_neg_integer(),
  encoding_versions: [encoding_version()],
  footer_compression: :uncompressed,
  footer_size: non_neg_integer(),
  logical_tree: logical_tree(),
  major_version: non_neg_integer(),
  metadata_size: non_neg_integer(),
  minor_version: non_neg_integer(),
  optional_sections: [section()],
  row_groups: [F3.RowGroup.t()],
  schema: F3.Schema.t(),
  schema_checksum: non_neg_integer(),
  shared_dictionary: shared_dictionary() | nil
}

Functions

to_map(meta)

@spec to_map(t()) :: map()

Map compared with the committed upstream golden files.

Keys are strings. Column payloads are not included. A value that is not metadata raises FunctionClauseError.

Examples

iex> {:ok, file} = F3.open("test/fixtures/small.f3")
iex> {:ok, meta} = F3.metadata(file)
iex> map = F3.Metadata.to_map(meta)
iex> map["postscript"]["major_version"]
0