F3.Batch (F3 v0.1.0)

Columnar batch.

columns maps a field name to an F3.Column. That is the storage form. to_rows/1 zips those columns into maps and is the convenience path for a small result. A large file should keep the columns.

Summary

Functions

to_rows(batch)

Materializes a batch as row maps.

Types

t()

@type t() :: %F3.Batch{
  columns: %{optional(String.t()) => term()},
  num_rows: non_neg_integer(),
  schema: F3.Schema.t() | nil
}

One batch.

columns maps a logical field name to an F3.Column. num_rows is the row count of every column. schema is the logical schema for those names. to_rows/1 requires it. A batch with num_rows 0 returns [] even when schema is nil.

Functions

to_rows(batch)

@spec to_rows(t()) :: {:ok, [map()]} | {:error, F3.Error.t()}

Materializes a batch as row maps.

Keys are the logical field names. Nested values are maps and lists. Nulls are nil. A batch that is not an F3.Batch is :invalid_argument. This does not raise.

Examples

iex> field = %{name: "id", type: {:int, 64, true}, nullable: false}
iex> {:ok, column} = F3.Column.from_native(:i64, 1, <<3::little-64>>, nil)
iex> batch = %F3.Batch{columns: %{"id" => column}, num_rows: 1, schema: %F3.Schema{fields: [field]}}
iex> F3.Batch.to_rows(batch)
{:ok, [%{"id" => 3}]}

iex> F3.Batch.to_rows(%F3.Batch{num_rows: 0})
{:ok, []}

iex> {:error, error} = F3.Batch.to_rows(%F3.Batch{num_rows: 1})
iex> error.message
"batch has no schema"