z3FDB - API#

Overview #

z3FDB enables to create views into FDB where the view is a Zarr array.

Views are defined by one or more MARS requests. Each keyword in the MARS request with more than one value defines an ‘Axis’. ‘Axis’ from MARS requests need to be mapped to ‘Axis’ in the Zarr array. This mapping can be a 1-1 or many-1 mapping, allowing to create a time based axis in the Zarr array that is composed from the ‘date’ and ‘time’ keyword when dealing with climate data.

For example, the request "..., date=1970-01-1/to/2020-12-31, time=00/06/12/18, ..." spans two axis ‘date’ and ‘time’. If you want to work on a unified time axis, then you can use the following AxisDefinition to map accordingly:

Example:

AxisDefinition(["date", "time"], Chunking.SINGLE_VALUE)

This defines an ‘Axis’ in the Zarr array that follows ‘date’ and ‘time’ from the MARS request, where the rightmost references ‘Axis’ (‘time’) is varying fastest.

You can combine multiple MARS request into one view. This is useful if you want to access surface and pressure level data in one view. In this case you need to select on which ‘Axis’ of the Zarr array the requests extend each other. The remaining axis have to have the same cardinality.

builder.add_part(
    "type=an,"
    "class=ea,"
    "domain=g,"
    "expver=0001,"
    "stream=oper,"
    "date=2020-01-01/2020-01-02,"
    "levtype=sfc,"
    "step=0,"
    "param=165/166,"
    "time=0/to/21/by/3",
    [
        AxisDefinition(["date", "time"], Chunking.SINGLE_VALUE),
        AxisDefinition(["param"], Chunking.SINGLE_VALUE)
    ],
    ExtractorType.GRIB,
)
builder.add_part(
    "type=an,"
    "class=ea,"
    "domain=g,"
    "expver=0001,"
    "stream=oper,"
    "date=2020-01-01/2020-01-02,"
    "levtype=pl,"
    "step=0,"
    "param=131/132,"
    "levelist=50/100,"
    "time=0/to/21/by/3",
    [
        AxisDefinition(["date", "time"], Chunking.SINGLE_VALUE),
        AxisDefinition(["param", "levelist"], Chunking.SINGLE_VALUE)
    ],
    ExtractorType.GRIB,
)
builder.extend_on_axis(1)
store = builder.build()

The created Zarr array will always have the actual data points available as the final ‘Axis’.

arr[0][0][0][0]
    ^  ^  ^  ^
    |  |  | Index in field  -> Implicit
    |  | Ensemble           -> Created from an AxisDefinition
    | Step                  -> Created from an AxisDefinition
   Date                     -> Created from an AxisDefinition

Exceptions #

z3fdb.Z3fdbError #

exception Z3fdbError#

Base Exception of all Z3fdb related errors.

Raised when an operation fails.

Initialize self. See help(type(self)) for accurate signature.

Classes #

z3fdb.SimpleStoreBuilder #

class SimpleStoreBuilder(fdb_config_file: pathlib.Path | None = None)#

Builder to create a Zarr store with FDB backing.

This builder will create a Zarr store with a Zarr Array at its root (“/”) containing the data from your MARS request(s).

Parameters:: fdb_config_file – Optional path to FDB config file. If not set normal FDB config file resolution is applied.

add_part(mars_request_key_values: str, axes: list[pychunked_data_view.AxisDefinition], extractor_type: pychunked_data_view.ExtractorType) → None#

Add a MARS request to the view.

Parameters:

mars_request_key_values (str) –
A string with fully spefified MARS key:values pairs.

For example:
type=an, class=ea, domain=g, expver=0001, stream=oper, date=2020-01-01/to/2020-01-04, levtype=sfc, step=0, param=167/131/132, time=0/to/21/by/3
axes (list of AxisDefinition) – List of AxisDefinitions that describe how axis in the MARS request are mapped to axis in the Zarr array.
extractor_type – Defines how to extract data from FDB. Currently only ExtractorType.GRIB is supported.

build() → zarr.abc.store.Store#

Build the store from the inputs.

Raises:: Z3fdbError if store cannot be created. –

extend_on_axis(axis: int) → None#

Defines the extension axis when multiple parts are added.

Parameters:: axis (int) – Index of the axis that is extendet when multiple parts have been added.

z3fdb.AxisDefinition #

See Dimension Mapping for how axis definitions map MARS keywords to Zarr dimensions.

class AxisDefinition(keys: list[str], chunking: Chunking)#

Defines which axis from a MARS Request form an axis in the Zarr array.

Also defines if the data is to be chunked.

Parameters:

keys (list of str) – mars keys that for this axis.
chunking (Chunking) – Define how this axis shall be chunked

property chunking: Chunking#

property keys: list[str]#

Enums #

z3fdb.Chunking #

class Chunking(*args, **kwds)#

Defines how a axis will be chunked

NONE#: Axis will not be chunked, accessing any value from this axis will load all values.

SINGLE_VALUE#: Axis will be chunked. One chunk per value.

NONE#

SINGLE_VALUE#

z3fdb.ExtractorType #

class ExtractorType(*args, **kwds)#

Suported data extractors.

Defines what storage format the caller expects to be stored in FDB.

GRIB#: Extract data from GRIB

z3FDB - API#

This Page