pychunked_data_view.chunked_data_view#

Attributes#

MarsSelection

Classes#

Chunking

Defines how an axis will be chunked.

AxisDefinition

Maps one or more MARS keys to a single zarr array axis with a given chunking strategy.

ChunkedDataView

Python wrapper around the C++ ChunkedDataView.

ExtractorType

Namespace for extractor configuration types.

ChunkedDataViewBuilder

Collects MARS request parts and builds a ChunkedDataView.

Functions#

_mars_selection_to_string(→ str)

Serialise a MarsSelection dict to a key=value,... MARS string.

Module Contents#

type MarsSelection = Mapping[str, str | int | float | Collection[str | int | float]]#
_mars_selection_to_string(request: MarsSelection) → str#

Serialise a MarsSelection dict to a key=value,... MARS string.

Parameters:

request (MarsSelection) – MARS key-value mapping to serialise.

Returns:

Comma-separated key=value pairs, with multi-valued entries

joined by /.

Return type:

str

class Chunking(*args, **kwds)#

Bases: enum.Enum

Defines how an axis will be chunked.

WHOLE_AXIS#

The entire axis is a single chunk; accessing any value loads all values on that axis.

SINGLE_VALUE#

Each value along the axis is its own chunk.

FixedSizeChunk#

Groups every chunk_shape consecutive values into one chunk.

WHOLE_AXIS#
SINGLE_VALUE#
class FixedSizeChunk(chunk_shape: int)#
chunk_shape#
class AxisDefinition(keys: list[str], chunking: Chunking | Chunking, name: str | None = None)#

Maps one or more MARS keys to a single zarr array axis with a given chunking strategy.

Defines which MARS keys form an axis in the zarr array, and how it is chunked.

Parameters:

  • keys (list[str]) – MARS keys that form this axis.

  • chunking (Chunking | FixedSizeChunk) – How this axis shall be chunked.

  • name (str | None) – Zarr dimension name. Defaults to the keys joined by "_".

static _translate_chunking(chunking: Chunking | Chunking) → chunked_data_view_bindings.AxisDefinition.WholeAxisChunking | chunked_data_view_bindings.AxisDefinition.SingleValueChunking | chunked_data_view_bindings.AxisDefinition.FixedSizeChunking#

Convert a Python Chunking value to the corresponding C++ binding type.

Parameters:

chunking (Chunking | FixedSizeChunk) – Chunking strategy to translate.

Returns:

The matching pdv.AxisDefinition chunking object.

Raises:

TypeError – If chunking is not a recognised Chunking value.

_obj#
property name: str | None#

The zarr dimension name for this axis, or None to derive it from the keys.

property keys: list[str]#

The MARS keys that form this axis.

property chunking: Chunking | Chunking#

The chunking strategy for this axis.

Raises:

InternalError – If the underlying C++ chunking type is unrecognised.

class ChunkedDataView(obj: chunked_data_view_bindings.ChunkedDataView)#

Python wrapper around the C++ ChunkedDataView.

Provides shape and chunk-count metadata, and per-chunk data access via at(). Instances are returned by ChunkedDataViewBuilder.build().

_obj#
at(index: list[int] | tuple[int, ...]) → numpy.ndarray#

Return the values of the chunk at index.

Parameters:

index (list[int] | tuple[int, ...]) – Per-dimension chunk coordinates, including the implicit grid-point dimension.

Returns:

1-D float32 array of chunk_shape() values, C-order.

Return type:

numpy.ndarray

Raises:

RuntimeError – If index is out of bounds or the FDB retrieval fails.

chunk_shape() → tuple[int, ...]#

Return the per-dimension element count of one chunk.

Returns:

Number of elements along each dimension within a single chunk.

Return type:

tuple[int, …]

chunkShape() → tuple[int, ...]#

Deprecated alias of chunk_shape().

Kept so existing callers keep working; every other method on this class is snake_case.

chunks() → tuple[int, ...]#

Return the per-dimension number of chunks.

Returns:

Number of chunks along each dimension.

Return type:

tuple[int, …]

shape() → tuple[int, ...]#

Return the total array shape in elements (not chunks).

Returns:

Total number of elements along each dimension.

Return type:

tuple[int, …]

fill_missing_value() → float#

Return the fill value used for bitmap-masked grid points.

Returns:

Value written into positions flagged as missing by the GRIB bitmap.

Return type:

float

class ExtractorType#

Namespace for extractor configuration types.

Each class wraps the matching C++ ExtractorDefinition, which is what ChunkedDataViewBuilder.add_part() takes.

One instance may be reused across as many parts and builders as you like: add_part stores a copy, so defaults the builder applies (e.g. its fdb_config) are never written back into your object.

class Grib(fdb_config: pathlib.Path | None = None)#

Reads full GRIB fields from FDB.

Parameters:

fdb_config (pathlib.Path | None) – Path to the FDB configuration YAML. None (default) uses the builder’s FDB config.

_obj#
class GribJump(fdb_config: pathlib.Path | None = None, gribjump_config: pathlib.Path | None = None, field_chunking: Chunking | Chunking.FixedSizeChunk | None = None)#

Reads grid-point values from FDB via GribJump.

GribJump avoids a full GRIB decode by jumping directly to the grid-point values inside each message.

Parameters:

  • fdb_config (pathlib.Path | None) – Path to the FDB configuration YAML. None (default) uses the builder’s FDB config.

  • gribjump_config (pathlib.Path | None) – Path to the GribJump configuration YAML. None (default) reads the GRIBJUMP_CONFIG_FILE environment variable.

  • field_chunking (Chunking | FixedSizeChunk | None) – How to sub-divide the implicit (grid-point) dimension into Zarr chunks. None (default) produces a single chunk covering the full field. The size must divide the grid exactly – that dimension cannot be left ragged.

_obj#
class ChunkedDataViewBuilder(fdb_config_file: pathlib.Path | None)#

Collects MARS request parts and builds a ChunkedDataView.

Wraps the C++ ChunkedDataViewBuilder. Call add_part() one or more times, then build() to obtain the view.

Parameters:

fdb_config_file (pathlib.Path | None) – Path to the FDB configuration YAML. None lets FDB resolve its configuration from the environment.

_obj#
_dim_names: list[str] | None = None#
add_part(mars_request: MarsSelection, axes: list[AxisDefinition], extractor: ExtractorType | ExtractorType) → ChunkedDataViewBuilder#

Validate axes against mars_request, record dimension names, and register the part.

Parameters:

  • mars_request (MarsSelection) – MARS key-value mapping describing the data to retrieve.

  • axes (list[AxisDefinition]) – Axis definitions; each must reference keys present in mars_request.

  • extractor (ExtractorType.Grib | ExtractorType.GribJump) – Extraction backend to use.

Returns:

self, for method chaining.

Return type:

ChunkedDataViewBuilder

Raises:

ValueError – If any axis key is not present in mars_request.

Note

Only the axis-key check happens here. Everything that needs FDB – field sizes, axis mapping, whether the parts fit together – is validated by build(), which raises RuntimeError on any of it.

dim_names() → list[str]#

Return the zarr dimension names derived from the first registered part.

Returns:

One name per axis (MARS keys joined by _), plus "values"

for the implicit grid-point axis. Empty if no part has been added yet.

Return type:

list[str]

extend_on_axis(axis: int) → ChunkedDataViewBuilder#

Set axis as the extension axis when multiple parts are added.

Parameters:

axis (int) – Zero-based index of the axis along which parts are concatenated.

Returns:

self, for method chaining.

Return type:

ChunkedDataViewBuilder

fill_missing_value(value: float) → ChunkedDataViewBuilder#

Set the fill value for bitmap-masked grid points.

Parameters:

value (float) – Value written into positions flagged as missing by the GRIB bitmap. Also used as the zarr array fill_value.

Returns:

self, for method chaining.

Return type:

ChunkedDataViewBuilder

build() → ChunkedDataView#

Build and return the ChunkedDataView.

Returns:

The assembled view, ready for chunk-level data access.

Return type:

ChunkedDataView

Raises:

MarsRequestFormattingError – If the MARS request string is malformed (trailing comma, missing comma between keys, or misspelled key).