pychunked_data_view =================== .. py:module:: pychunked_data_view Submodules ---------- .. toctree:: :maxdepth: 1 /autoapi/pychunked_data_view/chunked_data_view/index /autoapi/pychunked_data_view/exceptions/index Attributes ---------- .. autoapisummary:: pychunked_data_view.MarsSelection Exceptions ---------- .. autoapisummary:: pychunked_data_view.InternalError pychunked_data_view.MarsRequestFormattingError Classes ------- .. autoapisummary:: pychunked_data_view.AxisDefinition pychunked_data_view.ChunkedDataView pychunked_data_view.ChunkedDataViewBuilder pychunked_data_view.Chunking pychunked_data_view.ExtractorType Package Contents ---------------- .. py:class:: AxisDefinition(keys: list[str], chunking: Chunking | Chunking, name: str | None = None) Maps one or more MARS keys to a single zarr array axis with a given chunking strategy. Defines which MARS keys form an axis in the zarr array, and how it is chunked. :param keys: MARS keys that form this axis. :type keys: list[str] :param chunking: How this axis shall be chunked. :type chunking: ~pychunked_data_view.Chunking | ~pychunked_data_view.Chunking.FixedSizeChunk :param name: Zarr dimension name. Defaults to the keys joined by ``"_"``. :type name: str | None .. py:method:: _translate_chunking(chunking: Chunking | Chunking) -> chunked_data_view_bindings.AxisDefinition.WholeAxisChunking | chunked_data_view_bindings.AxisDefinition.SingleValueChunking | chunked_data_view_bindings.AxisDefinition.FixedSizeChunking :staticmethod: Convert a Python :class:`Chunking` value to the corresponding C++ binding type. :param chunking: Chunking strategy to translate. :type chunking: ~pychunked_data_view.Chunking | ~pychunked_data_view.Chunking.FixedSizeChunk :returns: The matching ``pdv.AxisDefinition`` chunking object. :raises TypeError: If *chunking* is not a recognised :class:`Chunking` value. .. py:attribute:: _obj .. py:property:: name :type: str | None The zarr dimension name for this axis, or None to derive it from the keys. .. py:property:: keys :type: list[str] The MARS keys that form this axis. .. py:property:: chunking :type: Chunking | Chunking The chunking strategy for this axis. :raises ~pychunked_data_view.exceptions.InternalError: If the underlying C++ chunking type is unrecognised. .. py:class:: ChunkedDataView(obj: chunked_data_view_bindings.ChunkedDataView) Python wrapper around the C++ ``ChunkedDataView``. Provides shape and chunk-count metadata, and per-chunk data access via :meth:`at`. Instances are returned by :meth:`ChunkedDataViewBuilder.build`. .. py:attribute:: _obj .. py:method:: at(index: list[int] | tuple[int, ...]) -> numpy.ndarray Return the values of the chunk at *index*. :param index: Per-dimension chunk coordinates, including the implicit grid-point dimension. :type index: list[int] | tuple[int, ...] :returns: 1-D ``float32`` array of ``chunk_shape()`` values, C-order. :rtype: numpy.ndarray :raises RuntimeError: If *index* is out of bounds or the FDB retrieval fails. .. py:method:: chunk_shape() -> tuple[int, ...] Return the per-dimension element count of one chunk. :returns: Number of elements along each dimension within a single chunk. :rtype: tuple[int, ...] .. py:method:: chunkShape() -> tuple[int, ...] Deprecated alias of :meth:`chunk_shape`. Kept so existing callers keep working; every other method on this class is snake_case. .. py:method:: chunks() -> tuple[int, ...] Return the per-dimension number of chunks. :returns: Number of chunks along each dimension. :rtype: tuple[int, ...] .. py:method:: shape() -> tuple[int, ...] Return the total array shape in elements (not chunks). :returns: Total number of elements along each dimension. :rtype: tuple[int, ...] .. py:method:: fill_missing_value() -> float Return the fill value used for bitmap-masked grid points. :returns: Value written into positions flagged as missing by the GRIB bitmap. :rtype: float .. py:class:: ChunkedDataViewBuilder(fdb_config_file: pathlib.Path | None) Collects MARS request parts and builds a :class:`ChunkedDataView`. Wraps the C++ ``ChunkedDataViewBuilder``. Call :meth:`add_part` one or more times, then :meth:`build` to obtain the view. :param fdb_config_file: Path to the FDB configuration YAML. ``None`` lets FDB resolve its configuration from the environment. :type fdb_config_file: pathlib.Path | None .. py:attribute:: _obj .. py:attribute:: _dim_names :type: list[str] | None :value: None .. py:method:: add_part(mars_request: MarsSelection, axes: list[AxisDefinition], extractor: ExtractorType | ExtractorType) -> ChunkedDataViewBuilder Validate *axes* against *mars_request*, record dimension names, and register the part. :param mars_request: MARS key-value mapping describing the data to retrieve. :type mars_request: MarsSelection :param axes: Axis definitions; each must reference keys present in *mars_request*. :type axes: list[AxisDefinition] :param extractor: Extraction backend to use. :type extractor: ExtractorType.Grib | ExtractorType.GribJump :returns: ``self``, for method chaining. :rtype: ChunkedDataViewBuilder :raises ValueError: If any axis key is not present in *mars_request*. .. note:: Only the axis-key check happens here. Everything that needs FDB -- field sizes, axis mapping, whether the parts fit together -- is validated by :meth:`build`, which raises ``RuntimeError`` on any of it. .. py:method:: dim_names() -> list[str] Return the zarr dimension names derived from the first registered part. :returns: One name per axis (MARS keys joined by ``_``), plus ``"values"`` for the implicit grid-point axis. Empty if no part has been added yet. :rtype: list[str] .. py:method:: extend_on_axis(axis: int) -> ChunkedDataViewBuilder Set *axis* as the extension axis when multiple parts are added. :param axis: Zero-based index of the axis along which parts are concatenated. :type axis: int :returns: ``self``, for method chaining. :rtype: ChunkedDataViewBuilder .. py:method:: fill_missing_value(value: float) -> ChunkedDataViewBuilder Set the fill value for bitmap-masked grid points. :param value: Value written into positions flagged as missing by the GRIB bitmap. Also used as the zarr array ``fill_value``. :type value: float :returns: ``self``, for method chaining. :rtype: ChunkedDataViewBuilder .. py:method:: build() -> ChunkedDataView Build and return the :class:`ChunkedDataView`. :returns: The assembled view, ready for chunk-level data access. :rtype: ChunkedDataView :raises ~pychunked_data_view.exceptions.MarsRequestFormattingError: If the MARS request string is malformed (trailing comma, missing comma between keys, or misspelled key). .. py:class:: Chunking(*args, **kwds) Bases: :py:obj:`enum.Enum` Defines how an axis will be chunked. .. attribute:: WHOLE_AXIS The entire axis is a single chunk; accessing any value loads all values on that axis. .. attribute:: SINGLE_VALUE Each value along the axis is its own chunk. .. attribute:: FixedSizeChunk Groups every ``chunk_shape`` consecutive values into one chunk. .. py:attribute:: WHOLE_AXIS .. py:attribute:: SINGLE_VALUE .. py:class:: FixedSizeChunk(chunk_shape: int) .. py:attribute:: chunk_shape .. py:class:: ExtractorType Namespace for extractor configuration types. * :class:`ExtractorType.Grib` - standard full-field GRIB extraction. * :class:`ExtractorType.GribJump` - partial-field extraction via GribJump. Each class wraps the matching C++ ``ExtractorDefinition``, which is what :meth:`ChunkedDataViewBuilder.add_part` takes. One instance may be reused across as many parts and builders as you like: ``add_part`` stores a copy, so defaults the builder applies (e.g. its ``fdb_config``) are never written back into your object. .. py:class:: Grib(fdb_config: pathlib.Path | None = None) Reads full GRIB fields from FDB. :param fdb_config: Path to the FDB configuration YAML. ``None`` (default) uses the builder's FDB config. :type fdb_config: pathlib.Path | None .. py:attribute:: _obj .. py:class:: GribJump(fdb_config: pathlib.Path | None = None, gribjump_config: pathlib.Path | None = None, field_chunking: Chunking | Chunking.FixedSizeChunk | None = None) Reads grid-point values from FDB via GribJump. GribJump avoids a full GRIB decode by jumping directly to the grid-point values inside each message. :param fdb_config: Path to the FDB configuration YAML. ``None`` (default) uses the builder's FDB config. :type fdb_config: pathlib.Path | None :param gribjump_config: Path to the GribJump configuration YAML. ``None`` (default) reads the ``GRIBJUMP_CONFIG_FILE`` environment variable. :type gribjump_config: pathlib.Path | None :param field_chunking: How to sub-divide the implicit (grid-point) dimension into Zarr chunks. ``None`` (default) produces a single chunk covering the full field. The size must divide the grid exactly -- that dimension cannot be left ragged. :type field_chunking: ~pychunked_data_view.Chunking | ~pychunked_data_view.Chunking.FixedSizeChunk | None .. py:attribute:: _obj .. py:type:: MarsSelection :canonical: Mapping[str, str | int | float | Collection[str | int | float]] .. py:exception:: InternalError Bases: :py:obj:`Exception` Indicates an internal error. You will only see this exception if there is something broken inside pychunked_data_view. Initialize self. See help(type(self)) for accurate signature. .. py:exception:: MarsRequestFormattingError Bases: :py:obj:`RuntimeError` Unspecified run-time error. Initialize self. See help(type(self)) for accurate signature.