pymetkit ======== .. py:module:: pymetkit Submodules ---------- .. toctree:: :maxdepth: 1 /autoapi/pymetkit/_version/index /autoapi/pymetkit/generate_parameter_metadata/index /autoapi/pymetkit/models/index /autoapi/pymetkit/pymetkit/index Attributes ---------- .. autoapisummary:: pymetkit._requests pymetkit._platformdirs pymetkit.ffi pymetkit.lib Exceptions ---------- .. autoapisummary:: pymetkit.AmbiguousParamError pymetkit.MetKitException pymetkit.CFFIModuleLoadFailed pymetkit.AmbiguousParamError Classes ------- .. autoapisummary:: pymetkit.ParameterEntry pymetkit.ParamIDCandidate pymetkit.MarsRequest pymetkit.PatchedLib pymetkit.ParamDB pymetkit.ParamDB pymetkit.ParamIDCandidate pymetkit.ParameterEntry pymetkit.MarsRequestContext Functions --------- .. autoapisummary:: pymetkit.ffi_encode pymetkit.ffi_decode pymetkit.parse_mars_request Package Contents ---------------- .. py:class:: ParameterEntry Bases: :py:obj:`pydantic.BaseModel` A single entry from the ECMWF parameter database. Accepts both the canonical field names produced by ``_normalise`` and the raw aliases that may appear in YAML or API responses. .. py:attribute:: model_config .. py:attribute:: id :type: int .. py:attribute:: shortname :type: str .. py:attribute:: longname :type: str .. py:attribute:: units :type: str .. py:attribute:: table :type: int | None .. py:attribute:: mars_request_context :type: list[MarsRequestContext] .. py:attribute:: origin_ids :type: list[int] .. py:attribute:: access_ids :type: list[str] .. py:method:: coerce_id_to_int(v: Any) -> int :classmethod: .. py:method:: normalise_shortname_key(v: Any) -> str :classmethod: .. py:method:: normalise_longname_key(v: Any) -> str :classmethod: .. py:method:: default_empty_units(v: Any) -> str :classmethod: .. py:method:: coerce_table(v: Any) -> int | None :classmethod: .. py:method:: coerce_origin_ids(v: Any) -> list[int] :classmethod: .. py:method:: coerce_access_ids(v: Any) -> list[str] :classmethod: .. py:method:: _normalise_aliases(data: Any) -> Any :classmethod: Accept legacy key spellings from raw YAML / API payloads. .. py:data:: _requests :value: None .. py:data:: _platformdirs :value: None .. py:data:: ffi .. py:function:: ffi_encode(data) -> bytes .. py:function:: ffi_decode(data: cffi.FFI.CData) -> str .. py:class:: ParamIDCandidate One possible paramid for a shortname, plus the context that selects it. .. attribute:: param_id The candidate numeric parameter ID. .. attribute:: table GRIB parameter table the id encodes to (e.g. ``128``, ``228``). .. attribute:: origin WMO originating centre ids associated with this candidate — the full list (e.g. ``[0, 34, 98]``), since several centres may share the id. .. attribute:: access Access categories (e.g. ``["dissemination"]``) — the full list. .. attribute:: mars_request_context The minimal set of MARS key/value pairs that, when passed as ``context=`` to :meth:`ParamDB.shortname_to_param_id`, selects this candidate (e.g. ``{"class": "ai"}``). Empty if no baked context is recorded or if no key-subset uniquely distinguishes this candidate. .. py:attribute:: param_id :type: int .. py:attribute:: table :type: int .. py:attribute:: origin :type: list[int] .. py:attribute:: access :type: list[str] .. py:attribute:: mars_request_context :type: dict .. py:exception:: AmbiguousParamError(shortname: str, candidates: list[ParamIDCandidate]) Bases: :py:obj:`KeyError` Raised when a shortname (given the supplied context) maps to >1 paramid. .. attribute:: shortname The shortname that could not be uniquely resolved. .. attribute:: candidates Every matching candidate, each carrying the context needed to select it, sorted by ``(table, origin, access, mars_request_context)``. The caller inspects these and re-calls with a narrowing ``context=``. .. attribute:: Initialize self. See help(type(self)) for accurate signature. .. py:attribute:: shortname .. py:attribute:: candidates .. py:class:: MarsRequest(verb: str | None = None, **kwargs) Create MetKit MarsRequest object. Parameters and values in the request can be specified through kwargs, noting that reserved words in Python must be suffixed with "_" e.g. "class_" .. py:attribute:: __request .. py:method:: ctype() -> cffi.FFI.CData .. py:method:: verb() -> str .. py:method:: expand(inherit: bool = True, strict: bool = False) -> MarsRequest Return expanded request Params ------ inherit: bool, if True, populates expanded request with default values strict: bool, if True, raise error instead of warning for invalid values :rtype: Request, resulting from expansion .. py:method:: resolved_param() -> int Return the single resolved ``param`` value as an ``int``. Convenience accessor for the shortname -> paramid resolution path: after building a request with ``param=`` plus context keys and calling :meth:`expand`, the C++ ``TypeParam::pass2`` rewrites ``param`` to the numeric id. This reads that id back. :returns: The single resolved paramid. :rtype: int :raises KeyError: If the request has no ``param`` key. :raises ValueError: If ``param`` resolved to zero or more than one value (i.e. the context did not collapse to a single id). .. py:method:: validate() Check if request is valid against MARS language definition. Does not inherit missing parameters. :raises Exception if request is incompatible with MARS language definition: .. py:method:: keys() -> Iterator[str] Get iterator over parameters in request :rtype: Iterator over parameter names .. py:method:: num_values(param: str) -> int Number of values for parameter Params ------ param: parameter name :rtype: int .. py:method:: merge(other: MarsRequest) -> MarsRequest Merge the values in another request with existing request and returns result as a new Request object. Does not modify inputs to merge. Both input requests must contain the same values and the resulting request object must be compatible with MARS language definition Params ------ other: Request, request to merge with self :rtype: Request, containing the result of the merge :raises ValueError if parameters in the two requests do not match: :raises MetKitException if resulting request is not compatible with MARS language definition: .. py:method:: __iter__() -> Iterator[tuple[str, list[str]]] .. py:method:: __getitem__(param: str) -> str | list[str] .. py:method:: __contains__(param: str) -> bool .. py:method:: __setitem__(param: str, values: int | str | list[str]) .. py:method:: __eq__(other: MarsRequest) -> bool .. py:function:: parse_mars_request(file_or_str: IO | str, strict: bool = False) -> list[MarsRequest] Function for parsing mars request from file object or string. Params ------ file_or_str: string or file-like object, containing mars request strict: bool, whether to raise error or warning when request is not compatible with MARS language definition. In the case of warning, when False, the incompatible parameters are unset from the request. :rtype: list of Request .. py:exception:: MetKitException Bases: :py:obj:`RuntimeError` Raised when MetKit library throws exception Initialize self. See help(type(self)) for accurate signature. .. py:exception:: CFFIModuleLoadFailed Bases: :py:obj:`ImportError` Raised when the shared library fails to load Initialize self. See help(type(self)) for accurate signature. .. py:class:: PatchedLib Patch a CFFI library with error handling Finds the header file associated with the MetKit C API and parses it, loads the shared library, and patches the accessors with automatic python-C error handling. .. py:method:: __read_header() .. py:method:: __check_error(fn, name: str) If calls into the MetKit library return errors, ensure that they get detected and reported by throwing an appropriate python exception. .. py:class:: ParamDB(mode: str = 'offline', cache_ttl: timedelta | None = None, cache_path: Path | str | None = None, yaml_path: Path | str | None = None) Parameter database providing metadata lookup for ECMWF parameters. Supports both online mode (fetching from the ECMWF parameter database API) and offline mode (loading from a bundled YAML file). When using ``mode="online"`` a local JSON cache is maintained so that repeated instantiations within the TTL window do not make a new HTTP request. The cache is stored under the OS user-cache directory (e.g. ``~/.cache/pymetkit/`` on Linux, ``~/Library/Caches/pymetkit/`` on macOS) using the fixed filename defined by ``_CACHE_FILENAME``. Shortname collision resolution -------------------------------- Some short names (e.g. ``t``, ``tp``, ``u``) are reused across different GRIB parameter tables and originating centres. When no context is given the default resolution priority is: 1. Prefer parameters with ``"dissemination"`` in their ``access_ids``. 2. Among those, prefer parameters whose ``origin_ids`` include an origin from ``_DEFAULT_ORIGIN_PREFERENCE`` (tried in order). 3. Fall back to the lowest param ID. Pass ``table=``, ``origin=``, or ``access=`` to :meth:`shortname_to_param_id` / :meth:`shortname_to_longname` to override this behaviour explicitly. Initialise the parameter database. The underlying data is loaded **lazily** — no file I/O or network request is made until the first lookup method is called. This makes instantiation cheap and safe to do at import time or inside hot paths. :param mode: Either ``"online"`` (fetch from the ECMWF API) or ``"offline"`` (load from a YAML file). :type mode: str :param cache_ttl: How long a previously fetched online result may be reused before a fresh HTTP request is made. Defaults to 1 hour. Only relevant when ``mode="online"``. Pass ``timedelta(0)`` to disable caching entirely (always fetch). :type cache_ttl: datetime.timedelta, optional :param cache_path: Directory in which to store the cache file. Defaults to the OS-appropriate user cache directory (requires ``platformdirs``). Only relevant when ``mode="online"``. :type cache_path: Path or str, optional :param yaml_path: Path to a custom YAML file to load instead of the bundled ``parameter_metadata.yaml``. The file must be a YAML list where each entry contains at minimum an ``id`` (integer), a short name (``shortname`` / ``shortName`` / ``short_name``), and a long name (``longname`` / ``longName`` / ``long_name`` / ``name``). Only valid with ``mode="offline"``; raises ``ValueError`` if combined with ``mode="online"``. :type yaml_path: Path or str, optional .. py:attribute:: _API_URL :value: 'https://codes.ecmwf.int/parameter-database/api/v1/param/' .. py:attribute:: _CACHE_FILENAME :value: 'paramdb_online_cache.json' .. py:attribute:: _DEFAULT_CACHE_TTL .. py:attribute:: _REQUEST_TIMEOUT :value: 30 .. py:attribute:: _DEFAULT_ORIGIN_PREFERENCE :type: list[int] :value: [98, 0] .. py:attribute:: _mode :value: 'offline' .. py:attribute:: _effective_cache_ttl .. py:attribute:: _cache_path :value: None .. py:attribute:: _yaml_path :value: None .. py:attribute:: _by_id :type: dict[int, dict] .. py:attribute:: _by_shortname :type: dict[str, dict] .. py:attribute:: _by_shortname_all :type: dict[str, list[dict]] .. py:attribute:: _by_longname :type: dict[str, dict] .. py:attribute:: _loaded :type: bool :value: False .. py:attribute:: _ctx_cache :type: dict .. py:method:: _table_from_id(param_id: int) -> int :staticmethod: Decode the GRIB parameter table number from an encoded param ID. The encoding scheme mirrors the C++ ``Param::paramId()`` logic: * IDs 1–999 → table 128 (classic ECMWF, table prefix suppressed) * IDs 1 000–999 999 → ``table * 1000 + param`` (e.g. 228228 → table 228) * IDs ≥ 1 000 000 → ``center * 1_000_000 + table * 1000 + param`` (e.g. 7001292 → center 7, table 1) .. py:method:: _center_from_id(param_id: int) -> int | None :staticmethod: Decode the originating WMO center from an encoded param ID. Returns ``None`` for IDs below 1 000 000 (ECMWF-local parameters). For IDs ≥ 1 000 000 the center is ``param_id // 1_000_000``. .. py:method:: _resolve_shortname_with_context(shortname: str, table: int | None = None, origin: int | None = None, access: str | None = None) -> dict Return the best-matching entry for *shortname* given optional context. :param shortname: The ECMWF short name to look up. :param table: GRIB parameter table number (e.g. ``128`` for classic ECMWF, ``140`` for ocean waves, ``228`` for "Standard 2"). When provided, only candidates whose encoded param ID belongs to this table are considered. :param origin: WMO originating centre ID (e.g. ``98`` for ECMWF, ``0`` for WMO, ``7`` for NCEP). When provided, only candidates whose ``origin_ids`` list includes this value are considered. :param access: Access category string (e.g. ``"dissemination"``). When provided, only candidates whose ``access_ids`` list includes this value are considered. :returns: The matched parameter metadata entry. :rtype: dict :raises KeyError: If *shortname* is not found, or if no candidate matches the supplied context. .. py:method:: _context_resolved_ids(shortname: str, context: dict) -> set[int] | None Resolve *shortname* + *context* to paramid(s) via the C++ engine. Builds ``MarsRequest(param=shortname, **context)``, expands it (which runs ``TypeParam::pass2`` and fills MARS defaults for unspecified keys), and reads back the resolved ``param`` value(s). Results are memoised on ``self._ctx_cache`` since each ``expand`` is a comparatively expensive (~ms) C++ round-trip and the minimal-context search repeats queries. The C++ library reads its language files (``params.yaml`` etc.) from ``~metkit/share/metkit`` — set ``METKIT_HOME`` to this repository root to resolve against the in-repo ``share/metkit`` data. :returns: The set of resolved numeric ids, or ``None`` if the MetKit C library is unavailable (caller should fall back to baked contexts). :rtype: set[int] | None .. py:method:: _entry_matches_context(entry: dict, context: dict) -> bool :staticmethod: Offline fallback: does any baked context of *entry* satisfy *context*? Used only when the C++ library is unavailable. An entry matches when at least one of its ``mars_request_context`` dicts contains every ``key == value`` pair in *context*. .. py:method:: _minimal_distinguishing_context(entry: dict, siblings: list[dict], shortname: str) -> dict Return the smallest MARS key-subset that selects *entry* over siblings. Searches the key-subsets of *entry*'s baked ``mars_request_context`` dicts for the smallest subset that uniquely selects this id. When the MetKit C library is available, ``expand`` is used as the authoritative oracle: a subset qualifies only if ``MarsRequest(param=shortname, **subset).expand()`` resolves to exactly this id. This guarantees the advertised context actually round-trips (bare ``{"class": "ai"}`` is rejected for ``tp`` because ``expand``'s inherited defaults resolve it to 228, not 228228). Offline (no library), it falls back to a set-membership heuristic against the baked contexts of the sibling candidates. Returns ``{}`` if no subset uniquely distinguishes *entry* (residual ambiguity) or if *entry* has no baked context. .. py:method:: _make_candidate(entry: dict, siblings: list[dict], shortname: str, minimal: bool = True) -> ParamIDCandidate Build a :class:`ParamIDCandidate` from a metadata *entry*. When *minimal* is ``False`` the (expensive, expand-oracle) minimal distinguishing context is skipped and left empty — used when the entry is already unambiguous so no disambiguating context is required. .. py:method:: _candidate_sort_key(cand: ParamIDCandidate) -> tuple :staticmethod: .. py:method:: _normalise(raw: dict) -> dict :staticmethod: Return a normalised and validated parameter dict with canonical key names. Uses :class:`~pymetkit.models.ParameterEntry` for field coercion and validation. Extra keys present in *raw* (e.g. ``"url"``, ``"units_id"``) are preserved in the returned dict so no information is discarded. .. py:method:: _index(entry: dict) -> None Insert a normalised entry into the internal lookup dicts. .. py:method:: _ensure_loaded() -> None Load parameter data on first access (lazy initialisation). Subsequent calls are no-ops — once ``_loaded`` is ``True`` the method returns immediately without any I/O. .. py:method:: _load_online(cache_ttl: datetime.timedelta, cache_path: Path | str | None) -> None .. py:method:: _load_offline(yaml_path: Path | str | None = None) -> None .. py:method:: _find_offline_yaml() -> pathlib.Path :staticmethod: Locate ``parameter_metadata.yaml``, searching in order: 1. Via ``importlib.resources`` from the installed package (reliable in both regular installs and zip-safe wheels). 2. Next to this module file (editable / development install layout). 3. ``/share/metkit/`` (development tree layout after the YAML files were moved out of the Python package directory). .. py:method:: _find_offline_json() -> Path | None :staticmethod: Locate ``parameter_metadata.json`` (fast-load format). Returns ``None`` if the JSON file is not found — caller should fall back to the YAML file. Searches the same locations as :meth:`_find_offline_yaml`. .. py:method:: _resolve_cache_dir(cache_path: Path | str | None) -> Path | None Return the directory to use for the cache file, or None if unavailable. .. py:method:: _cache_file(cache_path: Path | str | None) -> Path | None Return the full path to the cache file, or None if no cache dir is available. .. py:method:: _read_cache(cache_path: Path | str | None, cache_ttl: datetime.timedelta) -> list | None Return the cached parameter list if it exists and is still fresh, otherwise return None. .. py:method:: _write_cache(params: list, cache_path: Path | str | None) -> None Persist *params* to the cache file (best-effort; errors are silenced). .. py:method:: _resolve(identifier: int | str) -> dict Resolve *identifier* (param_id, shortname, or longname) to a metadata dict. .. py:method:: shortname_to_longname(shortname: str, table: int | None = None, origin: int | None = None, access: str | None = None) -> str Return the long name for *shortname*. :param shortname: ECMWF short name (e.g. ``"t"``, ``"tp"``). :param table: Optional GRIB parameter table number to disambiguate collisions (e.g. ``128`` for classic ECMWF, ``140`` for ocean waves). :param origin: Optional WMO originating centre ID (e.g. ``98`` for ECMWF, ``0`` for WMO, ``7`` for NCEP). :param access: Optional access category filter (e.g. ``"dissemination"``). .. py:method:: longname_to_shortname(longname: str) -> str .. py:method:: shortname_to_param_id_candidates(shortname: str, context: dict | None = None, *, table: int | None = None, origin: int | None = None, access: str | None = None) -> list[ParamIDCandidate] Return all candidate paramids for *shortname*, each with its context. The programmatic counterpart to :class:`AmbiguousParamError`: it returns the candidate + context information as a normal value, so callers can inspect the options and then call :meth:`shortname_to_param_id` with the narrowing ``context=`` (or ``table``/``origin``/``access``) they want. Two independent narrowing mechanisms are available and may be combined: * ``context`` — a dict of MARS keys resolved via the C++ ``expand`` engine (authoritative, cycle-correct). When the MetKit C library is unavailable, the baked ``mars_request_context`` metadata is used as a fallback. * ``table`` / ``origin`` / ``access`` — direct hard filters on the candidate metadata, applied WITHOUT constructing a MARS request. :param shortname: ECMWF short name (e.g. ``"t"``, ``"tp"``). :param context: Optional dict of MARS keys used to pre-narrow the candidate set (e.g. ``{"class": "ai"}``). When omitted, all candidates surviving the hard filters are returned. :param table: Optional hard filter — GRIB parameter table number. :param origin: Optional hard filter — WMO originating centre id (membership). :param access: Optional hard filter — access category string (membership). :returns: Every matching candidate, sorted by ``(table, origin, access, mars_request_context, param_id)``. Length 1 means the shortname (given any supplied context/filters) is unambiguous. :rtype: list[ParamIDCandidate] :raises KeyError: If *shortname* is unknown, or no candidate survives the filters. .. py:method:: _filter_shortname_entries(shortname: str, context: dict | None = None, *, table: int | None = None, origin: int | None = None, access: str | None = None) -> tuple[list[dict], list[dict]] Return ``(surviving_entries, all_siblings)`` for *shortname*. Applies the ``table``/``origin``/``access`` hard filters and the MARS ``context`` filter (via the C++ ``expand`` engine, or the baked-context offline fallback). Does NOT compute minimal contexts — that expensive step is deferred to :meth:`_make_candidate`. Raises ``KeyError`` if the shortname is unknown or no entry survives the filters. .. py:method:: shortname_to_param_id(shortname: str, context: dict | None = None, *, table: int | None = None, origin: int | None = None, access: str | None = None) -> int Return the single param ID for *shortname*, given optional context. Ambiguity is never resolved by guessing. When more than one candidate remains after applying ``context`` and the ``table``/``origin``/ ``access`` filters, :class:`AmbiguousParamError` is raised; its ``.candidates`` attribute lists every remaining id with the context needed to select it. :param shortname: ECMWF short name (e.g. ``"t"``, ``"tp"``). :param context: Optional dict of MARS keys resolved via the C++ ``expand`` engine (e.g. ``{"class": "ai"}``). Partial context is usually sufficient — ``expand`` fills defaults for unspecified keys. :param table: Optional hard filter — GRIB parameter table number. :param origin: Optional hard filter — WMO originating centre id (membership). :param access: Optional hard filter — access category string (membership). :returns: The uniquely resolved paramid. :rtype: int :raises KeyError: If *shortname* is unknown, or no candidate survives the filters. :raises AmbiguousParamError: If more than one candidate remains after applying context/filters. .. py:method:: param_id_to_shortname(param_id: int) -> str .. py:method:: _param_context_from_cpp(param_id: int) -> list[dict] | None :staticmethod: Return MARS contexts for *param_id* from the C++ layer, if available. Placeholder for the future ``metkit_param_context`` C API (outcomes §4). That hook does not exist yet, so this always returns ``None`` to signal "not available", causing :meth:`param_id_to_context` to fall back to the precomputed YAML data. Once the C API lands, implement it here and the public method will transparently prefer it. .. py:method:: param_id_to_context(param_id: int) -> list[dict] Return the MARS-key contexts in which *param_id* is valid. Each context is a dict of MARS keys (e.g. ``{"class": "ai", "stream": "enfo", "type": "cf", "levtype": "sfc"}``) drawn from the authoritative ``params.yaml`` map. These are the raw contexts used to disambiguate shortname collisions. Resolution order: 1. The C++ layer (``metkit_param_context``) when available — see :meth:`_param_context_from_cpp`. Not yet implemented. 2. Fallback: the precomputed ``mars_request_context`` field baked into the bundled parameter metadata. :param param_id: Numeric parameter id. :returns: Zero or more MARS context dicts. Empty if the id has no recorded context (e.g. not referenced in ``params.yaml``). :rtype: list[dict] :raises KeyError: If *param_id* is not in the database. .. py:method:: longname_to_param_id(longname: str) -> int .. py:method:: param_id_to_longname(param_id: int) -> str .. py:method:: get_metadata(identifier: int | str) -> dict Return the full metadata dictionary for a parameter. :param identifier: A param ID (int), shortname, or longname. :type identifier: int or str .. py:method:: get_units(identifier: int | str) -> str Return the units string for a parameter. :param identifier: A param ID (int), shortname, or longname. :type identifier: int or str :returns: The units string, or ``"unknown"`` if not available. :rtype: str .. py:method:: get_all_by_shortname(shortname: str) -> list[dict] Return *all* parameter entries that share *shortname*. Most short names map to exactly one param ID, but ~163 short names are reused across different GRIB parameter tables or originating centres. This method exposes every candidate so callers can inspect the collisions and choose the appropriate one. :param shortname: ECMWF short name to look up. :returns: List of metadata dicts, sorted by ascending param ID. Each dict contains at minimum ``id``, ``shortname``, and ``longname``. :rtype: list[dict] :raises KeyError: If *shortname* is not found in the database at all. .. rubric:: Examples >>> db = ParamDB() >>> entries = db.get_all_by_shortname("t") >>> [(e["id"], e["longname"]) for e in entries] [(130, 'Temperature'), (500014, 'Temperature')] .. py:method:: shortname_has_collisions(shortname: str) -> bool Return ``True`` if *shortname* maps to more than one param ID. :param shortname: ECMWF short name to check. :raises KeyError: If *shortname* is not found in the database at all. .. py:data:: lib .. py:class:: ParamDB(mode: str = 'offline', cache_ttl: timedelta | None = None, cache_path: Path | str | None = None, yaml_path: Path | str | None = None) Parameter database providing metadata lookup for ECMWF parameters. Supports both online mode (fetching from the ECMWF parameter database API) and offline mode (loading from a bundled YAML file). When using ``mode="online"`` a local JSON cache is maintained so that repeated instantiations within the TTL window do not make a new HTTP request. The cache is stored under the OS user-cache directory (e.g. ``~/.cache/pymetkit/`` on Linux, ``~/Library/Caches/pymetkit/`` on macOS) using the fixed filename defined by ``_CACHE_FILENAME``. Shortname collision resolution -------------------------------- Some short names (e.g. ``t``, ``tp``, ``u``) are reused across different GRIB parameter tables and originating centres. When no context is given the default resolution priority is: 1. Prefer parameters with ``"dissemination"`` in their ``access_ids``. 2. Among those, prefer parameters whose ``origin_ids`` include an origin from ``_DEFAULT_ORIGIN_PREFERENCE`` (tried in order). 3. Fall back to the lowest param ID. Pass ``table=``, ``origin=``, or ``access=`` to :meth:`shortname_to_param_id` / :meth:`shortname_to_longname` to override this behaviour explicitly. Initialise the parameter database. The underlying data is loaded **lazily** — no file I/O or network request is made until the first lookup method is called. This makes instantiation cheap and safe to do at import time or inside hot paths. :param mode: Either ``"online"`` (fetch from the ECMWF API) or ``"offline"`` (load from a YAML file). :type mode: str :param cache_ttl: How long a previously fetched online result may be reused before a fresh HTTP request is made. Defaults to 1 hour. Only relevant when ``mode="online"``. Pass ``timedelta(0)`` to disable caching entirely (always fetch). :type cache_ttl: datetime.timedelta, optional :param cache_path: Directory in which to store the cache file. Defaults to the OS-appropriate user cache directory (requires ``platformdirs``). Only relevant when ``mode="online"``. :type cache_path: Path or str, optional :param yaml_path: Path to a custom YAML file to load instead of the bundled ``parameter_metadata.yaml``. The file must be a YAML list where each entry contains at minimum an ``id`` (integer), a short name (``shortname`` / ``shortName`` / ``short_name``), and a long name (``longname`` / ``longName`` / ``long_name`` / ``name``). Only valid with ``mode="offline"``; raises ``ValueError`` if combined with ``mode="online"``. :type yaml_path: Path or str, optional .. py:attribute:: _API_URL :value: 'https://codes.ecmwf.int/parameter-database/api/v1/param/' .. py:attribute:: _CACHE_FILENAME :value: 'paramdb_online_cache.json' .. py:attribute:: _DEFAULT_CACHE_TTL .. py:attribute:: _REQUEST_TIMEOUT :value: 30 .. py:attribute:: _DEFAULT_ORIGIN_PREFERENCE :type: list[int] :value: [98, 0] .. py:attribute:: _mode :value: 'offline' .. py:attribute:: _effective_cache_ttl .. py:attribute:: _cache_path :value: None .. py:attribute:: _yaml_path :value: None .. py:attribute:: _by_id :type: dict[int, dict] .. py:attribute:: _by_shortname :type: dict[str, dict] .. py:attribute:: _by_shortname_all :type: dict[str, list[dict]] .. py:attribute:: _by_longname :type: dict[str, dict] .. py:attribute:: _loaded :type: bool :value: False .. py:attribute:: _ctx_cache :type: dict .. py:method:: _table_from_id(param_id: int) -> int :staticmethod: Decode the GRIB parameter table number from an encoded param ID. The encoding scheme mirrors the C++ ``Param::paramId()`` logic: * IDs 1–999 → table 128 (classic ECMWF, table prefix suppressed) * IDs 1 000–999 999 → ``table * 1000 + param`` (e.g. 228228 → table 228) * IDs ≥ 1 000 000 → ``center * 1_000_000 + table * 1000 + param`` (e.g. 7001292 → center 7, table 1) .. py:method:: _center_from_id(param_id: int) -> int | None :staticmethod: Decode the originating WMO center from an encoded param ID. Returns ``None`` for IDs below 1 000 000 (ECMWF-local parameters). For IDs ≥ 1 000 000 the center is ``param_id // 1_000_000``. .. py:method:: _resolve_shortname_with_context(shortname: str, table: int | None = None, origin: int | None = None, access: str | None = None) -> dict Return the best-matching entry for *shortname* given optional context. :param shortname: The ECMWF short name to look up. :param table: GRIB parameter table number (e.g. ``128`` for classic ECMWF, ``140`` for ocean waves, ``228`` for "Standard 2"). When provided, only candidates whose encoded param ID belongs to this table are considered. :param origin: WMO originating centre ID (e.g. ``98`` for ECMWF, ``0`` for WMO, ``7`` for NCEP). When provided, only candidates whose ``origin_ids`` list includes this value are considered. :param access: Access category string (e.g. ``"dissemination"``). When provided, only candidates whose ``access_ids`` list includes this value are considered. :returns: The matched parameter metadata entry. :rtype: dict :raises KeyError: If *shortname* is not found, or if no candidate matches the supplied context. .. py:method:: _context_resolved_ids(shortname: str, context: dict) -> set[int] | None Resolve *shortname* + *context* to paramid(s) via the C++ engine. Builds ``MarsRequest(param=shortname, **context)``, expands it (which runs ``TypeParam::pass2`` and fills MARS defaults for unspecified keys), and reads back the resolved ``param`` value(s). Results are memoised on ``self._ctx_cache`` since each ``expand`` is a comparatively expensive (~ms) C++ round-trip and the minimal-context search repeats queries. The C++ library reads its language files (``params.yaml`` etc.) from ``~metkit/share/metkit`` — set ``METKIT_HOME`` to this repository root to resolve against the in-repo ``share/metkit`` data. :returns: The set of resolved numeric ids, or ``None`` if the MetKit C library is unavailable (caller should fall back to baked contexts). :rtype: set[int] | None .. py:method:: _entry_matches_context(entry: dict, context: dict) -> bool :staticmethod: Offline fallback: does any baked context of *entry* satisfy *context*? Used only when the C++ library is unavailable. An entry matches when at least one of its ``mars_request_context`` dicts contains every ``key == value`` pair in *context*. .. py:method:: _minimal_distinguishing_context(entry: dict, siblings: list[dict], shortname: str) -> dict Return the smallest MARS key-subset that selects *entry* over siblings. Searches the key-subsets of *entry*'s baked ``mars_request_context`` dicts for the smallest subset that uniquely selects this id. When the MetKit C library is available, ``expand`` is used as the authoritative oracle: a subset qualifies only if ``MarsRequest(param=shortname, **subset).expand()`` resolves to exactly this id. This guarantees the advertised context actually round-trips (bare ``{"class": "ai"}`` is rejected for ``tp`` because ``expand``'s inherited defaults resolve it to 228, not 228228). Offline (no library), it falls back to a set-membership heuristic against the baked contexts of the sibling candidates. Returns ``{}`` if no subset uniquely distinguishes *entry* (residual ambiguity) or if *entry* has no baked context. .. py:method:: _make_candidate(entry: dict, siblings: list[dict], shortname: str, minimal: bool = True) -> ParamIDCandidate Build a :class:`ParamIDCandidate` from a metadata *entry*. When *minimal* is ``False`` the (expensive, expand-oracle) minimal distinguishing context is skipped and left empty — used when the entry is already unambiguous so no disambiguating context is required. .. py:method:: _candidate_sort_key(cand: ParamIDCandidate) -> tuple :staticmethod: .. py:method:: _normalise(raw: dict) -> dict :staticmethod: Return a normalised and validated parameter dict with canonical key names. Uses :class:`~pymetkit.models.ParameterEntry` for field coercion and validation. Extra keys present in *raw* (e.g. ``"url"``, ``"units_id"``) are preserved in the returned dict so no information is discarded. .. py:method:: _index(entry: dict) -> None Insert a normalised entry into the internal lookup dicts. .. py:method:: _ensure_loaded() -> None Load parameter data on first access (lazy initialisation). Subsequent calls are no-ops — once ``_loaded`` is ``True`` the method returns immediately without any I/O. .. py:method:: _load_online(cache_ttl: datetime.timedelta, cache_path: Path | str | None) -> None .. py:method:: _load_offline(yaml_path: Path | str | None = None) -> None .. py:method:: _find_offline_yaml() -> pathlib.Path :staticmethod: Locate ``parameter_metadata.yaml``, searching in order: 1. Via ``importlib.resources`` from the installed package (reliable in both regular installs and zip-safe wheels). 2. Next to this module file (editable / development install layout). 3. ``/share/metkit/`` (development tree layout after the YAML files were moved out of the Python package directory). .. py:method:: _find_offline_json() -> Path | None :staticmethod: Locate ``parameter_metadata.json`` (fast-load format). Returns ``None`` if the JSON file is not found — caller should fall back to the YAML file. Searches the same locations as :meth:`_find_offline_yaml`. .. py:method:: _resolve_cache_dir(cache_path: Path | str | None) -> Path | None Return the directory to use for the cache file, or None if unavailable. .. py:method:: _cache_file(cache_path: Path | str | None) -> Path | None Return the full path to the cache file, or None if no cache dir is available. .. py:method:: _read_cache(cache_path: Path | str | None, cache_ttl: datetime.timedelta) -> list | None Return the cached parameter list if it exists and is still fresh, otherwise return None. .. py:method:: _write_cache(params: list, cache_path: Path | str | None) -> None Persist *params* to the cache file (best-effort; errors are silenced). .. py:method:: _resolve(identifier: int | str) -> dict Resolve *identifier* (param_id, shortname, or longname) to a metadata dict. .. py:method:: shortname_to_longname(shortname: str, table: int | None = None, origin: int | None = None, access: str | None = None) -> str Return the long name for *shortname*. :param shortname: ECMWF short name (e.g. ``"t"``, ``"tp"``). :param table: Optional GRIB parameter table number to disambiguate collisions (e.g. ``128`` for classic ECMWF, ``140`` for ocean waves). :param origin: Optional WMO originating centre ID (e.g. ``98`` for ECMWF, ``0`` for WMO, ``7`` for NCEP). :param access: Optional access category filter (e.g. ``"dissemination"``). .. py:method:: longname_to_shortname(longname: str) -> str .. py:method:: shortname_to_param_id_candidates(shortname: str, context: dict | None = None, *, table: int | None = None, origin: int | None = None, access: str | None = None) -> list[ParamIDCandidate] Return all candidate paramids for *shortname*, each with its context. The programmatic counterpart to :class:`AmbiguousParamError`: it returns the candidate + context information as a normal value, so callers can inspect the options and then call :meth:`shortname_to_param_id` with the narrowing ``context=`` (or ``table``/``origin``/``access``) they want. Two independent narrowing mechanisms are available and may be combined: * ``context`` — a dict of MARS keys resolved via the C++ ``expand`` engine (authoritative, cycle-correct). When the MetKit C library is unavailable, the baked ``mars_request_context`` metadata is used as a fallback. * ``table`` / ``origin`` / ``access`` — direct hard filters on the candidate metadata, applied WITHOUT constructing a MARS request. :param shortname: ECMWF short name (e.g. ``"t"``, ``"tp"``). :param context: Optional dict of MARS keys used to pre-narrow the candidate set (e.g. ``{"class": "ai"}``). When omitted, all candidates surviving the hard filters are returned. :param table: Optional hard filter — GRIB parameter table number. :param origin: Optional hard filter — WMO originating centre id (membership). :param access: Optional hard filter — access category string (membership). :returns: Every matching candidate, sorted by ``(table, origin, access, mars_request_context, param_id)``. Length 1 means the shortname (given any supplied context/filters) is unambiguous. :rtype: list[ParamIDCandidate] :raises KeyError: If *shortname* is unknown, or no candidate survives the filters. .. py:method:: _filter_shortname_entries(shortname: str, context: dict | None = None, *, table: int | None = None, origin: int | None = None, access: str | None = None) -> tuple[list[dict], list[dict]] Return ``(surviving_entries, all_siblings)`` for *shortname*. Applies the ``table``/``origin``/``access`` hard filters and the MARS ``context`` filter (via the C++ ``expand`` engine, or the baked-context offline fallback). Does NOT compute minimal contexts — that expensive step is deferred to :meth:`_make_candidate`. Raises ``KeyError`` if the shortname is unknown or no entry survives the filters. .. py:method:: shortname_to_param_id(shortname: str, context: dict | None = None, *, table: int | None = None, origin: int | None = None, access: str | None = None) -> int Return the single param ID for *shortname*, given optional context. Ambiguity is never resolved by guessing. When more than one candidate remains after applying ``context`` and the ``table``/``origin``/ ``access`` filters, :class:`AmbiguousParamError` is raised; its ``.candidates`` attribute lists every remaining id with the context needed to select it. :param shortname: ECMWF short name (e.g. ``"t"``, ``"tp"``). :param context: Optional dict of MARS keys resolved via the C++ ``expand`` engine (e.g. ``{"class": "ai"}``). Partial context is usually sufficient — ``expand`` fills defaults for unspecified keys. :param table: Optional hard filter — GRIB parameter table number. :param origin: Optional hard filter — WMO originating centre id (membership). :param access: Optional hard filter — access category string (membership). :returns: The uniquely resolved paramid. :rtype: int :raises KeyError: If *shortname* is unknown, or no candidate survives the filters. :raises AmbiguousParamError: If more than one candidate remains after applying context/filters. .. py:method:: param_id_to_shortname(param_id: int) -> str .. py:method:: _param_context_from_cpp(param_id: int) -> list[dict] | None :staticmethod: Return MARS contexts for *param_id* from the C++ layer, if available. Placeholder for the future ``metkit_param_context`` C API (outcomes §4). That hook does not exist yet, so this always returns ``None`` to signal "not available", causing :meth:`param_id_to_context` to fall back to the precomputed YAML data. Once the C API lands, implement it here and the public method will transparently prefer it. .. py:method:: param_id_to_context(param_id: int) -> list[dict] Return the MARS-key contexts in which *param_id* is valid. Each context is a dict of MARS keys (e.g. ``{"class": "ai", "stream": "enfo", "type": "cf", "levtype": "sfc"}``) drawn from the authoritative ``params.yaml`` map. These are the raw contexts used to disambiguate shortname collisions. Resolution order: 1. The C++ layer (``metkit_param_context``) when available — see :meth:`_param_context_from_cpp`. Not yet implemented. 2. Fallback: the precomputed ``mars_request_context`` field baked into the bundled parameter metadata. :param param_id: Numeric parameter id. :returns: Zero or more MARS context dicts. Empty if the id has no recorded context (e.g. not referenced in ``params.yaml``). :rtype: list[dict] :raises KeyError: If *param_id* is not in the database. .. py:method:: longname_to_param_id(longname: str) -> int .. py:method:: param_id_to_longname(param_id: int) -> str .. py:method:: get_metadata(identifier: int | str) -> dict Return the full metadata dictionary for a parameter. :param identifier: A param ID (int), shortname, or longname. :type identifier: int or str .. py:method:: get_units(identifier: int | str) -> str Return the units string for a parameter. :param identifier: A param ID (int), shortname, or longname. :type identifier: int or str :returns: The units string, or ``"unknown"`` if not available. :rtype: str .. py:method:: get_all_by_shortname(shortname: str) -> list[dict] Return *all* parameter entries that share *shortname*. Most short names map to exactly one param ID, but ~163 short names are reused across different GRIB parameter tables or originating centres. This method exposes every candidate so callers can inspect the collisions and choose the appropriate one. :param shortname: ECMWF short name to look up. :returns: List of metadata dicts, sorted by ascending param ID. Each dict contains at minimum ``id``, ``shortname``, and ``longname``. :rtype: list[dict] :raises KeyError: If *shortname* is not found in the database at all. .. rubric:: Examples >>> db = ParamDB() >>> entries = db.get_all_by_shortname("t") >>> [(e["id"], e["longname"]) for e in entries] [(130, 'Temperature'), (500014, 'Temperature')] .. py:method:: shortname_has_collisions(shortname: str) -> bool Return ``True`` if *shortname* maps to more than one param ID. :param shortname: ECMWF short name to check. :raises KeyError: If *shortname* is not found in the database at all. .. py:class:: ParamIDCandidate One possible paramid for a shortname, plus the context that selects it. .. attribute:: param_id The candidate numeric parameter ID. .. attribute:: table GRIB parameter table the id encodes to (e.g. ``128``, ``228``). .. attribute:: origin WMO originating centre ids associated with this candidate — the full list (e.g. ``[0, 34, 98]``), since several centres may share the id. .. attribute:: access Access categories (e.g. ``["dissemination"]``) — the full list. .. attribute:: mars_request_context The minimal set of MARS key/value pairs that, when passed as ``context=`` to :meth:`ParamDB.shortname_to_param_id`, selects this candidate (e.g. ``{"class": "ai"}``). Empty if no baked context is recorded or if no key-subset uniquely distinguishes this candidate. .. py:attribute:: param_id :type: int .. py:attribute:: table :type: int .. py:attribute:: origin :type: list[int] .. py:attribute:: access :type: list[str] .. py:attribute:: mars_request_context :type: dict .. py:exception:: AmbiguousParamError(shortname: str, candidates: list[ParamIDCandidate]) Bases: :py:obj:`KeyError` Raised when a shortname (given the supplied context) maps to >1 paramid. .. attribute:: shortname The shortname that could not be uniquely resolved. .. attribute:: candidates Every matching candidate, each carrying the context needed to select it, sorted by ``(table, origin, access, mars_request_context)``. The caller inspects these and re-calls with a narrowing ``context=``. .. attribute:: Initialize self. See help(type(self)) for accurate signature. .. py:attribute:: shortname .. py:attribute:: candidates .. py:class:: ParameterEntry Bases: :py:obj:`pydantic.BaseModel` A single entry from the ECMWF parameter database. Accepts both the canonical field names produced by ``_normalise`` and the raw aliases that may appear in YAML or API responses. .. py:attribute:: model_config .. py:attribute:: id :type: int .. py:attribute:: shortname :type: str .. py:attribute:: longname :type: str .. py:attribute:: units :type: str .. py:attribute:: table :type: int | None .. py:attribute:: mars_request_context :type: list[MarsRequestContext] .. py:attribute:: origin_ids :type: list[int] .. py:attribute:: access_ids :type: list[str] .. py:method:: coerce_id_to_int(v: Any) -> int :classmethod: .. py:method:: normalise_shortname_key(v: Any) -> str :classmethod: .. py:method:: normalise_longname_key(v: Any) -> str :classmethod: .. py:method:: default_empty_units(v: Any) -> str :classmethod: .. py:method:: coerce_table(v: Any) -> int | None :classmethod: .. py:method:: coerce_origin_ids(v: Any) -> list[int] :classmethod: .. py:method:: coerce_access_ids(v: Any) -> list[str] :classmethod: .. py:method:: _normalise_aliases(data: Any) -> Any :classmethod: Accept legacy key spellings from raw YAML / API payloads. .. py:class:: MarsRequestContext Bases: :py:obj:`pydantic.BaseModel` A MARS-key context that (partly) selects a parameter id. This mirrors a single matcher rule from ``share/metkit/params.yaml``. Its schema is kept **separate** from :class:`ParameterEntry` so the context contract can evolve independently and so user-supplied context schemas can validate against it. All fields are optional because ``params.yaml`` rules do not always constrain every key (e.g. ``levtype`` is occasionally absent). .. py:attribute:: model_config .. py:attribute:: class_ :type: str | None .. py:attribute:: stream :type: str | None .. py:attribute:: type :type: str | None .. py:attribute:: levtype :type: str | None