pymetkit.generate_parameter_metadata#

Standalone script to generate:

parameter_metadata.yaml — one entry per ECMWF parameter (human-readable)
parameter_metadata.json — same data, compact JSON (fast-load, ~200× faster)
unit_metadata.yaml — one entry per ECMWF unit
parameter_entry_schema.json — JSON Schema for ParameterEntry validation

Usage#

python -m pymetkit.generate_parameter_metadata # or directly: python generate_parameter_metadata.py

Attributes#

`PARAM_URL`
`UNIT_URL`
`ORIGIN_URL`
`_REPO_ROOT`
`PARAM_OUTPUT`
`PARAM_JSON_OUTPUT`
`UNIT_OUTPUT`
`SCHEMA_OUTPUT`
`MARS_CONTEXT_SCHEMA_OUTPUT`
`LANGUAGE_PARAMS_YAML`
`REQUEST_TIMEOUT`
`_CONTEXT_KEYS`
`parameters`

Functions#

`fetch_units`(→ tuple[list[dict], dict[int, str]])	Fetch all units from the ECMWF parameter database API.
`write_unit_yaml`(→ None)	Write the unit list to a YAML file.
`fetch_origin_map`(→ tuple[dict[int, dict], dict[int, ...)	Fetch all origins and build a reverse map of param_id -> [origin_ids].
`fetch_parameters`(→ list[dict])	Fetch all parameters from the ECMWF parameter database API.
`write_param_yaml`(→ None)	Write the parameter list to a YAML file.
`write_param_json`(→ None)	Write the parameter list to a JSON file (fast-load format).
`table_from_id`(→ int)	Return the GRIB parameter table a param_id encodes to.
`build_param_context_map`(→ dict[int, list[dict]])	Invert `params.yaml` into `param_id -> [context dict, ...]`.
`enrich_parameters`(→ list[dict])	Add `table` and `mars_request_context` to each parameter entry.

Module Contents#

PARAM_URL = 'https://codes.ecmwf.int/parameter-database/api/v1/param/'#

UNIT_URL = 'https://codes.ecmwf.int/parameter-database/api/v1/unit/'#

ORIGIN_URL = 'https://codes.ecmwf.int/parameter-database/api/v1/origin/'#

_REPO_ROOT#

PARAM_OUTPUT#

PARAM_JSON_OUTPUT#

UNIT_OUTPUT#

SCHEMA_OUTPUT#

MARS_CONTEXT_SCHEMA_OUTPUT#

LANGUAGE_PARAMS_YAML#

REQUEST_TIMEOUT = 30#

fetch_units(url: str = UNIT_URL) → tuple[list[dict], dict[int, str]]#

Fetch all units from the ECMWF parameter database API.

Returns:

units (list[dict]) – Normalised unit records ready to be written to unit_metadata.yaml.
unit_map (dict[int, str]) – Mapping of unit id -> unit name string for use in parameter enrichment.

write_unit_yaml(units: list[dict], output_path: pathlib.Path = UNIT_OUTPUT) → None#: Write the unit list to a YAML file.

fetch_origin_map(origin_url: str = ORIGIN_URL, param_url: str = PARAM_URL) → tuple[dict[int, dict], dict[int, list[int]]]#

Fetch all origins and build a reverse map of param_id -> [origin_ids].

The /param/ endpoint does not include an origin field in its response, so we derive the mapping by querying each origin’s filtered parameter list via /param/?origin=<id>.

Returns:

origins (dict[int, dict]) – Mapping of origin_id -> origin metadata (id, abbreviation, name).
param_origin_map (dict[int, list[int]]) – Mapping of param_id -> sorted list of origin_ids that include it.

fetch_parameters(url: str = PARAM_URL, unit_map: dict[int, str] | None = None, param_origin_map: dict[int, list[int]] | None = None) → list[dict]#

Fetch all parameters from the ECMWF parameter database API.

Parameters:

url – The parameter API endpoint.
unit_map – Mapping of unit_id -> unit name string, used to resolve the units field. When None the units field is left empty.
param_origin_map – Mapping of param_id -> list of origin_ids, built by fetch_origin_map(). When provided, each entry gains an origin_ids field containing the sorted list of WMO originating centre IDs that include this parameter. When None the field is omitted.

write_param_yaml(params: list[dict], output_path: pathlib.Path = PARAM_OUTPUT) → None#: Write the parameter list to a YAML file.

write_param_json(params: list[dict], output_path: pathlib.Path = PARAM_JSON_OUTPUT) → None#

Write the parameter list to a JSON file (fast-load format).

This is functionally identical to the YAML but loads ~10-50× faster via json.load() compared to yaml.safe_load().

_CONTEXT_KEYS = ('class', 'stream', 'type', 'levtype')#

table_from_id(param_id: int) → int#

Return the GRIB parameter table a param_id encodes to.

Mirrors the C++ Param encoding / ParamDB._table_from_id:

< 1000 -> table 128 (classic ECMWF; prefix suppressed)
< 1_000_000 -> param_id // 1000
>= 1_000_000-> (param_id % 1_000_000) // 1000

build_param_context_map(params_yaml_path: pathlib.Path = LANGUAGE_PARAMS_YAML) → dict[int, list[dict]]#

Invert params.yaml into param_id -> [context dict, ...].

params.yaml lists, for each MARS context matcher ({class, stream, type, levtype}), the paramids valid in that context. This inverts it so each paramid maps to the list of contexts in which it appears — the raw material for disambiguating shortname collisions. The minimal distinguishing subset among a shortname’s candidates is computed at query time, not here.

enrich_parameters(params: list[dict], context_map: dict[int, list[dict]] | None = None) → list[dict]#

Add table and mars_request_context to each parameter entry.

table is derived from the id encoding; mars_request_context is looked up from context_map (built by build_param_context_map()). Both are fully offline derivations — no network access required.

parameters = []#