Hub API and administration design¶
The Hub owns platform actions and Site resources. It provides the UI and the HTTP API used by platform automation. The OpenAPI files are the complete endpoint references:
hub/src/ecmwf/sites/hub/static/specs/hub-api-spec.yaml(one file for both versions; it declares the v1 and v2 servers)hub/src/ecmwf/sites/hub/static/specs/site-api-spec.yaml
hub/tests/test_api_spec_coverage.py compares the specs with the routes in
both directions: every route must be documented, and the spec must not
document routes that do not exist. The Hub spec is compared against the
registered Flask rules of the v1 and v2 blueprints; the Site spec is compared
against the route registrations parsed from the Go source
(site/.../api/main.go). Both spec files are Jinja templates; the test
renders them with a fixed context. The Hub browser interface is documented in
Hub web interface.
The Go Site API owns Site-volume operations. The boundary between the two APIs is described in Site API design.
Application assembly¶
hub/.../server.py builds the Flask application and registers, in order: the
authentication blueprint, the redirects blueprint, the hub UI blueprint, the
api_v1 and api_v2 blueprints, and the admin blueprint. It also:
- installs
ProxyFixfor the ingress-provided host, proto, and port headers; - injects the branding template data and the
requests_availablecontext flag (true when a notification provider is active); - registers app-level error handlers for 403, 404, 405, 410, 500, 502, 503, and for unhandled exceptions;
- registers the read-only and security-header rules (below) and the cross-site request forgery (CSRF) rule (see Authorization);
- installs the metrics counter hook when the metrics integration is active;
the
/hub/metrics/route always exists and answers 503 while the integration is inactive (21).
Error shape. The two API blueprints register their own Exception
handler: API failures return JSON ({status: ERROR, message}, code taken from
the exception). The app-level handlers render the mapped error template
instead, except when the request accepts JSON or its path contains /api/.
The 405 handler is shared with read-only mode, so a blocked write returns
JSON on API paths (Platform operations — Read-only mode).
CORS and audit. Every API route carries @cross_origin, imported from
hub/src/ecmwf/sites/hub/cors.py. That wrapper forces
Access-Control-Allow-Origin: * and never sends
Access-Control-Allow-Credentials, whatever the call site passes. A browser
page on another origin can therefore read a response only when it sends no
cookie; session-cookie requests from other origins fail. The only routes
without CORS headers are the specification
routes of each version (/spec/ and /spec/swagger.json). Every API route
carries the audit.log_access decorator except the high-frequency lookups
that would flood the audit log: sites_status (polled by the Site view),
users_list, sites_users_list, groups_list, sites_groups_list, and
applications_list, in both versions. Those calls produce no action=access
audit entries (17 — Logs and audit).
Web Server Gateway Interface (WSGI) serving layer¶
The Hub container starts with python -m ecmwf.sites.hub
(hub/.../__main__.py): it reads /etc/build-date into BUILD_DATE, then
launches Gunicorn with hub/.../conf/hub-gunicorn.py. There is no
console_scripts entry point; python hub/.../server.py is the development
run path and re-reads GUNICORN_WORKERS.
| Setting | Value | Source |
|---|---|---|
| Worker class | gthread |
conf/hub-gunicorn.py |
| Workers | 4 | GUNICORN_WORKERS |
| Threads per worker | 10 | GUNICORN_THREADS |
| Request timeout | 120 seconds | GUNICORN_TIMEOUT |
| Graceful timeout | 30 seconds | fixed |
| Worker recycling | every 1024 requests, jitter 100 | GUNICORN_MAX_REQUESTS |
| Bind | 0.0.0.0:8080 |
GUNICORN_PORT |
| Keep-alive | 5 seconds | fixed |
| Request line / fields / field size limits | 65535 / 100 / 8190 bytes | fixed |
| Access log | disabled (the app writes its own audit lines) | fixed |
preload_app |
False |
fixed |
Scaling guidance for operators is in chart values — Hub concurrency.
Response minification. flask-minify (minify(app=...), no Go pipeline)
minifies every rendered HTML page, including its inline CSS and JavaScript.
Static assets. The Hub serves static/, including a robots.txt asset
(User-agent: *, Allow: /) reachable at /static/robots.txt. The
platform publishes no /robots.txt route at the host root; site robots
behaviour is the X-Robots-Tag feature of the site proxy
(site proxy).
Request path and ownership¶
flowchart LR
C["Hub UI or automation"] --> H["Hub API"]
H --> K["Kubernetes API"]
K --> CR["Site resource"]
CR --> O["Site operator"]
O --> W["Site workload"]
C --> S["Site API for content operations"]
The Hub validates the request, applies the caller's role and Site access rules, then writes the Site resource or performs a platform action. The operator reconciles the resource. The Hub does not proxy Site content.
API versions and route groups¶
The Hub exposes v1 and v2 route groups (/hub/api/v1/, /hub/api/v2/).
Both use the Hub authentication model; v2 is the current API surface for new
automation. v2 repeats all 52 v1 route rules and adds exactly four
endpoints; no route exists only in v1:
| v2-only route | Method | Purpose |
|---|---|---|
/sites/<space>/<site>/access-tokens/ |
DELETE | Revoke one HTTP access token by value |
/sites/<space>/<site>/access-tokens/history/ |
GET | Token history, including entries near expiry |
/sites/<space>/<site>/requests/publish/ |
POST | Request the publication of a private Site |
/sites/<space>/<site>/requests/storage/ |
POST | Request extra storage quota |
Differences that keep the same rules: v2 lists Sites with limit/offset
pagination and returns a {total, offset, limit, count, items} envelope,
while v1 returns the full filtered list with no pagination. The
continuation_token query parameter is parsed on list requests for
compatibility but never read: Hub pagination is limit/offset only. It is
not the Site API's working continuation token
(07 — Listing and download).
| Group | Main operations |
|---|---|
| Site resources | List, view, create, update (PUT and PATCH), delete, toggle, extend retention date, upgrade, restart, logs, status, health, export |
| Access | Request access to a Site; list users and groups; change access through Site update and sharing |
| Credentials | Regenerate master token, create/delete HTTP access tokens, read token history, create signed and session URLs |
| OpenID Connect | POST /openid-connect/token/ exchanges a username and password (JSON or form) for an identity-provider access token; DELETE /openid-connect/token/ revokes a token carried as a bearer header; neither requires a Hub session |
| Governance requests | Publish request and storage request (requests/publish/, requests/storage/); they need a working notification provider |
| Platform | Health (JSON or Nagios format via Accept), Site health, read-only state, system/service statistics, cache clearing, Hub logs, version |
| Administration | Backup (GET returns the current day's backup file, POST first runs a fresh backup), directory users, groups, Site owners and groups, on-demand applications, Site create templates (GET for staff; create and delete for platform administrators) |
| Utility | whoami/ (name, username, role, policies), .htpasswd/ password-file generation |
| Analytics | Weekly and monthly Site and platform analytics |
| Specifications | /spec/ (Swagger UI page) and /spec/swagger.json on both versions, public |
The exact paths, methods, request schemas, and response schemas are defined by
the OpenAPI files and the route modules in
hub/src/ecmwf/sites/hub/app/api/.
Tests¶
hub/tests/test_api_spec_coverage.py— both specs against both route sources, in both directions.hub/tests/test_api_request_validation.py— request binding, list parameters, and the add/edit pre-initialization as administrator and non-administrator.hub/tests/test_csrf.py— the session CSRF rule end to end.hub/tests/test_sites_upgrade.py— force annotation and validation, lock 409 and release, role gates, and the failure-summary logging.hub/tests/test_sites_service.py,test_edit_site_reverted.py,test_notifications_required.py,test_routing_rules.py,test_oidc_initializer.py— listing and search, the public-to-private revert message, the 503 request gate, the read-only block rules, and the OIDC init file.- Authentication internals:
core/tests/test_api_authorization.py,test_authentication_chain_builder.py,test_master_token_compare.py,test_auth_view_utils.py.
Most app/api/common.py handlers (health, status, logs, export, backup,
listings, analytics, restart, toggle) have no direct handler tests; they are
covered through the UI flows at best. The table below is honest about it.
Behaviour requirements¶
| ID | Requirement | Implemented in | Verified by |
|---|---|---|---|
| HUB-01 | The Hub API validates and authorizes requests before it writes a Site resource or performs a platform action. | hub/.../app/api/, core/.../auth/ |
hub/tests/test_api_request_validation.py, core/tests/test_api_authorization.py |
| HUB-02 | Site creation and update use one Site model for UI and API input. | core/.../model/site.py, core/.../model/site_builder.py, hub/.../app/api/common.py |
hub/tests/test_sites_service.py, hub/tests/test_api_request_validation.py |
| HUB-03 | Upgrade actions preserve custom images and update built-in application images. | core/.../model/site.py, hub/.../app/api/common.py |
hub/tests/test_sites_upgrade.py |
| HUB-04 | Read-only state blocks mutating requests in front of the application (Ingress snippet or gateway request rule, allow-list GET/HEAD/OPTIONS plus the v2 toggle path and the list pages), and the state update retries on resource-version conflicts. | core/.../kubernetes/platform_state.py, hub/.../routing_rules.py, core/.../kubernetes/ingress_utils.py |
hub/tests/test_routing_rules.py, core/tests/test_routing.py |
| HUB-05 | Hub platform backup exports Site definitions without status or resource-version fields. | core/.../backup/system.py, hub/.../app/api/ |
core/tests/test_backups.py (backup flow; the export stripping itself is not asserted), hub/tests/ |
| HUB-06 | The Hub returns Site and platform status from the Site resource, operator status, and integration services without proxying Site content. | hub/.../app/api/, hub/.../app/admin/ |
— (handlers tested through UI flows at best) |
| HUB-07 | The Hub init container creates the platform data directories and the OIDC client secrets file before the Hub container starts; an unresolvable OIDC endpoint fails the init step without writing the file. | hub/.../initializer/, helm/ecmwf-sites/templates/deployment-hub.yaml |
hub/tests/test_oidc_initializer.py (endpoint resolution, overrides, and the failure case; the directories ensurer is not tested) |
| HUB-08 | A forced upgrade writes a fresh sites.ecmwf.int/upgrade-request UUID annotation, which the operator treats as a full update trigger; a second force on the same Site is possible. |
core/.../model/site.py, core/.../service/sites.py, operator/.../kopf_diff.py |
hub/tests/test_sites_upgrade.py (annotation changes per call; no annotation on ordinary upgrades) |
| HUB-09 | Only one bulk upgrade runs at a time: a Redis lock (30-minute expiry) guards it, a second request answers 409, and the lock releases when the queued work completes. | hub/.../app/api/common.py, core/.../service/lock_service.py |
hub/tests/test_sites_upgrade.py (409 in progress; lock release on dispatch failure and on completion) |
| HUB-10 | v2 contains all v1 routes plus exactly four endpoints (access-token DELETE, token history, publish and storage requests); the OpenAPI specs match the registered routes in both directions. | hub/.../app/api/v1/views.py, v2/views.py, static/specs/ |
hub/tests/test_api_spec_coverage.py |
| HUB-11 | Unsafe requests that present an authentication cookie (the Hub session cookie, the OIDC id-token cookie, or the access_token cookie) must present the session CSRF token; bearer requests and requests without an authentication cookie are exempt; violations answer 403. |
hub/.../csrf.py |
hub/tests/test_csrf.py |
| HUB-12 | The token, signed-URL, and session-URL endpoints proxy the Site API with the Site master token; on archived Sites they answer 410 (master-token regeneration answers 200 with a null token). | hub/.../app/api/common.py, core/.../misc/requests_utils.py |
— (the 410/200 branches are not asserted) |
| HUB-13 | Access, publish, and storage requests require an active notification provider (503 otherwise) and carry the required comment; publish on a public Site answers 400. An access request requires a signed-in caller: an anonymous caller is refused with 401 before the owner is notified. | hub/.../app/api/common.py, core/.../form/forms.py |
hub/tests/test_notifications_required.py (the 503 gate; the body rules are not asserted), hub/tests/test_access_request_authorization.py |
| HUB-14 | A missing analytics report is queued for background generation (202); an existing report is served in the format chosen by Accept; a report without data answers 404. |
hub/.../app/api/common.py, core/.../analytics/report.py |
— |