Skip to content

CI/CD and release automation

This page documents the GitHub Actions workflows for the ecmwf-sites repository and the Makefile targets they rely on. The helper tests that protect these workflows are inventoried in 20-testing.

flowchart LR
    PR["pull request / push"] --> CI["CI workflow (make test)"]
    SEC["Security workflow (advisory: CodeQL, audits, zizmor, helper tests)"] -. "parallel, advisory" .- CI
    CI -- "push to develop + CI green" --> DEV["Develop: gate + publish develop images + pre-release chart"]
    MAIN["main, workflow_dispatch"] --> REL["Release: gate + tests + tag + GitHub release"]
    REL --> PUB["Publish: images + chart (tag)"]
    PUB --> DOCS["Publish docs (sitesctl upload)"]

Design goals

The repository keeps workflow logic simple by:

  • putting reusable execution logic in the root Makefile
  • keeping workflow YAML focused on triggers, permissions, and credentials
  • using pinned third-party actions
  • validating workflow syntax with actionlint and security patterns with zizmor
  • checking out code without a persisted token (persist-credentials: false, except the release tag job, which authenticates with a deploy key instead of a token)
  • passing workflow values to run: steps through environment variables, not inline ${{ }} expressions
  • avoiding pull_request_target
  • gating fork pull request execution with approved-for-ci
  • writing a concise job summary for every job that starts

Workflow map

Workflow Purpose Documented in
ci.yml Run the repository test suite on every branch and pull request Continuous integration
runner-compatibility.yml Weekly test-matrix check across Ubuntu images Continuous integration
develop.yml Publish develop images and the <VERSION>-develop chart after CI passes Continuous integration
release.yml Create the tag and GitHub release from main Releases
publish.yml Push release images, the chart, and this documentation site Releases
security.yml Advisory scans: CodeQL, dependency audits, workflow lint, helper tests; Go fuzzing, which fails on a finding Security scanning

The shared build entrypoints are in Makefile entrypoints, the documentation pipeline in Documentation site, and the local run recipes in Local testing.

Runner images

Jobs that use GitHub-hosted runners pin the ubuntu-24.04 label. This keeps production workflows independent of changes to the ubuntu-latest label. runner-compatibility.yml runs the repository test suite on both ubuntu-24.04 and ubuntu-26.04 on a schedule and by manual dispatch. It has read-only permissions and does not publish anything.

The image publishing jobs use the managed arc-dind-pet-vsphere-prod runner in the pet-privileged-runners group. The ubuntu-24.04 pinning rule applies to hosted-runner jobs only. .github/actionlint.yaml lists the self-hosted label plus ubuntu-26.04 as valid runner labels.

Job summaries

The local ./.github/actions/write-job-summary action writes Markdown to GITHUB_STEP_SUMMARY. Each summary includes the job status, workflow, job, event, ref, commit, workflow-run link, and job-specific details. Summary steps use always() so a failed check still reports its result.

Security scan helpers also append their findings to the same job summary. They keep the detailed scan result, while the shared action provides the common workflow context.

GitHub does not run steps in a job that is skipped by a job-level condition, so skipped jobs have no job summary. A reusable workflow caller has no steps of its own; its called workflow writes the summaries for the jobs that run.

Secrets and registry access

All Sites container images are stored in the ECMWF container registry (ECCR):

  • registry: eccr.ecmwf.int
  • repository prefix: eccr.ecmwf.int/sites-oss
  • Helm chart: oci://eccr.ecmwf.int/sites-oss/ecmwf-sites

This is the only registry for Sites images. The workflows, the Makefile, the Dockerfiles, and the Helm chart use it on purpose. Do not replace or remove eccr.ecmwf.int, also when an open-source audit reports it as an internal hostname.

The workflows log in to the registry host eccr.ecmwf.int. The Makefile uses sites-oss as the default project for image names and chart publication; SITES_PRIVATE_REGISTRY and SITES_PRIVATE_REGISTRY_PROJECT can override it.

The publishing workflows require:

  • ECCR_USERNAME
  • ECCR_PASSWORD

These credentials are only used in workflows that push images.

./.github/actions/prepare-credentials resolves each value in order: repository secret, then Vault. The workflows never read these credentials from repository variables. Vault is not configured for Sites at the moment: the role and the secret path in the action are placeholders, and a TODO in the action lists the steps to configure them. The Vault login uses GitHub OpenID Connect (OIDC) authentication (the job needs id-token: write). Two action inputs control it: vault_url (default https://secrets-test.ecmwf.int) and vault_role (default sites-publisher). The action validates and masks the resolved values and exposes them as outputs; the workflows pass them directly to the Docker and Helm login steps and never export them to the job environment. node --test .github/actions/prepare-credentials/credentials.test.cjs covers the resolution and masking rules.

Pinned tool versions

The automation pins the tooling it installs:

Tool Pin Where
actionlint v1.7.7 (run with go run) security.yml workflow-lint job
zizmor 1.31.0 (Python package, security_scan.py) security.yml workflow-lint job
Helm v4.3.0 via azure/setup-helm (SHA-pinned action) prepare-publishing, security.yml
pip-audit 2.10.1 utilities/release/dependency_audit_fix.py constants (shared with the audit scan)
govulncheck v1.9.0 utilities/release/dependency_audit_fix.py constants (shared with the audit scan)
Node.js 24 via actions/setup-node (SHA-pinned action); node:24-trixie-slim with a digest in the Hub image build security.yml dependency-audit-npm job, hub/Dockerfile, utilities/docker/local-dev/hub.Dockerfile
Hub frontend packages exact versions in hub/frontend/package.json, with sha512 integrity in package-lock.json hub/frontend/
sitesctl (docs upload and site-mirror image) SITESCTL_VERSION=20260817.0745, verified with SITESCTL_SHA256 publish.yml publish-docs job, site/mirror.Dockerfile

Workflow permissions model

The workflows are written with explicit least-privilege permissions:

  • read-only workflows use contents: read
  • the release workflow uses contents: write in the tag-and-release job only, because it creates the tag and the GitHub release
  • the CodeQL workflow grants only the permissions required to upload security results

Third-party actions are pinned to immutable commit SHAs rather than mutable version tags.

Where to look when changing workflow behavior

When updating the automation model, keep these files in sync:

  • .github/workflows/*.yml
  • .github/actions/*
  • Makefile
  • the pages in this directory
  • docs/design/16-testing.md
  • the root README.md

Behaviour requirements

ID Requirement Implemented in Verified by
CI-01 Fork pull requests do not check out or execute repository code until the approved-for-ci label is applied, and pull_request_target is never used. ci.yml, security.yml gate steps test_security_workflow.py (gate ordering)
CI-02 Hosted-runner jobs pin ubuntu-24.04; image publishing runs only on arc-dind-pet-vsphere-prod in pet-privileged-runners. workflow runs-on keys gate.test.cjs (the gate job pin), runner-compatibility.yml (hosted-image compatibility); the remaining pins are verified by workflow review
CI-03 develop and release images are published only after CI passed for the current branch head; a gate failure skips publishing instead of failing it. develop.yml gate, .github/actions/check-develop-gate gate.test.cjs
CI-04 Release publishing refuses a tag that does not equal VERSION, and requires lockstep versioning. publish.yml, release.yml, make release Workflow step review (no automated test)
CI-05 Every job that starts writes a job summary with status and context. .github/actions/write-job-summary test_workflow_summaries.py (summary output); workflow review (job coverage)
CI-06 Dependency, workflow, and secret scans report findings as warnings and fail only when the scan itself cannot run. utilities/release/security_scan.py test_security_scan.py
CI-07 Publishing credentials come from secrets/variables or Vault and reach only the login steps, masked. .github/actions/prepare-credentials, prepare-publishing credentials.test.cjs
CI-08 Third-party actions are pinned to commit SHAs and workflows are checked with actionlint and zizmor. .github/workflows/*, .github/actionlint.yaml workflow-lint job
CI-09 The Security workflow fuzzes the Site API Content-Range parser and path jail for a fixed time, fails on a finding, and keeps the failing inputs as an artifact. security.yml go-fuzz job, make fuzz-go, api/fuzz_test.go test_security_workflow.py (test_go_fuzz_job_is_blocking_and_keeps_failing_inputs); the seed inputs run in go test ./...
CI-10 Tracked files contain no 64-character hex literal that can be a site master token, in hex form or in base64 form. Tests use synthetic values, for example '0' * 64. Lock files, SHA-256 digests, OpenAPI examples, and low-entropy values are allowed. security.yml repository-tests job test_committed_tokens.py
CI-11 The Hub pages load third-party scripts, styles, and fonts only from the Hub static/vendor directory. The image build installs them from the npm lockfile without install scripts, and copies only the allow-listed files and their licences. Each package has an exact version, sha512 integrity, and an MIT, BSD-3-Clause, or Apache-2.0 licence. hub/frontend/, hub/Dockerfile, utilities/docker/local-dev/hub.Dockerfile, security.yml dependency-audit-npm job hub/tests/test_vendor_assets.py, copy-vendor.test.mjs, dockerfile-policy.test.sh, test_security_scan.py (NpmScanTest)
CI-12 The Security workflow scans the full Git history for secrets with a pinned gitleaks release whose SHA-256 it checks before use. Findings are redacted warnings. security.yml secret-scan job, .gitleaks.toml test_security_workflow.py (test_secret_scan_is_advisory_and_uses_a_verified_gitleaks), test_security_scan.py (SecretScanTest)
CI-13 The secret scan also reads the pull request refs. The Security workflow warns when the refs of the repository have more than one root commit, and fails when the clone is shallow. security.yml secret-scan job, utilities/release/security_scan.py (scan_refs) test_security_workflow.py (test_secret_scan_reads_pull_request_refs_and_checks_root_commits), test_security_scan.py (RefScanTest)
CI-14 The Site proxy image installs every OPM Lua package with an explicit =version, so the build does not take the newest release of the day. Renovate does not update these pins. site/proxy.Dockerfile utilities/tests/dockerfile-policy.test.sh
CI-15 The CI workflow builds the documentation with make build-docs, so a broken link, a broken anchor, or a page missing from the navigation fails a pull request before a release. The publish workflow does not publish a page set that does not build. .github/workflows/ci.yml, Makefile (build-docs), utilities/release/build_publish_docs.sh utilities/tests/test_security_workflow.py (test_ci_builds_the_documentation_after_the_tests), utilities/tests/build-publish-docs.test.sh