Site proxy¶

site-proxy runs OpenResty on port 8080. It is the Site gateway and serves the default file listing UI. It also routes requests to the API, admin, WebDAV, S3, and custom applications.

site-proxy exposes no Service port and no route. Inside the Pod, site-oauth2 reaches it at 127.0.0.1:8080.

This page gives the application contract first, then the full design.

site-proxy is the OpenResty gateway inside each Site Pod. It serves the default listing UI, applies request policy, and routes traffic to local applications. Its Python configuration class renders the NGINX and HTML files that the container uses.

Path shortcuts: site/.../applications/ is site/src/ecmwf/sites/website/applications/; Lua and nginx paths are relative to site/.../applications/proxy/.

Configuration assembly¶

site/src/ecmwf/sites/website/applications/proxy/app_config.py renders the files ConfigMap with Jinja: 23 NGINX files (all proxy/nginx/ templates, including custom-mime.types) and 3 HTML files (header.html, footer.html, retention-banner.html) — 26 keys total. The two Jinja environments differ: NGINX templates render without autoescaping; HTML templates render with autoescaping on. The rendered header.html comes from the template header-proxy.html, which includes the shared listing header (also used by the WebDAV listing) and adds the hidden index title.

The container mounts the rendered .conf files into /usr/local/openresty/nginx/conf/, server.conf into /etc/nginx/conf.d/, and the listing templates into /opt/proxy/html/template/. Every mount is read-only except location.conf (app_config.py, extra_volume_mounts). The fancyindex module reads header.html and footer.html from /opt/proxy/html/template/ one time, when NGINX starts (fancyindex_header ... local). On startup, bin/entrypoint.sh removes a legacy /www/template directory or symlink, exports the image BUILD_DATE, and starts NGINX in the foreground with nginx.conf.

The renderer provides these inputs:

nginx.conf raises variables_hash_max_size to 2048. The OpenResty image and this configuration use most of the default 1024 entries of the variables hash, so NGINX rebuilds the hash with a larger bucket size and warns for a site that adds a few more variables. The larger hash keeps the default bucket size and gives the configuration room to grow. | hostname, metrics_enabled, site_type (type) | Set the server name, and select the metrics and public/private configuration. | | js_code, css_code | Minified listing scripts and styles, inlined into the listing templates (core/.../misc/asset_utils.py). |

The renderer validates the values that become NGINX settings. remove_unsafe_nginx_settings deep-copies the configuration (the Site resource is never mutated), removes invalid entries, and logs one WARNING per affected key: Site <path>: ignored N invalid value(s) in <key>. The validated lists are then used for rendering. This also protects the proxy when a Site resource was changed directly in Kubernetes. The accepted formats are (core/.../config/constants.py):

The env declarations in nginx.conf pass these process variables to the Lua modules: OIDC_CLIENT_ID, OIDC_CLIENT_SECRET, OIDC_INTROSPECTION_URL, OIDC_EXTRA_AUDIENCES, AUTHORIZED_ADMIN_USERS, AUTHORIZED_ADMIN_GROUPS, AUTHORIZED_STAFF_USERS, AUTHORIZED_STAFF_GROUPS, MASTER_TOKEN, RETENTION_DATE, SITE_PATH, SITE_TYPE, and IMAGE_TAG. nginx passes only the variables that the env directive names to the worker; a variable that the Lua reads but the directive omits is nil in the worker.

Request path¶

flowchart LR
    R["Ingress or HTTPRoute"] --> O["OAuth2 Proxy :4180"]
    O --> P["OpenResty proxy :8080"]
    P --> A["Go API :10000"]
    P --> U["Admin :9000"]
    P --> W["WebDAV :7000"]
    P --> S["S3 :6000"]
    P --> C["Custom application :8000"]
    P --> F[("Static Site files")]

Browser traffic normally enters OAuth2 Proxy first and holds an OAuth2 session there. The API, WebDAV, S3, _m2m, and signed or session URL traffic uses the skip-auth list of site-oauth2: it crosses the site-oauth2 hop unauthenticated (the Site route points only at that container; nothing exposes site-proxy as a Service port — it is reached at 127.0.0.1:8080 inside the Pod) and is authenticated by the proxy's own Lua modules. The proxy selects the authentication module and backend from the request path.

Always-on proxy behaviour¶

The proxy always:

• listens on port 8080 with server_name set to the site's public host (absolute_redirect off, server_tokens off);
• serves /www/html and the generated directory-listing templates (index index.html);
• includes the API, admin, WebDAV, S3, health, metrics, CORS, error, and main Site location configuration files;
• writes access logs to stdout in the main format (which records the X-Forwarded-For client and the X-Remote-User identity, and writes REDACTED in place of the signed URL signature value and the session URL token in the request and referer fields) and error logs to stderr at the platform LOG_LEVEL; the health and metrics locations set access_log off, and the metrics log phase also skips their URIs;
• disables response buffering on the way to and from backends (proxy_buffering off, proxy_request_buffering off, proxy_cache off) and uses sendfile, TCP push, and TCP nodelay;
• sizes body and header buffers at 512 KiB and 4×64 KiB;
• supports proxy and FastCGI timeouts of up to 600 seconds;
• compresses responses with gzip (minimum 1 KiB, level 5, Vary set, listed text/font/SVG types), and serves pre-compressed .gz files (gzip_static) when they exist;
• overrides the response charset to UTF-8 and accepts underscores in request header names (underscores_in_headers);
• keeps client connections alive for 30 seconds or 1000 requests, and upgrades Websocket connections to the backend ($connection_upgrade map);
• forwards X-Real-IP, X-Forwarded-For/Proto/Host/Port, and X-Remote-User to backends (proxy.conf), always declaring the external scheme as https and port 443;
• follows a symlink in the Site content only when the link and its target have the same owner (disable_symlinks if_not_owner from=$document_root). The site containers create links and files as UID 65534, so links between site files work. A link to a file of another owner, for example /etc, a mounted Secret, or a file that root copied into the volume, gives 403. This also closes the race between a check and the open of the file, because NGINX checks each path component when it opens it;
• resolves names with the pod's own resolver (resolver local=on ipv6=off);
• uses the prometheus_metrics Lua shared dictionary (10 MiB) for metrics data when the metrics integration is active;
• serves the custom MIME type map nginx/custom-mime.types: ECMWF data formats (grib, grib1, grib2, bufr, odb, tfrecord) and csv as text/plain;
• verifies TLS certificates for Lua HTTPS requests with the system CA bundle (lua_ssl_trusted_certificate, depth 2).

The proxy does not impose a client body size limit. Upload limits and backpressure are enforced by the Go API.

Site type scenarios¶

The authentication chain and status-code rules are in security design. Public API path hiding is also enforced by the Go public router.

Backend selection¶

The default Site path serves static content. A configured site-custom application changes the backend:

The full custom application contract is in custom applications.

Endpoint locations¶

The health location always allows the pod loopback address, so the Go API probe passes whatever trusted_ips holds; all other sources need trusted_ips. An empty trusted_ips list denies metrics requests. The metrics allowlist evaluates the X-Forwarded-For real IP; the health location uses the direct peer address. The metrics configuration is not rendered when the metrics integration is disabled (see 07-site-api).

Authentication modules¶

Each location loads one entry module with access_by_lua_file. The modules and their policies (proxy/lua/):

All chain calls share one token extraction path: the configured header (Authorization by default, flattened by OAuth2 Proxy into <token>,Bearer <access-token> — the first comma-separated part wins; X-Amz-Security-Token for S3), then the OAuth2 session cookie s, then the BasicAuth cookie basic. Typed prefixes Bearer, Branca, and Master select the token kind inside the header. The chain order, JSON Web Token (JWT) introspection, and the authentication caches (150-second negative entries, at most 10-second positive Branca entries) are specified in 10-security; this page only maps modules to locations.

Site browsing user interface¶

When no site-custom application replaces it, the proxy serves a file-listing UI built from the fancyindex templates (proxy/html/template/) and browser scripts (proxy/html/js/). The configuration renderer minifies the scripts and inlines them: styles into header.html, scripts into footer.html (core/src/ecmwf/sites/core/misc/asset_utils.py). The listing uses fancyindex with date_desc sorting, human-readable sizes, and the parent-directory entry hidden (server.conf).

The scripts cooperate through two shared primitives: 0.js (named to load first) defines the AppReady promises for DOM and site-information readiness, and utilities.js publishes the site information (path, space, name, user identity, default signed/session URL durations in days, which the header template injects from platform configuration) and the Site API endpoint builders. Several features render as custom elements (<user-name>, <main-menu>, <file-toolbar>).

• Top menu and breadcrumbs render the Site branding, links, and current path; the username script shows the signed-in user from the identity headers that OAuth2 Proxy injects.
• The toolbar (file_toolbar.js, a <file-toolbar> element) adds a search box and sort controls. Search is a client-side, case-insensitive substring filter on entry names that applies to both the table and the grid view; it never calls the server. The sort controls write the fancyindex query parameters in the URL: C for the column (D date, N name, S size) and O for the order (D descending, A ascending), then reload the page; with no parameters the listing uses the configured default date_desc.
• Edit Mode (Ctrl+E) is visible to site administrators only. It adds delete buttons to entries, a directory-creation modal, and the upload drop area. The state persists in localStorage.
• Dropzone upload accepts files and directories by drag-and-drop and uploads them with PUT to the Site API, using resumable chunks (HEAD plus Content-Range, 4 MiB default chunk, 4 concurrent uploads).
• The right-click menu generates signed and session URLs in the browser: it calls s/api/v2/signed-urls/ and s/api/v2/session-urls/ with a scope and expiry, and copies the result to the clipboard. Default durations come from platform configuration through the header template.
• Thumbnails and grid layout render previews and an alternative layout; toasts report upload and action results; a retention banner is injected when the retention date has passed.

Headers, errors, and retention¶

• Server-level add_header directives — X-Site-Proxy-Server with the proxy identity line (<account>; <site version>; <namespace>;, custom-headers.conf, added with always) and X-Robots-Tag when no-robots is on — reach every response except the WebDAV locations: webdav.conf declares its own add_header directives, and nginx does not inherit server-level add_header into a location that sets them.
• CORS maps and headers are rendered only when CORS is enabled; OPTIONS then answers 204 preflights directly. Both branches set Strict-Transport-Security (two years, includeSubdomains, preload) and X-Permitted-Cross-Domain-Policies: none; the CORS-disabled branch also clears any upstream CORS headers.
• trusted_domains contributes the platform domains and their subdomains to the CSP frame-ancestor policy; CORS extra_domains extend it only when CORS is enabled.
• headers.no_cache adds expires -1 for all content or the configured file extensions.
• headers.no_robots adds X-Robots-Tag: noindex.
• Custom error entries use CODE=URL_OR_PATH; root-relative targets receive the Site path prefix; each entry becomes an error_page to a named location that answers 302. Without custom handling and without CORS, the defaults redirect 403/404/405/410/500/502/503 to https://$host/hub/error/page/<code>/.
• A request for favicon.ico under the Site path or the admin UI redirects (301) to the platform-root favicon, which the Hub serves from the branding (favicon.conf, included by location.conf and admin.conf).
• Retention: the retention module runs in every content authentication path. It always sets X-Site-Retention-Date-Expired to true or false (result cached 150 seconds per worker). When the date has passed, it loads the branded retention-banner.html from the files ConfigMap and nginx sub_filter injects it before the closing </html> tag of responses. A missing or empty banner file logs an ERROR and injects nothing; an invalid RETENTION_DATE logs a WARNING and behaves like no date (lua/retention_date_notification.lua).

Use cases¶

• One public entry point for a Site: listings, the Site API, WebDAV, S3, administration, and custom applications all arrive here.
• The default listing UI, with the branded header and footer, for Sites that publish static content.
• The retention banner and security headers, applied to every response without touching the content.
• Metrics and health: the /s/metrics and /s/healthz locations.

Example usage¶

A request for the Site root returns the listing with the security headers:

curl -sI "https://<domain>/<space>/<site>/"

Every other interface of the Site is a location on the same host:

Request flow¶

flowchart TD
    C["Client"] --> P["site-proxy :8080"]
    P --> L["listing UI and Site content"]
    P --> A["/s/api - site-api"]
    P --> D["/s/dav - site-webdav"]
    P --> S["/s/s3 - site-s3"]
    P --> AD["/s/admin - site-admin"]
    P --> X["custom application"]