Built-in Site applications¶
This page describes the built-in applications in a Site Pod. The registry and shared container rules are in site application architecture.
Path shortcuts: site/.../applications/ is
site/src/ecmwf/sites/website/applications/; Lua and nginx paths are relative
to site/.../applications/proxy/.
Application sets and selection¶
The platform splits the applications into three sets: the registry keeps
init applications apart from normal containers
(core/.../model/site_apps_registry.py); the basic and essential sets are
computed by SiteApplicationsBuilder (core/.../model/site.py) from the
site_apps flags in each app_config.py.
- Init applications: run before the normal containers. Only
site-initializeris in this set. - Essential applications:
site-initializer,site-oauth2,site-api,site-proxy.SiteConfiguration.set_applicationsre-adds them on every change, so a Site cannot remove them (12). - Basic applications: the default list for a new Site: the four essential
applications plus
site-admin.
A normal application creates a container only when the Site specification
lists it with an image (image_available()). A specification entry without
an image does not select the application; adding an image for
site-webdav or site-s3 enables that container.
The Site form's applications field processes selections in order
(SiteApplicationsBuilder.from_form, core/.../model/site.py):
- A name that starts with
-removes that application from the default list (for example-site-admin). The-syntax is processed only on create; on update the service resolves every token verbatim and rejects-names. site-adminis the only default application that this syntax can remove; the essential set is restored afterwards.- Unknown names are skipped on create; on update the service raises
Application <name> not supported!. site-customis added when the form carries a custom image (07).
When site-admin is removed: the container is absent, the
admin-logo.svg and admin-favicon.svg keys disappear from the files
ConfigMap, and the proxy does not render the admin location — the path falls
through to the default file-listing rule (06).
Site initializer¶
site-initializer is an init container. It creates the content directory
(/www/html) on the Site volume before the normal containers start
(initializer/bin/initializer.sh). It needs the SITE_PATH environment
variable and exits with an error when the variable is missing. It prints a
version banner (IMAGE_TAG and /etc/build-date). It has no Service port,
route, or probe. A failed initializer prevents the normal containers from
starting.
Site OAuth2¶
site-oauth2 runs OAuth2 Proxy on port 4180. The served prefix is
<site-path>s/oauth2 (proxy_prefix in oauth2.cfg).
This application is the only one that exposes a Service port and owns the
public route for the Site path. Browser traffic reaches it first; it
forwards to site-proxy on 127.0.0.1:8080 after the browser session is
established (the upstream is in the rendered alpha-oauth2.cfg). The
liveness probe is HTTP GET /ping (initial delay 10 seconds, period 60
seconds).
The operator renders three files into the Site files ConfigMap:
oauth2.cfg: the redirect URL, trusted IPs, session settings, and the proxy prefix. The skip-auth route list depends on the Site type: a public Site skips OAuth2 for every path except/s/admin; a private Site skips it for the machine paths/s/api,/s/dav,/s/s3,/s/signed,/s/session,/s/healthz,/s/_m2m, and/s/metrics(only when metrics is enabled). The proxy still enforces its own authentication rules on those paths (06).alpha-oauth2.cfg: the HTTP server wiring (bindAddress0.0.0.0:4180, upstreamhttp://127.0.0.1:8080with a 600-second timeout), the OIDC provider definition (client credentials, scope, issuer URL, and explicit login, redeem, profile, and JWKS endpoints when discovery is skipped), the request-header injection rules, and the claim mapping (10 — OAuth2 Proxy identity wiring).oauth2-error.html: the branded sign-in/error page, mounted read-only over the page in the image.
oauth2.cfg also sets operational behaviour: skip_provider_button (single
IdP, no chooser; the browser goes straight to the provider login page),
silence_ping_logging (the /ping probe stays out of the logs),
show_debug_on_error set to false (the error page never shows debug
details, also when the platform-wide debug setting is on), and
custom_templates_dir pointing at the image's oauth2/templates/
(sign_in.html and robots.txt), which keeps oauth2-proxy from logging
about default templates. Because of skip_provider_button, the
sign_in.html template is never rendered: the branded page users see is
oauth2-error.html. The identity-provider email domain
filter is ["*"]: any verified email passes the proxy, and Site access
rules make the real decisions.
The OIDC endpoint rules and session cookie behaviour are defined in security design.
Site proxy¶
site-proxy runs OpenResty on port 8080. It is the Site gateway and serves
the default file listing UI. It also routes requests to the API, admin,
WebDAV, S3, and custom applications.
site-proxy exposes no Service port and no route. Inside the Pod,
site-oauth2 reaches it at 127.0.0.1:8080.
Its complete configuration contract is in Site proxy.
Site API¶
site-api runs the Go API on port 8090. It binds only the loopback
address, and the proxy reaches it through that address. It does not
authenticate requests itself. The liveness probe runs /api healthcheck in
the container, because the kubelet cannot reach a loopback port.
It handles file and directory operations, token operations, signed and session URLs, checksums, and health. See Site API design.
The proxy gates <site-path>s/api/ with the same admin chain as the admin
path (admin_auth.lua); token and public paths use their own rules
(06).
The version endpoints /api/v1/version/ and /api/v2/version/ return the
IMAGE_TAG environment variable, or N/A when the variable is unset
(api/api/version.go).
Site admin¶
site-admin is a File Browser application on port 9000 under
<site-path>s/admin/. It is part of the basic application set: new Sites
include it by default, and the -site-admin selection removes it
(Application sets and selection). The
operator mounts the Site logo and favicon from the files ConfigMap
(admin-logo.svg, admin-favicon.svg) read-only into the branding
directory.
The container builds its File Browser configuration at startup
(admin/bin/initializer.sh), then starts the server:
| Setting | Value |
|---|---|
| Identity | trusts the proxy header X-Remote-User (--auth.method=proxy) |
| Listen and logs | binds 0.0.0.0 on the container PORT; logs to stdout; type detection by header disabled (--disableTypeDetectionByHeader=true) |
| Base URL / root | <site-path>s/admin/, root /www/html/ |
| Sign-up, login button | disabled |
| Command execution | disabled (--disableExec=true) |
| External symlinks | not followed (--followExternalSymlinks=false) |
| Share permission | disabled (config and seeded user) |
| Token lifetime | 12 hours (hard-coded in the script) |
| Seeded user | administrator, list view mode, password locked, no admin or share permission |
| File and directory modes | 0644 and 0755 |
| Branding | external resources disabled; name "Sites File Browser"; files from the mounted branding directory |
The proxy restricts this path to Site administrators: admin_auth.lua runs
the authentication chain with the admin_groups and admin_users rules,
which match the Site's admin users and groups plus the platform
hub.administrator_users and hub.administrator_groups
(SiteShare.get_admin_users(include_default=True),
10).
Site WebDAV¶
site-webdav is an optional WebDAV application on port 7000 under
<site-path>s/dav/. It uses the Site volume as its storage. The operator
renders its configuration (webdav.yaml) and the branded header and footer
(webdav-header.html, webdav-footer.html) into the Site files ConfigMap;
the header and footer are mounted read-only and referenced by path from
webdav.yaml. The container itself runs without user accounts
(noPassword: true, CRUD permissions): the proxy enforces all access
(webdav_auth.lua runs BasicAuth.handle(true)).
The WebDAV BasicAuth password must be a Site token: the master token
(Hex64 form) or a Branca token. JWT session tokens are not accepted by
the BasicAuth handler (lua/basic.lua). A directory that contains a
.htpasswd file is checked by nginx auth_basic instead; a master or
administrator token supplied as the password overrides that check
completely. On public Sites, directories without a .htpasswd file also
accept anonymous read-only WebDAV: the handler sets X-Remote-User:
anonymous for read-only methods (lua/basic.lua).
The rendered webdav.yaml carries the WebDAV server's CORS block: enabled,
with credentials, allowed_hosts: ['*'], empty exposed_headers, and the
method set COPY, DELETE, GET, HEAD, LOCK, MKCOL, MOVE, OPTIONS, PATCH, POST,
PROPFIND, PROPPATCH, PUT, UNLOCK. No client reaches the application
directly: the site proxy owns the s/dav/ route, enforces authentication,
and adds MS-Author-Via: DAV for classic WebDAV client discovery
(06 — Endpoint locations), so the
wildcard origin never widens access beyond what the proxy admits.
Site S3¶
site-s3 is an optional S3-compatible application on port 6000 under
<site-path>s/s3. Its filesystem backend is rooted at /www/html. The
operator exports the s3proxy contract to the container:
S3PROXY_ENDPOINT on http://0.0.0.0:6000, S3PROXY_AUTHORIZATION=none,
S3PROXY_SERVICE_PATH=<site-path>s/s3, JCLOUDS_PROVIDER=filesystem, and
the filesystem base directories JCLOUDS_FILESYSTEM_BASEDIR and
JCLOUDS_FILESYSTEM_NIO2_BASEDIR (s3/app_config.py).
The client authenticates at the proxy: s3_auth.lua runs the
authentication chain over the X-Amz-Security-Token request header with no
group requirements, so any valid Site token passes. The proxy then clears
both the Authorization and X-Amz-Security-Token headers before
forwarding (nginx/s3.conf). The application therefore runs without any
authentication material, and it receives no public route outside the Site
proxy.
Container and Service behaviour¶
| Application | Init | Optional | Container port | External route | Liveness probe | Site volume |
|---|---|---|---|---|---|---|
site-initializer |
yes | no | — | — | none | yes |
site-oauth2 |
no | no | 4180 |
Site path (owns the route) | HTTP GET /ping |
yes |
site-proxy |
no | no | 8080 |
none — loopback only | TCP | yes |
site-api |
no | no | 8090 (loopback only) |
/s/api through proxy |
exec /api healthcheck |
yes |
site-admin |
no | yes (default, removable) | 9000 |
/s/admin through proxy |
TCP | yes |
site-webdav |
no | yes | 7000 |
/s/dav through proxy |
TCP | yes |
site-s3 |
no | yes | 6000 |
/s/s3 through proxy |
TCP | yes |
Only selected optional applications are added to the container list. The Site
Service exposes exactly one port — site-oauth2 on 4180 — and
site-oauth2 owns the route for the Site path; other applications and
optional applications do not create independent Service ports, Ingress, or
HTTPRoute objects (SiteExporter.to_service, get_service_ports()). The
Service uses sessionAffinity: ClientIP with a 10800-second timeout, so one
client keeps its sticky session, which the OAuth2 session cookie needs.
Resource tiers¶
Each application module defines its own RESOURCE_LIMIT table
(<application>/app_config.py). All current modules override the base
table (domain model). The Site
setting settings.resource_limits selects one tier per Site, and the
same tier applies to every container. The tables below give the memory
and CPU request and limit of each built-in application per tier. A limit
of no limit means the code sets MAX: None, which leaves the
corresponding Kubernetes limit unset, so the container can burst without
a bound.
Memory (request / limit):
| Application | normal |
medium |
high |
|---|---|---|---|
site-initializer |
128 Mi / 128 Mi | 128 Mi / 128 Mi | 128 Mi / 128 Mi |
site-oauth2 |
64 Mi / 512 Mi | 128 Mi / 1024 Mi | 256 Mi / 2048 Mi |
site-proxy |
64 Mi / 512 Mi | 128 Mi / 1024 Mi | 256 Mi / 2048 Mi |
site-api |
64 Mi / 1024 Mi | 128 Mi / 1024 Mi | 256 Mi / 1024 Mi |
site-admin |
64 Mi / 1024 Mi | 128 Mi / 1536 Mi | 256 Mi / 2048 Mi |
site-webdav |
32 Mi / 512 Mi | 64 Mi / 768 Mi | 128 Mi / 1024 Mi |
site-s3 |
128 Mi / 256 Mi | 256 Mi / 512 Mi | 256 Mi / 768 Mi |
CPU (request / limit):
| Application | normal |
medium |
high |
|---|---|---|---|
site-initializer |
100 m / no limit | 100 m / no limit | 100 m / no limit |
site-oauth2 |
100 m / 1000 m | 200 m / 1500 m | 300 m / no limit |
site-proxy |
100 m / 1000 m | 200 m / 1500 m | 300 m / no limit |
site-api |
100 m / 2000 m | 100 m / 3000 m | 100 m / no limit |
site-admin |
100 m / 1000 m | 200 m / 1500 m | 300 m / no limit |
site-webdav |
100 m / 1000 m | 100 m / 1500 m | 100 m / no limit |
site-s3 |
250 m / 500 m | 500 m / 1000 m | 1000 m / no limit |
The initializer takes the same values in every tier. The custom application and the content mirror define their own tables (custom applications, content mirror).
Build metadata¶
Every Site image records its build at image-build time with the mandatory
TAG build argument and the IMAGE_TAG environment variable. Seven of the
eight images also write a /etc/build-date file; api.Dockerfile does not,
because its final stage is distroless. Only the initializer and the mirror
print a version banner; the proxy and OAuth2 entrypoints export BUILD_DATE
from /etc/build-date without printing. The Go API serves IMAGE_TAG at
/api/v2/version/ (see 11-site-api).
Behaviour requirements¶
| ID | Requirement | Implemented in | Verified by |
|---|---|---|---|
| BIA-01 | The initializer runs before normal Site containers and has no Service port or route. | site/.../applications/initializer/app_config.py, core/.../model/site_exporter.py |
site/tests/test_site_apps_registry.py (test_only_the_initializer_is_an_init_application; test_only_oauth2_has_a_service_port_and_a_route_path: no Service port or route; the start order itself is not asserted) |
| BIA-02 | OAuth2 Proxy receives browser traffic, uses the rendered Site configuration, and forwards authenticated requests to the Site proxy. | site/.../applications/oauth2/app_config.py |
site/tests/test_oauth_app_config.py |
| BIA-03 | The admin, WebDAV, and S3 applications are selected from the Site application list and are reached through proxy-owned paths. | site/.../applications/admin/, webdav/, s3/, proxy/ |
site/tests/test_admin_app_config.py, site/tests/test_webdav_app_config.py, site/tests/test_s3_app_config.py |
| BIA-04 | The Site API is reachable through the loopback network and exposes no independent external route. | site/.../applications/api/app_config.py, core/.../model/site_exporter.py |
site/tests/test_api_app_config.py |
| BIA-05 | Every Site keeps the essential set (site-initializer, site-oauth2, site-api, site-proxy); the application list re-adds it on every change, and site-admin is the only removable default application. |
core/.../model/site.py (set_applications), core/.../model/site_apps_registry.py |
site/tests/test_site_apps_registry.py |
| BIA-06 | A normal application creates a container only when the Site specification lists it with an image. | core/.../model/site_apps_registry.py (image_available()) |
site/tests/test_site_apps_registry.py |
| BIA-07 | The proxy renders the admin, API, WebDAV, and S3 locations only when the application is configured for the Site. | site/.../applications/proxy/app_config.py (is_application_configured filter), proxy/nginx/*.conf |
— |
| BIA-08 | WebDAV BasicAuth accepts the master token (Hex64) or a Branca token only; on public Sites, read-only methods on directories without .htpasswd run as anonymous. |
proxy/lua/basic.lua |
site/tests/test_proxy_lua_basic.py (anonymous fallback and Hex64 path; the Branca branch is not asserted) |
| BIA-09 | The S3 container receives no authentication material: the proxy authenticates the X-Amz-Security-Token header and clears it and Authorization before forwarding. |
proxy/nginx/s3.conf, proxy/lua/s3_auth.lua |
— |
| BIA-10 | The Site Service exposes exactly one port, site-oauth2 on 4180, with sessionAffinity: ClientIP. |
core/.../model/site_exporter.py (to_service), per-app get_service_port() |
site/tests/test_oauth_app_config.py (port and route present), site/tests/test_site_apps_registry.py (test_only_oauth2_has_a_service_port_and_a_route_path: absent for all other applications); the Service object itself is not asserted |
| BIA-11 | Every Site image sets IMAGE_TAG from the mandatory TAG build argument; all images except the Go API also record /etc/build-date. |
site/*.Dockerfile |
site/tests/test_dockerfile_licences.py, utilities/tests/dockerfile-policy.test.sh |