Skip to content

scoreboard — boundary record and attestations

Leaderboard service (their apps/scoreboard): a portal and an API for benchmark boards and score submissions. On dev, the hostname is an ADR-0005 §16 whole-public hostname on cf-public. Anyone can read the boards, under a per-address budget. A write needs a Cloudflare Access login on /v1/scores and passes Envoy's Access-JWT check, under a per-address write budget. Staging and production have had the same shape since 2026-10-07. Owner decision 2026-10-22 (ADR-0005 §16(j), OME-1286): on every environment the write-path Access application's policy becomes Include Everyone, so any person who can log in may submit. The attestation tables record each edit.

Environment Hostname Reads Writes Gateway and tunnel Status
dev leaderboard.dev.screamingface.ai anonymous; 600 a minute per client address, 6000 a minute for the route Access on /v1/scores (the dev leaderboard application's own AUD, §11 as amended 2026-10-30; policy Include Everyone) and Envoy's JWT check on every write method; 30 writes a minute per client address, 300 a minute for the route cf-public, through the tenant's public tunnel sf-public-dev live, single-homed on cf-public
staging leaderboard.staging.screamingface.ai anonymous since 2026-10-07; 600 a minute per client address, 6000 a minute for the route Access on /v1/scores (own AUD) and Envoy's JWT check on every write method; 30 writes a minute per client address, 300 a minute for the route cf-public, through tunnel sf-public-staging live (#1501, #1556; owner tunnel move and Access edit 2026-10-07)
production leaderboard.screamingface.ai anonymous since 2026-10-07; 600 a minute per client address, 6000 a minute for the route Access on /v1/scores (own AUD) and Envoy's JWT check on every write method; 30 writes a minute per client address, 300 a minute for the route cf-public, through tunnel sf-public-prod live (#1502, #1572; owner tunnel move and Access edit 2026-10-07)

Environments (2026-10-05): dev at pin c08ce63e (#1492); staging since 2026-09-09 and production since 2026-09-30, both at pin 68cad9d9, each with one user declaration that gates every method of the hostname.

Platform half: kubernetes/apps/sf-scoreboard/ (quota, NetworkPolicies, the database ExternalSecret and the ServiceAccount; a disposable Postgres in overlays/dev, managed Postgres on staging and production). The namespace sf-scoreboard is in the tenant registry kubernetes/tenants/screamingface/namespaces.yaml with the cf-public admission label openmined.org/public-routes: "true". Edge on dev: two route declarations, rendered by the tenant-edge chart (ADR-0005 §14, §16):

  • kubernetes/tenants/screamingface/routes/dev/leaderboard.yaml — tier anonymous-read, GET HEAD OPTIONS, path prefix /, 30s timeout, no SecurityPolicy, and the read budget (BackendTrafficPolicy leaderboard-public-budget).
  • kubernetes/tenants/screamingface/routes/dev/leaderboard-writes.yaml — tier user, the six write methods, the dev leaderboard application's own AUD, SecurityPolicy leaderboard-writes-cf-access, and the write budget (BackendTrafficPolicy leaderboard-writes-public-budget, ADR-0005 §16(j)).

The hostname is on the chart's tenant-edge.publicHostnames list and off tenant-edge.publicHostnamesDualHome, so both routes attach to cf-public only. The cf-public fleet may reach the backend on TCP 9106 and Cloudflare's published IPv4 ranges on TCP 443, for the Access signing keys (JWKS) (kubernetes/infrastructure/pulse-edge/base/cf-public-fleet-netpol.yaml). Traffic arrives through the tenant's public tunnel sf-public-dev (connector cloudflared-public in sf-edge). Workload: tenants-charts ticket kubernetes/tenants/screamingface/charts/scoreboard.yaml + values kubernetes/apps/sf-scoreboard/values/dev.yaml (config.authMode: cloudflare_headers).

The identity boundary

The scoreboard does no token work. In cloudflare_headers mode it takes the submitter of a score from X-User-Email. It trusts that header from any peer inside config.allowedNetworks, which is 10.244.0.0/16, the whole pod CIDR (tenant source apps/scoreboard/src/scoreboard/core/auth/cloudflare_identity.py). So three platform controls are the whole boundary:

  1. The identity strip on cf-public. The ClientTrafficPolicy cf-public-client-ip (namespace pulse-edge) removes X-User-Email, X-User-Id and X-Service-Id from every request before any filter runs. scripts/ci/k8s-validate.sh asserts this statically, and no overlay may patch the policy.
  2. The write route's SecurityPolicy. leaderboard-writes-cf-access verifies the Access JWT (issuer, signing keys, the AUD). Only then does it set X-User-Email from the verified email claim. A write without a valid token answers 401 before the app.
  3. The ingress selector. The NetworkPolicy allow-scoreboard-ingress-proxy admits only the cf-public and cf-protected proxy fleets to port 9106. Each peer names both gateway.envoyproxy.io/owning-gateway-name and gateway.envoyproxy.io/owning-gateway-namespace: pulse-edge. The platform Gatus pod is the only other peer (allow-scoreboard-ingress-gatus; it reads /healthz only, an accepted widening recorded in netpol.yaml).

The read route has no JWT filter. The strip alone keeps a forged X-User-Email off a read, and a read carries no verified identity.

Attestation record (leaderboard.dev.screamingface.ai)

The read path has no AUD, so its recorded boundary is this dated attestation set: facts that live in the tenant's Cloudflare account and cannot be verified from git. Behavioral verification: the four Gatus rows in the group Tenant · sf-scoreboard (ADR-0005 §16(h)):

  • Leaderboard public read — GET /v1/benchmarks answers 200. A 302 means an Access application crept over the read path.
  • Leaderboard write refusal — POST /v1/scores answers 302 to the Access login. Any other answer means the path-scoped application is gone.
  • Leaderboard write JWT — POST /v1/benchmarks answers 401 Jwt is missing from Envoy (a path that Access does not cover).
  • Leaderboard JWKS reachability — POST /v1/benchmarks with a crafted, unsigned token (key id probe, the full dev leaderboard AUD) answers 401 Jwks doesn't have key to match kid or alg from Jwt. Envoy answers from its cached key set, and an unknown key ID does not make it refetch. So the row lags a break in the fleet's TCP 443 egress: it turns red with Jwks remote fetch is failed only after the cached set expires and the refetch fails. The token is not a credential.

Re-attestation triggers: Cloudflare plan change · any tunnel change for the hostname (rule, CNAME, connector, token) · any Access application change on the zone (destination, policy, AUD) · the "Remove visitor IP headers" transform touched · a change to the read or write budget or to the chart's whole-public lists · any abuse incident on the hostname. After any Access edit, the real-client submission proof in ADR-0005 §16(j) runs again.

# Fact Status Attested by Date
1 On this hostname, the dev leaderboard Access application (created 2026-10-30) covers only leaderboard.dev.screamingface.ai/v1/scores: a path destination. Its AUD is the audience in routes/dev/leaderboard-writes.yaml. Its one policy is Allow, Include Everyone, login method one-time PIN. The shared dev application (fusion.dev, reports-internal.dev) no longer lists the path. No Bypass policy exists, and no other Access application covers the whole hostname. Outside-in, platform: POST /v1/scores answers 302 with the login kid b745c508… (this AUD), and fusion.dev keeps the shared f9d2d35a…. The policy and login method: PENDING — owner platform 2026-10-30
2 The hostname's rule on sf-public-dev (origin: the cf-public fleet's in-cluster Service) was added through the API (scripts/ops/cloudflare-tunnel-route.sh add), and both cloudflared-public pods logged the new configuration. Then the proxied CNAME was edited in place to the sf-public-dev cfargotunnel.com name, through the API (cname). PENDING — owner's tunnel move — —
3 The hostname's rule on the protected tunnel was removed through the API (remove), never with a dashboard route delete. The hostname appears in exactly one connector's latest configuration. PENDING — owner — —
4 The zone's "Remove visitor IP headers" managed transform is OFF. Required: cf-public reads the client address from CF-Connecting-IP with failClosed: true. With the transform on, every request to the hostname answers 403 at Envoy, and the read budget has no address to key on. PENDING — owner — —
5 Zone plan Free, and Bot Fight Mode status (zone-wide, unscopable on Free; it can challenge anonymous programmatic readers before our edge). PENDING — re-attest with the move (attested for the zone on 2026-08-31, report-intake rows 4 and 5) — —
6 The read budget is live: BackendTrafficPolicy sf-scoreboard/leaderboard-public-budget is Accepted on aks-dev. It allows 600 requests a minute per client address (IPv4, and IPv6 per /128) and 6000 a minute for the route. PENDING — platform, after the merge — —
7 Burst and spoof proofs (§16(f) step 7). From one address, requests above 600 in one minute answer 429, and the address answers 200 again after the window. A CF-Connecting-IP header that the caller sends through Cloudflare does not choose the caller's bucket. Recorded before the staging move starts. PENDING — platform, after the merge — —
8 Identity strip proof through the cf-public Envoy listener on aks-dev, by a port-forward probe before the tunnel move: a forged X-User-Email did not reach the app; a request without CF-Connecting-IP answered 403; POST /v1/benchmarks answered 401 Jwt is missing with no token, and 401 Jwks doesn't have key to match kid or alg from Jwt with the crafted token (a JWKS fetch on TCP 443 succeeded). ATTESTED (port-forward probe on dev) platform 2026-10-05
9 The write budget is live: BackendTrafficPolicy sf-scoreboard/leaderboard-writes-public-budget is Accepted on aks-dev. It allows 30 writes a minute per client address and 300 a minute for the route (ADR-0005 §16(j)). PENDING — platform, after the merge — —
10 Open submission (ADR-0005 §16(j)): the write path has its own dev Access application with one policy, Allow Include Everyone. One real submission with client 0.1.6 or later by a person on no allow list: the row shows that person's email, an anonymous POST /v1/scores answers 302, and fusion.dev still refuses that person. PENDING — owner (application) and platform (proof) — —

Attestation record (staging and production, 2026-10-07)

# Fact staging production
1 The leaderboard Access application was edited IN PLACE to the path destination <host>/v1/scores; same AUD, no Bypass, policy unchanged. owner, 2026-10-07 owner, 2026-10-07
2 The hostname's rule was added on the public tunnel through the API (cloudflare-tunnel-route.sh add); both public connectors logged the new version with the hostname before the CNAME switch. owner + platform (version 1), 2026-10-07 owner + platform (version 1), 2026-10-07
3 The rule was removed from the protected tunnel through the API; the protected connectors' latest configuration no longer lists the hostname. platform (protected tunnel version 5) platform (protected tunnel version 4)
4 A logged-in visit before the old rule's removal reached the app through cf-public only. 85 requests, all 200 151 requests, all 200
5 Read budget BackendTrafficPolicy leaderboard-public-budget Accepted. platform, 2026-10-07 platform, 2026-10-07
6 Outside-in matrix after the Access edit (16:19Z): GET /, GET /v1/benchmarks, GET /v1/leaderboard/<id> 200; GET /v1/scores, GET /v1/scores/1, POST /v1/scores 302 (Access); POST /v1/benchmarks 401 Jwt is missing; crafted RS256 token 401 Jwks doesn't have key… (JWKS reachable). platform platform
7 Identity strip from outside: a forged X-User-Email on the private board returns scoped_to_caller: true with no entries. platform platform
8 The tenant's public data surface (submitter local parts, full author emails on a local-part collision, /docs /redoc /openapi.json, the private board's metadata) and, for production, the §16(g) conditions (one limiter replica, fail-open; no paging provider). owner proceeded with the Access edit after being told it records acceptance, 2026-10-07 same
9 Burst test (429 above 600 a minute from one address). not run on staging (proven on dev, 2026-10-07: 600×200 then 429) not run (owner agreement needed for a burst on production)
10 Write budget BackendTrafficPolicy leaderboard-writes-public-budget Accepted (30 writes a minute per client address, 300 a minute for the route; ADR-0005 §16(j)). PENDING — platform, after the merge PENDING — platform, after the merge
11 The leaderboard Access application's policy edited IN PLACE to Allow, Include Everyone (ADR-0005 §16(j)); same AUD, no Bypass. PENDING — owner PENDING — owner, after the staging proof
12 One real submission with client 0.1.6 or later by a person on no allow list: the row shows that person's email; an anonymous POST /v1/scores answers 302; the environment's engine hostname still refuses that person (ADR-0005 §16(j)). PENDING PENDING

Known gaps

  • Every person who logs in takes a seat. Each Access login takes one seat from the openmined Zero Trust team, and every Access application (tenant apps, previews, PULSE, Firecall) draws from that one pool. On the free plan the pool is 50 seats; when it is full, Cloudflare refuses the next login of anyone, staff included. Owner decision 2026-10-29 (ADR-0005 §16(j)): open admission is capped at 50 people a month, seats are not reserved in advance, and seat expiry is one month. The seat watch warns in Slack from 80% of the plan's seats (docs/runbooks/cloudflare-seats.md).
  • Submitting needs client 0.1.6 or later. Earlier ScreamingFace clients end the Access login on /, which is public, so Cloudflare never transfers the token and a submission waits out the 300-second login (OME-1649; fixed by tenant #1486).
  • Score detail. Access matches the host and the path, not the method. So GET /v1/scores/{id} still answers 302 to the login. Anonymous readers use GET /v1/public/scores/{id} instead (tenant #1459): it sits outside /v1/scores, so Access does not cover it. It is on dev since pin main-5ba6785a; staging and production get it with their next promotion.
  • Logged-in readers lose their private-board rows on the read path. The read route has no SecurityPolicy, so no verified identity reaches the app on a read. Optional identity on the read route (jwt.optional) is a later, separate ADR item. The owner tells the tenant before the staging and production Access edits.
  • The read scope is the whole hostname. Path prefix / makes /healthz, /readyz (a database query), /docs, /redoc and /openapi.json public on dev, under the budget. The tenant confirms that data surface before the staging and production Access edits (ADR-0005 §16(g)).
  • New write endpoints. A new write endpoint must sit under /v1/scores before it ships. Cloudflare turns the SDK's Cf-Access-Token into Cf-Access-Jwt-Assertion only on a path that an Access application covers. On any other path, a write answers 401 at Envoy.
  • The limiter fails open. One envoy-ratelimit replica and one Redis, and no rateLimit.failClosed. If the limiter is down, reads pass with no budget (accepted in ADR-0005 §16(d)).
  • 429 back-off. Envoy Gateway answers 429 with x-ratelimit-* headers and no Retry-After (v1.5.0). An SDK reader must back off on that shape; the tenant verifies it (ADR-0005 §15 ladder item 11).
  • Dev pin defect (tenant). On 2026-10-05 the dev pin main-8c4db51e answered 500 on every GET /v1/leaderboard/{id} (a Tortoise ParamsError in the tenant's code). The same day the dev pin moved to main-c08ce63e (#1492), and seven board reads from outside answered 200 when this page was written. Do not promote 8c4db51e; the tenant confirms the fix.

Moving back, and the kill switch

  • Moving back to cf-protected. After the single-home change, cf-protected has no route for the hostname, so a CNAME back to the protected tunnel answers 404. Revert the single-home change first. Wait until both routes list both parents and each parent is Accepted. Then add the rule on the protected tunnel through the API, wait for both cloudflared pods, and switch the CNAME back (docs/runbooks/move-hostname-between-tunnels.md, step 7).
  • Kill switch for anonymous reads. In the dev leaderboard Access application, change the destination leaderboard.dev.screamingface.ai/v1/scores to the whole hostname. Every anonymous request then answers 302 to the login. Writes keep working, because the AUD does not change. No git change and no tunnel change are needed.
  • Closing open writes (any environment). In the hostname's leaderboard Access application, change the policy from Include Everyone back to an email list. The AUD does not change, so no git change is needed. People already logged in keep their session until it expires.

Open items

  • Staging and production follow ADR-0005 §16(f) in order: the public connector (#1495 staging, #1496 production), the split and the dual-home, the owner's tunnel move, single-home with the budget, the strip, spoof and burst proofs, and the Access edit last. The tenant confirms the public data surface first; on production the owner also signs the §16(g) conditions.
  • The cf-protected peer leaves allow-scoreboard-ingress-proxy after the last move. Any later fallback to cf-protected must restore it first.