scoreboard — boundary record and attestations
Leaderboard service (their apps/scoreboard): a portal and an API for
benchmark boards and score submissions. On dev, the hostname is an
ADR-0005 §16 whole-public hostname on cf-public. Anyone can read the
boards, under a per-address budget. A write needs a Cloudflare Access login
on /v1/scores and passes Envoy's Access-JWT check, under a per-address write
budget. Staging and production have had the same shape since 2026-10-07.
Owner decision 2026-10-22 (ADR-0005 §16(j), OME-1286): on every environment
the write-path Access application's policy becomes Include Everyone, so any
person who can log in may submit. The attestation tables record each edit.
| Environment | Hostname | Reads | Writes | Gateway and tunnel | Status |
|---|---|---|---|---|---|
| dev | leaderboard.dev.screamingface.ai |
anonymous; 600 a minute per client address, 6000 a minute for the route | Access on /v1/scores (the dev leaderboard application's own AUD, §11 as amended 2026-10-30; policy Include Everyone) and Envoy's JWT check on every write method; 30 writes a minute per client address, 300 a minute for the route |
cf-public, through the tenant's public tunnel sf-public-dev |
live, single-homed on cf-public |
| staging | leaderboard.staging.screamingface.ai |
anonymous since 2026-10-07; 600 a minute per client address, 6000 a minute for the route | Access on /v1/scores (own AUD) and Envoy's JWT check on every write method; 30 writes a minute per client address, 300 a minute for the route |
cf-public, through tunnel sf-public-staging |
live (#1501, #1556; owner tunnel move and Access edit 2026-10-07) |
| production | leaderboard.screamingface.ai |
anonymous since 2026-10-07; 600 a minute per client address, 6000 a minute for the route | Access on /v1/scores (own AUD) and Envoy's JWT check on every write method; 30 writes a minute per client address, 300 a minute for the route |
cf-public, through tunnel sf-public-prod |
live (#1502, #1572; owner tunnel move and Access edit 2026-10-07) |
Environments (2026-10-05): dev at pin c08ce63e (#1492); staging since
2026-09-09 and production since 2026-09-30, both at pin 68cad9d9, each with
one user declaration that gates every method of the hostname.
Platform half: kubernetes/apps/sf-scoreboard/ (quota, NetworkPolicies, the
database ExternalSecret and the ServiceAccount; a disposable Postgres in
overlays/dev, managed Postgres on staging and production). The namespace
sf-scoreboard is in the tenant registry
kubernetes/tenants/screamingface/namespaces.yaml with the cf-public
admission label openmined.org/public-routes: "true". Edge on dev: two route
declarations, rendered by the tenant-edge chart (ADR-0005 §14, §16):
kubernetes/tenants/screamingface/routes/dev/leaderboard.yaml— tieranonymous-read,GET HEAD OPTIONS, path prefix/, 30s timeout, no SecurityPolicy, and the read budget (BackendTrafficPolicyleaderboard-public-budget).kubernetes/tenants/screamingface/routes/dev/leaderboard-writes.yaml— tieruser, the six write methods, the dev leaderboard application's own AUD, SecurityPolicyleaderboard-writes-cf-access, and the write budget (BackendTrafficPolicyleaderboard-writes-public-budget, ADR-0005 §16(j)).
The hostname is on the chart's tenant-edge.publicHostnames list and off
tenant-edge.publicHostnamesDualHome, so both routes attach to cf-public
only. The cf-public fleet may reach the backend on TCP 9106 and Cloudflare's
published IPv4 ranges on TCP 443, for the Access signing keys (JWKS)
(kubernetes/infrastructure/pulse-edge/base/cf-public-fleet-netpol.yaml).
Traffic arrives through the tenant's public tunnel sf-public-dev (connector
cloudflared-public in sf-edge). Workload: tenants-charts ticket
kubernetes/tenants/screamingface/charts/scoreboard.yaml + values
kubernetes/apps/sf-scoreboard/values/dev.yaml (config.authMode:
cloudflare_headers).
The identity boundary
The scoreboard does no token work. In cloudflare_headers mode it takes the
submitter of a score from X-User-Email. It trusts that header from any
peer inside config.allowedNetworks, which is 10.244.0.0/16, the whole pod
CIDR (tenant source apps/scoreboard/src/scoreboard/core/auth/cloudflare_identity.py).
So three platform controls are the whole boundary:
- The identity strip on
cf-public. The ClientTrafficPolicycf-public-client-ip(namespacepulse-edge) removesX-User-Email,X-User-IdandX-Service-Idfrom every request before any filter runs.scripts/ci/k8s-validate.shasserts this statically, and no overlay may patch the policy. - The write route's SecurityPolicy.
leaderboard-writes-cf-accessverifies the Access JWT (issuer, signing keys, the AUD). Only then does it setX-User-Emailfrom the verifiedemailclaim. A write without a valid token answers 401 before the app. - The ingress selector. The NetworkPolicy
allow-scoreboard-ingress-proxyadmits only thecf-publicandcf-protectedproxy fleets to port 9106. Each peer names bothgateway.envoyproxy.io/owning-gateway-nameandgateway.envoyproxy.io/owning-gateway-namespace: pulse-edge. The platform Gatus pod is the only other peer (allow-scoreboard-ingress-gatus; it reads/healthzonly, an accepted widening recorded innetpol.yaml).
The read route has no JWT filter. The strip alone keeps a forged
X-User-Email off a read, and a read carries no verified identity.
Attestation record (leaderboard.dev.screamingface.ai)
The read path has no AUD, so its recorded boundary is this dated attestation
set: facts that live in the tenant's Cloudflare account and cannot be
verified from git. Behavioral verification: the four Gatus rows in the group
Tenant · sf-scoreboard (ADR-0005 §16(h)):
- Leaderboard public read —
GET /v1/benchmarksanswers 200. A 302 means an Access application crept over the read path. - Leaderboard write refusal —
POST /v1/scoresanswers 302 to the Access login. Any other answer means the path-scoped application is gone. - Leaderboard write JWT —
POST /v1/benchmarksanswers 401Jwt is missingfrom Envoy (a path that Access does not cover). - Leaderboard JWKS reachability —
POST /v1/benchmarkswith a crafted, unsigned token (key idprobe, the full dev leaderboard AUD) answers 401Jwks doesn't have key to match kid or alg from Jwt. Envoy answers from its cached key set, and an unknown key ID does not make it refetch. So the row lags a break in the fleet's TCP 443 egress: it turns red withJwks remote fetch is failedonly after the cached set expires and the refetch fails. The token is not a credential.
Re-attestation triggers: Cloudflare plan change · any tunnel change for the hostname (rule, CNAME, connector, token) · any Access application change on the zone (destination, policy, AUD) · the "Remove visitor IP headers" transform touched · a change to the read or write budget or to the chart's whole-public lists · any abuse incident on the hostname. After any Access edit, the real-client submission proof in ADR-0005 §16(j) runs again.
| # | Fact | Status | Attested by | Date |
|---|---|---|---|---|
| 1 | On this hostname, the dev leaderboard Access application (created 2026-10-30) covers only leaderboard.dev.screamingface.ai/v1/scores: a path destination. Its AUD is the audience in routes/dev/leaderboard-writes.yaml. Its one policy is Allow, Include Everyone, login method one-time PIN. The shared dev application (fusion.dev, reports-internal.dev) no longer lists the path. No Bypass policy exists, and no other Access application covers the whole hostname. |
Outside-in, platform: POST /v1/scores answers 302 with the login kid b745c508… (this AUD), and fusion.dev keeps the shared f9d2d35a…. The policy and login method: PENDING — owner |
platform | 2026-10-30 |
| 2 | The hostname's rule on sf-public-dev (origin: the cf-public fleet's in-cluster Service) was added through the API (scripts/ops/cloudflare-tunnel-route.sh add), and both cloudflared-public pods logged the new configuration. Then the proxied CNAME was edited in place to the sf-public-dev cfargotunnel.com name, through the API (cname). |
PENDING — owner's tunnel move | — | — |
| 3 | The hostname's rule on the protected tunnel was removed through the API (remove), never with a dashboard route delete. The hostname appears in exactly one connector's latest configuration. |
PENDING — owner | — | — |
| 4 | The zone's "Remove visitor IP headers" managed transform is OFF. Required: cf-public reads the client address from CF-Connecting-IP with failClosed: true. With the transform on, every request to the hostname answers 403 at Envoy, and the read budget has no address to key on. |
PENDING — owner | — | — |
| 5 | Zone plan Free, and Bot Fight Mode status (zone-wide, unscopable on Free; it can challenge anonymous programmatic readers before our edge). | PENDING — re-attest with the move (attested for the zone on 2026-08-31, report-intake rows 4 and 5) | — | — |
| 6 | The read budget is live: BackendTrafficPolicy sf-scoreboard/leaderboard-public-budget is Accepted on aks-dev. It allows 600 requests a minute per client address (IPv4, and IPv6 per /128) and 6000 a minute for the route. |
PENDING — platform, after the merge | — | — |
| 7 | Burst and spoof proofs (§16(f) step 7). From one address, requests above 600 in one minute answer 429, and the address answers 200 again after the window. A CF-Connecting-IP header that the caller sends through Cloudflare does not choose the caller's bucket. Recorded before the staging move starts. |
PENDING — platform, after the merge | — | — |
| 8 | Identity strip proof through the cf-public Envoy listener on aks-dev, by a port-forward probe before the tunnel move: a forged X-User-Email did not reach the app; a request without CF-Connecting-IP answered 403; POST /v1/benchmarks answered 401 Jwt is missing with no token, and 401 Jwks doesn't have key to match kid or alg from Jwt with the crafted token (a JWKS fetch on TCP 443 succeeded). |
ATTESTED (port-forward probe on dev) | platform | 2026-10-05 |
| 9 | The write budget is live: BackendTrafficPolicy sf-scoreboard/leaderboard-writes-public-budget is Accepted on aks-dev. It allows 30 writes a minute per client address and 300 a minute for the route (ADR-0005 §16(j)). |
PENDING — platform, after the merge | — | — |
| 10 | Open submission (ADR-0005 §16(j)): the write path has its own dev Access application with one policy, Allow Include Everyone. One real submission with client 0.1.6 or later by a person on no allow list: the row shows that person's email, an anonymous POST /v1/scores answers 302, and fusion.dev still refuses that person. |
PENDING — owner (application) and platform (proof) | — | — |
Attestation record (staging and production, 2026-10-07)
| # | Fact | staging | production |
|---|---|---|---|
| 1 | The leaderboard Access application was edited IN PLACE to the path destination <host>/v1/scores; same AUD, no Bypass, policy unchanged. |
owner, 2026-10-07 | owner, 2026-10-07 |
| 2 | The hostname's rule was added on the public tunnel through the API (cloudflare-tunnel-route.sh add); both public connectors logged the new version with the hostname before the CNAME switch. |
owner + platform (version 1), 2026-10-07 | owner + platform (version 1), 2026-10-07 |
| 3 | The rule was removed from the protected tunnel through the API; the protected connectors' latest configuration no longer lists the hostname. | platform (protected tunnel version 5) | platform (protected tunnel version 4) |
| 4 | A logged-in visit before the old rule's removal reached the app through cf-public only. |
85 requests, all 200 | 151 requests, all 200 |
| 5 | Read budget BackendTrafficPolicy leaderboard-public-budget Accepted. |
platform, 2026-10-07 | platform, 2026-10-07 |
| 6 | Outside-in matrix after the Access edit (16:19Z): GET /, GET /v1/benchmarks, GET /v1/leaderboard/<id> 200; GET /v1/scores, GET /v1/scores/1, POST /v1/scores 302 (Access); POST /v1/benchmarks 401 Jwt is missing; crafted RS256 token 401 Jwks doesn't have key… (JWKS reachable). |
platform | platform |
| 7 | Identity strip from outside: a forged X-User-Email on the private board returns scoped_to_caller: true with no entries. |
platform | platform |
| 8 | The tenant's public data surface (submitter local parts, full author emails on a local-part collision, /docs /redoc /openapi.json, the private board's metadata) and, for production, the §16(g) conditions (one limiter replica, fail-open; no paging provider). |
owner proceeded with the Access edit after being told it records acceptance, 2026-10-07 | same |
| 9 | Burst test (429 above 600 a minute from one address). | not run on staging (proven on dev, 2026-10-07: 600×200 then 429) | not run (owner agreement needed for a burst on production) |
| 10 | Write budget BackendTrafficPolicy leaderboard-writes-public-budget Accepted (30 writes a minute per client address, 300 a minute for the route; ADR-0005 §16(j)). |
PENDING — platform, after the merge | PENDING — platform, after the merge |
| 11 | The leaderboard Access application's policy edited IN PLACE to Allow, Include Everyone (ADR-0005 §16(j)); same AUD, no Bypass. | PENDING — owner | PENDING — owner, after the staging proof |
| 12 | One real submission with client 0.1.6 or later by a person on no allow list: the row shows that person's email; an anonymous POST /v1/scores answers 302; the environment's engine hostname still refuses that person (ADR-0005 §16(j)). |
PENDING | PENDING |
Known gaps
- Every person who logs in takes a seat. Each Access login takes one seat
from the
openminedZero Trust team, and every Access application (tenant apps, previews, PULSE, Firecall) draws from that one pool. On the free plan the pool is 50 seats; when it is full, Cloudflare refuses the next login of anyone, staff included. Owner decision 2026-10-29 (ADR-0005 §16(j)): open admission is capped at 50 people a month, seats are not reserved in advance, and seat expiry is one month. The seat watch warns in Slack from 80% of the plan's seats (docs/runbooks/cloudflare-seats.md). - Submitting needs client 0.1.6 or later. Earlier ScreamingFace clients
end the Access login on
/, which is public, so Cloudflare never transfers the token and a submission waits out the 300-second login (OME-1649; fixed by tenant #1486). - Score detail. Access matches the host and the path, not the method. So
GET /v1/scores/{id}still answers 302 to the login. Anonymous readers useGET /v1/public/scores/{id}instead (tenant #1459): it sits outside/v1/scores, so Access does not cover it. It is on dev since pinmain-5ba6785a; staging and production get it with their next promotion. - Logged-in readers lose their private-board rows on the read path. The
read route has no SecurityPolicy, so no verified identity reaches the app
on a read. Optional identity on the read route (
jwt.optional) is a later, separate ADR item. The owner tells the tenant before the staging and production Access edits. - The read scope is the whole hostname. Path prefix
/makes/healthz,/readyz(a database query),/docs,/redocand/openapi.jsonpublic on dev, under the budget. The tenant confirms that data surface before the staging and production Access edits (ADR-0005 §16(g)). - New write endpoints. A new write endpoint must sit under
/v1/scoresbefore it ships. Cloudflare turns the SDK'sCf-Access-TokenintoCf-Access-Jwt-Assertiononly on a path that an Access application covers. On any other path, a write answers 401 at Envoy. - The limiter fails open. One
envoy-ratelimitreplica and one Redis, and norateLimit.failClosed. If the limiter is down, reads pass with no budget (accepted in ADR-0005 §16(d)). - 429 back-off. Envoy Gateway answers 429 with
x-ratelimit-*headers and noRetry-After(v1.5.0). An SDK reader must back off on that shape; the tenant verifies it (ADR-0005 §15 ladder item 11). - Dev pin defect (tenant). On 2026-10-05 the dev pin
main-8c4db51eanswered 500 on everyGET /v1/leaderboard/{id}(a TortoiseParamsErrorin the tenant's code). The same day the dev pin moved tomain-c08ce63e(#1492), and seven board reads from outside answered 200 when this page was written. Do not promote8c4db51e; the tenant confirms the fix.
Moving back, and the kill switch
- Moving back to
cf-protected. After the single-home change,cf-protectedhas no route for the hostname, so a CNAME back to the protected tunnel answers 404. Revert the single-home change first. Wait until both routes list both parents and each parent isAccepted. Then add the rule on the protected tunnel through the API, wait for bothcloudflaredpods, and switch the CNAME back (docs/runbooks/move-hostname-between-tunnels.md, step 7). - Kill switch for anonymous reads. In the dev leaderboard Access
application, change the destination
leaderboard.dev.screamingface.ai/v1/scoresto the whole hostname. Every anonymous request then answers 302 to the login. Writes keep working, because the AUD does not change. No git change and no tunnel change are needed. - Closing open writes (any environment). In the hostname's leaderboard Access application, change the policy from Include Everyone back to an email list. The AUD does not change, so no git change is needed. People already logged in keep their session until it expires.
Open items
- Staging and production follow ADR-0005 §16(f) in order: the public connector (#1495 staging, #1496 production), the split and the dual-home, the owner's tunnel move, single-home with the budget, the strip, spoof and burst proofs, and the Access edit last. The tenant confirms the public data surface first; on production the owner also signs the §16(g) conditions.
- The
cf-protectedpeer leavesallow-scoreboard-ingress-proxyafter the last move. Any later fallback tocf-protectedmust restore it first.