Skip to content

ADR-0004: Platform tenancy — namespaces first, vcluster by trigger

Date: 2026-07-17 Status: accepted · amended 2026-07-18: the per-tenant Gateway edge (§1 table row 5, §2) is superseded by ADR-0005's shared tiered gateway. Tenants receive platform-authored hostname routes on a shared listener and Gateway. All other primitives here stand. · amended 2026-07-31: the NetworkPolicy row (§1 table row 3) is refined by ADR-0006 — the openings on top of the default-deny baseline are platform-authored, not "the tenant's own policies"; ADR-0006 also fixes which objects inside a tenant namespace the platform owns rather than the tenant's chart. · amended 2026-08-12: ADR-0008 supersedes manual pin approval for routine tenant application promotion. Every other primitive here stands.

Context

ScreamingFace is the first tenant: a product team whose workloads run on our clusters but whose identity is their own — their domain (fusion.screamingface.ai), their edge auth (Cloudflare Access), their release cadence. Tenant traffic does not flow through *.dev.openmined.org, and tenant prod/dev map onto OUR prod/dev clusters. The Fusion Edge brief (workload-deploy.md) also demands enforced NetworkPolicies and a Gateway API data path. We need a tenancy model that serves this tenant now and the next one without redesign.

Decision

1. A tenant is: an AppProject + a namespace set + a vault path + an edge

Primitive Concretely Enforces
Argo AppProject per tenant screamingface: allowed source repos (their repo + ours), allowed destination namespaces (sf-* only) a tenant's manifests cannot land outside their namespaces
Namespace set, tenant-prefixed sf-edge, sf-url4, sf-aigw (one prefix, N namespaces per their architecture) naming collisions impossible; policy selectors trivial
Default-deny NetworkPolicies stamped per tenant namespace deny-all ingress+egress baseline; the platform authors every approved opening tenant workloads are isolated by default, including from other tenants
Key Vault path per tenant (sf-* secret names) + ESO tenant Secrets materialize only in tenant namespaces secret isolation without per-tenant vaults (upgrade path exists)
Edge per tenant platform-authored routes and policies attach tenant hostnames to one shared tier Gateway the platform enforces hostname, identity, and traffic policy

2. Two data paths, by audience — this is ADR-0002 §4's pre-planned second path

Amended by ADR-0005 (2026-07-18): the tenant path below is now a shared, platform-owned tier Gateway and listener with hostname routes, not a Gateway per tenant. See ADR-0005 for the tier model, SecurityPolicy ownership, identity contract, and topology decisions.

  • Platform apps (Argo CD, portal, docs): ingress-nginx under *.dev.openmined.org. Unchanged.
  • Tenant apps: Gateway API via a shared, pinned Envoy Gateway controller (kubernetes/infrastructure/envoy-gateway/, enabled per cluster). The platform owns the shared tier Gateway, routes, policies, GatewayClass, and CRD versions. Tenant workloads receive traffic through approved routes.

3. Tenant DNS and TLS are the tenant's

fusion.screamingface.ai lives in THEIR Cloudflare zone. Preferred connectivity: cloudflared Tunnel per tenant edge (outbound-only — no public origin IP, no inbound exposure, no cert coupling; Envoy's listener can stay HTTP behind the tunnel per the brief; the tunnel mechanics are ADR-0005 §7). Fallback: proxied DNS to a dedicated LB IP. Either way, external-dns and our Azure zones are NOT involved in tenant hostnames. Tenant dev hosts (<svc>.dev.screamingface.ai, e.g. fusion.dev.screamingface.ai) are the tenant's naming call, terminating on our dev cluster; prod is <svc>.screamingface.ai.

4. Tenant environments map to platform clusters

Tenant-dev runs on aks-dev, tenant-prod on aks-prod (roadmap 3b — a prerequisite for any tenant prod; until it exists, trials run on aks-dev and are labeled trials). Kargo writes immutable version-pin pull requests and merges them only after required checks pass. ADR-0008 governs routine tenant promotion. Production infrastructure changes remain human-gated.

5. vcluster: defined triggers, not a default

Namespaces + AppProject + NetworkPolicy + Gateway API cover Fusion v1 entirely. Introduce a vcluster for a tenant only when one of these becomes true:

  1. CRD/operator conflict — the tenant needs cluster-scoped APIs (versions or operators) that collide with platform or other-tenant installs.
  2. Self-service k8s API — the tenant needs to create cluster-scoped objects or namespaces themselves, i.e. wants their own control plane to administer.
  3. Control-plane blast radius — tenant workloads are hostile enough that a shared API server is itself the risk (webhooks, aggressive controllers).

Untrusted code execution (url4's /python, subprocess workers — RSV-1) is NOT a vcluster trigger: vclusters share the node kernel. That is solved at the node layer — AKS Pod Sandboxing (Kata microVMs) on a dedicated tainted pool — which the platform provides per-tenant when RSV-1 lands.

6. Cluster capability required: enforced NetworkPolicies

aks-dev today runs Azure CNI overlay with networkPolicy: none — policies are silently unenforced, which invalidates any tenant isolation claim. The aks module gains network policy enforcement (Cilium dataplane preferred), applied to dev first; enforcement becomes part of the module's default shape for every future cluster. No tenant goes live before a positive deny-test passes.

Consequences

  • First-tenant cost is real (netpol enablement, Envoy add-on, AppProject scaffolding); every later tenant is a prefix, a project, a vault path, and a platform-authored attachment to a shared Gateway.
  • Two edge technologies to operate (nginx + Envoy Gateway). Accepted: different audiences, and consolidation onto Gateway API later remains open.
  • Argo remains the single deploy path for tenant manifests (their repo as a source, scoped by AppProject) — tenant CI never touches the cluster.
  • Rate-limit Redis and cloudflared run per tenant edge (in sf-edge), not as shared platform services — isolation over efficiency at this scale.