Skip to content

Upstream header injection

Pattern 1 is live; Pattern 1b's demo was retired on 2026-07-31

Pattern 1 (static injection) still runs in production shape — the HTTPRouteFilter is attached to the /whoami rule of the real tenant route, fusion.dev.screamingface.ai.

Pattern 1b (per-user, via ext_authz) was torn down: cred-service, redis-exp and the fusion-exp route on fusion.testsfapp.net, along with the second Cloudflare Access trust root they used. The pattern and its headline lesson — ext_authz sees client headers, so the service must verify the token itself — are unchanged and still binding; only the demo deployment is gone. Manifests (cred-service-exp.yaml) remain in git history.

The capability: the gateway can attach headers to the request it forwards to your backend — a credential reference, a tier, any custom header — that the client cannot set, cannot overwrite, and never sees returned. Two tiers: one credential for a whole route (static), or a different one per verified user (via an external authorization service).

Foundation: Identity & traffic flow explains how the caller's identity is verified and delivered as a header.


Why a reference, not the secret

Throughout, the gateway injects an opaque reference — keyref://… — not a real credential. The real secret lives in Key Vault; something downstream resolves the reference to it. This is deliberate:

  • The gateway (and the authorization service) only ever handle references — their blast radius is a pointer, not a key.
  • You can rotate the real key, or reassign a user, from Azure, live — the reference is stable, so the change is invisible to the caller.

Who touches the real secret

The reference indirection splits responsibilities cleanly — a product manager edits policy, an operator handles secrets, and only the resolver reads a key:

Credential lifecycle

The resolver (the backend that makes the upstream call) is the only component that reads the real key from Key Vault. The gateway never sees it. (In the demo the injected reference stops at an echo backend — the resolver is documented here, not built.)


Pattern 1a — static, per-route

One credential reference for every caller on a route. The mechanism is Envoy's HTTPRouteFilter with credentialInjection, sourced from a Kubernetes Secret, plus RequestHeaderModifier for arbitrary custom headers.

The flow

client → Envoy → [HTTPRouteFilter: set X-Upstream-Key from Secret, overwrite]
       → backend

Injection happens on the inward leg only (gateway → backend). overwrite: true replaces any client-sent value. Forgery fails on two stacked guarantees: the overwrite (filter) and the NetworkPolicy that makes the gateway the only path to the backend.

Platform-supported method

kind: HTTPRouteFilter          # gateway.envoyproxy.io/v1alpha1
spec:
  credentialInjection:
    header: X-Upstream-Key
    overwrite: true            # non-negotiable — strips the client's value
    credential:
      valueRef: {kind: Secret, name: <secret-with-key "credential">}
Attach it to the route rule via an ExtensionRef filter; add custom headers with a RequestHeaderModifier.

Proof

Static per-route injection

Injected though the client sent nothing; a forged value is overwritten; the header never appears in a response; every caller on the route gets the same value.


Pattern 1b — per-user, via ext_authz

A different reference per verified user. Envoy calls an external authorization service (SecurityPolicy.extAuth) on every request; the service resolves the caller to a reference and Envoy injects it.

The flow

client → Envoy → [jwt: verify identity]
       → [ext_authz: call cred-service → returns X-Upstream-Key]
       → [inject headersToBackend] → backend

The cred-service looks the caller up in Azure App Configuration (Redis-cached, short TTL) and returns X-Upstream-Key: keyref://user-<id> + X-Upstream-Tier. Envoy injects the whitelisted response headers upstream.

The security lesson (a real edge this demo surfaced)

Envoy's ext_authz check receives the original client request headers — before the JWT filter stamps or strips identity. So a client-set X-User-Email reaches the authorization service, and the JWT-derived identity does not. Keying per-user decisions on that header is spoofable — any authenticated caller could forge another user's identity.

The rule: the ext_authz service must verify the signed token itself (validate the JWT against the IdP's JWKS + audience) and derive identity from the verified claim — never trust a header Envoy hasn't stamped yet. Forward the signed assertion (headersToExtAuth: [cf-access-jwt-assertion]), not the identity header.

Platform-supported method

kind: SecurityPolicy           # gateway.envoyproxy.io/v1alpha1
spec:
  targetRefs: [{kind: HTTPRoute, name: <your-route>}]
  jwt: { providers: [ ... ] }              # verify identity
  extAuth:
    failOpen: false                        # service down → deny, not keyless
    headersToExtAuth: [cf-access-jwt-assertion]   # the SIGNED token (source of truth)
    http:
      backendRefs: [{name: cred-service, port: 8000}]
      path: /auth
      headersToBackend: [x-upstream-key, x-upstream-tier]

path is prepended to the original request path, so the service must answer the check at any path (Envoy calls it at /auth/<original>).

Security properties

  • Verified: identity comes from the signed JWT, validated in the service.
  • Fail-closed: failOpen: false — if the cred-service is unreachable, the request is denied, never forwarded keyless.
  • References only: the service returns keyref://…, never a real key.

Proof

Per-user injection via ext_authz

Per-identity reference; a forged X-User-Email is ignored; a value edited in the Azure App Config portal changes what's injected within seconds, no redeploy; cred-service down → 403.


Pattern 1c — the seamless swap

Because the injected value is a stable reference, an operator can change what it means from Azure, with no redeploy and no client-visible change:

Intent Edit in Azure Propagates via Invisible because
Rotate a compromised key the Key Vault secret ESO re-sync / resolver reference is stable; only the resolved key changes
Quarantine / reassign a user the App Configuration value cred-service poll (short TTL) user keeps sending the same thing; gateway changes what it attaches

App Configuration is the idiomatic surface for the second: a product manager gets Data Owner on the config store with no Key Vault access, edits a value in the portal, and the next request reflects it. Secrets stay in Key Vault; policy lives in App Config. (See the swap frames in the Pattern 1b GIF above.)


What to remember

  • Inject a reference, resolve the secret downstream — the gateway stays blast-radius-free.
  • ext_authz sees client headers — verify the token in the service.
  • overwrite: true / failOpen: false are non-negotiable.
  • Rotate keys in Key Vault, reassign users in App Config — both seamless.

Sources: the gateway-patterns experiment (2026-07); live sf-fusion HTTPRouteFilter / SecurityPolicy.extAuth; proof transcripts.