Skip to content

aigateway on PULSE

aigateway is ScreamingFace's private OpenAI-compatible model gateway. It maps a verified user to an account and profile, selects an encrypted provider credential, validates a provider-prefixed model request, and dispatches it. ScreamingFace owns the application and chart; PULSE owns its private runtime boundary and dev database.

Dependencies and ports

flowchart LR
    U[url4-cloud server] -->|catalog<br/>TCP 9105| A[aigateway server]
    R[Runner Job] -->|chat completion<br/>TCP 9105| A
    G[Gatus] -->|healthz<br/>TCP 9105| A
    M[Migration Job] -->|schema<br/>TCP 5432| D[(PostgreSQL)]
    A -->|state<br/>TCP 5432| D
    A -->|HTTPS 443<br/>server only| P[Model providers]

There is no public Ingress or Gateway route to aigateway. Network policy allows only the specific in-cluster callers and status probe. The migration Job can reach PostgreSQL but cannot reach model providers.

Surface Purpose Important boundary
GET /healthz Process health Does not test database credentials, profiles, or providers
GET /v1/models Aggregated canonical model catalog Requires accepted user identity in the deployed auth mode
POST /v1/chat/completions Streaming or non-streaming model dispatch Model must be provider-prefixed and profile must resolve
X-Profile request header Select a stored profile label Defaults to default; it is a selector, not a credential

Identity and profile selection

The deployed cloudflare_headers mode reads only X-User-Email. Envoy has already verified the Cloudflare assertion and stripped any client-supplied copy. aigateway lowercases the email into its account key and rejects a missing identity.

The trust remains valid only while the service is private. aigateway checks its direct network peer against the configured Pod network, and platform NetworkPolicy limits the actual callers. It never trusts X-Forwarded-For. Cloudflare service identity without a user email is currently out of scope for this auth mode.

For chat requests, X-Profile selects the caller's stored profile and defaults to default. The request body's model must include its provider prefix, such as provider/model. A bare model name is rejected before provider dispatch. The selected profile must contain an approved connection for that provider.

Credentials and state

Provider credentials are encrypted in PostgreSQL per account/profile. The platform supplies the database URL and application runtime keys through existing Secrets; the chart's secret generation is disabled.

The aigateway encryption master key protects stored credential blobs. It must be stable across restarts and replicas and cannot be replaced in place without an application-supported re-encryption operation. Losing or blindly rotating it makes existing provider credentials unreadable.

The current dev PostgreSQL is platform-owned, single-instance, and disposable. It uses persistent storage to survive ordinary pod replacement, but it is not a production backup, recovery, or high-availability design.

Migration and Kubernetes identity

The chart runs a schema migration before the Deployment during install and upgrade. It uses the same platform-owned ServiceAccount as the server. API token automount is explicitly disabled because neither process calls the Kubernetes API.

The application server is labeled separately from the migration Job so provider egress can select only the long-running server. Any chart bump must re-check those rendered labels and the hook's ServiceAccount.

Readiness versus functional proof

Signal Proves Does not prove
/healthz is 200 Process, Service, and allowed probe reachability Database access, profile existence, provider credential, completion
Migration hook succeeded Schema operation completed against the configured database Server request path or provider access
/v1/models succeeds Identity and model registry path A usable credential or model completion
Chat completion succeeds Identity, profile, credential, model dispatch, egress, and provider response Only valid for the tested profile/model pair

Provider completion is not yet an accepted dev capability. The deployed url4-cloud default used a bare model identifier, while a canonical request reached aigateway but had no matching approved profile. Health and transport remain green despite that application-level gap.

Common failure signatures

Symptom First contract to inspect
401 with an otherwise healthy service Verified X-User-Email did not reach the private service
400 “model must be provider-prefixed” Replace the bare model with a canonical ID from /v1/models
profile_not_found Requested/default profile is not provisioned for the derived account
Provider authentication failure Stored profile connection, credential lifecycle, and provider selection
Database connection failure Platform database Secret, Service, password generation, and NetworkPolicy
Migration fails before rollout Schema compatibility, database readiness, migration Secret, and hook ServiceAccount
Server can reach DB but not provider Server component label or provider-egress NetworkPolicy
Credentials become unreadable after restart Encryption master key changed or was not consistently supplied

Change and rollback boundary

An aigateway bump must move the chart commit and image tag together. Re-review the port, health path, identity mode, allowed-network contract, model/provider registry, profile schema, Secret keys, migration behavior, ServiceAccount, pod labels, database compatibility, and provider destinations.

Rollback must account for database migrations. Repinning an older image is safe only when its schema remains compatible; otherwise restore or migrate through an explicitly tested application path. Dev PostgreSQL is not a production rollback mechanism.

Source of truth