Skip to content

Auth: one IdP, authenticate at the gateway, apps trust headers

Authentication and authorization for humans and services — built around one idea: complete browser authentication once, at the Istio Gateway, so the apps behind it never implement login.

Design thesis: Keycloak is the single IdP; the Gateway is the single browser chokepoint — for apps that need it. oauth2-proxy sits on the Gateway as an ext_authz check and gives apps with no native auth (Prometheus, Hubble, Longhorn) OIDC for free. Apps that already have a real auth surface (Vault, Argo CD, Grafana, Backstage, Keycloak itself) are explicitly excluded from that Gateway check and authenticate natively against the same Keycloak realm instead — the Gateway passes their traffic straight through. One identity source, two enforcement tiers.

What you'll find here: how the SSO flow actually moves between Gateway, oauth2-proxy, and Keycloak; which apps bypass the Gateway check entirely and why; why Gateway-level auth won over per-service OIDC and Istio-native filters; and how LAN and internet traffic get different session friction on the same infrastructure.

Components

dir role namespace
keycloak/ OIDC IdP — realm, users, clients (backed by Postgres, creds via Vault dynamic) platform-auth-prod
oauth2-proxy/ ext_authz for the Istio Gateway — validates cookie / JWT against Keycloak (2 replicas + PDB) auth-system
vault-oidc-auth/ Vault's Keycloak OIDC mount, declared as VCO CRs — the SSO path for Vault CLI / UI vault

SSO flow (browser → platform UI)

sequenceDiagram
    autonumber
    participant U as Browser
    participant GW as Istio Gateway<br/>(gateway-platform)
    participant O2P as oauth2-proxy<br/>(ext_authz)
    participant KC as Keycloak
    participant APP as Backstage / Prometheus<br/>/ Hubble / Longhorn

    U->>GW: HTTPS /
    GW->>O2P: ext_authz check
    alt no session cookie
        O2P-->>U: 302 → Keycloak /auth
        U->>KC: login (password / PKCE)
        KC-->>U: 302 → /oauth2/callback?code=...
        U->>O2P: GET /oauth2/callback?code=...
        O2P->>KC: exchange code → token
        O2P-->>U: Set-Cookie + 302 → original URL
    end
    U->>GW: original URL (cookie present)
    GW->>O2P: ext_authz check (cookie OK)
    GW->>APP: forward + X-Auth-Request-* headers
    APP-->>U: 200

App-native bypass (Vault, Argo CD, Grafana, Keycloak)

vault.platform..., argocd.platform..., grafana.platform..., and auth.platform... (Keycloak itself) are not in the oauth2-proxy CUSTOM policy's host list at all — the Gateway's ALLOW policy passes their traffic straight through (see kubernetes/network/istio/authorizationpolicy-gateway-platform-allow.yaml, Category 1). Each has its own reason:

  • Vault — has its own Keycloak OIDC client (vault) and issues Vault tokens directly; both the CLI (vault login -method=oidc) and the browser UI use this, never oauth2-proxy.
  • Argo CD — has its own OIDC config and issues its own JWT (argocd login --sso); same shape as Vault.
  • Grafana — authenticates via auth.generic_oauth straight to Keycloak. It has to bypass the Gateway check for a concrete reason: oauth2-proxy would inject an Authorization: Bearer <id_token> header, and Grafana misreads that as its own API key, returning 403 before its native OIDC handler ever runs.
  • Keycloak — is the IdP; gating it behind oauth2-proxy would be circular.

Because these hosts skip the Gateway ext_authz check entirely, notPaths/CLI carve-outs aren't needed — pre-authenticated Bearer tokens and browser logins land on the app the same way:

yu (local) → vault login -method=oidc role=admin
            └─ opens browser → authenticate at Keycloak → callback returns a Vault token

Apps that are behind the Gateway check (Category 2/3: Prometheus, Hubble, Longhorn) instead receive oauth2-proxy's X-Auth-Request-* headers and trust those — that's the flow in the diagram above.

Design rationale

Three principles thread the whole design:

  1. Authenticate at the chokepoint, not per app. All ingress converges on the Istio Gateway (see kubernetes/network/README.md), so that is where browser auth lives. Apps with no native auth (Hubble, Prometheus, Longhorn, Alertmanager) are fully covered without ad-hoc basic auth; the hybrid gateway + per-service model is ADR-002.
  2. Choose the boring, documented mechanism. Istio's native Envoy oauth2 filter is officially "under active development" — not a surface to build a production-framed platform on. oauth2-proxy via envoy_ext_authz_http (GA, the documented Istio pattern, the homelab-ecosystem standard) won after a full comparison of SSO behavior, logout, and audit centralization (ADR-005 → ADR-010).
  3. Same identity, different friction per network. LAN (*.platform.yu-min3.com via L2 LB) and internet (*.yu-mins.com via Cloudflare Tunnel) land on the same Gateway, but the inner session runs 30 days so the LAN is frictionless, while external traffic keeps a second factor at the Cloudflare edge (Access OTP as interim gate — ADR-016).

Concrete choices:

  • Bypass at the Gateway when the app has its own auth surface. Apps with real native auth (Vault, Argo CD, Grafana, Keycloak) are excluded from the Gateway's ext_authz check entirely — see "App-native bypass" above — and authenticate against Keycloak on their own. Apps with no native auth ride the Gateway's oauth2-proxy check instead. Either way it's the same Keycloak realm — one IdP, zero duplicate user stores.
  • The IdP's own dependencies are treated as critical path. Keycloak's DB credentials are Vault dynamic with explicit cascade mitigations (ADR-013); its availability directly gates every login above.
  • Vault's OIDC mount is GitOps too. The vault-oidc-auth/ CRs (auth role, identity group, group alias) are reconciled by Vault Config Operator, so SSO wiring for Vault is reviewable in a PR like everything else (ADR-015 covers the one scripted exception).