ADR-001: TLS Termination Pattern Selection¶
Status¶
Accepted
Addendum (2026-06-30): The edge-termination decision still stands for Gateway certificate ownership and backend HTTP mode. The intra-cluster trust model has since evolved from "physically isolated and mostly trusted" toward a zero-trust baseline: Cilium enforces default-deny reachability, Istio provides PERMISSIVE mTLS for injected mesh-to-mesh traffic, and Gateway-level auth centralizes ingress control. See
kubernetes/network/README.mdfor the current network model.
Date¶
2025-11-07
Context¶
When exposing platform infrastructure services (Argo CD, Grafana, Prometheus, Keycloak, Hubble UI, etc.) externally, we need to decide where TLS termination occurs.
Three TLS Termination Patterns¶
1. Edge Termination¶
- Characteristics: Gateway manages TLS certificates and forwards HTTP to the backend
- Pros:
- Centralized certificate management at a single point (Gateway)
- No TLS configuration needed on backends
- Certificates can be shared across multiple services
- Good performance (encryption/decryption happens only once)
- Cons:
- Traffic between Gateway and Backend is plaintext (within the cluster)
2. Passthrough Termination¶
- Characteristics: Gateway forwards encrypted traffic as-is; backend terminates TLS
- Pros:
- End-to-end encryption
- Backend has full control over certificates
- Cons:
- Each backend needs its own certificate
- No L7 routing at the Gateway (traffic is encrypted)
- Cannot perform HTTP header manipulation or authentication
3. Re-encryption¶
[Browser] --HTTPS--> [Gateway] --HTTPS--> [Backend]
^ TLS termination ^ TLS termination
v TLS re-encryption
- Characteristics: Gateway decrypts then re-encrypts before forwarding to the backend
- Pros:
- End-to-end encryption
- L7 processing possible at the Gateway (header injection, authentication, etc.)
- Gateway-to-Backend communication is also encrypted
- Cons:
- Most complex approach
- Performance overhead (two rounds of encryption/decryption)
- Certificates needed on both Gateway and Backend
- Istio Sidecar required as a Service Mesh (heavyweight)
Decision¶
Adopt Edge Termination
Use the Edge Termination pattern with Istio Gateway for all platform infrastructure services.
Services in Scope¶
- Argo CD
- Grafana
- Prometheus
- Keycloak (Prod/Dev)
- Hubble UI
- Future platform services
Implementation Approach¶
-
Gateway side:
- Use Let's Encrypt certificates on Istio Gateway (auto-managed by cert-manager)
- Wildcard certificate
*.platform.your-org.com - TLS termination (
tls.mode: Terminate)
-
Backend side:
- Services operate in HTTP mode (insecure mode)
- Example: Argo CD
server.insecure: "true" - Exposed via ClusterIP Service
-
HTTPRoute:
- Targets the backend service's HTTP port (typically port 80)
Rationale¶
Why Edge Termination Was Chosen¶
-
Simplified Certificate Management
- Manage Let's Encrypt certificates in one place (cert-manager + Gateway)
- No need to distribute certificates to each backend service
- Certificate renewal is automated with limited blast radius
-
Performance
- Encryption/decryption happens only once (at the Gateway)
- Resource-efficient for bare-metal Raspberry Pi environment
-
Operational Simplicity
- No TLS configuration needed on backend services
- Helm charts and existing manifests can be used as-is
- Easier troubleshooting
-
Intra-Cluster Trust
- Kubernetes intra-cluster network is treated as trusted
- NetworkPolicy can control inter-namespace communication
- Physically isolated bare-metal environment
Why Re-encryption Was Not Chosen¶
-
Istio Sidecar Injection Required
- Service Mesh (mTLS) must be enabled
- Envoy Sidecar injected into each Pod
- Significant resource overhead (heavyweight for Raspberry Pi environment)
-
Increased Complexity
- Certificates managed in two places (Gateway + Backend)
- Difficult to debug
- Performance overhead (two rounds of encryption/decryption)
-
Low Necessity for This Environment
- Physically isolated bare-metal environment
- NetworkPolicy provides sufficient control
- Platform infrastructure services are accessed only by administrators
Why Passthrough Was Not Chosen¶
-
No L7 Routing
- HTTP header-based routing is not possible
- Future Keycloak JWT authentication integration would be difficult
-
Complex Certificate Management
- Each backend needs its own certificate
- Let's Encrypt certificates must be managed individually
Consequences¶
Implementation Notes¶
- Backend Service Configuration
Each service must explicitly indicate that "TLS termination is handled by the proxy":
- **Argo CD**: `server.insecure: "true"` in ConfigMap
- **Grafana**: HTTP by default (no additional configuration needed)
- **Prometheus**: HTTP by default (no additional configuration needed)
- **Keycloak**: `KC_PROXY=edge` environment variable
- Avoiding Redirect Loops
If a backend service enforces HTTPS redirects, an infinite loop will occur. Always enable insecure/edge mode.
-
Certificate Scope
*.platform.your-org.com: For platform infrastructure*.app.your-org.com: For applications (future)
Let's Encrypt wildcard certificates cover only one level of subdomain, so certificates should be separated according to hierarchical subdomain structure.
Trade-offs¶
Security vs Performance/Operability
- Adopted: Plaintext within the cluster, easy operations, good performance
- Not adopted: End-to-end encryption, complex, heavyweight
Edge Termination is appropriate for this environment for the following reasons:
- Physically isolated bare-metal environment
- Raspberry Pi resource constraints
- Platform infrastructure services accessed only by administrators
- Inter-namespace communication control via NetworkPolicy
Conditions for Future Reconsideration¶
Consider migrating to the Re-encryption pattern in the following cases:
-
Compliance Requirements
- When regulations mandate end-to-end encryption
-
Multi-Tenant Environment
- When hosting applications from multiple organizations on the same cluster
-
Zero-Trust Networking
- When adopting a security policy that requires encryption for intra-cluster communication
-
Hardware Upgrade
- When migrating to higher-performance nodes where Istio Sidecar overhead becomes acceptable