ADR-015: Vault Config Operator + Setup-Script Hybrid¶
Status¶
Accepted
This is the ADR that
docs/secret-management/index.mdreferenced as "ADR-012 (to be filed)" before ADR-012 was taken by the Kyverno policy-enforcement decision. The hybrid itself has been operating since the vault-transit chart work (PR #340); this document records the decision.
Date¶
2026-06-07 (practice established with PR #340)
Context¶
Vault-side configuration (mounts, roles, policies, engines) is GitOps-managed through the
Vault Config Operator (VCO) wherever possible: SecretEngineMount, DatabaseSecretEngineConfig/Role,
KubernetesAuthEngineRole, Policy CRs live under kubernetes/secrets/vault-*-engine/ and are synced by
Argo CD like everything else.
However, VCO does not cover everything we need:
| Vault operation | VCO gap | Current workaround |
|---|---|---|
| Transit key create / rotate | No TransitSecretEngine-family CR exists |
kubernetes/secrets/vault-transit-engine/temp/setup-transit-keys.sh, run once per new key; rotate manually via vault write -f transit/keys/<name>/rotate |
OIDC config (auth/oidc/config) |
VCO patch unsupported; full-write of provider URL etc. required | temp/post-bootstrap-oidc-ux.sh (Stage 4) full-write |
Options considered¶
Pattern A: Scripts everywhere (drop VCO)¶
Manage all Vault config via bootstrap shell scripts / Terraform.
- Pros: one mechanism, no operator to run
- Cons: loses GitOps reconciliation (drift detection, self-heal) for the 90% VCO covers well; scripts are imperative and unaudited by Argo CD
Pattern B: Replace VCO with a fuller tool (Terraform provider, bank-vaults, ...)¶
- Pros: potentially closes the gaps
- Cons: migration cost for all working VCO CRs; each alternative has its own gap profile and operational surface; nothing offered Transit CRs and the lightweight in-cluster reconcile model we already have
Pattern C: Hybrid — VCO for what it supports, one-shot scripts for the gaps (Adopted)¶
- Pros: keeps GitOps reconciliation where it works; gap surface is small (2 operations), explicit, and documented in one table; scripts are one-shot bootstrap, not steady-state operations
- Cons: two mechanisms; the script-managed slice is invisible to Argo CD drift detection
Decision¶
Adopt Pattern C. VCO CRs are the default for all Vault configuration; setup scripts are the documented exception, constrained as follows:
- The exception table in
docs/secret-management/index.mdis the exhaustive list of script-managed Vault state. Adding a script requires adding a row there. - Scripts must be one-shot / idempotent bootstrap operations, committed to the repo next to the capability they bootstrap. No steady-state reconciliation via scripts.
- Removal trigger: when VCO (or a successor) ships CR support for a gap (e.g. a
TransitSecretEngine/ transit-key CR, or OIDC config patch semantics), migrate that row from script to CR and delete the script in the same PR. Re-evaluate this ADR if the exception table grows beyond ~4 rows — at that point Pattern B's migration cost may be worth paying.
Consequences¶
Positive¶
- GitOps reconciliation retained for all engine mounts / roles / policies (the high-churn surface)
- The non-GitOps slice is enumerable in one table instead of being implicit tribal knowledge
- A clear, pre-agreed trigger line for retiring each script
Trade-offs¶
- Transit key rotation stays manual until VCO grows a CR for it
- Anyone auditing "what configures Vault" must read both the CRs and the exception table
References¶
docs/secret-management/index.md§Exceptions — the exception table (SoT for script-managed state)kubernetes/secrets/vault-transit-engine/README.md— the capability that motivated the hybrid (PR #340/#343)- ADR-012 — unrelated; took the number this decision was originally penciled in for