Skip to content

In-Cluster Vault: Replace vault-secrets-injector

If you already run Vault or OpenBao inside your cluster — or it's reachable on the cluster network — and you want to stop secrets ever landing in Kubernetes Secrets, etcd, or on disk, cloudtaser can replace HashiCorp's vault-secrets-injector and the Banzai vault-secrets-webhook. Pods are still annotated; the mutating webhook still rewrites the workload at admission time. The difference is where the secret ends up: with vault-secrets-injector it lands in a tmpfs file the application reads; with cloudtaser it lands in memfd_secret memory the application's process inherits — never written to disk, never readable by another process on the node, never present in kubectl get secret.

This page covers the in-cluster, single-vault topology. If you want EU-vault sovereignty across multiple clusters with no direct vault network exposure, see the Sovereign Deployment Guide for the beacon-relay (P2P) topology instead.


cloudtaser sits in the same architectural slot as vault-secrets-injector — a Kubernetes mutating admission webhook that intercepts annotated pods and wires them to your existing Vault. The compliance-relevant difference: vault-secrets-injector writes the secret to a shared volume the application reads; any process inside the pod (or the node, with sufficient privilege) can read that file. cloudtaser writes the secret into anonymous, sealed memory (memfd_secret) inherited only by the application process. There is no on-disk file to back up, no kubectl exec cat path, no etcd path. For audits that ask "show me the file the secret was written to" the answer is "there is no file."


When This Topology Fits

This is the right page if all of the following are true:

  • You already run Vault or OpenBao inside your Kubernetes cluster, or you operate it elsewhere but the cluster has a stable network route to it (peered VPC, in-cluster Service, sidecar Vault).
  • You want a drop-in replacement for vault-secrets-injector / vault-secrets-webhook — same annotation-driven model, no code changes in your applications.
  • You do not need the EU-sovereignty story (cluster never holds a vault address, secrets relayed via outbound-only beacon). For that posture see the Sovereign Deployment Guide.
  • You are deploying to a single cluster. Cross-cluster fan-out, audit-port, and multi-region beacon-relay all live in the broker/p2p topology, not this one.

If your cluster cannot reach Vault on the network at all, you want the broker/p2p topology — the wrapper still talks to a local in-cluster broker, and the broker tunnels outbound to your vault via a beacon relay.


How cloudtaser Compares to vault-secrets-injector

HashiCorp vault-secrets-injector cloudtaser (this page)
Trigger Annotation on pod Annotation on pod
Mutation mechanism Mutating admission webhook Mutating admission webhook
What gets injected Init container + sidecar (vault-agent) Init container that copies a static binary to a memory-backed emptyDir
Where the secret lives at runtime tmpfs file at /vault/secrets/<name> (and the vault-agent process's memory) memfd_secret anonymous memory inherited by the application's process
Application sees secret as A file (must be cat'd, sourced, or read by the app) Through its environment API, using the appropriate runtime delivery mode; Python's os.environ needs explicit snapshot configuration
PID 1 inside the pod Your application The cloudtaser wrapper (forks + execs your application)
kubectl exec cat path to secret Yes — the file is readable by anything in the pod No — memfd_secret is sealed; even root in the pod cannot map another process's memfd
etcd / Kubernetes Secret involvement Vault token is typically in a Kubernetes Secret; secret values can be cached in tmpfs Zero K8s Secrets, zero etcd, zero disk — even the wrapper's auth uses the projected ServiceAccount token (no Secret object)
Vault Agent cache file Optional, on disk Not applicable — no agent, no cache
Runtime enforcement Application's responsibility (file permissions on the tmpfs) Optional eBPF daemonset blocks /proc/<pid>/environ reads, ptrace, and core dumps at the kernel level
Annotation surface vault.hashicorp.com/* cloudtaser.io/* (see Annotations Reference)

The wins, summarised:

  • No on-disk artefact. vault-secrets-injector's biggest residual surface is the tmpfs file. Backups, kubectl cp, debug sidecars, and any compromised init script in the pod can read it. cloudtaser removes that surface.
  • No init container chain to debug. vault-secrets-injector pods have an init container plus a long-running vault-agent sidecar. cloudtaser's init container is a single cp; the wrapper IS your application's PID 1.
  • No K8s Secret for the vault token. vault-secrets-injector typically stores the vault auth token in a Kubernetes Secret bound to the ServiceAccount. cloudtaser uses the projected ServiceAccount JWT directly (Vault's Kubernetes auth method) — there is no token Secret object to compromise.
  • Optional kernel-level enforcement. The eBPF daemonset blocks the canonical exfiltration paths (/proc/<pid>/environ read, ptrace(PEEKDATA), core dump write) at the syscall level for all annotated pods.

Deployment

Enroll the workload namespace

The examples use namespace production. Create it if needed, then enroll its workload admission before deploying annotated pods:

kubectl label namespace production cloudtaser.io/inject=true --overwrite

The namespace label and pod injection annotation are both required. Pods in unenrolled namespaces do not call the workload webhooks. See namespace enrollment.

1. Vault-side configuration

Configure your Vault or OpenBao with the Kubernetes auth method bound to the ServiceAccount your workload runs under. This is the vault-secrets-injector setup with one addition: the role must accept the cloudtaser.io/secret-fetch audience of the wrapper's login token.

The example below is for a Vault or OpenBao that runs in the same cluster and reviews tokens with its own ServiceAccount. For a store outside the cluster, run cloudtaser-cli target connect --mode direct instead: it configures the auth mount with the cluster's endpoint and CA and a token reviewer that may only create TokenReviews (see the CLI reference).

# Enable the kubernetes auth method (skip if already enabled)
vault auth enable kubernetes

# Configure it against your cluster's API server
vault write auth/kubernetes/config \
  kubernetes_host="https://kubernetes.default.svc"

# Create a role bound to the workload's ServiceAccount. The wrapper logs in
# with a token issued for the cloudtaser.io/secret-fetch audience, so the
# role must name that audience.
vault write auth/kubernetes/role/payment-service \
  bound_service_account_names=payment-service \
  bound_service_account_namespaces=production \
  audience=cloudtaser.io/secret-fetch \
  policies=payment-service \
  ttl=1h

# A minimal policy — adjust paths to match your KV layout
vault policy write payment-service - <<EOF
path "secret/data/payments/*" {
  capabilities = ["read"]
}
EOF

2. Install cloudtaser with the in-cluster vault profile

Start from the unified CloudTaser chart's Port profile values and override the four settings that flip the operator from beacon-relay (P2P) mode into in-cluster direct-vault mode. The reference for these overrides lives at the top of charts/cloudtaser/values-port.yaml in the Helm source repository.

helm repo add cloudtaser https://charts.cloudtaser.io
helm repo update

helm install cloudtaser cloudtaser/cloudtaser \
  --namespace cloudtaser-system \
  --create-namespace \
  -f https://codeberg.org/cloudtaser/cloudtaser-helm/raw/branch/main/charts/cloudtaser/values-port.yaml \
  --set operator.secretBackend=vault \
  --set operator.enableReverseConnect=false \
  --set operator.broker.beacon.enabled=false \
  --set operator.vault.address=https://openbao.openbao.svc:8200 \
  --set networkPolicy.vaultCIDR=10.42.0.0/16

The overrides:

Override What it does
operator.secretBackend=vault Tells the operator to inject VAULT_ADDR + VAULT_AUTH_* env vars into wrapped pods, instead of broker env vars.
operator.enableReverseConnect=false Disables the operator's broker (the in-cluster TLS gateway that fronts the beacon relay). Not needed when the wrapper talks to vault directly.
operator.broker.beacon.enabled=false Disables the outbound beacon dialer.
operator.vault.address=https://... The vault address the wrapper should dial. Use the in-cluster Service DNS for an in-cluster vault, or any address reachable on the pod network.
networkPolicy.vaultCIDR=<CIDR> Allows the chart-managed operator policy's egress to the vault on networkPolicy.vaultPort (default 8200). With an empty CIDR and the default settings, the chart omits this store egress rule. Use the address range the vault's pods or endpoint have inside the cluster. This value does not configure the operator-generated policies for protected workloads; Service DNS addresses also require the workload egress policy below.

The same profile with the CLI is cloudtaser-cli target install --mode direct --secretstore-address https://openbao.openbao.svc:8200 --set networkPolicy.vaultCIDR=10.42.0.0/16. It also sets the operator's auth path to auth/kubernetes. For an IP-literal address the CLI derives networkPolicy.vaultCIDR itself.

TLS is mandatory in production

The example above uses https://. The vault Service almost always terminates TLS — both Vault Helm chart and OpenBao Helm chart default to TLS-on. If you are running an http:// listener, that's a configuration issue on the vault side; fix it before adding cloudtaser. The wrapper will dial whatever scheme you set, but a plain http:// vault leaks every secret to anything with east-west pod network reach.

Required workload egress for Service DNS

With the direct profile and a Service DNS store address, apply an additive, selector-based NetworkPolicy in every protected workload namespace. The shipped operator does not resolve a Service DNS address such as https://openbao.openbao.svc:8200 into an allow rule for its protected-pod policy. Setting networkPolicy.vaultCIDR in Helm does not add that workload rule. Without the additional policy, an enforcing CNI can block the wrapper's store connection during startup.

Save the following as cloudtaser-direct-openbao.yaml and apply it before starting the protected workload. This example allows protected Pods in production to reach OpenBao Pods in openbao on TCP 8200:

cloudtaser-direct-openbao.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: cloudtaser-direct-openbao
  namespace: production
spec:
  podSelector:
    matchLabels:
      cloudtaser.io/protected: "true"
  policyTypes:
    - Egress
  egress:
    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: openbao
          podSelector:
            matchLabels:
              app.kubernetes.io/name: openbao
      ports:
        - protocol: TCP
          port: 8200
kubectl apply -f cloudtaser-direct-openbao.yaml

Adjust the workload namespace, store namespace, and destination Pod labels to your deployment; the destination labels should match the store Service's spec.selector. Keep namespaceSelector and podSelector in the same to entry so both must match. The source label cloudtaser.io/protected: "true" is added by the operator to protected Pods.

NetworkPolicy allow rules are additive: retain the operator-managed policies and eBPF enforcement, and add this narrowly scoped store rule alongside them. The store's ingress policy must also admit these clients. In particular, the cloudtaser-openbao chart requires workload Pods to have the label cloudtaser.io/openbao-client: enabled for server ingress on TCP 8200. For cross-namespace clients, their namespace must have that label too. Add it to the workload's Pod template labels and label the workload namespace:

kubectl label namespace production cloudtaser.io/openbao-client=enabled --overwrite
Workload Pod template labels
spec:
  template:
    metadata:
      labels:
        cloudtaser.io/openbao-client: enabled

Without these ingress labels, applying the egress policy alone can still leave the wrapper receiving ECONNREFUSED. Keep the HTTPS Service DNS address: the chart-issued certificate has DNS SANs only. An IP-literal ClusterIP produces only a /32 IPBlock in the operator-generated store rule; post-DNAT CNIs such as kube-router see the destination Pod IP and never match that ClusterIP rule.

3. Annotate your workload

For compatibility with releases that do not inherit the operator's store address and CA, configure both explicitly on the workload. Publish only the store's public CA certificate chain, never a private key, in the same namespace as the workload:

kubectl -n production create configmap secretstore-ca \
  --from-file=ca.crt=/path/to/secret-store-ca.pem

Set these annotations on the pod template, using the same store URL as the operator. Setting operator.secretstore.caCert in Helm configures operator trust; do not assume an older injector propagates it into the workload:

annotations:
  cloudtaser.io/secretstore-address: "https://openbao.openbao.svc:8200"
  cloudtaser.io/vault-ca-configmap: "secretstore-ca"
  cloudtaser.io/vault-ca-configmap-key: "ca.crt"

Use a workload image compatible with the selected wrapper delivery mode. The glibc-based debian:bookworm-slim image supplies the libdl.so.2 dependency missing from the old BusyBox quickstart. Check shell entrypoints and the application's environment API using the runtime compatibility table.

Per-pod annotations follow the same model as vault-secrets-injector — add them to the pod template, not the Deployment metadata. The full reference is at Annotations Reference; the settings used by this example are:

Annotation Value Purpose
secretstore.cloudtaser.io/inject "true" Triggers the mutating webhook for this pod.
cloudtaser.io/secretstore-address https://openbao.openbao.svc:8200 Explicit workload endpoint; required on older injectors that do not default it from Helm. Also use it when a workload needs a different endpoint.
cloudtaser.io/vault-ca-configmap / cloudtaser.io/vault-ca-configmap-key secretstore-ca / ca.crt Mount the public CA into the wrapper and configure VAULT_CACERT; ConfigMap must exist in the workload namespace.
cloudtaser.io/secretstore-role payment-service The Vault Kubernetes-auth role created in step 1.
secretstore.cloudtaser.io/secret-paths secret/data/payments/db KV v2 path(s) to read from. Comma-separate multiple paths.
secretstore.cloudtaser.io/env-map username=DB_USER,password=DB_PASS Optional. Maps Vault KV fields to environment variable names; without it, fields are exposed under their original Vault field names.

4. Before / After example

A typical Deployment that previously used vault-secrets-injector looks like this:

before — vault-secrets-injector
apiVersion: apps/v1
kind: Deployment
metadata:
  name: payment-service
  namespace: production
spec:
  selector:
    matchLabels:
      app: payment-service
  template:
    metadata:
      labels:
        app: payment-service
      annotations:
        vault.hashicorp.com/agent-inject: "true"
        vault.hashicorp.com/role: "payment-service"
        vault.hashicorp.com/agent-inject-secret-db: "secret/data/payments/db"
        vault.hashicorp.com/agent-inject-template-db: |
          {{- with secret "secret/data/payments/db" -}}
          export DB_USER={{ .Data.data.username }}
          export DB_PASS={{ .Data.data.password }}
          {{- end -}}
    spec:
      serviceAccountName: payment-service
      containers:
        - name: api
          image: example/payment-service:v2.1.0
          # The application has to source /vault/secrets/db before exec'ing
          command: ["/bin/sh", "-c", "source /vault/secrets/db && exec /app/payment-service"]
          ports:
            - containerPort: 8080

The cloudtaser equivalent:

after — cloudtaser in-cluster vault
apiVersion: apps/v1
kind: Deployment
metadata:
  name: payment-service
  namespace: production
spec:
  selector:
    matchLabels:
      app: payment-service
  template:
    metadata:
      labels:
        app: payment-service
      annotations:
        secretstore.cloudtaser.io/inject: "true"
        cloudtaser.io/secretstore-address: "https://openbao.openbao.svc:8200"
        cloudtaser.io/secretstore-role: "payment-service"
        cloudtaser.io/vault-ca-configmap: "secretstore-ca"
        cloudtaser.io/vault-ca-configmap-key: "ca.crt"
        secretstore.cloudtaser.io/secret-paths: "secret/data/payments/db"
        secretstore.cloudtaser.io/env-map: "username=DB_USER,password=DB_PASS"
    spec:
      serviceAccountName: payment-service
      containers:
        - name: api
          image: example/payment-service:v2.1.0
          # Original command is preserved verbatim — no source-the-file shim
          command: ["/app/payment-service"]
          ports:
            - containerPort: 8080

The application reads DB_USER / DB_PASS through its environment API, with no source, template file, or /vault/secrets directory. Choose the runtime delivery mode for that API: Python's os.environ requires explicit snapshot configuration and does not receive mapped secrets in the default strict interposer mode.


Verifying

After applying the annotated Deployment:

# 1. The pod should reach Ready. The webhook adds an init container.
kubectl -n production get pod -l app=payment-service
# Expect: STATUS Running, READY 1/1, INIT 1/1.

# 2. The wrapper status annotation should report 'injected'.
kubectl -n production get pod -l app=payment-service \
  -o jsonpath='{.items[0].metadata.annotations.cloudtaser\.io/status}'
# Expect: injected

# 3. There should be no Kubernetes Secret holding the vault token.
kubectl -n production get secret
# Expect: only the default ServiceAccount token (or none, on K8s >= 1.24).

# 4. The wrapper should be PID 1 inside the container.
POD=$(kubectl -n production get pod -l app=payment-service -o name | head -1)
kubectl -n production exec "$POD" -- cat /proc/1/comm
# Expect: cloudtaser-wrapper

# 5. The application's environment should NOT show DB_USER in /proc/<pid>/environ
#    (the wrapper passes secrets via execve, then drops PR_SET_DUMPABLE=0 +
#    relies on the eBPF agent to block /proc/<pid>/environ reads from outside
#    the process).
kubectl -n production exec "$POD" -- sh -c 'cat /proc/$(pgrep -f payment-service | head -1)/environ' 2>&1 | tr '\0' '\n' | grep -E '^DB_(USER|PASS)='
# Expect: empty result (when eBPF enforcement is active) or the env var
#   (without eBPF — still memory-only, just not blocked from /proc reads).

# 6. There should be no /vault/secrets file (we are not vault-secrets-injector).
kubectl -n production exec "$POD" -- ls /vault/ 2>&1 || echo "no /vault directory — expected"

What CI Tests on Every Release

This deployment lane is not a documented possibility — it's a CI-tested commitment. Every PR against cloudtaser-operator, cloudtaser-wrapper, cloudtaser-helm, and the e2e harness itself runs the in-cluster vault matrix in cloudtaser-e2e-test. The matrix exercises:

  • Both OpenBao and HashiCorp Vault OSS as the in-cluster vault backend.
  • Multiple workload shapes: Deployment (single + multi-container), StatefulSet (per-pod auth), Job (short-lived), and Pod-with-init-containers (verifying the wrapper does not fight init chains).
  • Both Kubernetes auth (canonical, recommended) and token auth (fallback) paths.
  • HTTPS vault with a mounted self-signed CA, plus the explicit --allow-insecure-tls opt-out.

Each matrix leg asserts the security-claim proof points listed below — both that the secret IS where we say it is (in memfd_secret memory) and that it is NOT where it shouldn't be (no Kubernetes Secret, no ConfigMap, no /proc/<pid>/environ exposure when eBPF enforcement is on).

Negative assertions the fixture enforces

A regression in any of these fails the build — they are the contract this deployment lane carries:

  1. kubectl get secrets -n <workload-ns> returns no Secret object containing the secret material — only ServiceAccount tokens and registry creds (asserted by name, not just by absence of value).
  2. The secret string never appears in any Kubernetes API object: kubectl get all,configmaps,secrets,serviceaccounts -A -o yaml is grepped for the secret value and returns zero matches.
  3. The secret IS present in the wrapped process's memfd_secret: cat /proc/$(pidof <app>)/maps | grep memfd finds the mapping; reading the corresponding /proc/<pid>/fd/<n> returns the secret value (proof the wrapper's memory-only delivery actually delivered).
  4. The secret is NOT present in /proc/<pid>/environ when eBPF runtime enforcement is enabled — the read is blocked at the kernel level.
  5. After the workload pod is deleted, the secret material is unrecoverable from the cluster — no orphan ConfigMap, no orphan Secret, no annotation holding the value.

If a customer follows the deployment steps above, these are the guarantees that hold — and the build artefacts in cloudtaser-e2e-test prove it on every release.


Caveats

  • Single-vault, single-cluster. This page describes one cluster talking to one vault. For cross-cluster fan-out, multi-region beacon relay, or audit-port in the secret-fetch path, use the Sovereign Deployment Guide instead.
  • No EU-sovereignty boundary. With direct in-cluster vault, the cluster network reaches vault. If the cluster's kube-apiserver is hosted by a US cloud provider, that provider can in principle observe vault traffic at the network layer (TLS-encrypted, but timing/volume metadata is visible). The beacon-relay topology removes the cluster's vault address entirely.
  • No audit-port in the path. Vault audit log is the only audit source. With the broker topology, the operator's broker can attach a per-cluster audit header (cluster ID, pod identity) before forwarding to vault — that information is also available via vault's existing Kubernetes-auth metadata in this topology, but the port gives you a richer per-request hook.
  • eBPF daemonset still recommended. Without the eBPF daemonset, secrets in memfd_secret are still memory-only but /proc/<pid>/environ is readable to anyone with kubectl exec and pid namespace visibility. The daemonset closes that gap. See eBPF Enforcement.