Volume Shield Installation¶

Managed host-Port path is not supported

Volume Shield fails admission on the current Linux host Port + customer relay path; its PVC did not bind in the live test. Secret delivery through that path does not qualify a Volume Shield mount or encrypted data path. Do not enable it for this customer topology. See Supported Paths.

This guide enables transparent PVC encryption for annotated workloads. Read the architecture overview first if you are new to Volume Shield.

Prerequisites¶

Requirement	Notes
CloudTaser operator installed	See Operator Installation — Volume Shield is injected by the same webhook
Workload namespace enrolled	Label it `cloudtaser.io/inject=true` and retain pod injection annotations; see namespace enrollment
EU-hosted OpenBao reachable	Same instance used for secret injection
Nodes allow a privileged DaemonSet	The vs-broker owns `/dev/fuse` and `mount(2)`; workload pods stay unprivileged
Wrapper image `v0.2.207` or newer	Broker and sidecar binaries ship in the wrapper image
Components with first-use key provisioning	Operator v0.13.172+ (bundled with chart 1.0.295 and later), a Volume Shield sidecar image of v0.2.180+ (the `volumeShield.sidecar.image.tag` you set in Step 2; the chart does not default it), and CloudTaser Port v0.4.109+ on the EU side — see First-use key provisioning

Serverless node pools are not supported

Like the eBPF agent, the vs-broker DaemonSet requires host-level access. GKE Autopilot and AWS Fargate cannot run Volume Shield.

Pod Security restrictions

Volume Shield workloads are not admitted under Kyverno baseline/restricted Pod Security policies. The chart's socket hostPath exception covers only the eBPF socket paths; it does not exempt Volume Shield's additional hostPath mounts or mount propagation. Earlier admission through an exception that matched just one allowed hostPath was a bypass, not supported compatibility. The operator also rejects Volume Shield in namespaces enforcing Kubernetes PSA baseline or restricted.

Step 1 — Vault policy and role¶

Which identity reads the key

The DEK read grant below is used when keys are read directly from OpenBao: by the direct-vault DEK backend, and by the serve-mode broker with its default volumeShield.broker.dekBackend: vault. In the default fd-pass mode with the broker DEK backend (the default whenever workloads fetch secrets through the operator broker), the sidecar reads its key through the operator broker and the Port instead, and that read is authorized per workload — see Authorize the key read.

The sidecar (and broker, for lifecycle state) authenticates via the Kubernetes auth backend using the cloudtaser-vs-broker role. Grant read access to the per-volume DEK path:

# volume DEKs
path "cloudtaser/data/volumes/*" {
  capabilities = ["read"]
}

Also grant read + check-and-set write on the Volume Shield state path — this is required: trusted freshness (vault mode) is the default and cannot be switched off, because current wrapper releases refuse the local mode (see the threat model):

path "cloudtaser/data/volumeshield-state/*" {
  capabilities = ["create", "read", "update"]
}
path "cloudtaser/metadata/volumeshield-state/*" {
  capabilities = ["read", "list"]
}

Missing state-path access fails closed: mounts with nonceBudget.mode or freshness.mode set to vault (the default) will refuse to serve.

Step 2 — Enable in Helm values¶

volumeShield:
  enabled: true
  broker:
    mode: "fdpass"          # default — privilege-separated architecture
    daemonset:
      enabled: true          # per-node broker DaemonSet
    image:
      tag: "v0.2.207"
    vaultAddr: "https://your-openbao.example.eu:8200"
    vaultRole: "cloudtaser-vs-broker"
  sidecar:
    image:
      tag: "v0.2.207"
  # freshness.mode defaults to "vault" (trusted anti-rollback) — shown explicitly for clarity.
  # nonceBudget.mode: "vault" is an additional recommended production setting (not default):
  nonceBudget:
    mode: "vault"
  freshness:
    mode: "vault"

Then upgrade the release:

helm repo update
helm upgrade cloudtaser cloudtaser/cloudtaser \
  --namespace cloudtaser-system \
  --version 1.0.359 \
  -f values.yaml

Verify the broker is running on every node:

kubectl get daemonset -n cloudtaser-system -l app.kubernetes.io/component=vs-broker

Step 3 — Annotate PVCs¶

Volume Shield is opt-in per PVC. Annotate the PVCs you want encrypted before the consuming pod starts:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: postgres-data
  annotations:
    cloudtaser.io/encrypt-pvc: "true"
    # optional:
    # cloudtaser.io/pvc-chunk-size-kb: "64"
    # cloudtaser.io/vs-io-mode: "cached"   # or "strict"
spec:
  accessModes: ["ReadWriteOnce"]
  resources:
    requests:
      storage: 10Gi

The consuming pod must also have wrapper injection enabled: configure the wrapper for the workload and set secretstore.cloudtaser.io/inject: "true", or use a matching SecretMapping that enables injection. A proxy-only annotation does not enable the wrapper. From operator v0.13.190 the webhook refuses consumers of encrypted Filesystem claims when wrapper injection is absent or namespace injection is disabled (older operators admit such a pod unencrypted; see Supported Paths); it does not start an ordinary application against the raw volume. A generic ephemeral volume whose claim template requests encryption is also rejected; use a pre-created Filesystem PVC. Create the claim before the consuming pod: a missing or unreadable mounted claim causes a retryable admission error because its encryption intent cannot be verified.

For an eligible wrapped pod in a watched namespace, the sidecar is injected at admission, the PVC is shadow-remounted to a ciphertext path, and the application sees a transparent plaintext view at its original mount path. Pods with multiple annotated PVCs get one FUSE mount per PVC, each with its own DEK and its own per-mount I/O mode.

Filesystem volume mode only

Volume Shield encrypts volumeMode: Filesystem PVCs. Block-mode PVCs are ignored.

Authorize the key read¶

Provisioning creates a key; it never grants permission to read one. The pod that mounts the PVC must be allowed to read cloudtaser/data/volumes/<namespace>/<pvc-uid>, the same requirement as before first-use provisioning existed. A pod without that permission gets HTTP 403 and fails closed, and a 403 never triggers provisioning.

Broker token auth (cloudtaser.io/secretstore-auth-method: "token"): add the exact key path of each PVC to spec.vaultPaths of the namespace's CloudTaserPolicy. The PVC UID exists once the claim has been created:

NS=<namespace>
PVC_UID=$(kubectl get pvc postgres-data -n "$NS" -o jsonpath='{.metadata.uid}')
printf 'cloudtaser/data/volumes/%s/%s\n' "$NS" "$PVC_UID"

apiVersion: api.cloudtaser.io/v1alpha1
kind: CloudTaserPolicy
metadata:
  name: postgres
  namespace: payments
spec:
  vaultPaths:
    - secret/data/payments/postgres
    - cloudtaser/data/volumes/payments/11111111-2222-3333-4444-555555555555

Replace payments and the example PVC UID with the actual namespace and the UID returned above. Use one exact volume path per encrypted PVC.

Exact key paths only

The Port (v0.4.110 and later) writes each per-pod OpenBao policy itself and accepts only exact volume key paths. A wildcard such as cloudtaser/data/volumes/<namespace>/* would expose every key under that namespace name, in every cluster that shares the Secretstore, so the Port refuses the whole per-pod policy: every secret read for the affected pods is denied, not only the key read. If several CloudTaserPolicy objects apply to the namespace, their paths are intersected, so list the key path in each of them.

StatefulSet volumeClaimTemplates create each claim when its pod is first scheduled, so the UID is only known then. Add the path once the claim exists; until then the sidecar's key read is denied and the sidecar fails closed.

Kubernetes auth (cloudtaser.io/secretstore-auth-method: kubernetes, the default): the Port reads with a token from its cloudtaser Kubernetes-auth login role, so that role's policies must allow the read. Every workload the role admits shares those policies. Use broker token auth for Volume Shield workloads to keep the grant to one volume.

The direct-vault DEK backend and a serve-mode broker with dekBackend: vault read with the cloudtaser-vs-broker role from Step 1.

First-use key provisioning¶

When a pod first mounts an annotated PVC that has no key yet, CloudTaser creates the key on the EU side before the mount serves any I/O. You do not create keys yourself.

The fd-pass sidecar reads cloudtaser/volumes/<namespace>/<pvc-uid> through the operator broker. Only a clean not-found continues to step 2.
The sidecar asks the operator broker for provisioning. It sends only the namespace and PVC UID, authenticated with a pod-bound projected ServiceAccount token whose audience is cloudtaser.io/vs-dek-provision.
The operator verifies the token with an audience-bound TokenReview, then checks live objects: the calling pod has that UID, is not terminating, runs as the token's ServiceAccount, carries the Volume Shield sidecar, and mounts a Filesystem PVC with the requested UID that is annotated cloudtaser.io/encrypt-pvc: "true". Only then does it contact the Port.
The Port generates a 32-byte key with crypto/rand and writes it with KV v2 cas=0: it creates the key only if none exists and never overwrites, rotates, or deletes one. Concurrent first starts converge on a single key.
The sidecar reads the key through the same authorized /v1/proxy/secret path as in step 1. The provisioning response carries only a status; key material never reaches the cluster through provisioning.

Restarts, retries, and further pods read the existing key; provisioning runs only while the key is absent. Keys that already exist, including keys created before first-use provisioning shipped, are used unchanged.

Requirements: operator v0.13.172 or later (bundled with chart 1.0.295 and later); a Volume Shield sidecar image of v0.2.180 or later (the chart does not default volumeShield.sidecar.image.tag, so the sidecar runs whatever tag you set in Step 2, or your installer set for you); and CloudTaser Port v0.4.109 or later (v0.4.110 or later once you follow the Section L upgrade order). The Port must serve the cluster over a per-cluster connection, and the cloudtaser mount must be KV v2.

Existing installations. The Port's own policy narrows to create on cloudtaser/data/volumes/* and read on cloudtaser/metadata/volumes/* the next time cloudtaser-cli source configure or the Port bootstrap runs. Provisioning also works under the older, broader grant, because a cas=0 write never replaces an existing key.

Where provisioning does not apply. First-use provisioning is wired only for the fd-pass sidecar with the broker DEK backend. In the following configurations nothing is provisioned, and a missing key fails closed exactly as before:

the direct-vault DEK backend (CLOUDTASER_VS_DEK_BACKEND=vault, used when the workload's secret backend is direct Vault);
the serve-mode broker (volumeShield.broker.mode: serve) and in-wrapper FUSE without a broker;
a Vault namespace configured for key reads: the Port creates keys only in its own namespace, so the sidecar disables provisioning and logs vs-dek-provision-vault-namespace;
sidecar migration runs (--migrate), which never create a key;
a Volume Shield sidecar image older than v0.2.180, which has no provisioning client.

Step 4 — Verify¶

Start a workload and confirm:

# sidecar injected and ready (readiness = ALL mounts serving)
kubectl get pod <pod> -o jsonpath='{.spec.containers[*].name}'
# ... should include cloudtaser-vs-sidecar

# the application path shows plaintext
kubectl exec <pod> -c <app> -- head -c 100 /data/somefile

# the backing store holds ciphertext (CTVS magic)
NODE=$(kubectl get pod <pod> -o jsonpath='{.spec.nodeName}')
BROKER=$(kubectl get pod -n cloudtaser-system -l app.kubernetes.io/component=vs-broker \
  --field-selector spec.nodeName=$NODE -o name | head -1)
kubectl exec -n cloudtaser-system $BROKER -- \
  head -c 4 /var/lib/cloudtaser/raw/<pvc-name>/somefile   # prints: CTVS

First-use provisioning needs a sidecar image of v0.2.180 or later. Check the image the webhook injected:

kubectl get pod <pod> -o jsonpath='{.spec.containers[?(@.name=="cloudtaser-vs-sidecar")].image}'

On the first start of a new PVC, the sidecar log shows the provisioning round trip:

kubectl logs <pod> -c cloudtaser-vs-sidecar | grep -F 'first-use'
# volume shield DEK not found; requesting first-use provisioning
# volume shield first-use DEK provisioning completed ... "status":"created"

status is created for the pod that created the key and exists when another pod created it first. Later starts read the key directly and log neither line. The operator and the Port count outcomes in cloudtaser_operator_vs_dek_provision_total{result} and cloudtaser_port_vs_dek_provision_total{result}, and the Port writes a vs_dek_provision_request audit event with its provision_result.

To confirm the key on the EU side without displaying it, read only its metadata:

bao kv metadata get cloudtaser/volumes/<namespace>/<pvc-uid>   # current_version 1

Do not run bao kv get on key paths: it prints the key.

First-use failure modes¶

Every failure below fails closed: the mount does not serve, and no key is replaced.

What you see	Cause	What to do
Sidecar log `fetching DEK from vault path` with HTTP 403, and no provisioning attempt	The pod may not read the key path	Add the exact key path; see Authorize the key read. Provisioning never grants reads.
Sidecar log `fetching DEK from vault path` with a not-found error, and no `first-use` lines	The sidecar image is older than v0.2.180, or provisioning does not apply to this configuration	Check the injected sidecar image (above) and the list under First-use key provisioning
`DEK exists but is deleted or destroyed` (operator HTTP 409, status `unavailable`), or a key read error for that path	The key's current version was soft-deleted or destroyed. CloudTaser never re-creates it.	Soft-deleted: find the version with `bao kv metadata get`, then restore it with `bao kv undelete -versions=<n> cloudtaser/volumes/<namespace>/<pvc-uid>`. Destroyed: data sealed under it cannot be decrypted; see Recovery options.
A new key is created for a PVC that already holds encrypted data; reads fail with `EIO` or the mount is refused	An administrator deleted the key's metadata, for example during crypto-shredding. The path counts as absent, so the Port creates a new, unrelated key.	Expected after revocation. The old ciphertext stays unreadable: it fails AES-GCM authentication under the new key, and retained trusted state no longer matches its `dek_fingerprint`. Restore data through the mount from a backup, or treat the volume as new.
`first-use DEK provisioning denied (HTTP 400/401/403, …)`	The operator could not tie the request to a live pod that mounts an annotated Filesystem PVC with that UID	Check the PVC annotation, `volumeMode`, and UID, and that the pod is not terminating
`denied by the Secretstore` (HTTP 502)	The Port's own policy lacks `create` on `cloudtaser/data/volumes/` or `read` on `cloudtaser/metadata/volumes/`	Re-run `cloudtaser-cli source configure` or the Port bootstrap to reconcile the Port policy
`unsupported by cloudtaser-port` (HTTP 502)	The cluster is served over a legacy single Port connection, or the `cloudtaser` mount is not KV v2	Serve the cluster over a per-cluster Port connection and use a KV v2 `cloudtaser` mount
`operator broker has no provisioning endpoint` (HTTP 404)	Operator older than v0.13.172	Upgrade the chart to 1.0.295 or later
`did not complete in time`	Port older than v0.4.109 (the operator answers 504 after 15 s), Port or Secretstore unreachable, or rate limiting. The sidecar retries transient errors for `CLOUDTASER_VS_DEK_PROVISION_TIMEOUT` (default `2m`).	Upgrade the Port; check Port health and Secretstore reachability

Encrypting pre-existing data¶

PVCs with existing plaintext data adopt encryption on write: reads pass through until a file is first written via the mount, at which point it is encrypted. To bulk-encrypt existing data instead, run the migration mode of the sidecar binary as a Kubernetes Job against the raw path — see bulk migration and TOFU-window graduation.

Trusted freshness no longer waits on this step

Before the 2026-07-06 default flip, bootstrapping via this migration was a prerequisite to trusted freshness meaning anything on a populated volume. Now that vault mode is the default, a populated volume with no prior freshness state gets protection immediately via a trust-on-first-use (TOFU) window — see upgrade cohorts. Running this migration is how you graduate out of that window (bulk-encrypt remaining plaintext and flip the volume to full strict enforcement), not how you turn trusted freshness on in the first place.

Limitations¶

Key paths are not partitioned by cluster. Keys live at cloudtaser/volumes/<namespace>/<pvc-uid> and trusted state at cloudtaser/volumeshield-state/<namespace>/<pvc-uid>, with no cluster component. When several clusters share one Secretstore, an identity granted these paths, such as the cloudtaser-vs-broker role from Step 1, can reach the matching paths of every cluster that shares the store. Provisioning itself never returns key material. Cluster-partitioned paths are not available in this release.
cloudtaser.io/pvc-transit-key is reserved and not active. The annotation has no effect; keys come from first-use provisioning or already exist in KV (see the configuration reference).
No automatic rotation. Provisioning creates only the first key; it never rotates, replaces, or deletes one. Rotation still requires re-encryption (known operational limits).

Next steps¶

Configuration reference — I/O modes, nonce budget, freshness, chunk sizing
Threat model — what Volume Shield does and does not protect against
Troubleshooting and DEK revocation — operating encrypted volumes