GKE Deployment Guide¶

This guide separates node capability from a working protected deployment. The wrapper's maximum score is 175/175, but a score is not proof that every required vector is enforced.

COS passed default strict readiness in the live test

On 29 October 2026, COS 6.12.105+ reported Tier 1 lsm+fmod_ret, active BPF LSM, and HTTP 200 readiness with no accepted or unaccepted gaps under the chart defaults. Kprobe override was unavailable; fmod_ret supplied synchronous paths. The measured GKE Ubuntu image (Ubuntu 24.04.5, 6.8.0-1062-gke) still lacks active bpf and is refused by default ebpf.requireLSM=true. See the recorded cohort and boundaries.

Use the measured platform matrix and qualify the exact image and released components before moving workloads. The setup below is a qualification workflow; one tested COS cohort does not qualify all GKE images or workloads.

Choose the node image using measured evidence¶

Property	Measured GKE Ubuntu `6.8.0-1062-gke`	COS milestones 121 / 125 / 129 / 133
Kernel series	6.8	6.6 / 6.12 / 6.12 / 6.18
`CONFIG_BPF_LSM`	Built	Built
Active/default LSM stack	Measured active list excludes `bpf`	Pinned default configs include `bpf`; verify the actual node
`CONFIG_BPF_KPROBE_OVERRIDE`	Enabled	Disabled
`CONFIG_FUNCTION_ERROR_INJECTION`	Enabled	Enabled
Lockdown	Measured integrity mode	Config default is `LOCK_DOWN_KERNEL_FORCE_NONE=y`; verify runtime state
Default agent outcome	Startup refused by `requireLSM`	COS `6.12.105+` measured Tier 1 and Ready via `lsm+fmod_ret`; qualify other images

The COS configuration evidence is pinned to 6.6, 6.12 and 6.18. COS supports function-error injection even though kprobe override is off. The live test confirmed the released agent uses fmod_ret; verify actual attachments and readiness on each node rather than inferring them from the kernel config.

Confidential Computing is a separate control. Memory encryption does not activate BPF LSM, repair enforcement gaps, or qualify the workload's secret-handling paths.

Step 1: Select a qualification cluster¶

For a COS evaluation cluster with explicit SEV-SNP, the configuration is:

gcloud container clusters create cloudtaser-prod \
  --region europe-west4 \
  --node-locations europe-west4-a,europe-west4-b,europe-west4-c \
  --num-nodes 3 \
  --image-type COS_CONTAINERD \
  --confidential-node-type=sev_snp \
  --machine-type n2d-standard-2 \
  --min-cpu-platform="AMD Milan" \
  --workload-pool "$(gcloud config get-value project).svc.id.goog" \
  --release-channel regular

This selects infrastructure. Do not migrate protected workloads until Step 3 and Step 5 pass on the actual node pool.

--enable-confidential-nodes alone selects SEV, not SEV-SNP. Use --confidential-node-type=sev_snp for SNP, or tdx for TDX. COS SEV-SNP and TDX require GKE 1.32.2-gke.1297000 or later; check the versions available in your chosen release channel. See Confidential GKE Nodes requirements.
SEV-SNP uses N2D with AMD Milan, including europe-west4-a/b/c. C3 uses Intel TDX; it is not an AMD SNP machine family. The current table lists C3D under SEV, not SNP. Confirm the machine/technology/zone combination in supported configurations.
The gcloud cluster-create reference documents the technology, node-location and minimum-CPU flags. Verify the resulting node configuration; the generic confidential-node label does not identify SNP versus SEV.

Step 2: Connect the Cluster to Your OpenBao¶

Use the cloudtaser CLI to configure Kubernetes auth on your EU-hosted OpenBao:

# Connect to the cluster
gcloud container clusters get-credentials cloudtaser-prod --region europe-west4

# Connect the cluster to your vault
cloudtaser-cli target connect \
  --secretstore-address https://vault.eu.example.com \
  --secretstore-token hvs.YOUR_ROOT_TOKEN \
  --auth-path kubernetes/gke-prod

This configures OpenBao's Kubernetes auth method to accept ServiceAccount JWTs from the GKE cluster.

Step 3: Install cloudtaser¶

Install the operator and eBPF daemonset via Helm. This uses the direct secret-store profile; the chart default is the Port and beacon-relay profile, so the address alone is not enough (see Operator Installation):

helm repo add cloudtaser https://charts.cloudtaser.io
helm install cloudtaser cloudtaser/cloudtaser \
  --namespace cloudtaser-system \
  --create-namespace \
  --set operator.secretBackend=vault \
  --set operator.enableReverseConnect=false \
  --set operator.broker.beacon.enabled=false \
  --set operator.secretstore.address=https://vault.eu.example.com \
  --set networkPolicy.vaultCIDR=<secret-store CIDR> \
  --set ebpf.enabled=true \
  --set ebpf.enforceMode=true \
  --set ebpf.requireLSM=true \
  --set ebpf.failClosedOnEnforcementGap=true

For a single pinned Helm walkthrough, see the short quickstart.

The two default gates have different purposes:

Setting	Current behavior
`ebpf.requireLSM: true`	Refuses startup when BPF LSM is unavailable. The measured GKE Ubuntu active stack fails this prerequisite.
`ebpf.failClosedOnEnforcementGap: true`	Returns HTTP 503 from health/readiness while a required enforcement gap remains. The measured COS node passed this gate through `lsm+fmod_ret`.

The chart's failClosedOnEnforcementGap default is null, which inherits the deprecated requireFullEnforcement: true alias; the effective default is true. The measured COS result needed no gap opt-out. Setting the gate false changes readiness policy, not kernel enforcement. Do not blanket-accept gaps or disable requireLSM to make this guide pass.

Inspect the installation:

kubectl get pods -n cloudtaser-system

Running is insufficient. Require the eBPF DaemonSet to be Ready and inspect its health/readiness response and per-vector status using Step 5. Stop on refusal diagnostics. The measured COS cohort passes; the measured stock GKE Ubuntu image does not.

Step 4: Deploy a Protected Workload¶

Only after the node and agent qualification passes, annotate your deployment with cloudtaser annotations:

First enroll the workload namespace; keep the pod injection annotation below:

kubectl create namespace production
kubectl label namespace production cloudtaser.io/inject=true --overwrite

Workload admission does not run in unenrolled namespaces. See namespace enrollment.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp
  namespace: production
spec:
  replicas: 2
  selector:
    matchLabels:
      app: myapp
  template:
    metadata:
      annotations:
        secretstore.cloudtaser.io/inject: "true"
        cloudtaser.io/ebpf: "true"
        cloudtaser.io/secretstore-address: "https://vault.eu.example.com"
        cloudtaser.io/secretstore-role: "cloudtaser"
        secretstore.cloudtaser.io/secret-paths: "secret/data/myapp/config"
        secretstore.cloudtaser.io/env-map: "db_password=PGPASSWORD,api_key=API_KEY"
      labels:
        app: myapp
    spec:
      containers:
        - name: myapp
          image: myorg/myapp:v1.2.3

Required Annotations¶

Annotation	Required	Description
`secretstore.cloudtaser.io/inject`	Yes	Enables cloudtaser injection (`"true"`)
`cloudtaser.io/ebpf`	No	Enables eBPF runtime enforcement (`"true"`)
`cloudtaser.io/secretstore-address`	Yes	URL of the EU-hosted OpenBao
`cloudtaser.io/secretstore-role`	Yes	OpenBao Kubernetes auth role name
`secretstore.cloudtaser.io/secret-paths`	Yes	Comma-separated OpenBao secret paths
`secretstore.cloudtaser.io/env-map`	Yes	Maps OpenBao fields to environment variable names

Apply the deployment:

kubectl apply -f deployment.yaml

Step 5: Verify enforcement before interpreting the score¶

Through your approved node-access procedure, confirm bpf in /sys/kernel/security/lsm; a built-in CONFIG_BPF_LSM=y or a nonzero loaded-program count alone is insufficient. Using the agent's configured TLS and authentication, inspect each node's /status kernel_tier and /healthz enforcement_vectors, plus the /readyz result and enforcement_gaps. Require every relevant vector to have the reviewed state for the exact release. A coarse tier, lsm_count > 0, or a high wrapper score cannot distinguish complete enforcement from Tier 2 with gaps. See node checks.

Then inspect the wrapper's score as additional control telemetry:

kubectl logs -n production deploy/myapp -c myapp 2>&1 | grep "protection"

The following is a synthetic all-checks score example, separate from the measured COS Tier 1 readiness result. The measured COS node lacks kprobe override; a missing kprobe score does not cancel its effective fmod_ret paths:

[cloudtaser-wrapper] Protection score: 175/175
[cloudtaser-wrapper]   memfd_secret:          OK  (+15)
[cloudtaser-wrapper]   mlock:                 OK  (+10)
[cloudtaser-wrapper]   core_dump_exclusion:   OK  (+5)
[cloudtaser-wrapper]   dumpable_disabled:     OK  (+5)
[cloudtaser-wrapper]   guard_pages:           OK  (+5)
[cloudtaser-wrapper]   buffer_canary:         OK  (+5)
[cloudtaser-wrapper]   token_protected:       OK  (+10)
[cloudtaser-wrapper]   vault_tls_verified:    OK  (+20)
[cloudtaser-wrapper]   broker_tls_verified:   OK  (+20)
[cloudtaser-wrapper]   environ_scrubbed:      OK  (+5)
[cloudtaser-wrapper]   getenv_interposer:     OK  (+10)
[cloudtaser-wrapper]   ebpf_agent_connected:  OK  (+10)
[cloudtaser-wrapper]   cpu_mitigations:       OK  (+5)
[cloudtaser-wrapper]   ebpf_enforce_mode:     OK  (+15)
[cloudtaser-wrapper]   ebpf_lsm_enforce:      OK  (+15)
[cloudtaser-wrapper]   ebpf_kprobes:          OK  (+10)
[cloudtaser-wrapper]   confidential_vm:       OK  (+10)

The old 135 maximum and 125/115 platform ceilings predate the two 20-point TLS checks. Their replacement is not a promise of a 175-point GKE deployment: readiness, per-vector enforcement and runtime evidence remain separate gates.

Mixed node pools and workload placement¶

The confidential-node label describes hardware eligibility, not agent enforcement readiness. Restrict protected workloads to a qualified node pool and verify the agent on every eligible node; a label alone does not establish SNP/TDX technology, active BPF LSM, or required-vector coverage.

If you have a mixed cluster with both confidential and non-confidential node pools, the operator still detects the capability per-node and reports the confidential_vm check accordingly.

Container Image Requirements¶

The getenv_interposer check (10 points) is an optional glibc-only enhancement. The LD_PRELOAD interposer (libcloudtaser.so) blocks getenv() from returning secrets on the heap by returning pointers to memfd_secret-backed memory instead. It does not activate on musl or statically linked binaries -- those use the default env-var delivery path, which works for all binaries without code changes.

Base Image	getenv_interposer	Recommendation
Debian / Ubuntu	Supported	Recommended
Red Hat / Fedora	Supported	Recommended
Alpine (musl)	Not supported	Use debian-slim instead
Distroless (glibc)	Supported	Works
Distroless (static)	Not supported	Use glibc variant
Scratch (static binary)	Not supported	Uses default env-var delivery

Switch from Alpine to Debian slim

If your application uses alpine as the base image, consider switching to debian:bookworm-slim or ubuntu:24.04 for the same small footprint with glibc support. This enables the optional getenv interposer and adds 10 points to your protection score. Alpine and musl-based images still receive secrets through the default env-var delivery path.

Workload migration gate¶

Complete Steps 1–3 and the checks in Step 5 before applying Step 4 or running cloudtaser-cli target protect. Stop if the actual node reports refusal diagnostics. The measured COS cohort passed with default strict settings; qualify your own release/configuration before using the migration workflow. Retain the exact node image, component versions, active LSM list, agent vector table and readiness evidence with the deployment record.

Troubleshooting¶

Symptom	Cause	Fix
`confidential_vm: FAIL`	Confidential technology absent or not detected	Verify the actual machine/technology pair. Explicit SNP uses N2D Milan and `--confidential-node-type=sev_snp`; C3 uses TDX.
`ebpf_kprobes: MISSING` on COS	Kprobe override is disabled	Inspect the required-vector gaps and readiness. Do not switch to Ubuntu expecting active BPF LSM, or bypass the gap gate.
`ebpf_lsm_enforce: MISSING`	BPF LSM absent from the active stack, or hooks unavailable	Check `/sys/kernel/security/lsm` on the actual node. The measured GKE Ubuntu stack lacks `bpf`; COS config includes it but still needs runtime verification.
Agent HTTP 503 on COS	A required enforcement gap or another readiness prerequisite is failing	Retain the reasons and exact release/image. The measured COS cohort passed defaults; compare actual hook attachments and keep the gap gate enabled.
`getenv_interposer: FAIL`	musl or static application	Use the supported glibc delivery mode if appropriate; this does not resolve node enforcement refusal.
`ebpf_agent_connected: FAIL`	Agent unavailable or attestation not verified	Inspect DaemonSet readiness and startup diagnostics.
`ebpf_enforce_mode: FAIL`	Enforce mode disabled or status unverified	Check `ebpf.enforceMode=true`, verified status, and the independent readiness gates.

Other runtime security tools¶

Tetragon and KubeArmor can provide additional runtime policy, but their presence does not satisfy CloudTaser's missing vectors or readiness gates. Qualify the combined policies, attachment order and kernel configuration on the actual node. Do not infer CloudTaser support from another tool's COS support.

NetworkPolicy and NodeLocal DNSCache¶

If you use Kubernetes NetworkPolicy to restrict egress from protected namespaces, wrapper pods may fail with context deadline exceeded during the broker secret fetch. The root cause is GKE's NodeLocal DNSCache (node-local-dns), which runs as a hostNetwork: true DaemonSet listening on 169.254.20.10. Because it uses the host network, it does not match a namespaceSelector targeting kube-system — even though the node-local-dns pods live in the kube-system namespace.

The problem¶

A NetworkPolicy like this looks correct but silently blocks DNS when NodeLocal DNSCache is active:

# BROKEN: does not match hostNetwork pods
egress:
  - to:
      - namespaceSelector:
          matchLabels:
            kubernetes.io/metadata.name: kube-system
    ports:
      - protocol: UDP
        port: 53
      - protocol: TCP
        port: 53

Pods resolve DNS via the node-local cache at 169.254.20.10, not via the kube-dns ClusterIP. The namespaceSelector rule only matches traffic to pod IPs in kube-system, which does not include host-network addresses.

The fix¶

Add an explicit egress rule for the NodeLocal DNSCache IP in addition to the kube-system selector (the kube-system rule is still needed as a fallback when NodeLocal DNSCache is not present or for TCP DNS to upstream):

egress:
  # NodeLocal DNSCache (hostNetwork, 169.254.20.10)
  - to:
      - ipBlock:
          cidr: 169.254.20.10/32
    ports:
      - protocol: UDP
        port: 53
      - protocol: TCP
        port: 53
  # kube-dns fallback (pod network)
  - to:
      - namespaceSelector:
          matchLabels:
            kubernetes.io/metadata.name: kube-system
    ports:
      - protocol: UDP
        port: 53
      - protocol: TCP
        port: 53

Self-check¶

Verify DNS works from within a restricted namespace:

kubectl run dns-test --rm -it --restart=Never \
  --namespace <your-namespace> \
  --image=busybox:1.36 -- nslookup kubernetes.default

If the command hangs or returns server can't find kubernetes.default: SERVFAIL, the NetworkPolicy is still blocking DNS.

Applies to any workload behind NetworkPolicy

This is not cloudtaser-specific — any pod behind a restrictive egress NetworkPolicy on GKE with NodeLocal DNSCache enabled will hit the same issue. CloudTaser wrapper pods surface it early because the broker fetch is the first network call after pod start.

Reference: DNS egress diagnosis