## Kubernetes controls stop at the infrastructure layer
The Cloud Native Computing Foundation published a significant advisory in late March 2026 that deserves careful attention from anyone deploying large language models on Kubernetes in production environments. The core message is direct: Kubernetes excels at orchestrating and isolating workloads, but it does not inherently understand or control the behavior of AI systems, creating a fundamentally different and more complex threat model than traditional cloud-native deployments. For DevOps consultants advising enterprise clients on AI infrastructure strategy, this is not a theoretical concern — it is a practical security gap that organizations are already encountering as LLM adoption scales across regulated industries.
Practical tip: write a threat model for each AI assistant before production rollout. List every tool the model can call, every data store it can retrieve from, and the authorization check that runs before output leaves the service; then align the platform side with your [Kubernetes AI infrastructure plan](/blog/kubernetes-ai-infrastructure-2026/).
The distinction between traditional and AI workloads that makes this gap so critical comes down to how LLMs process untrusted input and make dynamic decisions. A conventional microservice receives structured requests, executes defined logic, and returns predictable outputs. An LLM ingests free-form text that can contain maliciously crafted prompts designed to manipulate model behavior, exfiltrate training data, or manipulate connected tools and downstream systems. Kubernetes knows nothing about the semantic content of prompts, the sensitivity of data flowing through a model, or the consequences of model outputs being passed to external APIs. Infrastructure can be perfectly healthy while an LLM layer is actively leaking customer data — and cluster metrics will show no indication anything is wrong.
## Where LLM threats bypass cluster signals
Consider what happens when an LLM is deployed on Kubernetes with access to internal tools, documentation, or APIs — a common enterprise AI assistant pattern. The cluster handles pod scheduling, enforces resource limits, and applies network policies correctly. However, a carefully constructed prompt injection attack can cause the model to invoke tooling functions it was not designed to expose, retrieve sensitive internal documents, or pass credentials to external systems. Kubernetes cannot evaluate whether a model's tool invocation is legitimate or malicious. The model becomes a programmable layer an attacker can steer through text input, and the entire trust model depends on application-layer output filtering rather than infrastructure enforcement.
The threat landscape extends beyond prompt injection into retrieval-augmented generation pipelines, where an LLM queries internal vector databases to ground responses in proprietary knowledge. If a retrieval query is manipulated through injection, the system may retrieve documents the requesting user is not authorized to access and pass them to the model. Kubernetes network policies control pod-to-pod communication at the network layer but cannot enforce document-level access controls on retrieval operations. Output filtering — scanning responses for sensitive data before returning them — operates entirely outside Kubernetes' visibility and requires application-layer implementation that most Helm charts do not provide.
Traditional Kubernetes security controls remain necessary but are insufficient as a standalone defense for AI workloads. Role-based access control restricts which service accounts can access which secrets, but it cannot prevent a model with legitimate secret access from being manipulated into misusing those credentials through a crafted prompt. Pod Security Standards and seccomp profiles limit container privileges but do not govern what a model does with the data it receives. Network policies segment traffic between pods but cannot prevent data from flowing through a model to an unauthorized destination. Each control addresses a real attack vector, but none addresses the new class of risk that arises from models interpreting instructions and acting on untrusted input.
## Practical controls for platform teams
AI-specific controls must be layered on top of Kubernetes foundations to create a coherent security posture for LLM deployments. Prompt validation and sanitization at the input boundary — rejecting or neutralizing known injection patterns before content reaches the model — is one foundational layer that Kubernetes does not provide. Output filtering, content classification, and data loss prevention checks on model responses represent another essential layer that must be implemented at the application tier. Tool access policies should define explicit, auditable rules about which functions a model can invoke under which conditions, with human-in-the-loop approval required for high-stakes operations. Runtime monitoring should track behavioral signals such as unusual retrieval patterns, atypical tool invocation sequences, or response characteristics that suggest data leakage.
The OWASP Top 10 for Large Language Model Applications provides a structured framework for identifying and prioritizing these risks, and it has become a reference standard for AI security practitioners advising enterprise clients. Organizations deploying LLMs on Kubernetes should treat the OWASP LLM Top 10 as a required audit checklist, mapping each vulnerability class to compensating controls that complement existing Kubernetes hardening. Policy-as-code approaches — encoding AI usage policies in machine-readable definitions that can be enforced, version-controlled, and reviewed — are well-aligned with GitOps workflows platform teams already use for Kubernetes infrastructure. Integrating AI policy-as-code with ArgoCD or Flux pipelines creates a unified control plane where both infrastructure and model behavior are governed by the same declarative definitions.
The CNCF advisory corrects a pattern that has become common in enterprise AI deployments: treating Kubernetes adoption as equivalent to having a secure AI platform. Platform teams that have invested heavily in cluster hardening, CIS benchmark compliance, and multi-tenancy isolation have built a strong foundation, but they have not necessarily addressed risks specific to intelligent, programmable systems. Security reviews for LLM deployments must expand beyond the standard Kubernetes audit checklist to include prompt injection vectors, retrieval pipeline access controls, output filtering, tool invocation governance, and behavioral monitoring. Operational cluster health and AI-layer security are distinct concerns requiring distinct controls — conflating them is precisely the gap the CNCF is now warning about.
## Sources
Sources for the risk model and infrastructure boundary: [OWASP Top 10 for LLM Applications](https://owasp.org/www-project-top-10-for-large-language-model-applications/) and [Kubernetes RBAC documentation](https://kubernetes.io/docs/reference/access-authn-authz/rbac/).