Kubernetes security discussions often focus on what happens at the boundary: admission control validates manifests before they land, network policies restrict pod-to-pod communication, and RBAC governs who can touch the API server. These are all necessary controls, but they share a critical limitation — they operate at policy enforcement time, not at runtime. Once a pod is running, none of these mechanisms can tell you that a Java process just spawned a reverse shell, that a container is mining cryptocurrency, or that a compromised workload is reading secrets it was never meant to access. For the growing number of organizations running production Kubernetes across multiple clusters, this runtime visibility gap is no longer acceptable.
Extended Berkeley Packet Filter has fundamentally changed what is possible for runtime security on Linux, and by extension on Kubernetes. eBPF allows safe, sandboxed programs to run inside the kernel in response to system events — syscalls, network packets, file operations, process creation — without loading kernel modules or modifying kernel source code. For security teams, this means the kernel itself can be instrumented to observe every syscall, every file open, every network connection, and every process execution across all containers on a node, with minimal performance overhead. eBPF is the engine that makes modern runtime security tools possible, and Falco is the project that has made it accessible to Kubernetes platform teams.
Falco, originally created by Sysdig and now a CNCF graduated project, uses an eBPF probe to capture system calls on each Kubernetes node and evaluates them against a configurable rules engine. Rules are expressed in a declarative YAML syntax and can detect patterns such as a shell spawned inside a container that was not designed to run shells, a process reading a file in /etc/shadow, a container making an outbound network connection to a known mining pool, or a privileged container performing actions its developers never intended. When a rule matches, Falco generates an alert that can be routed to stdout for log aggregation, to a webhook for incident response automation, or to a chat platform where the on-call engineer can triage it immediately.
The operational model for Falco on Kubernetes is well-defined and battle-tested. A DaemonSet deploys the Falco agent to every node in the cluster, where it attaches the eBPF probe (or loads the kernel module on older kernels) and begins streaming system call events. This architecture ensures complete node coverage regardless of pod scheduling decisions. The Falco agent itself runs in a privileged container because it needs to inspect the kernel event stream — an unavoidable requirement, and one that should be carefully scoped through Pod Security Standards and RBAC to limit what the Falco service account can do beyond attaching the probe. For clusters where kernel-level access is restricted, the userspace instrumentation driver provides a fallback that captures syscall events in userspace using a ptrace-based approach, though with higher overhead and slightly different event coverage.
The rules engine is where Falco distinguishes between useful detection and noisy alerting. Out of the box, Falco ships with a comprehensive set of default rules maintained by the CNCF community that cover common attack patterns. These include unexpected spawned processes, filesystem reads and writes below sensitive directories, usage of suspicious syscalls like ptrace, changes to Linux capabilities, and network activity that deviates from expected patterns. The rules use a macro system that allows platform teams to define reusable conditions — for example, a macro that defines which containers are allowed to run shells as part of their normal operation — and then reference those macros across multiple rules. This significantly reduces the duplication and maintenance burden that would otherwise make a large rule set difficult to manage.
Customization is essential because every environment has different baselines. A CI/CD runner container spawning shells is normal; a production web server doing the same is an incident. Falco rules can scope detection to specific namespaces, pod labels, container images, or Kubernetes service accounts. Tags can be applied to rules to control alert routing and severity classification. For infrastructure teams managing multiple clusters — staging, production, customer environments — rules can be maintained as code in a Git repository, versioned alongside other infrastructure definitions, and deployed through the same GitOps pipeline that manages the rest of the cluster configuration. This brings runtime detection into the scope of infrastructure-as-code governance rather than leaving it as a manually configured afterthought.
The alert pipeline requires deliberate design to avoid alert fatigue. Raw Falco alerts can be verbose, especially during initial deployment when rules have not yet been tuned to the specific environment. A practical approach is to deploy Falco in a passive observation mode first, aggregating alerts into a centralized logging system such as Loki or Elasticsearch, and then reviewing the top alerts to identify false positives and tune rules before enabling active notification channels. Once the rule set is tuned, critical alerts — unexpected shells in production namespaces, writes to /etc or /root, connections to known malicious IPs — should trigger immediate notifications through PagerDuty or Slack. Lower-severity informational events can remain available for forensic investigation without triggering alerting.
Performance is often the first concern raised about kernel-level instrumentation, and the eBPF ecosystem has made substantial progress here. Modern eBPF probes run verified code inside the kernel with near-zero overhead for common workloads when the probe is well-designed. Falco's default eBPF probe is engineered to be efficient, and in practice most clusters see single-digit percentage CPU overhead on the node from the Falco agent. For extremely latency-sensitive workloads such as high-frequency trading or real-time streaming, additional tuning is available through syscall event filtering — telling the probe to skip events from specific containers, namespaces, or syscall types that are not relevant to the security detection use case. The key takeaway is that runtime security does not require a trade-off between protection and performance when implemented with modern eBPF tools.
Integration with the broader Kubernetes security stack is where Falco becomes a component of a defense-in-depth strategy rather than a standalone tool. Admission control prevents the deployment of workloads that violate policy, but it does not catch a workload that was benign at deploy time and becomes compromised later. Network policies prevent unauthorized network flows but do not detect a malicious process inside an allowed pod. RBAC prevents unauthorized API access but does not stop a process from reading a secret file it has legitimate filesystem access to. Falco provides the detection layer that catches what the prevention layers miss. Its alerts can feed into Kubernetes audit logs, be correlated with admission denial events, and be consumed by Security Information and Event Management platforms for cross-cluster threat detection.
For consulting engineers and platform teams responsible for customer or multi-tenant Kubernetes environments, runtime security with Falco provides evidence that goes beyond checkbox compliance. Being able to demonstrate that every syscall on every node is monitored for known attack patterns, that unexpected behavior generates immediate alerts with full forensic context, and that detection rules are maintained as version-controlled code changes the security conversation from 'we think we are secure' to 'here is what we detect and how we respond.' It also provides a foundation for incident response that is often missing in Kubernetes environments — when an engineer receives a Falco alert showing the exact command, container, pod, namespace, and timestamp of a suspicious event, investigation time drops from hours of log correlation to minutes of focused analysis.
The practical path forward for organizations that have not yet deployed runtime security on Kubernetes starts with deploying Falco in a staging cluster to build familiarity with the rules engine and alert pipeline. Use the default rule set as a starting point, observe the alerts for a week, and tune out the noise specific to your environment. Then deploy to production clusters in observe-only mode, leaving existing monitoring and alerting unchanged. After another tuning cycle, enable production alerting for critical rules and iterate on the rule set as the threat landscape and workload patterns evolve. The goal is not perfection on day one; it is closing the runtime visibility gap that static defenses cannot address, and building the detection muscle that every production Kubernetes platform needs.