Monitoring Kubernetes (K8s) clusters at scale remains a top challenge for many organizations, even as K8s deployments continue to make their way into the mainstream. Why the disparity between K8s growth, and the ability for teams to successfully monitor their deployments? Bottom like, it's... just more complicated than before, since the systems are diverse, and each deployment can be made up of many different components, tools, architectures and more. Suffice to say, runtime monitoring in K8s is *unique* compared to systems such as Linux or Windows.
- Distributed Architecture: K8s clusters containing multiple nodes run a high number of different kernels and container types with different operating systems, obscuring direct visibility into network and resource usage.
- K8s Internal Reporting: While K8s reports metrics at different levels such as containers, clusters, and services, it does not record data on individual processes – only the state of hosted applications – or persist any log data for wiped containers.
- Unique Metrics for Services and Components: The K8s API, scheduler, and key-value store each report their own unique metrics but lack any integrated performance metric.
While limited observability into resource requirements and performance at scale can easily result in unnecessary expenses due to over-provisioning or application performance issues due to under-provisioning, it also creates a persistent blind spot in which attackers can operate, siphoning resources or moving laterally to find routes to privilege escalation. For security operations, this lack of environment-wide observability highlights the need for K8s and container monitoring capabilities based on ground-level data.
eBPF and Kernel Space Observability
The Berkeley Packet Filter (BPF) is a network traffic analysis tool developed for Linux systems in the 90s. Since Linux 4.x, an extended form of BPF has allowed programs to run in kernel space without requiring changes to the kernel source code or additional modules. Originally developed to filter network packets, eBPF can also detect network events, system calls, function entries, and kernel tracepoints in kernel space and execute attached programs in response. These capabilities have made it a coincidentally elegant solution for runtime visibility challenges in today’s distributed, containerized environments.
eBPF and Kubernetes
As a virtual sandbox running in kernel space, eBPF hooks into and reports on exactly the kind of granular container runtime data that is missing from logs and traditional monitoring tools.
- Specific Services: eBPF can consistently target specific services and processes across clusters
- Greater Logging Depth with Minimal Resource Consumption: Not only does eBPF have access to ground-level data that existing logging tools cannot see, but it also operates at the OS level and does not require sidecar containers that tax resource overhead.
As eBPF is installed by default in Linux 4.x and after, deploying eBPF programs is simple and unintrusive, even at any Kubernetes scale.
Capitalizing on eBPF for Kubernetes Runtime Monitoring with Spyderbat
Spyderbat employs a lightweight nano agent leveraging an eBPF program to enable runtime monitoring throughout K8s and containerized environments. The nano agent transmits targeted eBPF-captured system calls live to Spyderbat’s SaaS solution where individual activities are connected via their causal relationships. Operators can view these relationships via Spyderbat’s interactive visual interface to see causal connections between process activities, user sessions and network connections within and across containers and systems – to its root source with a few mouse clicks. Spyderbat automatically fingerprints workload behaviors, enabling real-time automated application drift detection for new behaviors. In addition, Spyderbat monitors each trace for threat indicators, eradicating attackers in their tracks with surgical precise blocking actions that eliminate threats early and thoroughly.
Check out our eBPF for Cloud Runtime Security page to learn more about how Spyderbat turns eBPF data into actionable context now!
