The multi-cloud Kubernetes promise was simple: write once, run anywhere, and scale infinitely. The reality for modern enterprise infrastructure teams, however, is a chaotic operational nightmare. As clusters sprawl across AWS EKS, GCP GKE, and Azure AKS, cloud budgets disintegrate into a fog of unallocated resources, over-provisioned CPU requests, and idle buffer capacity.
Traditional FinOps methodologies have hit a wall. Dashboards that generate month-end CSV reports do not save money; they simply digitize post-mortem financial grief. Meanwhile, standard Kubernetes autoscaling mechanisms—such as the Horizontal Pod Autoscaler (HPA) and standard Vertical Pod Autoscaler (VPA)—rely on high-level, coarse metrics scraped from /metrics endpoints or cAdvisor. These traditional scrapers miss kernel-level context, suffer from scrape-interval latency, and force developers to vastly over-provision requests and limits out of fear of out-of-memory (OOM) kills and P99 latency degradation.
To eliminate compute waste across multi-cloud environments without compromising application reliability, we must move from reactive cost tracking to an autonomous, kernel-native FinOps engine.
At Ecstaticloud, we built an engine that pairs zero-overhead eBPF (Extended Berkeley Packet Filter) kernel telemetry with predictive ML forecasting models to execute zero-downtime, in-place pod right-sizing across heterogeneous cloud providers. Here is how it works.
System Architecture Overview
The autonomous FinOps engine consists of four distinct operational planes working in an event-driven loop:
+-----------------------------------------------------------------------------------+
| DATA PLANE |
| +-----------------------------------------------------------------------------+ |
| | eBPF Kernel Probes (cgroup v2, tracepoints) | |
| +-----------------------------------------------------------------------------+ |
+------------------------------------------+----------------------------------------+
| Low-overhead RINGBUF streams
v
+-----------------------------------------------------------------------------------+
| INGESTION & METRICS PLANE |
| +-----------------------------------------------------------------------------+ |
| | eBPF User-Space Agent (Go + libbpf-go) | |
| +-----------------------------------------------------------------------------+ |
| |
| v
| +-----------------------------------------------------------------------------+ |
| | High-Throughput Time-Series Engine (Prometheus/Vector) | |
| +-----------------------------------------------------------------------------+ |
+------------------------------------------+----------------------------------------+
|
v
+-----------------------------------------------------------------------------------+
| INTELLIGENCE PLANE |
| +-----------------------------------------------------------------------------+ |
| | Predictive ML Inference Engine (XGBoost / Temporal Transformer) | |
| +-----------------------------------------------------------------------------+ |
+------------------------------------------+----------------------------------------+
| Dynamic resource recommendation
v
+-----------------------------------------------------------------------------------+
| ACTUATION PLANE |
| +-----------------------------------------------------------------------------+ |
| | Ecstaticloud FinOps Controller (Go Operator using InPlacePodResize) | |
| +-----------------------------------------------------------------------------+ |
+-----------------------------------------------------------------------------------+
- Kernel Telemetry Layer (eBPF): Attached to tracepoints, raw sockets, and cgroup kernel structures. Captures actual CPU cycles consumed, CFS throttling latency, memory page cache pressure, and socket-buffer wait times at nanosecond resolution.
- Aggregation Pipeline: Normalizes raw kernel events and streams metric vectors into a centralized high-throughput telemetry bus, mapping PIDs and thread IDs directly to Kubernetes Pod UID and Container cgroup context.
- Predictive Inference Engine: A lightweight time-series model that forecasts resource demands over $N$-minute horizons, factoring in periodicity, historical burn-rates, and kernel-level bottleneck indicators.
- Autonomous Controller: A custom Kubernetes Operator that executes
InPlacePodVerticalScaling(mutating container resources without pod eviction) and manages node-level bin-packing optimization.
Layer 1: Zero-Overhead Kernel Telemetry via eBPF
Standard metrics scrapers ask the Linux kernel: "How many total CPU seconds has this cgroup used over the last 30 seconds?" This broad question masks severe performance bugs. A process might sit idle for 29 seconds, spike brutally for 100 milliseconds—triggering severe Completely Fair Scheduler (CFS) throttling—and yet report a completely fine 5% average usage.
Using eBPF, our probes attach directly to kernel tracepoints like sched:sched_stat_throttled, sched:sched_switch, and memctl:mm_vmscan_direct_reclaim. This gives us exactVisibility into off-CPU time (when a thread wants to run but cannot) and CFS period bandwidth exhaustion.
eBPF C Probe: Tracking CPU Throttling and Off-CPU Latency
Below is an abbreviated C implementation of an eBPF program utilizing BPF CO-RE (Compile Once – Run Everywhere) to track exact container throttling duration and off-CPU latency per cgroup.
// +build ignore
#include "vmlinux.h"
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>
#include <bpf/bpf_core_read.h>
struct key_t {
u64 cgroup_id;
u32 pid;
};
struct metrics_t {
u64 cpu_off_time_ns;
u64 throttle_time_ns;
u64 nr_throttled;
};
// Hash map to store metric accumulators per cgroup/PID
struct {
__uint(type, BPF_MAP_TYPE_HASH);
__uint(max_entries, 10240);
__type(key, struct key_t);
__type(value, struct metrics_t);
} cgroup_metrics SEC(".maps");
// Track start time of off-CPU events
struct {
__uint(type, BPF_MAP_TYPE_HASH);
__uint(max_entries, 10240);
__type(key, u32);
__type(value, u64);
} start_off_cpu SEC(".maps");
SEC("tp/sched/sched_stat_throttled")
int handle_sched_stat_throttled(struct trace_event_raw_sched_stat_throttled *ctx)
{
u64 cgroup_id = bpf_get_current_cgroup_id();
u32 pid = bpf_get_current_pid_tgid() >> 32;
struct key_t key = { .cgroup_id = cgroup_id, .pid = pid };
struct metrics_t *val = bpf_map_lookup_elem(&cgroup_metrics, &key);
if (!val) {
struct metrics_t zero = {};
bpf_map_update_elem(&cgroup_metrics, &key, &zero, BPF_NOEXIST);
val = bpf_map_lookup_elem(&cgroup_metrics, &key);
if (!val) return 0;
}
// ctx->delay corresponds to time spent waiting due to CFS bandwidth limits
__sync_fetch_and_add(&val->throttle_time_ns, ctx->delay);
__sync_fetch_and_add(&val->nr_throttled, 1);
return 0;
}
SEC("tp/sched/sched_switch")
int handle_sched_switch(struct trace_event_raw_sched_switch *ctx)
{
u64 ts = bpf_ktime_get_ns();
u32 prev_pid = ctx->prev_pid;
u32 next_pid = ctx->next_pid;
// Record when prev_pid went off-CPU
if (prev_pid != 0) {
bpf_map_update_elem(&start_off_cpu, &prev_pid, &ts, BPF_ANY);
}
// Calculate time next_pid spent off-CPU
u64 *start_ts = bpf_map_lookup_elem(&start_off_cpu, &next_pid);
if (start_ts) {
u64 delta = ts - *start_ts;
u64 cgroup_id = bpf_get_current_cgroup_id();
struct key_t key = { .cgroup_id = cgroup_id, .pid = next_pid };
struct metrics_t *val = bpf_map_lookup_elem(&cgroup_metrics, &key);
if (val) {
__sync_fetch_and_add(&val->cpu_off_time_ns, delta);
}
bpf_map_delete_elem(&start_off_cpu, &next_pid);
}
return 0;
}
char LICENSE[] SEC("license") = "GPL";
Why This Matters for FinOps
With this eBPF program deployed:
- We distinguish between desired underutilization (a pod idling safely) and covert starvation (a pod under-allocated CPU limit running into heavy CFS throttling despite low average CPU telemetry).
- We detect memory pressure before OOM events happen by inspecting kernel direct memory reclaim cycles (
mm_vmscan_direct_reclaim). If reclaim cycles stay at zero, we can aggressively trim memory allocations down to the working set size (WSS) + a minimal safety buffer.
Layer 2: Predictive Inference & Allocation Cost Function
Static right-sizing algorithms (such as picking the P95 usage over 7 days) lead to either unnecessary headroom waste or catastrophic dynamic outages during unexpected traffic surges.
Our FinOps Engine uses a time-series model (a pruned XGBoost regressor or lightweight Temporal Fusion Transformer running inside an inference service) that runs every 60 seconds against collected eBPF metrics vectors.
The Objective Function
The optimization model minimizes the Cost/SLO Risk Function ($J$):
$$J(R_{cpu}, R_{mem}) = \underbrace{\alpha \cdot \text{Cost}(R_{cpu}, R_{mem})}{\text{Financial Expenditure}} + \underbrace{\beta \cdot P(\text{CFS_Throttling} > \tau)}{\text{CPU SLO Risk}} + \underbrace{\gamma \cdot P(\text{Direct_Reclaim} > 0)}_{\text{OOM / Memory Latency Risk}}$$
Where:
- $R_{cpu}, R_{mem}$ are the requested CPU and Memory capacity vectors.
- $\alpha$ is the dynamic unit cost factor based on cloud provider billing API (e.g., AWS On-Demand vs Spot vs GCP Committed Use Discounts).
- $\beta, \gamma$ are penalty coefficients mapped directly to application SLO sensitivity thresholds.
- $\tau$ is the acceptable micro-throttling latency limit (e.g., $< 5\text{ms}$).
If the ML inference predicts that the tail workload probability $P(\text{Spike})$ over the next 15-minute window is low, and kernel metrics show zero off-CPU wait time and zero memory reclaim pressure, the cost function $J$ forces $R_{cpu}$ and $R_{mem}$ to converge rapidly toward actual consumption levels.
Layer 3: Zero-Downtime Actuation via InPlacePodVerticalScaling
Traditionally, changing a Pod’s resource requests meant deleting and re-creating the pod. In a dynamic production ecosystem, rolling restarts cause mass cache invalidation, deployment churn, and high risks of elevated errors.
With Kubernetes 1.27+, the InPlacePodVerticalScaling feature gate allows us to mutate container resources spec fields without restarting the container process.
The Go Actuation Controller
Below is an extract from the custom Ecstaticloud Go Operator using client-go to perform an in-place patch on an active, running Pod based on eBPF ML engine recommendations.
package controller
import (
"context"
"encoding/json"
"fmt"
"log"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/api/resource"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/types"
"k8s.io/client-go/kubernetes"
)
type ResourcePatch struct {
Spec SpecPatch `json:"spec"`
}
type SpecPatch struct {
Containers []ContainerPatch `json:"containers"`
}
type ContainerPatch struct {
Name string `json:"name"`
Resources corev1.ResourceRequirements `json:"resources"`
}
// DynamicResizePod applies in-place resource mutations based on eBPF recommendations
func DynamicResizePod(ctx *context.Context, clientset *kubernetes.Clientset, pod *corev1.Pod, targetCPU string, targetMem string) error {
// Verify if Pod policies allow InPlaceResize
if pod.Spec.ResizePolicy == nil {
log.Printf("Pod %s does not specify explicit ResizePolicy; falling back to default control execution.", pod.Name)
}
patchData := ResourcePatch{
Spec: SpecPatch{
Containers: []ContainerPatch{
{
Name: pod.Spec.Containers[0].Name,
Resources: corev1.ResourceRequirements{
Requests: corev1.ResourceList{
corev1.ResourceCPU: resource.MustParse(targetCPU),
corev1.ResourceMemory: resource.MustParse(targetMem),
},
Limits: corev1.ResourceList{
corev1.ResourceCPU: resource.MustParse(targetCPU),
corev1.ResourceMemory: resource.MustParse(targetMem),
},
},
},
},
},
}
payloadBytes, err := json.Marshal(patchData)
if err != nil {
return fmt.Errorf("failed to marshal JSON patch: %w", err)
}
// Apply Patch using Strategic Merge Patch to trigger InPlacePodVerticalScaling
_, err = clientset.CoreV1().Pods(pod.Namespace).Patch(
*ctx,
pod.Name,
types.StrategicMergePatchType,
payloadBytes,
metav1.PatchOptions{},
)
if err != nil {
return fmt.Errorf("failed to execute in-place resource patch on pod %s: %w", pod.Name, err)
}
log.Printf("Successfully performed zero-downtime resize on Pod [%s/%s] -> CPU: %s, Mem: %s",
pod.Namespace, pod.Name, targetCPU, targetMem)
return nil
}
Multi-Cloud Cost Synthesis: Normalizing AWS, GCP, and Azure
Downscaling Kubernetes pods is only half the battle. The true goal of multi-cloud FinOps is driving down absolute infrastructure spend across diverse hyperscalers.
Compute costs vary wildly across providers and region profiles. A vCPU on an AWS c6i.xlarge instance carries a different price-to-performance profile than an n2-standard-4 on GCP or a D4s_v5 on Azure.
Our FinOps Engine solves this by introducing the Normalized Compute Unit (NCU).
+-----------------------+ +-----------------------+ +-----------------------+
| AWS EKS Cluster | | GCP GKE Cluster | | Azure AKS Cluster |
| c6i.xlarge (x86_64) | | n2-standard (x86_64) | | D4s_v5 (x86_64) |
+-----------+-----------+ +-----------+-----------+ +-----------+-----------+
| | |
| eBPF Hardware Metrics | eBPF Hardware Metrics | eBPF Hardware Metrics
| (IPC, L3 Cache Misses) | (IPC, L3 Cache Misses) | (IPC, L3 Cache Misses)
v v v
+-----------------------------------------------------------------------------------+
| NORMALIZED COMPUTE UNIT ENGINE |
| |
| Standardized Score = (Raw CPU Cycles * Instructions Per Cycle [IPC]) |
| / Provider Cost Per Hour |
+-----------------------------------------------------------------------------------+
|
v
+-----------------------------------+
| Global Bin-Packing Scheduler |
| (Moves workloads to lowest $/NCU) |
+-----------------------------------+
By profiling low-level instructions-per-cycle (IPC) and memory bus latency via eBPF performance counters (perf_event_open), the Engine maps raw execution capability to actual cloud cost.
If AWS us-east-1 spot instance capacity delivers a higher IPC per Dollar than GCP us-central1 on-demand capacity, the engine signals the multi-cloud orchestrator (e.g., via Crossplane or custom cluster autoscaler providers) to expand node pools on AWS while gracefully draining and right-sizing workloads on GCP.
Production Lessons & Edge Cases
Deploying an autonomous, kernel-driven controller that modifies production container resources in real time requires robust safety guardrails. Here are key hard-won operational realities to consider:
1. The Runtime Memory Shrinkage Paradox (GC Languages)
Lowering memory.limits on a live container running Java (JVM) or Go runtime code can be fatal if the runtime's Garbage Collector isn't explicitly notified. While the kernel cgroup boundary shrinks instantly, the JVM heap size ($Xmx$) remains fixed, triggering an unexpected OOMKill.
- Solution: The Ecstaticloud Engine injects dynamic runtime configuration handlers (e.g., mutating
GOMEMLIMITdynamically via signals or interacting with the JVM Attach API to lower heap limits prior to issuing the Kubernetes in-place resize patch).
2. Kernel BTF (BPF Type Format) Portability
Older cloud kernel images lack built-in CONFIG_DEBUG_INFO_BTF=y, breaking BPF CO-RE capability across hybrid nodes.
- Solution: Maintain a central repository of pre-compiled eBPF object files indexed by kernel release headers, or ensure all target node pool AMIs utilize modern Linux kernels (5.15+) with native BTF support enabled.
3. Circuit Breaker Fallbacks
If the ML Inference Engine experiences network partition or anomalous metric spikes, automated actuation loops must instantly freeze.
- Solution: Implement strict hysteresis thresholds. Resource changes are capped at a maximum step-down percentage (e.g., no more than a 20% reduction per 10-minute period), and any observed pod restart immediately locks the resizing engine into a read-only observability state for that namespace for 24 hours.
Conclusion: The Era of Kernel-Native FinOps
The era of static YAML resource declarations and reactive cost-reporting dashboards is over. Managing multi-cloud infrastructure sprawl manually is a losing battle that wastes millions of dollars in idle capacity and thousands of engineering hours in endless tuning cycles.
By pairing zero-overhead eBPF kernel observability with predictive inference and non-disruptive Kubernetes actuation, cloud engineering teams can finally build infrastructure that scales dynamically to match true demand—achieving optimal performance at the absolute lowest cost.