Zero-Trust Microsegmentation at Scale: Implementing Cilium eBPF and SPIFFE/SPIRE Across Multi-Region EKS Clusters | Ecstaticloud Blog

The shift to multi-region Amazon EKS deployments has exposed a fundamental flaw in enterprise cloud networking: traditional perimeter-based security model and basic Kubernetes NetworkPolicies are entirely inadequate for distributed microservices. Relying on static IP addresses, AWS Security Groups at the ENI boundary, or iptables-based network filtering introduces unacceptable operational toil, latency penalties, and security gaps.

When operating thousands of pods across multiple AWS regions, IPs are ephemeral, subnets overlap, and iptables rules scale with $O(N^2)$ complexity, leading to packet processing bottlenecks and kernel CPU spikes. True Zero-Trust requires two non-negotiable primitives:

Cryptographic Workload Identity: Every running process must prove who it is via short-lived, verifiable credentials rather than relying on IP addresses or network topology.
Kernel-Level, Packet-by-Packet Microsegmentation: Enforcement must occur directly in the Linux kernel network stack without the overhead of user-space proxies or sequential iptables traversals.

In this guide, we will design and deploy a production-grade, multi-region Zero-Trust architecture by pairing SPIFFE/SPIRE for strong, cryptographically attested identity with Cilium eBPF for high-performance microsegmentation across multi-region Amazon EKS clusters (us-east-1 and eu-west-1).

Architectural Deep Dive: eBPF Meets Cryptographic Identity

To understand why this dual-engine approach is necessary, let us break down how eBPF and SPIFFE/SPIRE operate at the low level and how they seamlessly bridge network layers 3 through 7.

+---------------------------------------------------------------------------------------+
| EKS Node (us-east-1)                                                                  |
|                                                                                       |
|  +--------------------+                         +----------------------------------+  |
|  | Pod A: Payment API |                         | SPIRE Agent                      |  |
|  |  (App Process)     |                         |  - Attests node via AWS IID      |  |
|  +---------+----------+                         |  - Obtains Trust Bundle          |  |
|            | Mounts Unix Domain Socket          +----------------+-----------------+  |
|            v                                                     |                    |
|  /run/spire/sockets/agent.sock <---------------------------------+                    |
|            |                                                     |                    |
|            v                                                     v                    |
|  +---------------------------------------------------------------------------------+  |
|  | Cilium Agent                                                                    |  |
|  |  - Fetches SVID / Map SPIFFE ID <-> eBPF Identity Map                          |  |
|  +-------------------------------------+-------------------------------------------+  |
|                                        | Syncs Maps                                   |
| =======================================v============================================ |
| Linux Kernel                                                                          |
|                                                                                       |
|  +---------------------------------------------------------------------------------+  |
|  | eBPF Program Engine (TC / XDP / Sockops)                                        |  |
|  |                                                                                 |  |
|  |  [ eBPF Map: Identity Lookup ]                                                  |  |
|  |  Src ID: 1042 (spiffe://ecstaticloud.io/ns/prod/sa/payment)                      |  |
|  |  Dst ID: 2089 (spiffe://ecstaticloud.io/ns/prod/sa/ledger)                       |  |
|  |  Action: ALLOW (L3/L4 Fast Path / L7 Envoy Proxy Redirection if HTTP rule)      |  |
|  +---------------------------------------------------------------------------------+  |
+---------------------------------------------------------------------------------------+

1. Cilium & eBPF: Bypassing the Kernel Network Bottleneck

Traditional Kubernetes CNI plugins rely on kube-proxy and iptables (or IPVS) to route and filter packets. Every packet traversing a node must traverse chains of rule evaluations. At scale, updating thousands of sequential iptables entries stalls kernel routines and causes massive latency jitter.

Cilium completely bypasses iptables by attaching eBPF (Extended Berkeley Packet Filter) programs directly to network hooks within the Linux kernel:

XDP (eXpress Data Path): Processes packets directly at the network interface card (NIC) driver level for extreme DDoS mitigation and packet dropping.
Traffic Control (tc): Intercepts ingress/egress packets before network buffers reach the full kernel IP stack.
Socket Layer (sockops / sk_msg): Bypasses the TCP/IP stack entirely for local pod-to-pod communication on the same node, streaming socket buffers directly from writer to reader socket.

Instead of matching IPs, Cilium assigns a numeric Security Identity to a set of pods. Map updates are $O(1)$ hash table lookups stored in BPF maps inside kernel memory space (cilium_policy_v2).

2. SPIFFE/SPIRE: The Cryptographic Plane

While Cilium identifies workloads using Kubernetes labels by default, labels are soft metadata stored in the Kubernetes API server; they do not prove authenticity. If a worker node is compromised, an attacker can spoof pod labels or poison local network routes.

SPIFFE (Secure Production Identity Framework for Everyone) defines a standard format for workload identity: spiffe://<trust-domain>/ns/<namespace>/sa/<service-account>

SPIRE (SPIFFE Runtime Environment) enforces this standard through precise workload attestation:

Node Attestation: The SPIRE Agent proves the underlying EC2 instance identity to the SPIRE Server using AWS Instance Identity Documents (IID) signed by AWS KMS.
Workload Attestation: When a pod requests an identity, the local SPIRE Agent queries the Linux kernel (/proc filesystem, cgroups, socket UID/GID) and Kubernetes API to verify the pod's exact identity parameters (Image SHA, PID, Namespace, Service Account) before issuing an X.509 SVID (SPIFFE Verifiable Identity Document).

3. The Fusion: Cryptographically Anchored eBPF Policies

By coupling SPIRE with Cilium, identity validation shifts from static, dynamic label mappings to cryptographically attested SPIFFE IDs.

SPIRE Agent maintains local X.509 SVIDs and exports them over a UNIX Domain Socket (/run/spire/sockets/agent.sock).
Cilium Agent interacts with the SPIRE Workload API to read SVIDs and map verified SPIFFE IDs directly to internal Cilium Security Identities.
The underlying eBPF maps are populated only when SPIRE successfully attests the workload and issues an SVID. If a pod fails attestation, its identity is denied at the kernel level before a single network packet can leave the node.

Multi-Region Architecture Topology

Our production reference architecture spans two AWS regions: us-east-1 (Primary) and eu-west-1 (Secondary).

                      +---------------------------------------+
                      |       SPIRE Trust Federation          |
                      |  Exchanges keys/bundles over TLS      |
                      +-------------------+-------------------+
                                          |
             +----------------------------+----------------------------+
             |                                                         |
             v                                                         v
+------------------------------------+                    +------------------------------------+
| EKS Cluster: us-east-1             |                    | EKS Cluster: eu-west-1             |
| Trust Domain: us-east.ecstaticloud |                    | Trust Domain: eu-west.ecstaticloud |
|                                    |                    |                                    |
| +--------------------------------+ |                    | +--------------------------------+ |
| | SPIRE Server (Primary)         | |                    | | SPIRE Server (Secondary)       | |
| +---------------+----------------+ |                    | +---------------+----------------+ |
|                 |                  |                    |                 |                  |
|                 v                  |                    |                 v                  |
| +--------------------------------+ |  Cilium ClusterMesh| +--------------------------------+ |
| | Cilium Agent + SPIRE Agent     | |<------------------>| | Cilium Agent + SPIRE Agent     | |
| | (eBPF Encapsulation / WireGuard| | WireGuard Tunnel   | | (eBPF Encapsulation / WireGuard| |
| +--------------------------------+ |                    | +--------------------------------+ |
+------------------------------------+                    +------------------------------------+

Network Topology Details

Underlay Networking: AWS VPC Peering or AWS Transit Gateway establishes cross-region IP connectivity between VPC subnets without NAT.
Overlaid Control & Data Planes:
- Cilium ClusterMesh: Links the eBPF control planes across both EKS clusters. Cross-cluster pod-to-pod traffic is encrypted in-transit using Native eBPF WireGuard transparent encryption.
- SPIRE Federation: SPIRE Servers in us-east-1 and eu-west-1 federate trust bundles. Pods in us-east-1 can validate X.509 SVIDs presented by pods in eu-west-1 without hitting a centralized identity bottleneck.

Step-by-Step Implementation Guide

Prerequisites

Two EKS Clusters running Kubernetes v1.28+ (eks-us-east-1, eks-eu-west-1).
AWS IAM roles configured for Service Accounts (IRSA).
helm, kubectl, and cilium-cli installed locally.

Step 1: Deploy SPIRE Server and Agent with AWS Node Attestation

Deploy SPIRE Server in us-east-1 (spiffe://us-east.ecstaticloud). Configure it to use the AWS Instance Identity Document (aws_iid) attestor for hardware-level node identity verification.

Save the following configuration manifest as spire-server-config.yaml:

apiVersion: v1
kind: ConfigMap
metadata:
  name: spire-server
  namespace: spire
data:
  spire-server.conf: |
    server {
      bind_address = "0.0.0.0"
      bind_port = "8081"
      trust_domain = "us-east.ecstaticloud"
      data_dir = "/run/spire/data"
      log_level = "INFO"
      ca_ttl = "168h"
      default_x509_svid_ttl = "1h"
    }

    plugins {
      DataStore "sql" {
        plugin_data {
          database_type = "sqlite3"
          connection_string = "/run/spire/data/datastore.sqlite3"
        }
      }

      NodeAttestor "aws_iid" {
        plugin_data {
          accountId = "123456789012"
          cluster = "eks-us-east-1"
          partition = "aws"
        }
      }

      KeyManager "disk" {
        plugin_data {
          keys_path = "/run/spire/data/keys"
        }
      }

      UpstreamAuthority "aws_pca" {
        plugin_data {
          region = "us-east-1"
          certificate_authority_arn = "arn:aws:acm-pca:us-east-1:123456789012:certificate-authority/a1b2c3d4-5678-90ab-cdef-111122223333"
        }
      }
    }

Now, configure the SPIRE Agent DaemonSet configuration to expose its API via a shared hostPath directory accessible by Cilium:

apiVersion: v1
kind: ConfigMap
metadata:
  name: spire-agent
  namespace: spire
data:
  spire-agent.conf: |
    agent {
      data_dir = "/run/spire"
      log_level = "INFO"
      server_address = "spire-server.spire.svc"
      server_port = "8081"
      socket_path = "/run/spire/sockets/agent.sock"
      trust_domain = "us-east.ecstaticloud"
    }

    plugins {
      NodeAttestor "aws_iid" {
        plugin_data {
          partition = "aws"
        }
      }

      KeyManager "memory" {}

      WorkloadAttestor "k8s" {
        plugin_data {
          node_name_env = "MY_NODE_NAME"
          skip_kubelet_verification = true
        }
      }
    }

Step 2: Deploy Cilium with SPIRE Socket Integration & WireGuard Encryption

Deploy Cilium via Helm on the eks-us-east-1 cluster. We will disable kube-proxy, enable eBPF replacement, enable transparent WireGuard encryption, and configure SPIRE workload integration.

Create cilium-values.yaml:

cluster:
  name: eks-us-east-1
  id: 1

kubeProxyReplacement: true

# Enable Transparent Network Encryption via eBPF WireGuard
encryption:
  enabled: true
  type: wireguard

# Enable Hubble Observability Framework
hubble:
  enabled: true
  relay:
    enabled: true
  ui:
    enabled: true

# Configure Authentication & SPIFFE/SPIRE Integration
authentication:
  enabled: true
  mutual:
    spire:
      enabled: true
      install:
        enabled: false # Use externally deployed SPIRE instance
      agentSocketPath: /run/spire/sockets/agent.sock

# Host path mount for SPIRE Workload API socket
extraMounts:
  - name: spire-agent-socket
    mountPath: /run/spire/sockets
    hostPath: /run/spire/sockets
    readOnly: true

Install Cilium using Helm:

helm repo add cilium https://helm.cilium.io/
helm install cilium cilium/cilium \
  --version 1.15.2 \
  --namespace kube-system \
  -f cilium-values.yaml

Step 3: Establish Cross-Region SPIRE Federation & Cilium ClusterMesh

To extend Zero-Trust boundaries across clusters, federate the us-east-1 and eu-west-1 SPIRE Servers.

1. SPIRE Federation Declaration

Apply this declaration on the us-east-1 SPIRE server to establish federated trust with eu-west-1:

apiVersion: spire.spiffe.io/v1alpha1
kind: ClusterFederatedTrustDomain
metadata:
  name: eu-west.ecstaticloud
spec:
  trustDomain: eu-west.ecstaticloud
  bundleEndpointURL: https://spire-federation.eu-west.ecstaticloud.io:8443
  bundleEndpointProfile:
    https_spiffe:
      endpointSPIFFEID: spiffe://eu-west.ecstaticloud/spire/server

2. Enable Cilium ClusterMesh

Connect the two clusters' eBPF control planes using the Cilium CLI:

# Export kubeconfig contexts
export CTX_US="arn:aws:eks:us-east-1:123456789012:cluster/eks-us-east-1"
export CTX_EU="arn:aws:eu-west-1:123456789012:cluster/eks-eu-west-1"

# Enable ClusterMesh on both environments
cilium clustermesh enable --context $CTX_US
cilium clustermesh enable --context $CTX_EU

# Connect the clusters over secure WireGuard cross-region link
cilium clustermesh connect --context $CTX_US --destination-context $CTX_EU

Verify status across the fabric:

cilium clustermesh status --context $CTX_US

Expected Output:

✅ ClusterMesh is ready
✨ Nodes: 12/12 online
🔒 Encryption: WireGuard (Active)
🌐 Combined identities: 48 identities synchronized across 2 clusters

Step 4: Authoring Fine-Grained Cryptographic Network Policies

Now that our network overlay, cryptographic identity provider, and eBPF kernel paths are established, we can implement Level 3 through Level 7 Microsegmentation Policies.

The policy below enforces the following security posture:

Cryptographic Identity Matching: Only workloads holding an SVID issued to spiffe://us-east.ecstaticloud/ns/prod/sa/payment-service OR spiffe://eu-west.ecstaticloud/ns/prod/sa/payment-service are permitted to initiate connections.
L7 Protocol Restriction: Traffic is restricted to strictly HTTP POST requests to /v2/ledger/credit. All other HTTP paths or methods (e.g., DELETE, GET /admin) are dropped instantly by eBPF/Envoy in kernel-space.

apiVersion: "cilium.io/v2"
kind: CiliumClusterwideNetworkPolicy
metadata:
  name: enforce-payment-to-ledger-strict-zero-trust
spec:
  description: "Strict L3-L7 Zero-Trust enforcement between Payment API and Ledger Service across regions"
  endpointSelector:
    matchLabels:
      app.kubernetes.io/name: ledger-service
      io.kubernetes.pod.namespace: prod
  
  ingress:
    # 1. Cryptographic Authentication Layer (SPIFFE)
    - authentication:
        mode: "spire"
      
      # 2. Match Network Identities across federated regions
      fromEndpoints:
        - matchLabels:
            "k8s:io.cilium.k8s.policy.serviceaccount": payment-service
            "k8s:io.kubernetes.pod.namespace": prod
      
      # 3. Layer 7 Protocol Level Enforcement
      toPorts:
        - ports:
            - port: "8443"
              protocol: TCP
          rules:
            http:
              - method: "POST"
                path: "/v2/ledger/credit"
                headers:
                  - "X-Client-Region: (us-east-1|eu-west-1)"

Operational Best Practices, Performance & Benchmarks

1. Packet Processing Overhead: eBPF vs. Legacy Security

Transitioning from traditional IP/iptables-based network security to Cilium eBPF + SPIRE delivers significant structural performance improvements.

2. High-Availability & Failure Modes

When running zero-trust at scale, you must plan for control-plane outages.

SPIRE Server Outages: If the SPIRE Server becomes unreachable, SPIRE Agents continue serving cached SVIDs until their expiration.
- Setting SVID TTL: Tune default_x509_svid_ttl to 1h. This guarantees a safe window for recovery without leaving wide security windows open.
- Cilium eBPF Map Resilience: BPF maps reside inside the kernel. Even if both the Cilium Agent and SPIRE Agent pods drop offline, existing network connections and existing identity maps in the kernel continue filtering packets uninterrupted.
WireGuard Key Rotation: Enable automatic WireGuard key rotation in Cilium via Helm values:
```
encryption:
  wireguard:
    persistentKeepalive: 10s
    userspaceFallback: false
```

3. Deep Observability via Hubble CLI

Verify network policies and cryptographic handshakes in real time using the Cilium Hubble CLI.

To stream all dropped packets attempting to breach our strict L7 policy across clusters:

hubble observe \
  --namespace prod \
  --label app.kubernetes.io/name=ledger-service \
  --verdict DROPPED \
  --follow

Example Live Output:

TIMESTAMP             SOURCE                                    DESTINATION                              TYPE          VERDICT   REASON
2024-10-24T14:02:11Z  prod/rogue-pod-58df8-x9p2q (10.0.14.22)   prod/ledger-service-10a-z8x (10.0.8.11)  HTTP/REST     DROPPED   Policy denied (HTTP POST /v1/admin/reset)
2024-10-24T14:02:15Z  prod/payment-service-1a-12q (10.2.1.44)   prod/ledger-service-10a-z8x (10.0.8.11)  Policy Auth   DROPPED   SPIFFE SVID validation failed: Expired Token

Conclusion & Key Takeaways

Combining Cilium eBPF with SPIFFE/SPIRE fundamentally redefines cloud-native security architecture. By moving away from brittle IP ranges, security groups, and CPU-intensive user-space sidecar proxies, you build an architecture capable of supporting enterprise workloads across multi-region EKS environments without compromising performance.

Summary Checklist for Production Readiness

Kernel Enforcement: Move all packet filtering and encapsulation out of user-space into kernel-level eBPF TC/XDP hooks.
Hardware Attestation: Use AWS Instance Identity Documents (aws_iid) in SPIRE to tie node identities directly to hardware and IAM primitives.
Federated Identity: Federate SPIRE trust domains across regions so microservices can validate X.509 SVIDs locally without regional cross-dependencies.
Fine-Grained Segmentation: Combine identity-based L3/L4 filters with dynamic L7 HTTP rules in unified CiliumNetworkPolicy declarations.

By implementing this architecture, your infrastructure satisfies the strictest requirements of NIST SP 800-207 Zero-Trust Framework: identity is verified dynamically, access is granular and least-privileged, and performance remains blazing fast at cloud scale.