CareerByteCode
ByteBlogby CareerByteCode

Cilium CNI: eBPF Networking and Security for Kubernetes

What you build: Cilium network policies on GKE Dataplane V2: default deny, dropped packets seen in the kernel, egress control and a DNS mistake fixed. Every step tested.

Cilium CNI: eBPF Networking and Security for Kubernetes

✍️ Translate this blog into Hindi, Tamil or Telugu

Cilium is the Kubernetes networking layer built on eBPF: small programs that run inside the Linux kernel and decide, packet by packet, what may pass. Google uses Cilium for GKE Dataplane V2, so if you create a GKE cluster with Dataplane V2 you are already running Cilium. In this lab you check that, lock a namespace down with network policies, watch the kernel drop forbidden packets, control outgoing traffic, break a policy in a way that catches many teams (DNS), and find out which Cilium features GKE's managed version does not offer.

Who it is for: Kubernetes users who want to understand pod networking and network policies. Time: about 1 hour. Cost: a small GKE cluster for about 10 minutes, covered by the free trial credit. Delete it at the end.

Every command below was run for real by the CareerByteCode Team, and every output is the real result we got. We typed the commands in a Cloud Shell-like Linux VM (Debian with gcloud and kubectl); Cloud Shell in your browser has the same tools.

🧪⭐ Recommended by CareerByteCode · ByteLabs project bundleCloud and DevOps Realtime Starterโ‚น499Build the projects →

Part A: Cilium, eBPF and network policies in plain English

By default every pod in Kubernetes can talk to every other pod. A NetworkPolicy says who may talk to whom. The CNI (the network plugin) enforces it. Older plugins use long lists of iptables rules; Cilium loads eBPF programs into the kernel, which is faster at scale and can tell you exactly which packet it dropped and why.

Network policies with Cilium
Three pods in one namespace: the policies allow frontend to backend, block the intruder, and limit the frontend's outgoing traffic

Words you will use:

  • eBPF: a safe way to run small programs inside the Linux kernel. Cilium uses them to route, filter and observe packets.
  • Dataplane V2: GKE's networking built on Cilium. On GKE the Cilium agent runs as the DaemonSet anetd (one per node), managed by Google.
  • Ingress / egress: traffic coming into a pod / going out of a pod.
  • Default deny: a policy that selects every pod and allows nothing, so only what later rules allow can pass.
  • Identity: Cilium gives each set of pod labels a number and filters by that number, not by IP addresses that change.
  • L3/L4 vs L7: standard NetworkPolicy filters by pod, IP and port (layers 3 and 4). Cilium's own CiliumNetworkPolicy can also filter HTTP methods and paths (layer 7).
  • Real-life example: a NetworkPolicy is the guest list at a building's door. eBPF is a guard who checks every visitor in a split second and writes down everyone turned away.

Before you start

  1. A Google Cloud project with billing linked (the free trial works) and Cloud Shell open.
  2. One small GKE cluster with Dataplane V2 (--enable-dataplane-v2). We created it once for this batch of Kubernetes labs:
gcloud container clusters create lab-cluster \
  --zone=us-east1-b --num-nodes=2 --machine-type=e2-standard-4 \
  --disk-type=pd-standard --disk-size=50 --release-channel=regular \
  --no-enable-autoupgrade --enable-dataplane-v2

Part B: the lab

Step 1: is Cilium really there?

Why: ADVANCED_DATAPATH is GKE's name for Dataplane V2. The anetd DaemonSet runs one Cilium agent per node, and the agent reports its Cilium version.

# The cluster runs GKE Dataplane V2: Cilium (eBPF) does the networking. On GKE its agent is the "anetd" DaemonSet
gcloud container clusters get-credentials lab-cluster --zone=us-east1-b > /dev/null 2>&1
gcloud container clusters describe lab-cluster --zone=us-east1-b --format="value(networkConfig.datapathProvider)"
kubectl -n kube-system get ds anetd -o custom-columns=NAME:.metadata.name,READY:.status.numberReady,NODES:.status.desiredNumberScheduled
kubectl -n kube-system exec ds/anetd -c cilium-agent -- cilium version 2>/dev/null | head -2
ADVANCED_DATAPATH
NAME    READY   NODES
anetd   2       2
Client: 1.19.4 ad1290bf77 2026-09-10T00:38:30+00:00 go version go1.26.8 linux/amd64
Daemon: 1.19.4 ad1290bf77 2026-09-10T00:38:30+00:00 go version go1.26.8 linux/amd64

Step 2: three workloads, no rules yet

Why: we need someone allowed and someone not. backend is a tiny web API; frontend is allowed to call it; intruder is not. Save this as apps.yaml:

# Three workloads in namespace netpol: a backend API, a frontend allowed to call it, and an "intruder" that should not
apiVersion: apps/v1
kind: Deployment
metadata: {name: backend, namespace: netpol}
spec:
  replicas: 1
  selector: {matchLabels: {app: backend}}
  template:
    metadata: {labels: {app: backend}}
    spec:
      containers:
      - {name: api, image: hashicorp/http-echo:1.0, args: ["-text=hello from backend", "-listen=:5678"], resources: {requests: {cpu: 10m, memory: 16Mi}}}
---
apiVersion: v1
kind: Service
metadata: {name: backend, namespace: netpol}
spec: {selector: {app: backend}, ports: [{port: 80, targetPort: 5678}]}
---
apiVersion: v1
kind: Pod
metadata: {name: frontend, namespace: netpol, labels: {role: frontend}}
spec:
  containers: [{name: c, image: curlimages/curl:8.11.1, command: ["sleep", "3600"]}]
---
apiVersion: v1
kind: Pod
metadata: {name: intruder, namespace: netpol, labels: {role: other}}
spec:
  containers: [{name: c, image: curlimages/curl:8.11.1, command: ["sleep", "3600"]}]
# Three workloads: backend, frontend, intruder. With no policy, everything can reach everything
kubectl create namespace netpol > /dev/null && kubectl apply -f apps.yaml > /dev/null
kubectl -n netpol rollout status deploy/backend --timeout=120s > /dev/null && kubectl -n netpol wait pod/frontend pod/intruder --for=condition=Ready --timeout=120s > /dev/null
for p in frontend intruder; do echo "$p -> backend: $(kubectl -n netpol exec $p -- curl -s --max-time 3 http://backend/ || echo 'BLOCKED (timeout)')"; done
frontend -> backend: hello from backend
intruder -> backend: hello from backend
Cilium on GKE
Step 1 and 2: Dataplane V2, one Cilium agent per node, and no rules yet

Without a policy, the intruder reaches the backend just as easily as the frontend.

Step 3: default deny, then allow only the frontend

Why: the safe pattern is "deny everything, then allow what is needed". The first policy selects every pod (podSelector: {}) and allows no ingress. The second allows ingress to the backend only from pods labelled role: frontend, only on port 5678. Save this as policies.yaml:

# 1) deny all ingress to every pod in the namespace, 2) allow only frontend pods to reach the backend on port 5678
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata: {name: default-deny-ingress, namespace: netpol}
spec:
  podSelector: {}
  policyTypes: [Ingress]
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata: {name: allow-frontend-to-backend, namespace: netpol}
spec:
  podSelector: {matchLabels: {app: backend}}
  policyTypes: [Ingress]
  ingress:
  - from: [{podSelector: {matchLabels: {role: frontend}}}]
    ports: [{port: 5678, protocol: TCP}]
# Default deny, then allow only the frontend to call the backend on port 5678
kubectl apply -f policies.yaml && sleep 10
for p in frontend intruder; do echo "$p -> backend: $(kubectl -n netpol exec $p -- curl -s --max-time 3 http://backend/ || echo 'BLOCKED (timeout)')"; done
networkpolicy.networking.k8s.io/default-deny-ingress created
networkpolicy.networking.k8s.io/allow-frontend-to-backend created
frontend -> backend: hello from backend
command terminated with exit code 28
intruder -> backend: BLOCKED (timeout)

The frontend still works; the intruder's request times out (curl exit code 28 = timeout). A NetworkPolicy drop is silent: the packet is simply discarded, so the caller waits until it gives up.

Step 4: watch the kernel drop the packets

Why: "it times out" is not a reason. Cilium can show every dropped packet with the reason. We run cilium monitor --type drop in the agent on the backend's node while the intruder tries again:

# See the drops in Cilium itself: watch policy drops on the backend's node while the intruder tries again
NODE=$(kubectl -n netpol get pod -l app=backend -o jsonpath='{.items[0].spec.nodeName}'); AGENT=$(kubectl -n kube-system get pod -l k8s-app=cilium -o jsonpath="{.items[?(@.spec.nodeName=='$NODE')].metadata.name}")
[ -z "$AGENT" ] && AGENT=$(kubectl -n kube-system get pod -o wide | grep anetd | grep $NODE | awk '{print $1}')
(kubectl -n kube-system exec $AGENT -c cilium-agent -- timeout 12 cilium monitor --type drop 2>/dev/null > drops.txt &) ; sleep 3
kubectl -n netpol exec intruder -- curl -s --max-time 3 http://backend/ > /dev/null || echo "intruder: blocked"
sleep 10; grep -m 2 -E "Policy denied|drop" drops.txt | sed -E 's/^xx drop //' | cut -c1-180
echo "drop events seen: $(grep -c -E 'Policy denied|drop' drops.txt)"
command terminated with exit code 28
intruder: blocked
(Policy denied) flow 0xaee432c4 to endpoint 935, ifindex 185, file bpf_lxc.c:2877, , identity 28009->54100: 10.8.1.93:43282 -> 10.8.1.91:5678 tcp SYN
(Policy denied) flow 0x25502591 to endpoint 935, ifindex 185, file bpf_lxc.c:2877, , identity 28009->54100: 10.8.1.93:43282 -> 10.8.1.91:5678 tcp SYN
drop events seen: 3

Policy denied from the eBPF program bpf_lxc.c, with both identities (28009 for the intruder's labels, 54100 for the backend's) and the TCP SYN (the very first packet of the connection). This is the evidence you want when someone asks "is the network blocking us?".

Policies and drops
Default deny: the frontend passes, the intruder times out, and Cilium shows the exact dropped SYN packets

Step 5: control outgoing traffic (egress)

Why: if a frontend pod is hacked, the attacker will try to reach the internet. An egress policy allows only what the pod needs. Our first version allows DNS to the kube-dns pods and the backend on port 5678. Save this as egress.yaml:

# Egress control for the frontend: it may talk to DNS and to the backend only (no internet)
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata: {name: frontend-egress, namespace: netpol}
spec:
  podSelector: {matchLabels: {role: frontend}}
  policyTypes: [Egress]
  egress:
  - to: [{namespaceSelector: {}, podSelector: {matchLabels: {k8s-app: kube-dns}}}]
    ports: [{port: 53, protocol: UDP}, {port: 53, protocol: TCP}]
  - to: [{podSelector: {matchLabels: {app: backend}}}]
    ports: [{port: 5678, protocol: TCP}]
# Egress control: the frontend may reach DNS and the backend, but not the internet
kubectl apply -f egress.yaml && sleep 10
echo "frontend -> backend: $(kubectl -n netpol exec frontend -- curl -s --max-time 3 http://backend/ || echo BLOCKED)"
echo "frontend -> internet: $(kubectl -n netpol exec frontend -- curl -s -o /dev/null -w '%{http_code}' --max-time 4 https://www.google.com/ || echo ' BLOCKED (timeout)')"
echo "intruder -> internet: $(kubectl -n netpol exec intruder -- curl -s -o /dev/null -w '%{http_code}' --max-time 4 https://www.google.com/ || echo ' BLOCKED')"
networkpolicy.networking.k8s.io/frontend-egress created
command terminated with exit code 28
frontend -> backend: BLOCKED
command terminated with exit code 28
frontend -> internet: 000 BLOCKED (timeout)
intruder -> internet: 200

The internet is blocked, as planned. But the frontend's call to the backend is blocked too, which we did not want.

Break it: the egress rule broke the app

This is one of the most common network policy mistakes. Let us find out why instead of guessing. The call needs two things: a DNS answer for backend, then a connection to the pod:

# Why is frontend -> backend blocked now? Test the two things the call needs: a DNS answer, then a TCP connection
echo "DNS lookup of backend from frontend: $(kubectl -n netpol exec frontend -- nslookup -timeout=3 backend.netpol.svc.cluster.local 2>&1 | grep -m1 -oE 'Address: 10\.[0-9.]+|timed out|no servers could be reached' || echo failed)"
BIP=$(kubectl -n netpol get pod -l app=backend -o jsonpath='{.items[0].status.podIP}')
echo "frontend -> backend pod IP directly: $(kubectl -n netpol exec frontend -- curl -s --max-time 3 http://$BIP:5678/ || echo BLOCKED)"
kubectl -n netpol exec frontend -- cat /etc/resolv.conf | grep nameserver
DNS lookup of backend from frontend: timed out
no servers could be reached
failed
frontend -> backend pod IP directly: hello from backend
nameserver 34.118.224.10

The DNS lookup timed out, but a direct call to the backend pod's IP works. So the backend rule is fine, and the DNS rule does not match the way DNS is reached here. The pod's resolver is 34.118.224.10, the address of the cluster's DNS Service, and on this cluster our rule "to pods labelled k8s-app: kube-dns" did not allow that traffic. We did not dig deeper into why; the fix below works without depending on it.

Fix it: allow DNS to any destination on port 53

Port 53 is DNS only. Allowing it to any destination is the common, safe pattern, and it does not depend on how the cluster's DNS is reached. Save this as egress-fixed.yaml:

# Egress control for the frontend, version 2: DNS on port 53 to any destination, the backend on 5678, nothing else
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata: {name: frontend-egress, namespace: netpol}
spec:
  podSelector: {matchLabels: {role: frontend}}
  policyTypes: [Egress]
  egress:
  - ports: [{port: 53, protocol: UDP}, {port: 53, protocol: TCP}]
  - to: [{podSelector: {matchLabels: {app: backend}}}]
    ports: [{port: 5678, protocol: TCP}]
📝⭐ Recommended by CareerByteCode · Certification examCIPS Level 5 Advanced Diploma in Procurement and Supply PracticeTake the examTake the exam →
# Fix it: allow DNS (port 53) to ANY destination, so the rule does not depend on how the cluster's DNS is reached
kubectl apply -f egress-fixed.yaml && sleep 10
echo "frontend -> backend: $(kubectl -n netpol exec frontend -- curl -s --max-time 3 http://backend/ || echo BLOCKED)"
echo "frontend -> internet: $(kubectl -n netpol exec frontend -- curl -s -o /dev/null -w '%{http_code}' --max-time 4 https://www.google.com/ || echo ' BLOCKED (timeout)')"
networkpolicy.networking.k8s.io/frontend-egress configured
frontend -> backend: hello from backend
command terminated with exit code 28
frontend -> internet: 000 BLOCKED (timeout)

The frontend reaches the backend again, and the internet stays blocked.

Break it and fix it
The egress rule blocked DNS; diagnosed in two commands; fixed with port 53 to any destination

Step 6: Cilium's own L7 policies on GKE?

Why: self-managed Cilium can filter HTTP methods and paths with a CiliumNetworkPolicy, for example "the frontend may only call GET /health". Save this as l7.yaml:

# A Cilium L7 (HTTP) rule: allow only GET /health from the frontend. Works with self-managed Cilium, see the post.
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata: {name: http-get-only, namespace: netpol}
spec:
  endpointSelector: {matchLabels: {app: backend}}
  ingress:
  - fromEndpoints: [{matchLabels: {role: frontend}}]
    toPorts:
    - ports: [{port: "5678", protocol: TCP}]
      rules:
        http: [{method: GET, path: "/health"}]
# Break it: try a Cilium L7 (HTTP method and path) policy. GKE Dataplane V2 does not offer Cilium's own policy types
kubectl apply -f l7.yaml 2>&1 | cut -c1-200
kubectl get crd -o name | grep -c cilium.io | sed 's/$/ Cilium policy CRDs installed on this cluster/'
error: resource mapping not found for name: "http-get-only" namespace: "netpol" from "l7.yaml": no matches for kind "CiliumNetworkPolicy" in version "cilium.io/v2"
ensure CRDs are installed first
5 Cilium policy CRDs installed on this cluster

GKE Dataplane V2 does not offer Cilium's own policy type: no matches for kind "CiliumNetworkPolicy". GKE installs a few Cilium object types for its own use, but not this one. For HTTP-level rules on GKE, use a service mesh or GKE's own features; with self-managed Cilium (installed by you, on a cluster without Dataplane V2) this policy is available. We did not test self-managed Cilium here.

Step 7: the final rules

# Fix it the GKE way: express what you can with standard NetworkPolicy (L3/L4). Check the final rules
kubectl -n netpol get networkpolicy -o custom-columns=NAME:.metadata.name,POD-SELECTOR:.spec.podSelector.matchLabels,TYPES:.spec.policyTypes
kubectl -n netpol describe networkpolicy allow-frontend-to-backend | grep -A6 "Allowing ingress" | sed 's/^ *//' | head -6
NAME                        POD-SELECTOR         TYPES
allow-frontend-to-backend   map[app:backend]     [Ingress]
default-deny-ingress        <none>               [Ingress]
frontend-egress             map[role:frontend]   [Egress]
Allowing ingress traffic:
To Port: 5678/TCP
From:
PodSelector: role=frontend
Not affecting egress traffic
Policy Types: Ingress
L7 and the final rules
CiliumNetworkPolicy is not available on GKE; the three standard policies in place

POC: prove it works

  • Dataplane V2 runs Cilium (anetd, Cilium 1.19) on every node (Step 1).
  • Default deny plus one allow rule let the frontend in and kept the intruder out (Step 3), and Cilium showed the dropped SYN packets with the reason (Step 4).
  • An egress rule blocked the internet; its first version also broke DNS, found and fixed (Step 5, Break it, Fix it).
  • CiliumNetworkPolicy (L7) is not available on GKE Dataplane V2 (Step 6).

Clean up (delete everything)

# Clean up: the namespace with its pods and policies
kubectl delete namespace netpol --wait=true 2>&1 | tail -1
rm -f drops.txt
namespace "netpol" deleted

When you have finished all your Kubernetes labs, delete the cluster:

gcloud container clusters delete lab-cluster --zone=us-east1-b --quiet

FAQ

Do I need to install Cilium on GKE? No. Create the cluster with --enable-dataplane-v2 and Google runs Cilium for you. You cannot change its settings or use its L7 policies.

Why did the intruder time out instead of getting "connection refused"? A policy drop throws the packet away without an answer, so the caller waits. That is why timeouts in a cluster with network policies often mean "a policy blocked this".

Should every namespace have default deny? In production, yes, together with explicit allow rules. Add egress rules too, and always allow DNS.

What is Hubble? Cilium's observability tool: a live map of who talks to whom and what was dropped. GKE offers it as "Dataplane V2 observability". We used cilium monitor in this lab instead.

Is eBPF faster than iptables? At scale, usually yes: iptables checks long rule lists in order, while eBPF uses hash maps. In a small lab you will not see a difference.

Troubleshooting

You seeWhyFix
Calls time out after adding a policyThe policy drops them silentlyCheck the drops with cilium monitor --type drop in the node's agent
Everything breaks after an egress policyDNS is blockedAllow port 53 (UDP and TCP) to any destination
no matches for kind "CiliumNetworkPolicy"GKE Dataplane V2 does not offer itUse standard NetworkPolicy, or self-managed Cilium
A policy seems to do nothingIts podSelector matches no podsCheck the labels: kubectl get pods --show-labels
anetd not foundThe cluster does not use Dataplane V2It cannot be switched on for an existing cluster; create a new one

What next

Prepared by the CareerByteCode Team from our tested lab, with AI help for the write-up.

Read next →Site-to-Site VPN Between Two Google Cloud Regions⏱ 13 min

⭐ Keep learning with CareerByteCode

Hand-picked to go deeper on this topic
News, guides and updates from the CareerByteCode team
🏅 Expert writer

The CareerByteCode team writes about what is new on careerbytecode.in: courses, labs, exams, events and the community, which is free to join for influencers and content creators in tech.

Get CareerByteCode Team's next blog by email

One email a week when they publish. Add the weekly Best of ByteBlog too. No spam, unsubscribe with one click.

We email you once to confirm. By subscribing you agree to the privacy policy.

💬 Comments (0)

Top · Newest

Sign in to comment

Related blogs

🧪⭐ Recommended by CareerByteCode · ByteLabs project bundleCloud and DevOps Realtime Starterโ‚น499Build the projects →