Flannel, flannel, flannel…

Flannel is a simple and popular CNI 😢.

It’s the default CNI in k3s, the one half the kubeadm tutorials use, and you’ll find it in quite a few managed offerings too.

OK, it’s simple, it routes packets between pods, it supports VXLAN and WireGuard, it takes 2 minutes to set up. What more could you ask for?

Well, exactly. There’s one thing flannel does not do: NetworkPolicies. (And that’s a big deal).

Flannel is focused on networking. For network policy, other projects such as Calico can be used.

[Edit] Contrary to what I wrote here, flannel actually does support NetworkPolicies and has for at least 2 years, via the reference implementation kube-network-policies from the Kubernetes project. The option isn’t prominently featured in the README and is probably not more recommended for production, but it does exist. See the official documentation.

And the trap is that if you haven’t read that one little line in the README.md, nothing tells you explicitly.

You can perfectly well create NetworkPolicy objects in your cluster, kubectl apply won’t complain, kubectl get netpol will happily list them. Except… they’re not enforced. Traffic still flows. Your deny-all doesn’t deny anything at all.

Let’s verify to be sure

Before solving anything, let’s verify the problem exists. Starting from the prerequisite that we have a Kubernetes cluster with flannel as the CNI and everything is working, we’ll deploy two pods in two different namespaces: a client (curl) and a server (nginx).

Let’s check that the client can reach the server:

kubectl exec -n netpol-test-a client -- \
  curl -s --max-time 5 http://server.netpol-test-b.svc.cluster.local

We get the nginx welcome page. So far, so normal. Now, let’s apply a deny-all NetworkPolicy on the server’s namespace:

# 01-deny-all-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: deny-all-ingress
  namespace: netpol-test-b
spec:
  podSelector: {}
  policyTypes:
    - Ingress

kubectl apply -f 01-deny-all-ingress.yaml

And let’s test again:

kubectl exec -n netpol-test-a client -- \
  curl -s --max-time 5 http://server.netpol-test-b.svc.cluster.local

Result: the nginx page still shows up. The NetworkPolicy was created (kubectl get netpol -n netpol-test-b shows it), but it’s not enforced. Traffic flows as if nothing happened.

This is the expected behavior with flannel (by default). Flannel only does L3 routing (VXLAN or WireGuard overlay). It doesn’t implement a NetworkPolicy controller. The objects exist in etcd, but nobody translates them into filtering rules.

Let’s clean up the policy before moving on:

kubectl delete -f 01-deny-all-ingress.yaml

Alternatives for adding support

For a long time I thought this was just the way it was. I still think it’s not a good choice for any production.

BUT recently, I discovered that it was possible to chain CNIs within the same cluster, and thus have one CNI handling most tasks (here Flannel) and another one taking care of other tasks, such as enforcing NetworkPolicies.

This is actually the principle behind Canal, which I knew by name but had never explored. In fact, it’s nothing more than a manifest that deploys Flannel as the main CNI with Calico for NetworkPolicy enforcement!

Canal was the name of Tigera and CoreOS’s project to integrate Calico and flannel.

Note: the GitHub project was archived in October 2025 but in theory it should still work, if you can find the correct documentation (I couldn’t, the links are broken, but I didn’t really look that hard either).

So you get the idea: we’re going to add a component that will watch NetworkPolicy objects and translate them into effective filtering rules (iptables, eBPF, nftables…), without touching the existing flannel setup. This is called “CNI chaining” or “policy-only” mode.

There are several solutions:

Calico (Canal); Historically, the flannel + Calico combination is called “Canal”, which I just mentioned. But the official Canal manifest bundles its own flannel in the same DaemonSet as calico-node. If your flannel is already installed and managed (by you, by an operator, by a provider…), you probably don’t want to replace it. And the Tigera operator (the “official” Helm method) doesn’t support policy-only deployment on an existing flannel either. In short, it’s doable but requires some effort. Can’t be bothered.

kube-router; kube-router can run in firewall-only mode (--run-firewall=true) and only needs iptables/ipset. This is actually what k3s uses by default for NetworkPolicies. It’s the lightest solution (supposedly ~50 MB of RAM per node). Make sure your kernel has the ip_set module, otherwise it won’t work.

Cilium in CNI chaining mode; This is the solution I chose and that we’ll detail here. Cilium attaches to the veth interfaces created by flannel and adds its eBPF programs for policy enforcement. No dependency on iptables or ipset, and as a bonus we get Hubble for network observability.

Installing Cilium in CNI chaining mode

Prerequisites

• A working Kubernetes cluster with flannel
• helm v3+
• A kernel >= 4.19 (ideally >= 5.10 for all eBPF features)

Retrieve flannel’s CNI configuration

Cilium in chaining mode needs to know the existing CNI configuration. Let’s retrieve it from a node:

kubectl debug node/$(kubectl get nodes -o jsonpath='{.items[0].metadata.name}') \
  -it --image=busybox -- cat /host/etc/cni/net.d/10-flannel.conflist

On my cluster, this gives:

{
  "name": "cbr0",
  "cniVersion": "0.3.1",
  "plugins": [
    {
      "type": "flannel",
      "delegate": {
        "hairpinMode": true,
        "isDefaultGateway": true
      }
    },
    {
      "type": "portmap",
      "capabilities": {
        "portMappings": true
      }
    }
  ]
}

Note the name field (here cbr0). We’ll need it.

Create the chaining ConfigMap

We’ll create a ConfigMap that takes the flannel config and adds the cilium-cni plugin in chaining mode:

apiVersion: v1
kind: ConfigMap
metadata:
  name: cni-configuration
  namespace: kube-system
data:
  cni-config: |-
    {
      "name": "cbr0",
      "cniVersion": "0.3.1",
      "plugins": [
        {
          "type": "flannel",
          "delegate": {
            "hairpinMode": true,
            "isDefaultGateway": true
          }
        },
        {
          "type": "portmap",
          "capabilities": {
            "portMappings": true
          }
        },
        {
          "type": "cilium-cni",
          "chaining-mode": "generic-veth"
        }
      ]
    }

Warning: the name field must match your flannel conflist. If yours is called cni0 or something else, adjust accordingly.

Before applying, also verify that your CNI uses veth interfaces (this is the default with flannel, but better safe than sorry). From a node:

ip -d link | grep veth

You should see veth-type interfaces corresponding to your pods, for example:

103: lxcb3901b7f9c02@if102: <BROADCAST,MULTICAST,UP,LOWER_UP> ...
    veth addrgenmode eui64 numtxqueues 1 numrxqueues 1

If that’s the case, Cilium’s generic-veth mode will work. Let’s apply:

kubectl apply -f cilium-cni-configmap.yaml

Install Cilium via Helm

helm repo add cilium https://helm.cilium.io/
helm repo update

Here are the values for chaining mode:

# cilium-values.yaml
cni:
  chainingMode: generic-veth
  customConf: true
  configMap: cni-configuration
  install: true

routingMode: native
enableIPv4Masquerade: false
enableIPv6Masquerade: false

hubble:
  enabled: true
  relay:
    enabled: true
  ui:
    enabled: true

The important points:

• cni.chainingMode: generic-veth, this is chaining mode, Cilium attaches to existing veth interfaces
• cni.customConf: true + cni.configMap, we provide our own CNI config
• routingMode: native, flannel handles routing, not Cilium
• enableIPv4Masquerade: false, flannel handles masquerading
• hubble.enabled: true, network observability, the big bonus of Cilium

helm install cilium cilium/cilium --version 1.19.1 \
  --namespace kube-system \
  -f cilium-values.yaml

Wait for everything to be ready:

kubectl rollout status daemonset/cilium -n kube-system --timeout=120s

Verification

kubectl exec -n kube-system ds/cilium -- cilium status

What we’re looking for:

Kubernetes:              Ok                1.35 (v1.35.0) [linux/amd64]
CNI Chaining:            generic-veth
Cilium:                  Ok                1.19.1
Hubble:                  Ok

The CNI Chaining: generic-veth line confirms that Cilium is running in chaining mode and isn’t replacing flannel.

Important: pods that existed before Cilium was installed are not automatically managed by Cilium. You need to restart them so that Cilium can attach its eBPF programs. Remember to kubectl rollout restart your test workloads (or recreate them).

Testing NetworkPolicies

This is the moment of truth. Let’s re-apply our deny-all:

kubectl apply -f 01-deny-all-ingress.yaml

kubectl exec -n netpol-test-a client -- \
  curl -s --max-time 5 http://server.netpol-test-b.svc.cluster.local

Result: timeout! This time, the NetworkPolicy is properly enforced. Traffic is blocked.

I followed up with the other classic NetworkPolicy scenarios, and everything works:

• Selective ingress allow by namespace; by adding a policy that allows traffic from netpol-test-a only, curl works from that namespace but remains blocked from default. Namespace isolation works.
• Deny-all egress; by blocking all outgoing traffic from the client, even DNS resolution is blocked (immediate timeout).
• Selective egress allow; by allowing only DNS (port 53) and the server (port 80 in the netpol-test-b namespace), curl to the server works but curl http://example.com remains blocked.

In short, ingress, egress, selective by namespace, everything works as expected.

Bonus: Hubble, network observability

This is for me the real advantage of Cilium over the alternatives. Hubble lets you see network flows and policy verdicts in real time:

kubectl exec -n kube-system ds/cilium -- \
  hubble observe --namespace netpol-test-b --last 5

Mar 15 13:20:42.287: netpol-test-a/client:40066 (ID:9745) ->
  netpol-test-b/server:80 (ID:22271)
  policy-verdict:none ALLOWED (TCP Flags: SYN)

You can see the source pod, destination pod, port, Cilium identity, and policy verdict. When you’re debugging a NetworkPolicy that isn’t behaving as expected, this is incredibly useful.

How much does it cost in resources?

On my cluster (3 nodes), here’s what Cilium consumes right after installation:

That’s roughly 180 MB per node for the agent + envoy. I don’t have a point of comparison with Calico or kube-router, but it seems acceptable to me, and being able to have full visibility into all network flows with Hubble more than justifies the overhead (in my opinion).

Conclusion

If you have no choice and have to work with flannel, and you want to plug the gaping security hole that is the lack of NetworkPolicy enforcement, know that it is possible to chain another CNI to handle it.

Short of having it as CNI for everything, Cilium in CNI chaining mode (generic-veth) is a pretty nice solution to fill this gap. It doesn’t touch the existing flannel setup, it grafts onto it. And as a bonus, you get Hubble for network observability, which is genuinely valuable.