| pick your distro, get ZFS on root
kldload — your platform, your way, free
Source
Documentation

Networking on kldload

kldload systems use NetworkManager for network configuration across all supported distros. This guide starts with the basics — static IPs, bridges, VLANs, bonds, firewalls — then takes you into the modern networking stack: VXLAN overlays, BGP routing, eBPF dataplane programming, and multi-cloud connectivity. The old boundaries between "local network" and "cloud network" don't exist anymore. A packet on your homelab ZFS server and a packet in AWS traverse the same logical fabric if you build it that way.

kldload installs nine distros but ships one network stack: NetworkManager everywhere. CentOS, RHEL, Rocky, Fedora, Debian, Ubuntu, Alpine, Arch — all get nmcli. That's a deliberate choice. If your fleet runs mixed distros, you don't want to remember which box uses ifupdown, which uses netplan, and which uses NM. One tool, one muscle memory, every distro. The firewall layer differs (firewalld vs nftables) because the distros ship different defaults, but the plumbing underneath is always NM.

How networking actually changed. Ten years ago, networking meant physical cables, physical switches, and a VLAN for each purpose. Your servers lived in one building. Your network ended at the building's edge. The firewall was a box in a rack.

That world is gone. Today a "network" is a software construct that can span a rack, a continent, or five cloud providers. Your containers talk to each other over virtual switches that exist entirely in kernel memory. Your edge nodes announce BGP routes to cloud routers. Your firewall rules are eBPF programs compiled and loaded at boot. The physical wire is just a transport — the network is defined in code.

The boundaries that disappeared: LAN vs WAN (WireGuard makes every link encrypted and flat), physical vs virtual (VXLAN puts 16 million virtual networks on one wire), hardware vs software firewalls (eBPF/XDP filters packets at line rate in the kernel), on-prem vs cloud (BGP peers with AWS/GCP the same way it peers with your rack switch). If you learn these four technologies, you can build any network topology that exists in production today.

This page teaches all of them. Start with the basics if you're new. Skip to VXLAN/BGP/eBPF if you're not.

Check current state

# Show all connections
nmcli connection show

# Show device status
nmcli device status

# Detailed info on a connection
nmcli connection show "Wired connection 1"

# IP addresses
ip addr show
Before you change anything, run nmcli connection show and read it. Most broken-network tickets start with someone modifying a connection they didn't identify first. NM names connections by detection order — "Wired connection 1" is just a label, not a guarantee. If you plugged in two NICs, "Wired connection 1" might be your management interface or your storage interface. Know which is which before you touch it.

Static IP

Using nmcli

# Set static IP on the primary interface
nmcli connection modify "Wired connection 1" \
  ipv4.method manual \
  ipv4.addresses 10.100.10.50/24 \
  ipv4.gateway 10.100.10.1 \
  ipv4.dns "1.1.1.1 8.8.8.8"

# Apply
nmcli connection up "Wired connection 1"

Verify

ip addr show
ip route show
cat /etc/resolv.conf

Switch back to DHCP

nmcli connection modify "Wired connection 1" \
  ipv4.method auto \
  ipv4.addresses "" \
  ipv4.gateway "" \
  ipv4.dns ""
nmcli connection up "Wired connection 1"

Network bridge (for KVM VMs)

A bridge is just a virtual switch. Your physical NIC becomes a port on that switch, and your VMs plug into other ports. The host's IP moves from the physical interface to the bridge — that's why you set the IP on br0, not eth0. If you skip this and use the default NAT network, your VMs live in an invisible subnet that nothing outside the host can reach. Every production KVM deployment uses bridges. It's not optional — it's how VMs become real citizens on your network.

VMs need a bridge interface to connect to the physical LAN:

# Create a bridge
nmcli connection add type bridge ifname br0 con-name br0 \
  ipv4.method manual \
  ipv4.addresses 10.100.10.50/24 \
  ipv4.gateway 10.100.10.1 \
  ipv4.dns "1.1.1.1"

# Attach the physical interface to the bridge
nmcli connection add type bridge-slave ifname eth0 master br0

# Bring down the old connection and bring up the bridge
nmcli connection down "Wired connection 1"
nmcli connection up br0

After this, your host gets its IP from br0 and VMs can bridge through eth0 to the LAN.

Verify

nmcli connection show
bridge link show
ip addr show br0

VLAN tagging

VLANs let one physical wire carry multiple isolated networks. Think of it as painting each Ethernet frame with a color — the switch only delivers frames to ports that speak the same color. In a kldload homelab, you'd put management traffic on VLAN 10, storage on VLAN 20, and VM guest traffic on VLAN 100. One NIC, three networks, complete isolation. If you're bridging VMs onto a VLAN, you build a bridge on top of the VLAN interface — the layering is physical NIC → VLAN → bridge → VM. Each layer is just another NM connection object.
# Create a VLAN interface (VLAN 100 on eth0)
nmcli connection add type vlan ifname eth0.100 dev eth0 id 100 \
  ipv4.method manual \
  ipv4.addresses 10.100.100.10/24

nmcli connection up vlan-eth0.100

VLAN on a bridge (for VMs on a specific VLAN)

# Create VLAN interface
nmcli connection add type vlan ifname eth0.200 dev eth0 id 200

# Create bridge on that VLAN
nmcli connection add type bridge ifname br-vlan200 con-name br-vlan200 \
  ipv4.method manual \
  ipv4.addresses 10.100.200.1/24

# Attach VLAN to bridge
nmcli connection add type bridge-slave ifname eth0.200 master br-vlan200

nmcli connection up br-vlan200
Bonding is insurance. active-backup means if one cable gets yanked or a switch port dies, traffic fails over in milliseconds with zero config on the switch side. LACP (802.3ad) gives you aggregated bandwidth but needs the switch to participate — your switch admin has to configure a LAG group. For a ZFS storage server doing replication over the network, active-backup is the sane default: you want the replication to never stop, and you rarely saturate a single 10GbE link with zfs send anyway.

Combine multiple NICs for redundancy or throughput:

# Create a bond (active-backup mode for redundancy)
nmcli connection add type bond ifname bond0 con-name bond0 \
  bond.options "mode=active-backup,miimon=100" \
  ipv4.method manual \
  ipv4.addresses 10.100.10.50/24 \
  ipv4.gateway 10.100.10.1

# Add slave interfaces
nmcli connection add type bond-slave ifname eth0 master bond0
nmcli connection add type bond-slave ifname eth1 master bond0

nmcli connection up bond0

Bond modes: - active-backup — one active NIC, failover to the other (no switch config needed) - 802.3ad (LACP) — aggregated throughput (requires switch support) - balance-rr — round-robin (basic load balancing)

Firewall

This is where kldload can't hide the distro difference. RPM distros ship firewalld (a wrapper around nftables with zones and services). Debian ships raw nftables. Both ultimately write nftables rules — firewalld is just a higher-level API. The one rule that matters everywhere: if you're running WireGuard, put wg0 in the trusted zone (firewalld) or explicitly accept UDP on your WG port (nftables). WireGuard authenticates at the crypto layer, so firewalling it is redundant — but you still need the port open for the initial handshake.

CentOS/RHEL (firewalld)

# Check status
firewall-cmd --state

# List open ports
firewall-cmd --list-all

# Open a port
firewall-cmd --permanent --add-port=8080/tcp
firewall-cmd --reload

# Open a service
firewall-cmd --permanent --add-service=http
firewall-cmd --permanent --add-service=https
firewall-cmd --reload

# Allow traffic on a WireGuard interface
firewall-cmd --permanent --zone=trusted --add-interface=wg0
firewall-cmd --reload

# Remove a rule
firewall-cmd --permanent --remove-port=8080/tcp
firewall-cmd --reload

Debian (nftables)

# Check status
nft list ruleset

# Open port 8080
nft add rule inet filter input tcp dport 8080 accept

# Make persistent
cat > /etc/nftables.d/kldload-custom.nft << 'EOF'
table inet filter {
  chain input {
    tcp dport 8080 accept
    tcp dport { 80, 443 } accept
    udp dport { 51820, 51821, 51822, 51823 } accept  # WireGuard
  }
}
EOF
systemctl reload nftables

Cross-distro shortcut

# Both distros: allow SSH, HTTP, HTTPS, WireGuard
for port in 22/tcp 80/tcp 443/tcp 51820/udp; do
  if command -v firewall-cmd &>/dev/null; then
    firewall-cmd --permanent --add-port="$port"
  fi
done
firewall-cmd --reload 2>/dev/null || true

DNS configuration

DNS is the number-one cause of "the network is broken" tickets that aren't actually broken. You can ping 1.1.1.1 but not google.com? That's DNS, not networking. On kldload, NM owns /etc/resolv.conf — don't hand-edit it, it gets overwritten on the next connection cycle. Always set DNS through nmcli connection modify. If you need split DNS (different resolvers for internal vs external), set ipv4.dns-search for your internal domain and let systemd-resolved route queries.
# Set DNS servers via NetworkManager
nmcli connection modify "Wired connection 1" ipv4.dns "1.1.1.1 8.8.8.8"
nmcli connection up "Wired connection 1"

# Set search domain
nmcli connection modify "Wired connection 1" ipv4.dns-search "infra.local"
nmcli connection up "Wired connection 1"

# Verify
resolvectl status

IP forwarding (for routing/NAT)

IP forwarding is off by default on every Linux box for good reason — a machine that forwards packets is a router, and you should only be a router on purpose. But if you're running WireGuard site-to-site, KVM with bridged guests, or containers that need outbound NAT, you need this on. kldload enables it automatically when WireGuard is configured with kwg, but if you're doing manual networking, you'll need to flip it yourself. The sysctl.d drop-in survives reboots; the bare sysctl -w does not.

Enable if this node acts as a router, VPN gateway, or container host:

# Enable immediately
sysctl -w net.ipv4.ip_forward=1

# Make persistent
echo "net.ipv4.ip_forward = 1" > /etc/sysctl.d/99-forwarding.conf
sysctl --system

NAT masquerade (share internet from one interface to another)

# CentOS/RHEL
firewall-cmd --permanent --add-masquerade
firewall-cmd --reload

# Debian (nftables)
nft add table nat
nft add chain nat postrouting { type nat hook postrouting priority 100 \; }
nft add rule nat postrouting oifname "eth0" masquerade

Troubleshooting

The troubleshooting ladder: Layer 1 (is the link up?), Layer 2 (does the interface have an address?), Layer 3 (is there a route?), Layer 4 (can you reach an IP?), Layer 7 (can you resolve a name?). Work from the bottom up. If ip link says the interface is DOWN, nothing above matters. If ping 1.1.1.1 works but ping google.com doesn't, stop looking at routes and look at DNS. Most people skip straight to Layer 7 and waste an hour.
# Can't reach the network
ip link show          # is the interface UP?
ip addr show          # does it have an IP?
ip route show         # is the gateway set?
ping 1.1.1.1          # can you reach the internet?
ping google.com       # DNS working?

# NetworkManager logs
journalctl -u NetworkManager --since "10 minutes ago"

# Reset a broken connection
nmcli connection delete "Wired connection 1"
nmcli device wifi connect SSID password PASS   # WiFi
# or let NM auto-configure:
nmcli device connect eth0

Beyond the basics: modern networking

Everything above this line is what networking looked like in 2010. Everything below is what it looks like now. The difference isn't complexity — it's abstraction. VLANs give you ~4,000 isolated networks on a wire. VXLAN gives you 16 million. BGP lets your Linux box participate in the same routing protocol that runs the internet. eBPF lets you write firewall rules that execute at near-hardware speed inside the kernel. These aren't exotic — they're what every cloud provider runs under the hood. AWS VPC is VXLAN. Kubernetes networking is eBPF (Cilium). Cloud interconnects are BGP. You're already using this stuff. You just didn't know it.

VXLAN overlay networks

VXLAN (Virtual Extensible LAN) wraps Layer 2 Ethernet frames inside UDP packets. This means you can create a virtual Layer 2 network between machines that are on completely different physical networks — different subnets, different data centers, different continents. The machines think they're on the same switch.

Why VXLAN matters: VLANs (802.1Q) top out at 4,094 IDs. That's fine for a homelab but useless for a cloud provider with millions of tenants. VXLAN uses a 24-bit VNI (VXLAN Network Identifier) — that's 16,777,216 isolated networks on the same physical infrastructure. Every major cloud provider uses VXLAN (or its cousin GENEVE) as the backbone of their virtual networking. When you create a "VPC" in AWS or a "VNet" in Azure, you're getting a VXLAN segment. When you run Kubernetes with Flannel or Calico in VXLAN mode, your pod network is VXLAN. Understanding this means understanding how every cloud network actually works.

Concept: what's happening on the wire

# Normal Ethernet frame:
# [Eth Header] [IP Header] [TCP/UDP] [Payload]

# VXLAN-encapsulated frame:
# [Outer Eth] [Outer IP] [Outer UDP:4789] [VXLAN Header (VNI)] [Inner Eth] [Inner IP] [Inner TCP/UDP] [Payload]
#  ^-- physical network sees this                                ^-- virtual network sees this

# The outer headers route the packet across the physical network.
# The inner headers are the "real" packet — completely untouched.
# The VNI in the VXLAN header says which virtual network this belongs to.

Example: connect two machines on different subnets

Machine A is on 192.168.1.0/24. Machine B is on 10.0.5.0/24. They can't talk directly. VXLAN makes them neighbors on 172.16.0.0/24:

# ── Machine A (physical IP: 192.168.1.10) ──

# Create a VXLAN interface
# VNI 100 = our virtual network ID
# remote = Machine B's physical IP
# dstport 4789 = standard VXLAN UDP port
ip link add vxlan100 type vxlan \
  id 100 \
  remote 10.0.5.20 \
  dstport 4789 \
  dev eth0

# Give it an IP on the overlay network
ip addr add 172.16.0.1/24 dev vxlan100
ip link set vxlan100 up
# ── Machine B (physical IP: 10.0.5.20) ──

ip link add vxlan100 type vxlan \
  id 100 \
  remote 192.168.1.10 \
  dstport 4789 \
  dev eth0

ip addr add 172.16.0.2/24 dev vxlan100
ip link set vxlan100 up
# ── Test it ──

# From Machine A:
ping 172.16.0.2    # reaches Machine B over the overlay

# What's actually on the wire (tcpdump on the physical interface):
# 192.168.1.10:random -> 10.0.5.20:4789 UDP (VXLAN VNI 100)
#   inside: 172.16.0.1 -> 172.16.0.2 ICMP echo request
Read that again. Machine A is on 192.168.1.x. Machine B is on 10.0.5.x. They're on completely different networks — maybe different buildings, different cities. But to applications running on them, they're on the same 172.16.0.0/24 LAN. The physical network is just plumbing. This is exactly how your cloud VM "sees" other VMs in the same VPC even though the physical servers are in different racks. The network you think you're on and the network you're actually on are two different things.

VXLAN with multiple peers (multicast or head-end replication)

# For more than two peers, use multicast group instead of "remote":
ip link add vxlan100 type vxlan \
  id 100 \
  group 239.1.1.1 \
  dstport 4789 \
  dev eth0

ip addr add 172.16.0.1/24 dev vxlan100
ip link set vxlan100 up

# Every machine that joins VNI 100 with group 239.1.1.1
# automatically discovers all other peers via multicast.
# No need to list every remote IP.

VXLAN + bridge (give VMs an overlay network)

# Create the VXLAN tunnel
ip link add vxlan100 type vxlan id 100 remote 10.0.5.20 dstport 4789 dev eth0
ip link set vxlan100 up

# Create a bridge and attach the VXLAN
ip link add br-overlay type bridge
ip link set vxlan100 master br-overlay
ip link set br-overlay up

# Now attach VM tap interfaces to br-overlay.
# VMs on Machine A and Machine B share the same Layer 2 segment
# even though the physical machines are on different networks.
# VM migration between sites? Just move the VM — the network follows.

Make VXLAN survive reboots (NetworkManager)

# NM can manage VXLAN interfaces natively:
nmcli connection add type vxlan \
  ifname vxlan100 \
  con-name vxlan100 \
  vxlan.id 100 \
  vxlan.remote 10.0.5.20 \
  vxlan.destination-port 4789 \
  vxlan.parent eth0 \
  ipv4.method manual \
  ipv4.addresses 172.16.0.1/24

nmcli connection up vxlan100

VXLAN over WireGuard (encrypted overlay)

Here's where kldload networking gets powerful. VXLAN by itself has zero encryption — it's UDP in the clear. But WireGuard is a Layer 3 encrypted tunnel. Stack them: WireGuard provides the encrypted transport, VXLAN provides the virtual Layer 2 on top. Now you have an encrypted overlay network that spans the internet. This is how you build a multi-site cluster where every node thinks it's on the same switch, and every packet is encrypted. No VPN appliance. No cloud-specific networking. Just Linux kernel modules doing what they do.
# Step 1: WireGuard tunnel between sites (already encrypted)
# Site A: wg0 = 10.200.0.1/24
# Site B: wg0 = 10.200.0.2/24
# (See the WireGuard Basics tutorial for setup)

# Step 2: VXLAN over the WireGuard tunnel
# On Site A:
ip link add vxlan100 type vxlan \
  id 100 \
  remote 10.200.0.2 \
  dstport 4789 \
  dev wg0               # <-- runs over WireGuard, not eth0

ip addr add 172.16.0.1/24 dev vxlan100
ip link set vxlan100 up

# On Site B:
ip link add vxlan100 type vxlan \
  id 100 \
  remote 10.200.0.1 \
  dstport 4789 \
  dev wg0

ip addr add 172.16.0.2/24 dev vxlan100
ip link set vxlan100 up

# Result: 172.16.0.0/24 is a flat Layer 2 network spanning both sites,
# fully encrypted by WireGuard. VMs, containers, ZFS replication —
# anything that needs Layer 2 adjacency works as if both racks
# were in the same room.

BGP: the routing protocol that runs the internet

BGP (Border Gateway Protocol) is how routers on the internet decide where to send packets. Every ISP, every cloud provider, every CDN uses BGP. The reason it matters to you: your Linux box can speak BGP. That means your kldload server can announce routes to cloud routers, receive routes from other sites, and participate in the same routing fabric as AWS and Google. No proprietary hardware. No expensive licenses. Just FRRouting (FRR), which is open source and runs on every distro kldload supports.

Why BGP and not OSPF? OSPF is great inside a single site — it converges fast and auto-discovers neighbors. But OSPF trusts everyone and shares everything. BGP is the opposite: it trusts nothing and shares only what you explicitly permit. That's why it's the protocol between networks (ISPs, clouds, enterprises). If you're connecting two sites, two clouds, or a site to a cloud, you want BGP. If you're connecting switches inside a single building, you might want OSPF. For multi-cloud? BGP is the only game in town. AWS Direct Connect, GCP Cloud Interconnect, Azure ExpressRoute — they all speak BGP. Your homelab can too.

Install FRRouting

# CentOS/RHEL/Rocky/Fedora
dnf install -y frr

# Debian/Ubuntu
apt install -y frr

# Enable the BGP daemon
sed -i 's/^bgpd=no/bgpd=yes/' /etc/frr/daemons
systemctl enable --now frr

Example: announce your network to a peer

Your kldload server has network 10.100.10.0/24. You want to announce it to a router at 10.100.10.1 (your upstream router or cloud interconnect). Your ASN is 65001 (private range), the peer's is 65000.

# Enter FRR's shell
vtysh

# Configure BGP
configure terminal
router bgp 65001
  bgp router-id 10.100.10.50
  neighbor 10.100.10.1 remote-as 65000

  address-family ipv4 unicast
    network 10.100.10.0/24
    neighbor 10.100.10.1 activate
  exit-address-family
end
write memory
# Verify
vtysh -c "show bgp summary"
vtysh -c "show bgp ipv4 unicast"
vtysh -c "show ip route"
What just happened: your Linux server is now a BGP router. It told the upstream router "I own 10.100.10.0/24 — send traffic for those IPs to me." The upstream router told its peers, and now the entire network knows how to reach your subnet. This is identical to what happens when AWS announces a VPC CIDR over a Direct Connect — same protocol, same mechanics. The difference is you're running it on a $200 server instead of a $50,000 router.

Example: multi-site BGP over WireGuard

Two kldload sites connected by WireGuard. Each site announces its local networks to the other via BGP. Routes update automatically — if a subnet appears or disappears, BGP propagates the change.

# ── Site A (WG IP: 10.200.0.1, local network: 10.100.10.0/24) ──
vtysh
configure terminal
router bgp 65001
  bgp router-id 10.200.0.1
  neighbor 10.200.0.2 remote-as 65002    # Site B over WireGuard

  address-family ipv4 unicast
    network 10.100.10.0/24               # announce Site A's network
    neighbor 10.200.0.2 activate
  exit-address-family
end
write memory
# ── Site B (WG IP: 10.200.0.2, local network: 10.100.20.0/24) ──
vtysh
configure terminal
router bgp 65002
  bgp router-id 10.200.0.2
  neighbor 10.200.0.1 remote-as 65001    # Site A over WireGuard

  address-family ipv4 unicast
    network 10.100.20.0/24               # announce Site B's network
    neighbor 10.200.0.1 activate
  exit-address-family
end
write memory
# ── Result ──

# On Site A:
ip route
# ...
# 10.100.20.0/24 via 10.200.0.2 dev wg0 proto bgp  <-- learned from Site B

# On Site B:
ip route
# ...
# 10.100.10.0/24 via 10.200.0.1 dev wg0 proto bgp  <-- learned from Site A

# Add a third site? It learns all routes automatically.
# No manual route tables. No static routes to maintain. BGP does it.

BGP route filtering (don't leak everything)

# Only accept specific prefixes from a neighbor:
vtysh
configure terminal
ip prefix-list ALLOWED seq 10 permit 10.100.0.0/16 le 24
ip prefix-list ALLOWED seq 100 deny any

router bgp 65001
  address-family ipv4 unicast
    neighbor 10.200.0.2 prefix-list ALLOWED in
  exit-address-family
end
write memory

# This says: accept any /24 (or shorter) within 10.100.0.0/16.
# Reject everything else. Never accept a default route you didn't ask for.
# Never announce a route you don't own. BGP without filters is a gun
# pointed at your own network.
BGP is the most powerful routing protocol in existence and also the most dangerous if misconfigured. A BGP misconfiguration took down Facebook for six hours in 2021 — they accidentally withdrew their own routes and vanished from the internet. The rule: always filter inbound and outbound. Use prefix lists. Never accept more-specific routes than you expect. Never announce routes you don't own. Start with explicit permits and deny everything else. BGP's power comes from its trust model — it trusts what you tell it. Make sure you're telling it the truth.

eBPF: programmable networking in the kernel

eBPF (extended Berkeley Packet Filter) lets you run custom programs inside the Linux kernel — at the network layer, the syscall layer, the scheduler, anywhere. For networking, this means you can write packet processing logic that runs at near-hardware speed without leaving kernel space. No copying packets to userland. No context switches. Just your code, running on every packet, at line rate.

This is the part everyone fails on because the docs jump straight into C code and BPF bytecode. Let's slow down. eBPF is not a networking technology — it's a programming environment that happens to be incredibly good at networking. Think of it this way: iptables is a fixed set of rules. nftables is a more flexible set of rules. eBPF is "write any rule you can imagine, and it runs in the kernel at wire speed." That's why Cilium replaced kube-proxy in Kubernetes. That's why Cloudflare uses XDP to stop DDoS attacks. That's why Facebook used eBPF to load-balance every packet. It's not a niche tool — it's the future of the Linux network stack.

The three eBPF network hooks you need to know

XDP (eXpress Data Path)

Runs on the NIC driver, before the kernel even allocates an sk_buff. This is as fast as software gets — process or drop packets before the kernel network stack sees them. Used for DDoS mitigation, load balancing, packet filtering.

Hook point: NIC → XDP → kernel. Fastest possible interception.

TC (Traffic Control)

Runs after the kernel builds the sk_buff but before routing. You can inspect, modify, redirect, or drop packets. More features than XDP (you can see full packet metadata) but slightly slower. Used for policy enforcement, traffic shaping, service mesh.

Hook point: NIC → kernel → TC → routing. Full packet context.

Socket-level (sk_msg, sockops)

Runs on socket operations — connect, send, receive. You can redirect traffic between sockets without it ever hitting the network stack. Used for service mesh acceleration (Cilium) and transparent proxying.

Hook point: application → socket → eBPF. Bypasses the entire network stack.

Example: drop all traffic from a specific IP (XDP)

# Install bpftool and dependencies
# CentOS/RHEL/Rocky/Fedora:
dnf install -y bpftool clang llvm

# Debian/Ubuntu:
apt install -y bpftool clang llvm linux-headers-$(uname -r)
# Write a minimal XDP program (drop_ip.c)
cat > /tmp/drop_ip.c << 'XDPEOF'
#include <linux/bpf.h>
#include <linux/if_ether.h>
#include <linux/ip.h>
#include <bpf/bpf_helpers.h>

// Drop all packets from 192.168.1.100
#define BLOCKED_IP 0x6401A8C0  // 192.168.1.100 in network byte order

SEC("xdp")
int xdp_drop_ip(struct xdp_md *ctx) {
    void *data = (void *)(long)ctx->data;
    void *data_end = (void *)(long)ctx->data_end;

    struct ethhdr *eth = data;
    if ((void *)(eth + 1) > data_end)
        return XDP_PASS;

    if (eth->h_proto != __constant_htons(ETH_P_IP))
        return XDP_PASS;

    struct iphdr *ip = (void *)(eth + 1);
    if ((void *)(ip + 1) > data_end)
        return XDP_PASS;

    if (ip->saddr == BLOCKED_IP)
        return XDP_DROP;  // silently drop — never reaches the kernel

    return XDP_PASS;
}

char _license[] SEC("license") = "GPL";
XDPEOF
# Compile and load
clang -O2 -g -target bpf -c /tmp/drop_ip.c -o /tmp/drop_ip.o

# Attach to eth0
ip link set dev eth0 xdpgeneric obj /tmp/drop_ip.o sec xdp

# Verify it's loaded
ip link show eth0
# eth0: ... xdpgeneric ...

# Check stats
bpftool prog list
bpftool prog show id <PROG_ID>

# Remove when done
ip link set dev eth0 xdpgeneric off
That program runs on every single packet before the kernel allocates memory for it. An iptables DROP rule does the same thing, but it runs after the kernel has already done header parsing, connection tracking, routing lookup, and memory allocation. XDP skips all of that. On a 10GbE link, the difference between iptables DROP and XDP DROP can be millions of packets per second. For DDoS mitigation, that's the difference between your server staying up and going down.

Example: count packets per source IP (TC with maps)

# This uses a BPF map — shared memory between the eBPF program
# and userland tools. The program counts; you read the counts.

cat > /tmp/pkt_count.c << 'TCEOF'
#include <linux/bpf.h>
#include <linux/pkt_cls.h>
#include <linux/if_ether.h>
#include <linux/ip.h>
#include <bpf/bpf_helpers.h>

struct {
    __uint(type, BPF_MAP_TYPE_HASH);
    __uint(max_entries, 10240);
    __type(key, __u32);      // source IP
    __type(value, __u64);    // packet count
} pkt_count SEC(".maps");

SEC("tc")
int count_packets(struct __sk_buff *skb) {
    void *data = (void *)(long)skb->data;
    void *data_end = (void *)(long)skb->data_end;

    struct ethhdr *eth = data;
    if ((void *)(eth + 1) > data_end)
        return TC_ACT_OK;

    if (eth->h_proto != __constant_htons(ETH_P_IP))
        return TC_ACT_OK;

    struct iphdr *ip = (void *)(eth + 1);
    if ((void *)(ip + 1) > data_end)
        return TC_ACT_OK;

    __u32 src = ip->saddr;
    __u64 *count = bpf_map_lookup_elem(&pkt_count, &src);
    if (count) {
        __sync_fetch_and_add(count, 1);
    } else {
        __u64 init = 1;
        bpf_map_update_elem(&pkt_count, &src, &init, BPF_ANY);
    }

    return TC_ACT_OK;  // pass all traffic, just count
}

char _license[] SEC("license") = "GPL";
TCEOF
# Compile
clang -O2 -g -target bpf -c /tmp/pkt_count.c -o /tmp/pkt_count.o

# Attach to eth0 ingress
tc qdisc add dev eth0 clsact
tc filter add dev eth0 ingress bpf da obj /tmp/pkt_count.o sec tc

# Read the map — see packet counts per source IP
bpftool map dump name pkt_count
# key: 0a 64 0a 01  value: 00 00 00 00 00 00 03 e8    <-- 10.100.10.1: 1000 packets
# key: 0a 64 0a 32  value: 00 00 00 00 00 00 00 2a    <-- 10.100.10.50: 42 packets

# Remove
tc filter del dev eth0 ingress
tc qdisc del dev eth0 clsact
You just wrote a custom network monitoring tool that runs inside the kernel. No tcpdump. No pcap. No copying packets to userland. The counts update in real time in a BPF map that you read with bpftool. This is the same primitive that Cilium uses for its network policy enforcement, that Falco uses for security monitoring, and that every observability platform is moving towards. eBPF maps are the bridge between kernel-speed packet processing and human-readable output.

Practical eBPF without writing C (bcc and bpftrace)

# Install bcc tools (pre-built eBPF programs for common tasks)
# CentOS/RHEL/Rocky/Fedora:
dnf install -y bcc-tools

# Debian/Ubuntu:
apt install -y bpfcc-tools

# ── Examples ──

# Watch TCP connections in real time (who's connecting to what)
/usr/share/bcc/tools/tcpconnect

# Watch TCP accept events (who's connecting to your server)
/usr/share/bcc/tools/tcpaccept

# Trace DNS queries
/usr/share/bcc/tools/tcptracer

# Measure TCP retransmits (network quality indicator)
/usr/share/bcc/tools/tcpretrans

# Watch packet drops with stack traces (why is the kernel dropping packets?)
/usr/share/bcc/tools/tcpdrop
# bpftrace: one-liners for network diagnostics

# Count packets per destination port
bpftrace -e 'kprobe:tcp_v4_connect { @[arg2] = count(); }'

# Histogram of TCP send sizes
bpftrace -e 'kprobe:tcp_sendmsg { @bytes = hist(arg2); }'

# Watch all DNS lookups with latency
bpftrace -e 'tracepoint:net:net_dev_xmit { @[comm] = count(); }'
You don't need to write C to use eBPF. The bcc toolkit ships dozens of production-ready tools. tcpconnect shows you every outbound TCP connection your server makes — in real time, with PID and process name. tcpretrans shows retransmits, which is how you find network quality problems that ping won't show. tcpdrop shows kernel packet drops with full stack traces — that's how you find out whether your firewall, conntrack, or socket buffer is the bottleneck. These tools replace weeks of strace/tcpdump debugging with one command.

Multi-cloud routing: putting it all together

Here's the real payoff. Combine WireGuard (encrypted transport) + VXLAN (virtual Layer 2) + BGP (dynamic routing) and you get a multi-cloud network fabric that works the same whether your nodes are in a rack, in AWS, in GCP, or in your basement.

This is what a "software-defined network" actually means. Not a marketing term. Not a product you buy. Three open-source Linux kernel features combined into a fabric that does what Cisco ACI, VMware NSX, and AWS Transit Gateway do — but on hardware you own, with code you can read. The reason this matters: cloud networking locks you in. AWS VPC peering doesn't work with GCP. Azure VNet doesn't work with AWS. But WireGuard + VXLAN + BGP works with everything because it's just Linux. Your network becomes portable.

Architecture: three sites, full mesh

# Site A: On-prem kldload (ASN 65001)
#   Physical: 192.168.1.0/24
#   WireGuard: 10.200.0.1
#   Overlay:   172.16.0.0/24
#
# Site B: AWS EC2 kldload instance (ASN 65002)
#   Physical: 10.0.1.0/24 (VPC)
#   WireGuard: 10.200.0.2
#   Overlay:   172.16.1.0/24
#
# Site C: Hetzner bare metal kldload (ASN 65003)
#   Physical: 88.99.x.x
#   WireGuard: 10.200.0.3
#   Overlay:   172.16.2.0/24

# Layer 1: WireGuard mesh (encrypted transport)
# Each site has WG peers to the other two sites.
# This gives you encrypted IP connectivity between all three.

# Layer 2: VXLAN over WireGuard (virtual Layer 2)
# VMs on all three sites share the same broadcast domain.
# A VM on Site A can ARP for a VM on Site C.

# Layer 3: BGP over WireGuard (dynamic routing)
# Each site announces its local + overlay networks.
# Add a fourth site? It learns all routes in seconds.

Concrete setup: Site A (on-prem)

# ── WireGuard (transport layer) ──
# /etc/wireguard/wg0.conf is already configured with peers
# Site A: 10.200.0.1, Site B: 10.200.0.2, Site C: 10.200.0.3
wg-quick up wg0

# ── VXLAN over WireGuard (overlay layer) ──
# Create VXLAN tunnel to each peer over WireGuard
ip link add vxlan100 type vxlan id 100 local 10.200.0.1 dstport 4789 nolearning

# Populate FDB (forwarding database) for each remote peer
bridge fdb append 00:00:00:00:00:00 dev vxlan100 dst 10.200.0.2
bridge fdb append 00:00:00:00:00:00 dev vxlan100 dst 10.200.0.3

# Bridge the VXLAN with local VMs
ip link add br-overlay type bridge
ip link set vxlan100 master br-overlay
ip addr add 172.16.0.1/24 dev br-overlay
ip link set vxlan100 up
ip link set br-overlay up

# ── BGP (routing layer) ──
vtysh
configure terminal
router bgp 65001
  bgp router-id 10.200.0.1
  neighbor 10.200.0.2 remote-as 65002
  neighbor 10.200.0.3 remote-as 65003

  address-family ipv4 unicast
    network 192.168.1.0/24     # physical network
    network 172.16.0.0/24      # overlay network
    neighbor 10.200.0.2 activate
    neighbor 10.200.0.3 activate

    # Safety: only accept /16 or longer from peers
    neighbor 10.200.0.2 prefix-list SAFE in
    neighbor 10.200.0.3 prefix-list SAFE in
  exit-address-family

ip prefix-list SAFE seq 10 permit 10.0.0.0/8 le 24
ip prefix-list SAFE seq 20 permit 172.16.0.0/12 le 24
ip prefix-list SAFE seq 30 permit 192.168.0.0/16 le 24
ip prefix-list SAFE seq 100 deny any
end
write memory
# ── Verify the full stack ──

# WireGuard: are the tunnels up?
wg show

# VXLAN: is the overlay interface up?
ip -d link show vxlan100
bridge fdb show dev vxlan100

# BGP: are all peers established? Are routes learned?
vtysh -c "show bgp summary"
# Neighbor        AS   Up/Down  Pfx
# 10.200.0.2   65002  00:05:23    2    <-- Site B: 2 routes learned
# 10.200.0.3   65003  00:05:20    2    <-- Site C: 2 routes learned

vtysh -c "show ip route"
# B>* 10.0.1.0/24 via 10.200.0.2 dev wg0      <-- AWS VPC, learned via BGP
# B>* 172.16.1.0/24 via 10.200.0.2 dev wg0    <-- AWS overlay, learned via BGP
# B>* 172.16.2.0/24 via 10.200.0.3 dev wg0    <-- Hetzner overlay, learned via BGP

# End-to-end: can you reach a VM on Site C from Site A?
ping 172.16.2.10   # reaches Hetzner VM over encrypted VXLAN-over-WG with BGP routing

What you just built

Encrypted multi-cloud network fabric in ~30 commands. No proprietary hardware. No cloud-specific networking. No vendor lock-in. Every component is a Linux kernel feature or an open-source daemon.

WireGuard encrypts the transport. VXLAN creates virtual Layer 2 segments across sites. BGP dynamically routes between all sites. eBPF lets you monitor and filter at wire speed. Add a new site? Configure WireGuard peer, add a BGP neighbor, done — it learns all routes automatically.

This is the same architecture that powers Cloudflare's edge network, Kubernetes service meshes (Cilium), and cloud provider backbones. The tools are identical. The scale is different. But if you can build it on three kldload nodes, you understand how it works everywhere.

The boundaries that don't exist anymore

If you read this whole page, here's what you should take away. The old networking model had hard boundaries: LAN was inside, WAN was outside. Physical switches were real, virtual switches were fake. The firewall was a box in a rack. Routing was something Cisco sold you. Every one of those boundaries is gone. LAN vs WAN? WireGuard makes every link encrypted and point-to-point. Your "LAN" can span continents. Physical vs virtual? VXLAN puts 16 million virtual networks on one physical wire. The virtual network is the real one. Hardware firewall vs software? eBPF/XDP processes packets at 10 million per second in the kernel. The software is faster than the hardware. On-prem vs cloud? BGP peers with AWS, GCP, Azure, and your home router using the exact same protocol. The network is code. The code runs in the kernel. The kernel runs on kldload. That's it. That's the whole story. Everything else is just configuration.
Old boundary What replaced it Linux feature
LAN vs WAN Encrypted flat network WireGuard
Physical vs virtual switch Overlay networks (16M segments) VXLAN / GENEVE
Hardware firewall Programmable kernel dataplane eBPF / XDP
On-prem vs cloud routing Same protocol everywhere BGP (FRRouting)
Per-host iptables rules Network-wide policy Cilium / eBPF
Static routes Dynamic route distribution BGP + BFD
~4,000 VLANs ~16 million virtual networks VXLAN VNI

Where to go next

← Secure Boot & Encryption WireGuard Basics →