Appliance Recipe: Advanced Firewall & Gateway

A kldload firewall where every configuration change is protected by ZFS snapshots. Snapshot before every rule change. Rollback bad configs in seconds. Replicate firewall state to a standby node. Your network's front door deserves the same transactional guarantees as your databases.

Why ZFS for a firewall:

• Snapshot before every nft rule change — rollback in seconds
• Boot environments: bad firewall rule? Boot the previous environment
• Replicate entire firewall state to a standby node with zfs send
• Atomic config changes — no half-applied rulesets
• Full audit trail via snapshot history
• ZFS compression on logs saves 70%+ on verbose firewall logging

Architecture

                          ┌─────────────────────────────────────────────────────┐
         ISP              │           kldload Firewall / Gateway               │
          │               │                                                     │
          ▼               │  WAN (enp1s0) ─── 203.0.113.1/24                    │
    ┌───────────┐         │       │                                             │
    │  Modem /  │─────────│───────┘                                             │
    │  ONT      │         │       │                                             │
    └───────────┘         │       ▼                                             │
                          │  ┌──────────────────────────────────────────────┐    │
                          │  │              nftables engine                 │    │
                          │  │                                              │    │
                          │  │  ┌─────────┐ ┌──────────┐ ┌─────────────┐   │    │
                          │  │  │ FORWARD │ │ NAT/PAT  │ │ Rate Limit  │   │    │
                          │  │  │ filter  │ │ masq     │ │ conntrack   │   │    │
                          │  │  └─────────┘ └──────────┘ └─────────────┘   │    │
                          │  └──────────────────────────────────────────────┘    │
                          │       │           │           │          │           │
                          │       ▼           ▼           ▼          ▼           │
                          │  ┌────────┐ ┌─────────┐ ┌────────┐ ┌────────┐      │
                          │  │  DMZ   │ │ Trusted │ │  IoT   │ │ Guest  │      │
                          │  │ enp2s0 │ │ enp3s0  │ │ enp4s0 │ │ enp5s0 │      │
                          │  │.1/24   │ │.1/24    │ │.1/24   │ │.1/24   │      │
                          │  └────────┘ └─────────┘ └────────┘ └────────┘      │
                          │  10.10.1.0   10.10.10.0  10.10.20.0  10.10.30.0    │
                          │                                                     │
                          │  wg0: 10.200.0.1/24  ── WireGuard site-to-site     │
                          │  Unbound: recursive DNS + sinkhole (:53)            │
                          │  Kea DHCP4: all zones (:67)                         │
                          │                                                     │
                          │  rpool/firewall         ← config snapshots          │
                          │  rpool/firewall/logs    ← compressed log storage    │
                          └─────────────────────────────────────────────────────┘
                                    │
                                    │ ZFS replication (syncoid)
                                    │ VRRP (keepalived)
                                    ▼
                          ┌─────────────────────────┐
                          │  kldload Standby FW    │
                          │  rpool/firewall-replica  │
                          │  VRRP backup priority    │
                          └─────────────────────────┘

Step 1: Install kldload

cat > /tmp/answers.env << 'EOF'
KLDLOAD_DISTRO=debian
KLDLOAD_DISK=/dev/sda
KLDLOAD_HOSTNAME=fw-primary
KLDLOAD_USERNAME=admin
KLDLOAD_PASSWORD=changeme
KLDLOAD_PROFILE=server
KLDLOAD_NET_METHOD=static
KLDLOAD_NET_IP=203.0.113.1
KLDLOAD_NET_MASK=255.255.255.0
KLDLOAD_NET_GW=203.0.113.254
KLDLOAD_NET_DNS=1.1.1.1
EOF
kldload-install-target --config /tmp/answers.env

Step 2: ZFS dataset layout

# Firewall configuration dataset — small, frequent snapshots
zfs create -o mountpoint=/etc/nftables -o compression=lz4 rpool/firewall
zfs create -o mountpoint=/etc/wireguard -o compression=lz4 rpool/firewall/wireguard
zfs create -o mountpoint=/etc/unbound -o compression=lz4 rpool/firewall/dns
zfs create -o mountpoint=/etc/kea -o compression=lz4 rpool/firewall/dhcp

# Log storage — verbose firewall logs compress extremely well
zfs create -o mountpoint=/var/log/firewall -o compression=zstd -o recordsize=128k rpool/firewall/logs

# Sinkhole blocklists
zfs create -o mountpoint=/etc/unbound/blocklists -o compression=lz4 rpool/firewall/blocklists

Step 3: Configure network interfaces

# /etc/systemd/network/10-wan.network
cat > /etc/systemd/network/10-wan.network << 'EOF'
[Match]
Name=enp1s0

[Network]
Address=203.0.113.1/24
Gateway=203.0.113.254
DNS=127.0.0.1
EOF

# Zone interfaces
for zone in "enp2s0:10.10.1.1/24:dmz" "enp3s0:10.10.10.1/24:trusted" \
            "enp4s0:10.10.20.1/24:iot" "enp5s0:10.10.30.1/24:guest"; do
  IFS=':' read -r iface addr name <<< "$zone"
  cat > "/etc/systemd/network/20-${name}.network" << EOF
[Match]
Name=${iface}

[Network]
Address=${addr}
EOF
done

# Enable IP forwarding
cat > /etc/sysctl.d/99-firewall.conf << 'EOF'
net.ipv4.ip_forward = 1
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv6.conf.all.forwarding = 1
net.netfilter.nf_conntrack_max = 262144
net.netfilter.nf_conntrack_tcp_timeout_established = 7200
EOF
sysctl --system

Step 4: nftables stateful firewall

# Snapshot before writing rules — always
ksnap "before nftables initial config"

cat > /etc/nftables/firewall.nft << 'NFTEOF'
#!/usr/sbin/nft -f

flush ruleset

# ─── Definitions ─────────────────────────────────────────────
define WAN    = enp1s0
define DMZ    = enp2s0
define TRUST  = enp3s0
define IOT    = enp4s0
define GUEST  = enp5s0

define TRUSTED_NET = 10.10.10.0/24
define DMZ_NET     = 10.10.1.0/24
define IOT_NET     = 10.10.20.0/24
define GUEST_NET   = 10.10.30.0/24
define VPN_NET     = 10.200.0.0/24

# ─── Tables ──────────────────────────────────────────────────
table inet filter {

    # conntrack: allow established, drop invalid
    chain ct_state {
        type filter hook prerouting priority -150; policy accept;
        ct state invalid drop
    }

    chain input {
        type filter hook input priority 0; policy drop;

        # Loopback
        iif lo accept

        # Established / related
        ct state established,related accept

        # ICMP (rate limited)
        ip protocol icmp limit rate 10/second accept
        ip6 nexthdr icmpv6 limit rate 10/second accept

        # SSH from trusted only
        iifname $TRUST tcp dport 22 accept

        # DNS from all internal zones
        iifname { $TRUST, $DMZ, $IOT, $GUEST } udp dport 53 accept
        iifname { $TRUST, $DMZ, $IOT, $GUEST } tcp dport 53 accept

        # DHCP from all internal zones
        iifname { $TRUST, $DMZ, $IOT, $GUEST } udp dport 67 accept

        # WireGuard on WAN
        iifname $WAN udp dport 51820 accept

        # Web UI from trusted (for monitoring)
        iifname $TRUST tcp dport { 8080, 443 } accept

        # Log and drop everything else
        log prefix "nft-input-drop: " limit rate 5/minute
        drop
    }

    chain forward {
        type filter hook forward priority 0; policy drop;

        # Established / related
        ct state established,related accept

        # ─── Zone policies ───────────────────────────
        # Trusted → everywhere
        iifname $TRUST accept

        # DMZ → WAN only (no lateral movement)
        iifname $DMZ oifname $WAN accept

        # IoT → WAN only (no LAN access)
        iifname $IOT oifname $WAN accept

        # Guest → WAN only, rate limited
        iifname $GUEST oifname $WAN limit rate 100 mbytes/second accept

        # VPN → trusted + DMZ
        iifname "wg0" oifname { $TRUST, $DMZ } accept

        # Trusted → IoT (for management)
        iifname $TRUST oifname $IOT accept

        # Log and drop inter-zone violations
        log prefix "nft-forward-drop: " limit rate 5/minute
        drop
    }

    chain output {
        type filter hook output priority 0; policy accept;
    }
}

table inet nat {
    chain prerouting {
        type nat hook prerouting priority -100; policy accept;

        # Port forward: WAN:443 → DMZ web server
        iifname $WAN tcp dport 443 dnat to 10.10.1.10:443

        # Port forward: WAN:25565 → IoT Minecraft (example)
        iifname $WAN tcp dport 25565 dnat to 10.10.20.50:25565
    }

    chain postrouting {
        type nat hook postrouting priority 100; policy accept;

        # Masquerade all internal zones to WAN
        oifname $WAN masquerade
    }
}

# ─── Rate limiting table ─────────────────────────────────────
table inet ratelimit {
    set ssh_meter { type ipv4_addr; flags dynamic; timeout 5m; }

    chain input {
        type filter hook input priority -10; policy accept;

        # SSH brute-force protection
        tcp dport 22 ct state new \
            add @ssh_meter { ip saddr limit rate 3/minute } accept

        tcp dport 22 ct state new drop
    }
}
NFTEOF

# Apply and enable
nft -f /etc/nftables/firewall.nft
systemctl enable nftables

# Snapshot after successful apply
ksnap "nftables initial config applied"

Step 5: WireGuard site-to-site VPN

# Generate keys
wg genkey | tee /etc/wireguard/private.key | wg pubkey > /etc/wireguard/public.key
chmod 600 /etc/wireguard/private.key

cat > /etc/wireguard/wg0.conf << EOF
[Interface]
Address = 10.200.0.1/24
ListenPort = 51820
PrivateKey = $(cat /etc/wireguard/private.key)

# Site B — remote office
[Peer]
PublicKey = <SITE_B_PUBLIC_KEY>
AllowedIPs = 10.200.0.2/32, 192.168.1.0/24
Endpoint = 198.51.100.1:51820
PersistentKeepalive = 25

# Site C — home office
[Peer]
PublicKey = <SITE_C_PUBLIC_KEY>
AllowedIPs = 10.200.0.3/32, 192.168.2.0/24
Endpoint = 198.51.100.2:51820
PersistentKeepalive = 25

# Road warrior — laptop
[Peer]
PublicKey = <LAPTOP_PUBLIC_KEY>
AllowedIPs = 10.200.0.10/32
EOF

systemctl enable --now wg-quick@wg0

The firewall is the VPN endpoint. No separate VPN appliance. Every site connects through encrypted WireGuard tunnels, and the same nftables rules control what VPN traffic can reach which zone.

Step 6: Unbound DNS + sinkhole

apt install -y unbound

cat > /etc/unbound/unbound.conf << 'EOF'
server:
    interface: 0.0.0.0
    port: 53
    access-control: 10.10.0.0/16 allow
    access-control: 10.200.0.0/24 allow
    access-control: 127.0.0.0/8 allow

    # Performance
    num-threads: 4
    msg-cache-size: 64m
    rrset-cache-size: 128m
    cache-min-ttl: 300
    cache-max-ttl: 86400
    prefetch: yes
    prefetch-key: yes

    # Privacy
    hide-identity: yes
    hide-version: yes
    qname-minimisation: yes
    aggressive-nsec: yes

    # DNSSEC validation
    auto-trust-anchor-file: "/var/lib/unbound/root.key"

    # Sinkhole — block ads, trackers, malware
    include: /etc/unbound/blocklists/blocklist.conf

    # Logging
    verbosity: 1
    log-queries: yes
    logfile: /var/log/firewall/unbound.log

forward-zone:
    name: "."
    forward-tls-upstream: yes
    # Cloudflare DNS over TLS
    forward-addr: 1.1.1.1@853#cloudflare-dns.com
    forward-addr: 1.0.0.1@853#cloudflare-dns.com
EOF

# Build blocklist from Steven Black's unified hosts
cat > /usr/local/bin/update-blocklist << 'SCRIPT'
#!/bin/bash
# Download and convert hosts file to unbound local-zone format
URL="https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts"

curl -fsSL "$URL" | \
    grep '^0\.0\.0\.0' | \
    awk '{print "local-zone: \""$2"\" always_nxdomain"}' | \
    grep -v 'localhost' \
    > /etc/unbound/blocklists/blocklist.conf

echo "Blocked $(wc -l < /etc/unbound/blocklists/blocklist.conf) domains"
systemctl reload unbound
SCRIPT
chmod +x /usr/local/bin/update-blocklist

# Initial blocklist build
/usr/local/bin/update-blocklist

# Weekly update
echo '0 4 * * 1 root /usr/local/bin/update-blocklist 2>&1 | logger -t dns-sinkhole' \
    > /etc/cron.d/dns-sinkhole

systemctl enable --now unbound

Every device on every zone uses this firewall as its DNS server. Ads, trackers, and malware domains return NXDOMAIN. No Pi-hole needed — the firewall does it natively.

Step 7: Kea DHCP4 with static leases

apt install -y kea-dhcp4-server

cat > /etc/kea/kea-dhcp4.conf << 'EOF'
{
  "Dhcp4": {
    "interfaces-config": {
      "interfaces": [ "enp2s0", "enp3s0", "enp4s0", "enp5s0" ]
    },
    "lease-database": {
      "type": "memfile",
      "persist": true,
      "lfc-interval": 3600
    },
    "valid-lifetime": 28800,
    "subnet4": [
      {
        "subnet": "10.10.1.0/24",
        "pools": [ { "pool": "10.10.1.100-10.10.1.200" } ],
        "option-data": [
          { "name": "routers", "data": "10.10.1.1" },
          { "name": "domain-name-servers", "data": "10.10.1.1" }
        ],
        "reservations": [
          { "hw-address": "aa:bb:cc:dd:ee:01", "ip-address": "10.10.1.10",
            "hostname": "webserver" }
        ]
      },
      {
        "subnet": "10.10.10.0/24",
        "pools": [ { "pool": "10.10.10.100-10.10.10.200" } ],
        "option-data": [
          { "name": "routers", "data": "10.10.10.1" },
          { "name": "domain-name-servers", "data": "10.10.10.1" }
        ]
      },
      {
        "subnet": "10.10.20.0/24",
        "pools": [ { "pool": "10.10.20.100-10.10.20.200" } ],
        "option-data": [
          { "name": "routers", "data": "10.10.20.1" },
          { "name": "domain-name-servers", "data": "10.10.20.1" }
        ],
        "reservations": [
          { "hw-address": "aa:bb:cc:dd:ee:10", "ip-address": "10.10.20.10",
            "hostname": "thermostat" },
          { "hw-address": "aa:bb:cc:dd:ee:11", "ip-address": "10.10.20.11",
            "hostname": "doorbell" }
        ]
      },
      {
        "subnet": "10.10.30.0/24",
        "pools": [ { "pool": "10.10.30.100-10.10.30.200" } ],
        "option-data": [
          { "name": "routers", "data": "10.10.30.1" },
          { "name": "domain-name-servers", "data": "10.10.30.1" }
        ],
        "valid-lifetime": 3600
      }
    ]
  }
}
EOF

systemctl enable --now kea-dhcp4-server

Step 8: eBPF-based traffic monitoring

# Install bcc tools (included in kldload)
# These are already available on a kldload server profile install

# Monitor TCP connection lifecycle — who's connecting where
tcplife -D -T >> /var/log/firewall/tcplife.log &

# Watch connection tracking table in real time
conntrack -E -o timestamp 2>&1 | \
    rotatelogs /var/log/firewall/conntrack.%Y%m%d 86400 &

# nftables counter summary — run periodically
cat > /usr/local/bin/fw-stats << 'SCRIPT'
#!/bin/bash
echo "=== nftables counter summary ==="
nft list ruleset | grep -E "(packets|bytes)" | grep -v "0 packets"
echo ""
echo "=== Connection tracking ==="
conntrack -C
echo "Active connections: $(conntrack -L 2>/dev/null | wc -l)"
echo ""
echo "=== Top talkers (last hour) ==="
journalctl -u nftables --since "1 hour ago" --no-pager | \
    grep -oP 'SRC=\K[0-9.]+' | sort | uniq -c | sort -rn | head -10
echo ""
echo "=== DNS query stats ==="
unbound-control stats_noreset | grep -E "(total\.|num\.)"
echo ""
echo "=== Zone bandwidth ==="
for iface in enp2s0 enp3s0 enp4s0 enp5s0; do
    rx=$(cat /sys/class/net/$iface/statistics/rx_bytes 2>/dev/null || echo 0)
    tx=$(cat /sys/class/net/$iface/statistics/tx_bytes 2>/dev/null || echo 0)
    echo "$iface: RX=$(numfmt --to=iec $rx) TX=$(numfmt --to=iec $tx)"
done
SCRIPT
chmod +x /usr/local/bin/fw-stats

Step 9: HA failover with keepalived + ZFS replication

apt install -y keepalived

# Primary node keepalived config
cat > /etc/keepalived/keepalived.conf << 'EOF'
global_defs {
    router_id FW_PRIMARY
    script_user root
    enable_script_security
}

vrrp_script chk_nftables {
    script "/usr/bin/nft list ruleset > /dev/null 2>&1"
    interval 5
    weight 10
    fall 3
    rise 2
}

vrrp_instance VI_TRUSTED {
    state MASTER
    interface enp3s0
    virtual_router_id 10
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass kldloadfw
    }
    virtual_ipaddress {
        10.10.10.254/24
    }
    track_script {
        chk_nftables
    }
}

vrrp_instance VI_DMZ {
    state MASTER
    interface enp2s0
    virtual_router_id 11
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass kldloadfw
    }
    virtual_ipaddress {
        10.10.1.254/24
    }
}

vrrp_instance VI_IOT {
    state MASTER
    interface enp4s0
    virtual_router_id 12
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass kldloadfw
    }
    virtual_ipaddress {
        10.10.20.254/24
    }
}
EOF

systemctl enable --now keepalived

# Replicate firewall config to standby every 15 minutes
cat > /etc/cron.d/fw-replicate << 'EOF'
*/15 * * * * root syncoid -r rpool/firewall 10.200.0.2:rpool/firewall-replica 2>&1 | logger -t fw-replicate
EOF

Both firewalls run identical kldload installs. The primary replicates its entire /etc/nftables, /etc/wireguard, /etc/unbound, and /etc/kea datasets to the standby via syncoid. If the primary dies, keepalived promotes the standby and it already has the latest configuration.

Step 10: Snapshot workflow for rule changes

This is where kldload earns its keep. Every firewall change follows the same pattern:

# 1. Snapshot before the change
ksnap "before adding IoT camera rules"

# 2. Make the change
nft add rule inet filter forward iifname "enp4s0" ip saddr 10.10.20.15 \
    oifname "enp3s0" ip daddr 10.10.10.50 tcp dport 554 accept

# 3. Test — does the camera stream work?
#    If yes: snapshot the success state
ksnap "IoT camera RTSP rule verified"

#    If no: rollback instantly
zfs rollback rpool/firewall@before-adding-iot-camera-rules

Boot environments for major changes

# Before a major network redesign — create a boot environment
kbe create pre-network-redesign

# Make sweeping changes to nftables, WireGuard, DNS, DHCP...
# Everything breaks? Boot the previous environment:
kbe activate pre-network-redesign
reboot

# Everything works? You have a rollback point forever:
kbe list
# NAME                    ACTIVE  MOUNTPOINT  CREATION
# pre-network-redesign    -       -           2026-03-27 14:30
# post-network-redesign   NR      /           2026-03-27 16:45

Bill of materials

Compare to a commercial firewall appliance with annual license renewals, zero snapshot capability, and zero boot environments. The kldload firewall costs a fraction, does more, and you own every bit of it.