| your Linux re-packer
kldload — your platform, your way, anywhere, free
Source
Documentation

IPsec Tunnels Masterclass

This guide covers IPsec from first principles to production deployment: the IKE negotiation phases, Security Associations, the Security Parameter Index, ESP and AH protocols, transport vs tunnel mode, hashing and integrity verification, the four deployment types (and what each one actually does to every packet), and complete strongSwan and Libreswan configurations on kldload. By the end you will understand exactly what happens to a packet from the moment it enters an IPsec tunnel to the moment it exits the other side — and why every field in the SA exists.

The premise: WireGuard is the right answer for most tunnelling needs — it is simple, fast, and hard to misconfigure. But IPsec is the standard that every firewall, router, cloud provider VPN gateway, and government network speaks. If you need to connect to an AWS VPN Gateway, a Cisco ASA, a Juniper SRX, or a partner's network, you need IPsec. It is also the only tunnel protocol with FIPS 140-3 validated implementations, which matters for compliance.

What this page covers: IPsec architecture, IKE Phase 1 and Phase 2, Security Associations (SA), Security Parameter Index (SPI), ESP encryption and authentication, AH integrity, transport mode vs tunnel mode, the four IPsec deployment types, hashing algorithms and integrity checking, anti-replay protection, Dead Peer Detection, strongSwan site-to-site and roadwarrior configs, Libreswan examples, IPsec with BGP, XFRM interface, nftables integration, and troubleshooting. All on kldload.

Prerequisites: a running kldload system. Networking fundamentals from the Backplane Networks masterclass. Firewall rules from the nftables Masterclass.

WireGuard is elegant because it throws away everything IPsec accumulated over 30 years of RFCs. But that accumulated complexity exists for a reason — IPsec handles scenarios WireGuard does not: interoperability with hardware appliances, FIPS-validated encryption, transport-mode encryption for host-to-host without tunnelling, and standards-based key exchange that every network vendor on earth implements. You do not need to love IPsec. You need to understand it well enough to configure it when the situation demands it, and to know why each piece exists.

1. IPsec Architecture

IPsec is not a single protocol. It is a framework of protocols that work together to provide confidentiality (encryption), integrity (tamper detection), authentication (peer identity), and anti-replay (packet sequence protection) at the IP layer.

IKE — Internet Key Exchange

The control plane. IKE negotiates the cryptographic parameters and establishes shared keys between two peers. IKEv2 (RFC 7296) is the current version. IKEv1 is legacy. IKE runs on UDP port 500 (and UDP 4500 when NAT traversal is needed).

// IKE = the handshake. "What ciphers do we both support? Let's agree on keys." Happens before any data flows.

ESP — Encapsulating Security Payload

The data plane for encryption + authentication. ESP encrypts the packet payload and appends an Integrity Check Value (ICV) — a cryptographic hash that proves the packet was not tampered with. ESP is IP protocol 50. It provides confidentiality, integrity, and anti-replay.

// ESP = the armoured envelope. The contents are encrypted. The seal proves nobody opened it.

AH — Authentication Header

Integrity and authentication only — no encryption. AH hashes the entire IP packet including most header fields, proving nothing was modified in transit. AH is IP protocol 51. It is rarely used alone because ESP with NULL encryption achieves the same thing.

// AH = a tamper-evident seal with no envelope. Everyone can read the letter, but nobody can change it.

SA — Security Association

A one-way agreement between two peers: the encryption algorithm, the key, the integrity algorithm, the SPI, and the lifetime. An IPsec tunnel needs at least two SAs (one per direction). SAs live in the kernel's Security Association Database (SAD).

// SA = "For traffic from A to B, use AES-256-GCM with this key, tagged with this SPI."

SPI — Security Parameter Index

A 32-bit identifier in every ESP/AH packet header. The receiving host uses the SPI + destination IP + protocol (ESP or AH) to look up the correct SA in its SAD. Without the SPI, the receiver would not know which key to use for decryption.

// SPI = the label on the envelope that tells the mailroom which decryption key to use.

SP — Security Policy

Rules that tell the kernel what to do with traffic: encrypt it (PROTECT), let it pass (BYPASS), or drop it (DISCARD). Policies live in the Security Policy Database (SPD) and match on source/destination IP, port, and protocol. Policies reference SAs.

// "All traffic to 10.0.0.0/24 must go through the tunnel. Everything else passes normally."
The key mental model: IKE negotiates, ESP/AH protects, SAs hold the state, SPIs identify the SA on the wire, and SPs decide which traffic gets protected. Every IPsec packet carries an SPI in its header. The receiver uses that SPI to find the matching SA, which tells it the algorithm and key. If the SPI does not match any SA, the packet is dropped. This is the core mechanism — everything else is negotiation detail.

2. IKE Negotiation — Phase 1 and Phase 2

IKEv2 negotiates in two exchanges (not "phases" technically, but the Phase 1/Phase 2 terminology from IKEv1 is still universally used). Phase 1 establishes a secure channel between the peers. Phase 2 negotiates the actual IPsec SAs for data traffic.

Phase 1 — IKE SA (the control channel)

IKE_SA_INIT

The initiator sends proposed cipher suites, a Diffie-Hellman public value, and a nonce. The responder picks a suite, sends its DH value and nonce. After this exchange both sides derive the IKE SA keys (encryption and integrity keys for the IKE channel itself). Two packets total.

// "Here are my cipher options and my half of the key exchange." — "I pick this cipher, here's my half."

IKE_AUTH

Both sides authenticate. They send their identity (IP, FQDN, or certificate DN), the authentication payload (pre-shared key MAC or certificate signature), and the first Child SA proposal (the Phase 2 negotiation piggybacks on this exchange). Two more packets.

// "I'm gateway-a.example.com, here's my certificate. Also, let's set up the data tunnel now."

Result of Phase 1: an encrypted, authenticated IKE channel (the IKE SA). All further negotiation happens inside this channel. The IKE SA has its own lifetime (typically 4–24 hours) and is rekeyed before it expires.

Phase 2 — Child SA (the data tunnel)

CREATE_CHILD_SA

Negotiates the IPsec SAs (Child SAs) for actual data traffic. Each side proposes encryption and integrity algorithms, the traffic selectors (which IP ranges and ports to protect), and optionally a new DH exchange for Perfect Forward Secrecy (PFS). The result is a pair of SAs — one for each direction, each with its own SPI.

// "For traffic between 10.1.0.0/24 and 10.2.0.0/24, let's use AES-256-GCM with this key pair."

Traffic Selectors (TS)

Define what traffic flows through the tunnel. TSi (initiator's traffic selector) and TSr (responder's traffic selector) specify IP ranges, protocol, and port ranges. A tunnel from 10.1.0.0/24 to 10.2.0.0/24 has TSi=10.1.0.0/24 and TSr=10.2.0.0/24.

// "This tunnel is for traffic between these two networks. Everything else stays outside."

Perfect Forward Secrecy (PFS)

Optional DH exchange during Phase 2 rekeying. Without PFS, all Child SAs derive keys from the Phase 1 DH material — if Phase 1 keys leak, all sessions are compromised. With PFS, each rekey does a fresh DH exchange, so past sessions stay secure even if current keys leak.

// PFS = a new key exchange every time you rekey. Yesterday's traffic is safe even if today's key is stolen.

IKEv2 in four packets

# Complete IKEv2 exchange:
#
# Packet 1: Initiator → Responder  IKE_SA_INIT request
#   - SA proposal (ciphers), KE (DH public value), Nonce
#
# Packet 2: Responder → Initiator  IKE_SA_INIT response
#   - SA chosen cipher, KE (DH public value), Nonce
#   — Both sides now derive IKE SA keys —
#
# Packet 3: Initiator → Responder  IKE_AUTH request (encrypted)
#   - Identity, Auth (PSK or cert), SA proposal for Child SA, TSi, TSr
#
# Packet 4: Responder → Initiator  IKE_AUTH response (encrypted)
#   - Identity, Auth, SA chosen for Child SA, TSi, TSr
#   — Child SA established, data can flow —

# Total: 4 packets, 2 round trips. Compare to IKEv1: 6-9 packets.
IKEv2 is a massive improvement over IKEv1. Four packets instead of six or nine. One exchange format instead of Main Mode, Aggressive Mode, and Quick Mode. Built-in NAT traversal. Built-in Dead Peer Detection. If you are configuring IPsec from scratch, always use IKEv2. The only reason to use IKEv1 is if the remote peer is a legacy device that does not support IKEv2.

3. The SPI — How Every Packet Finds Its Key

The Security Parameter Index is the most important field in an IPsec packet. Without it, the receiving host cannot decrypt or verify anything.

How the SPI works

# An ESP packet on the wire looks like:
#
# ┌──────────────────────────────────────────────┐
# │ IP Header (proto=50 for ESP)                 │
# ├──────────────────────────────────────────────┤
# │ ESP Header                                   │
# │   SPI: 0xc0a80123  (32 bits)                │
# │   Sequence Number: 00000042  (32 bits)       │
# ├──────────────────────────────────────────────┤
# │ Encrypted Payload                            │
# │   (original IP packet, padded)               │
# ├──────────────────────────────────────────────┤
# │ ESP Trailer                                  │
# │   Padding + Pad Length + Next Header          │
# ├──────────────────────────────────────────────┤
# │ Integrity Check Value (ICV)                  │
# │   (HMAC or GCM tag, covers ESP header +      │
# │    encrypted payload + trailer)              │
# └──────────────────────────────────────────────┘

# The receiver:
# 1. Reads the SPI (0xc0a80123) from the ESP header
# 2. Looks up SPI + destination IP + protocol in its SAD
# 3. Finds the SA: AES-256-GCM, key=..., anti-replay window=64
# 4. Checks the sequence number against the anti-replay window
# 5. Verifies the ICV (integrity check)
# 6. Decrypts the payload
# 7. Delivers the original IP packet to the stack

SPI assignment

# Each peer generates its own inbound SPI during IKE negotiation
# Peer A says: "Send ESP to me with SPI 0xaaa11111"
# Peer B says: "Send ESP to me with SPI 0xbbb22222"
#
# Traffic A→B carries SPI 0xbbb22222 (B's inbound SPI)
# Traffic B→A carries SPI 0xaaa11111 (A's inbound SPI)

# View current SAs and their SPIs on Linux:
ip xfrm state
# src 203.0.113.1 dst 198.51.100.1
#   proto esp spi 0xc0a80123 reqid 1 mode tunnel
#   replay-window 64 flag af-unspec
#   auth-trunc hmac(sha256) 0x... 128
#   enc cbc(aes) 0x...
#   encap type espinudp sport 4500 dport 4500 addr 0.0.0.0
#   anti-replay context: seq 0x0, oseq 0x2a, bitmap 0x0000003f
The SPI is not secret — it travels in the clear in every ESP header. Its job is identification, not security. Think of it as a session ID. The security comes from the key material inside the SA, which never travels on the wire (it is derived from the IKE DH exchange). If an attacker sees the SPI, they know which SA the packet belongs to, but they still cannot decrypt it without the key. The SPI also enables multiple tunnels between the same pair of hosts — each tunnel has different SPIs.

4. ESP vs AH — Encryption and Integrity

ESP — Encapsulating Security Payload

ESP provides encryption + integrity + anti-replay. Modern ESP uses AEAD (Authenticated Encryption with Associated Data) ciphers like AES-256-GCM, which combine encryption and integrity into a single operation. Older configurations separate them: AES-CBC for encryption + HMAC-SHA256 for integrity.

Algorithm Type Status Notes
AES-256-GCM AEAD (enc + auth) Recommended Single operation, hardware accelerated (AES-NI), FIPS approved
AES-256-CBC + HMAC-SHA256 Encrypt-then-MAC Acceptable Widely supported, FIPS approved, slower than GCM
ChaCha20-Poly1305 AEAD Good Excellent on CPUs without AES-NI, not FIPS approved
3DES Encrypt Deprecated 56-bit effective security. Legacy only. Never use for new tunnels.

AH — Authentication Header

AH provides integrity and authentication but no encryption. It hashes the entire IP packet including immutable header fields (source IP, destination IP, protocol). This means AH detects modifications to the IP header itself — something ESP does not do in transport mode. However, AH breaks with NAT (because NAT modifies the IP header, invalidating the hash). For this reason, AH is rarely used in modern deployments.

# AH packet structure:
#
# ┌──────────────────────────────────────────────┐
# │ IP Header (proto=51 for AH)                 │
# ├──────────────────────────────────────────────┤
# │ AH Header                                   │
# │   Next Header, Payload Length                │
# │   SPI: 0xdead0001                           │
# │   Sequence Number: 00000001                  │
# │   ICV: HMAC-SHA256 over entire packet        │
# │        (IP header + AH header + payload)     │
# ├──────────────────────────────────────────────┤
# │ Original Payload (NOT encrypted)             │
# └──────────────────────────────────────────────┘
#
# AH covers the IP header → breaks with NAT
# ESP does NOT cover the outer IP header → works with NAT
In practice, always use ESP. If you need integrity without encryption, use ESP with NULL encryption (RFC 2410) — it provides the same integrity guarantees as AH but works through NAT. AH exists in the specs and you will see it in exam questions and old documentation, but modern IPsec deployments are ESP-only. The only edge case for AH is when you need to authenticate the IP header itself and NAT is not in the path — a rare scenario.

5. Transport Mode vs Tunnel Mode

Transport Mode

Encrypts only the payload of the original IP packet. The original IP header stays intact (visible). Used for host-to-host encryption where both endpoints are the IPsec peers. No new IP header is added — lower overhead.

// Transport = sealing the letter but leaving the envelope visible. The postman can see who it's from and to.

Tunnel Mode

Encrypts the entire original IP packet (header + payload) and wraps it in a new IP header. The outer header has the tunnel endpoint addresses; the inner header (encrypted) has the actual source and destination. Used for site-to-site VPNs and roadwarrior access.

// Tunnel = putting the sealed letter inside a new envelope with different addresses. Nobody sees the original.
# Transport mode packet:
# [Original IP Header] [ESP Header] [Encrypted Original Payload] [ESP Trailer] [ICV]
#  src=10.1.0.5          SPI           (TCP/UDP data)
#  dst=10.2.0.5

# Tunnel mode packet:
# [New IP Header] [ESP Header] [Encrypted: Original IP Header + Payload] [ESP Trailer] [ICV]
#  src=203.0.113.1   SPI          (entire original packet)
#  dst=198.51.100.1

# In tunnel mode, the original source (10.1.0.5) and destination (10.2.0.5)
# are invisible to anyone observing the outer packet. They only see the
# tunnel endpoints (203.0.113.1 ↔ 198.51.100.1).

When to use which

Scenario Mode Why
Site-to-site VPN between two gateways Tunnel Hosts behind each gateway are not IPsec-aware
Roadwarrior laptop to corporate gateway Tunnel Laptop gets a virtual IP inside corporate network
Host-to-host encryption (e.g. database replication) Transport Both hosts are IPsec peers, no tunnelling needed
L2TP/IPsec (legacy remote access) Transport IPsec encrypts the L2TP tunnel, L2TP handles the tunnelling

6. Hashing, Integrity Checking & Anti-Replay

Every ESP packet includes an Integrity Check Value (ICV) — a cryptographic hash that proves the packet was not modified in transit. The receiver recomputes the hash and compares it to the ICV. If they do not match, the packet is dropped silently.

How integrity verification works

# For AEAD ciphers (AES-GCM):
# The GCM authentication tag IS the ICV. Encryption and integrity are one operation.
# AES-GCM-256 produces a 128-bit (16-byte) authentication tag.
#
# For separate integrity (HMAC):
# The sender computes HMAC-SHA256 over: ESP Header + Encrypted Payload + ESP Trailer
# and appends the truncated result (128 bits for HMAC-SHA256-128) as the ICV.
#
# The receiver:
# 1. Reads the ICV from the end of the packet
# 2. Recomputes the HMAC over the same fields using the SA's integrity key
# 3. Compares computed HMAC to received ICV
# 4. If mismatch → DROP (tampered or corrupted)
# 5. If match → proceed to decryption

# This is MAC-then-Encrypt for old configs, or AEAD for modern configs.
# AEAD is always preferred — it's faster and avoids padding oracle attacks.

Anti-replay protection

# Every ESP packet has a 32-bit (or 64-bit with ESN) sequence number.
# The receiver maintains a sliding window (typically 64 or 128 packets).
#
#  Window: [sequence 100 ───────── sequence 163]
#
# Packet with seq=150 → inside window → check if already seen → accept
# Packet with seq=99  → before window → DROP (too old, possible replay)
# Packet with seq=164 → ahead of window → advance window → accept
# Packet with seq=150 (again) → inside window, already marked → DROP (replay)
#
# This prevents an attacker from recording and replaying legitimate packets.

# View the anti-replay state:
ip xfrm state | grep -A2 "anti-replay"
# anti-replay context: seq 0x0, oseq 0x2a, bitmap 0x0000003f
#   oseq = outbound sequence counter (42 packets sent)
#   bitmap = which sequence numbers in the window have been received
Anti-replay is subtle but critical. Without it, an attacker who can capture packets could replay a valid "transfer $10,000" packet a hundred times. The sequence number and sliding window make this impossible — each packet is accepted exactly once. Extended Sequence Numbers (ESN, RFC 4304) use 64-bit counters to avoid wraparound on high-throughput links. At 10 Gbps with 1500-byte packets, a 32-bit counter wraps in about 50 minutes. ESN extends that to billions of years.

7. The Four IPsec Deployment Types

IPsec deployments are categorised by where encryption happens and what gets protected. These are often called Type 1 through Type 4, though the terminology varies across vendors and standards bodies. Understanding the types is understanding the architectural choices.

Type 1 — Gateway-to-Gateway (Site-to-Site)

Two IPsec gateways protect traffic between their respective networks. Hosts behind each gateway are unaware of IPsec. The gateways encrypt/decrypt at the network boundary. This is the classic site-to-site VPN: office A talks to office B through an encrypted tunnel between the edge firewalls.

// Two post offices with a locked courier bag between them. Senders and receivers never see the bag.

Type 2 — Host-to-Gateway (Roadwarrior)

A single host (laptop, mobile device) establishes a tunnel to a gateway to access the network behind it. The host is an IPsec peer; hosts on the remote network are not. The roadwarrior gets a virtual IP from the gateway's pool and appears to be on the remote network.

// A diplomat with a secure phone calling the embassy. The embassy connects them to internal extensions.

Type 3 — Host-to-Host (Transport Encryption)

Two hosts encrypt traffic directly between each other, typically in transport mode. No gateway involved. Both hosts run IPsec. Used for database replication, cluster heartbeats, or any host-pair communication that needs encryption without a tunnel overlay. The original IP headers stay visible.

// Two people whispering directly to each other in a room. No courier, no intermediary.

Type 4 — Gateway with Integrity Verification (Hashing Service)

A gateway that inspects every packet's integrity (ICV) against the SPI-keyed hash before forwarding. This is the "hashing service" deployment — the gateway does not necessarily decrypt the payload, but it verifies that every packet is authentic and untampered before allowing it through. Packets that fail the integrity check are dropped at the gateway, never reaching the destination. Used in high-security environments where a dedicated security appliance sits in the path and validates traffic integrity on behalf of the network.

// A security checkpoint that X-rays every sealed envelope. It can't read the contents, but it verifies the seal hasn't been broken. Tampered envelopes are destroyed.

Type 4 in detail — the integrity gateway

Type 4 is the least understood deployment type. The gateway receives ESP packets, extracts the SPI from the header, looks up the corresponding SA (it has the integrity key but may or may not have the encryption key), recomputes the ICV hash, and compares it to the packet's ICV. If the hash matches, the packet is forwarded. If not, it is dropped.

# Type 4 gateway packet flow:
#
# 1. ESP packet arrives at the integrity gateway
#    ┌─────────────────────────────────────────┐
#    │ IP Header → Integrity GW address         │
#    │ ESP Header → SPI: 0xabc12345             │
#    │ Encrypted Payload                         │
#    │ ICV: 0x9f8e7d6c5b4a3928                  │
#    └─────────────────────────────────────────┘
#
# 2. Gateway reads SPI 0xabc12345
# 3. Looks up SA in its SAD → finds integrity key (HMAC-SHA256)
# 4. Computes HMAC-SHA256 over ESP Header + Encrypted Payload
# 5. Compares computed hash to ICV 0x9f8e7d6c5b4a3928
#
# 6a. Match → FORWARD packet to destination (still encrypted)
# 6b. Mismatch → DROP packet (integrity violation)
#
# The gateway NEVER decrypts the payload.
# It only verifies that the packet was not tampered with.
# The destination host performs the actual decryption.

# This provides:
# - Tamper detection at the network perimeter
# - Anti-replay validation (sequence number checking)
# - Protection against packet injection attacks
# - All without giving the gateway access to plaintext traffic

Deployment type comparison

Type Peers Mode Gateway Decrypts? Use Case
Type 1 Gateway ↔ Gateway Tunnel Yes (both ends) Site-to-site VPN, multi-office
Type 2 Host ↔ Gateway Tunnel Yes (gateway end) Remote access, roadwarrior
Type 3 Host ↔ Host Transport N/A (no gateway) DB replication, cluster comms
Type 4 Host ↔ Host via Gateway Transport/Tunnel No (verify only) Integrity validation, high-sec perimeter
Type 4 is what people mean when they talk about "IPsec as a packet integrity service." The gateway has the SPI and the integrity key, so it can verify every packet's hash without decrypting the payload. This is a zero-trust pattern: even the network perimeter device does not get access to plaintext. It just guarantees that everything passing through has not been tampered with and was sent by an authenticated peer. You see this in military networks, financial trading floors, and air-gapped environments where the perimeter must validate traffic but must not have the ability to read it.

8. strongSwan — Site-to-Site Configuration

strongSwan is the standard IPsec implementation on Linux. It handles IKE negotiation and installs SAs into the kernel's XFRM subsystem.

Install

# CentOS / RHEL / Rocky
dnf install -y strongswan

# Debian / Ubuntu
apt install -y strongswan strongswan-pki

# Enable and start
systemctl enable --now strongswan

Site-to-site tunnel with PSK

# Gateway A: 203.0.113.1, LAN 10.1.0.0/24
# Gateway B: 198.51.100.1, LAN 10.2.0.0/24

# === Gateway A: /etc/swanctl/conf.d/site-to-site.conf ===
connections {
    site-b {
        version = 2
        local_addrs = 203.0.113.1
        remote_addrs = 198.51.100.1

        local {
            auth = psk
            id = gateway-a@example.com
        }
        remote {
            auth = psk
            id = gateway-b@example.com
        }

        children {
            lan-to-lan {
                local_ts = 10.1.0.0/24
                remote_ts = 10.2.0.0/24
                esp_proposals = aes256gcm128-sha256-modp2048
                start_action = start
                dpd_action = restart
                close_action = restart
                rekey_time = 3600
            }
        }

        proposals = aes256-sha256-modp2048
        rekey_time = 14400
        dpd_delay = 30
    }
}

secrets {
    ike-site-b {
        id-a = gateway-a@example.com
        id-b = gateway-b@example.com
        secret = "CHANGE_THIS_TO_A_STRONG_RANDOM_PSK_64_CHARS_MINIMUM"
    }
}
# === Gateway B: /etc/swanctl/conf.d/site-to-site.conf ===
connections {
    site-a {
        version = 2
        local_addrs = 198.51.100.1
        remote_addrs = 203.0.113.1

        local {
            auth = psk
            id = gateway-b@example.com
        }
        remote {
            auth = psk
            id = gateway-a@example.com
        }

        children {
            lan-to-lan {
                local_ts = 10.2.0.0/24
                remote_ts = 10.1.0.0/24
                esp_proposals = aes256gcm128-sha256-modp2048
                start_action = start
                dpd_action = restart
                close_action = restart
                rekey_time = 3600
            }
        }

        proposals = aes256-sha256-modp2048
        rekey_time = 14400
        dpd_delay = 30
    }
}

secrets {
    ike-site-a {
        id-a = gateway-a@example.com
        id-b = gateway-b@example.com
        secret = "CHANGE_THIS_TO_A_STRONG_RANDOM_PSK_64_CHARS_MINIMUM"
    }
}
# Load and start
swanctl --load-all
swanctl --initiate --child lan-to-lan

# Verify
swanctl --list-sas
# site-b: #1, ESTABLISHED, IKEv2, ...
#   lan-to-lan: #1, reqid 1, INSTALLED, TUNNEL, ESP:AES_GCM_16_256
#     10.1.0.0/24 === 10.2.0.0/24
Pre-shared keys are fine for a pair of gateways you control, but they do not scale. For more than two or three tunnels, use X.509 certificates from your internal CA (step-ca from the TLS Masterclass). Certificate authentication eliminates shared secrets and lets you add new peers without distributing keys to every existing peer.

9. strongSwan with Certificates

# Generate a CA and gateway certificates using strongSwan PKI
# (Or use step-ca from the TLS masterclass)

# Create CA
pki --gen --type rsa --size 4096 --outform pem > /etc/swanctl/private/ca-key.pem
pki --self --ca --lifetime 3650 \
    --in /etc/swanctl/private/ca-key.pem \
    --dn "C=CA, O=kldload, CN=IPsec CA" \
    --outform pem > /etc/swanctl/x509ca/ca-cert.pem

# Generate Gateway A key and certificate
pki --gen --type rsa --size 2048 --outform pem > /etc/swanctl/private/gw-a-key.pem
pki --req --type rsa \
    --in /etc/swanctl/private/gw-a-key.pem \
    --dn "C=CA, O=kldload, CN=gateway-a.example.com" \
    --san gateway-a.example.com --san 203.0.113.1 \
    --outform pem > /tmp/gw-a-req.pem
pki --issue --cacert /etc/swanctl/x509ca/ca-cert.pem \
    --cakey /etc/swanctl/private/ca-key.pem \
    --in /tmp/gw-a-req.pem --lifetime 365 \
    --outform pem > /etc/swanctl/x509/gw-a-cert.pem
# /etc/swanctl/conf.d/site-to-site-certs.conf (Gateway A)
connections {
    site-b {
        version = 2
        local_addrs = 203.0.113.1
        remote_addrs = 198.51.100.1

        local {
            auth = pubkey
            certs = gw-a-cert.pem
            id = gateway-a.example.com
        }
        remote {
            auth = pubkey
            id = gateway-b.example.com
        }

        children {
            lan-to-lan {
                local_ts = 10.1.0.0/24
                remote_ts = 10.2.0.0/24
                esp_proposals = aes256gcm128-modp2048
                start_action = start
                dpd_action = restart
            }
        }

        proposals = aes256-sha256-x25519
    }
}

10. Roadwarrior Configuration

# Gateway: /etc/swanctl/conf.d/roadwarrior.conf
connections {
    roadwarrior {
        version = 2
        local_addrs = %any
        remote_addrs = %any

        pools = rw-pool

        local {
            auth = pubkey
            certs = gw-cert.pem
            id = vpn.example.com
        }
        remote {
            auth = eap-mschapv2
            eap_id = %any
        }

        children {
            rw-child {
                local_ts = 10.0.0.0/8
                esp_proposals = aes256gcm128-modp2048
                dpd_action = clear
            }
        }

        proposals = aes256-sha256-x25519
        send_certreq = yes
    }
}

pools {
    rw-pool {
        addrs = 10.99.0.0/24
        dns = 10.1.0.53
    }
}

secrets {
    eap-alice {
        id = alice
        secret = "alice_password_here"
    }
    eap-bob {
        id = bob
        secret = "bob_password_here"
    }
}
# Client (laptop) — strongSwan or built-in IKEv2 client
# macOS/iOS/Windows all have native IKEv2 support
# Android: strongSwan app from F-Droid/Play Store

# Linux client: /etc/swanctl/conf.d/client.conf
connections {
    office {
        version = 2
        remote_addrs = vpn.example.com
        vips = 0.0.0.0

        local {
            auth = eap-mschapv2
            eap_id = alice
        }
        remote {
            auth = pubkey
            id = vpn.example.com
        }

        children {
            office-net {
                remote_ts = 10.0.0.0/8
                start_action = trap
                dpd_action = restart
            }
        }
    }
}

secrets {
    eap-alice {
        id = alice
        secret = "alice_password_here"
    }
}
Every major operating system has a built-in IKEv2 client. This is the killer advantage of IPsec over WireGuard for remote access — you do not need to install anything on the client. macOS, iOS, Windows, and Android can all connect to a strongSwan gateway using the native VPN settings. For Linux clients, strongSwan or NetworkManager with the strongswan plugin works. This makes IPsec the practical choice for bring-your-own-device environments where you cannot install WireGuard on every device.

11. Libreswan Alternative

Libreswan is the other major Linux IPsec implementation. It is the default on RHEL/CentOS/Rocky (installed as ipsec) and has FIPS 140-3 validation. The configuration syntax is different from strongSwan.

# Install
dnf install -y libreswan
systemctl enable --now ipsec

# Initialise the NSS database (Libreswan uses NSS for crypto)
ipsec initnss

# /etc/ipsec.d/site-to-site.conf
conn site-b
    type=tunnel
    authby=secret
    left=203.0.113.1
    leftsubnet=10.1.0.0/24
    leftid=@gateway-a
    right=198.51.100.1
    rightsubnet=10.2.0.0/24
    rightid=@gateway-b
    ike=aes256-sha2_256-modp2048
    esp=aes256gcm128
    ikelifetime=4h
    salifetime=1h
    dpddelay=30
    dpdtimeout=120
    dpdaction=restart
    auto=start

# /etc/ipsec.d/site-b.secrets
@gateway-a @gateway-b : PSK "STRONG_RANDOM_KEY_HERE"

# Load and start
ipsec auto --add site-b
ipsec auto --up site-b

# Verify
ipsec status
ipsec trafficstatus

12. XFRM Interface — Modern Route-Based IPsec

Traditional IPsec on Linux uses policy-based routing: traffic selectors determine which packets enter the tunnel. XFRM interfaces (since kernel 4.19) create a virtual tunnel interface that you route traffic through, just like a WireGuard or GRE interface. This is much easier to manage with BGP and dynamic routing.

# Create an XFRM interface
ip link add ipsec0 type xfrm dev eth0 if_id 42
ip addr add 169.254.0.1/30 dev ipsec0
ip link set ipsec0 up

# Route traffic through the XFRM interface
ip route add 10.2.0.0/24 dev ipsec0

# strongSwan config with XFRM interface
connections {
    site-b {
        version = 2
        local_addrs = 203.0.113.1
        remote_addrs = 198.51.100.1
        if_id_in = 42
        if_id_out = 42

        local {
            auth = pubkey
            certs = gw-a-cert.pem
        }
        remote {
            auth = pubkey
        }

        children {
            pass-all {
                local_ts = 0.0.0.0/0
                remote_ts = 0.0.0.0/0
                esp_proposals = aes256gcm128-modp2048
                start_action = trap
            }
        }
    }
}

# Now you can run BGP over ipsec0 with BIRD
# This is the kldload way: XFRM interface + BIRD BGP = route-based IPsec VPN
XFRM interfaces are how you do IPsec properly on modern Linux. Policy-based IPsec (traffic selectors) works for simple site-to-site tunnels, but it becomes unmanageable when you need dynamic routing, multiple tunnels, or BGP. With XFRM interfaces you treat IPsec tunnels exactly like any other network interface — add routes, run BGP, apply nftables rules, monitor with ip -s link show ipsec0. This is the same model that WireGuard uses, and it is the model that the backplane and BIRD BGP masterclasses assume.

13. nftables Rules for IPsec

# Allow IKE and ESP through the firewall
nft add rule inet filter input udp dport 500 accept comment "IKE"
nft add rule inet filter input udp dport 4500 accept comment "IKE NAT-T"
nft add rule inet filter input ip protocol esp accept comment "ESP"
nft add rule inet filter input ip protocol ah accept comment "AH"

# Allow traffic on the XFRM interface
nft add rule inet filter input iifname "ipsec0" accept
nft add rule inet filter forward iifname "ipsec0" accept
nft add rule inet filter forward oifname "ipsec0" accept

# NAT exemption for IPsec traffic (if you have masquerading)
nft add rule inet nat postrouting oifname "ipsec0" accept

# Match on IPsec policy (for policy-based routing)
nft add rule inet filter forward meta ipsec exists accept comment "IPsec-protected traffic"

# Log dropped IPsec-related traffic for debugging
nft add rule inet filter input udp dport { 500, 4500 } log prefix "ipsec-ike: " accept
nft add rule inet filter input ip protocol esp log prefix "ipsec-esp: " accept

14. Dead Peer Detection & Rekeying

# DPD — checks if the remote peer is alive
# Sends an IKE INFORMATIONAL request at the configured interval
# If no response after timeout, takes the configured action

# strongSwan DPD settings:
connections {
    site-b {
        dpd_delay = 30          # Send DPD every 30 seconds of inactivity
        children {
            lan-to-lan {
                dpd_action = restart    # restart = tear down and re-establish
                                         # clear = tear down only
                                         # trap = install trap policy, rekey on demand
            }
        }
    }
}

# Rekeying — replace keys before they expire
# IKE SA rekey: negotiates new IKE keys (default: 4 hours)
# Child SA rekey: negotiates new ESP keys (default: 1 hour)
# Rekeying happens automatically, no traffic interruption

# strongSwan rekey settings:
connections {
    site-b {
        rekey_time = 14400      # IKE SA lifetime: 4 hours
        children {
            lan-to-lan {
                rekey_time = 3600   # Child SA lifetime: 1 hour
                rekey_bytes = 0     # Rekey after N bytes (0 = disabled)
                rekey_packets = 0   # Rekey after N packets (0 = disabled)
            }
        }
    }
}

# Monitor rekey events
swanctl --log

15. Monitoring & Troubleshooting

# === strongSwan ===
swanctl --list-sas         # Show all active SAs
swanctl --list-conns       # Show configured connections
swanctl --list-certs       # Show loaded certificates
swanctl --stats            # IKE statistics
swanctl --log              # Real-time log (IKE events, rekeys, DPD)

# === Kernel XFRM ===
ip xfrm state              # Show all SAs (SPIs, algorithms, keys, counters)
ip xfrm policy             # Show all security policies
ip -s xfrm state           # SA statistics (bytes, packets, errors)
ip xfrm monitor            # Real-time XFRM events

# === Packet capture ===
# Capture IKE negotiation
tcpdump -ni eth0 udp port 500 or udp port 4500

# Capture ESP packets (encrypted — you won't see payloads)
tcpdump -ni eth0 esp

# Capture decrypted traffic on the XFRM interface
tcpdump -ni ipsec0 -w /tmp/decrypted.pcap

# === Libreswan ===
ipsec status               # Connection status
ipsec trafficstatus         # Bytes/packets per tunnel
ipsec auto --status        # Detailed SA info
ipsec look                 # Quick overview

# === Common issues ===
# "no proposal chosen" → cipher mismatch between peers
# "peer not responding" → firewall blocking UDP 500/4500 or ESP
# "authentication failed" → wrong PSK or certificate
# "traffic selector mismatch" → local_ts/remote_ts don't match
# "DPD timeout" → peer is down or unreachable

Verify SPI and integrity on the wire

# View SPI values in kernel state
ip xfrm state | grep spi
# proto esp spi 0xc0a80123 reqid 1 mode tunnel
# proto esp spi 0xdead5678 reqid 1 mode tunnel

# Watch packets and confirm SPI matches
tcpdump -ni eth0 esp -v 2>&1 | grep SPI
# SPI 0xc0a80123

# Check packet counts per SA
ip -s xfrm state
# stats:
#   replay-window 64 replay 0 failed 0
#   bytes: 1234567  packets: 8901  errors: 0

# "failed" = integrity check failures (ICV mismatch)
# "replay" = replayed packets detected and dropped
# Both should be 0 in normal operation. Non-zero = attack or corruption.
The ip -s xfrm state output is your single most important diagnostic tool. The failed counter tells you how many packets failed integrity verification — meaning either the path is corrupting packets, there is a key mismatch between peers, or someone is injecting traffic. The replay counter tells you how many replay attempts were detected. In a healthy tunnel, both are zero. In a tunnel under attack, they are not. This is the Type 4 deployment in action: every packet is verified against its SPI-keyed hash, and failures are counted and dropped.

16. IPsec vs WireGuard — When to Use Which

Criterion IPsec WireGuard
Configuration complexity High (IKE proposals, traffic selectors, SA lifetimes) Low (public key, endpoint, allowed IPs)
Interoperability Universal (every router, firewall, cloud gateway) Limited (needs WireGuard on both ends)
FIPS 140-3 Yes (Libreswan validated) No (ChaCha20/Curve25519 not FIPS)
Native OS support Built-in on macOS, iOS, Windows, Android Requires app/driver on most platforms
Performance Good (AES-NI accelerated) Excellent (in-kernel, minimal overhead)
Transport mode Yes No (tunnel only)
Key exchange IKE (automated, with rekeying) Static keys (no built-in rekeying protocol)
Code audit surface ~400K lines (strongSwan) ~4K lines

Use IPsec when: connecting to third-party equipment (cloud VPN gateways, partner networks, hardware firewalls), FIPS compliance is required, native OS VPN clients are needed, or transport mode encryption is needed. Use WireGuard when: you control both ends, simplicity and performance are priorities, and compliance is not a factor.

In a kldload stack, the typical pattern is WireGuard for your internal backplane mesh and IPsec for connections to the outside world. Your nodes talk to each other over WireGuard because it is fast and simple. When you need to connect to AWS, Azure, a partner's Cisco ASA, or a government network that requires FIPS, you bring up an IPsec tunnel. The two can coexist on the same host. Route internal traffic through wg0, route external VPN traffic through ipsec0, and let BIRD manage the routing table.

17. Quick Reference — Troubleshooting Table

Symptom Likely Cause Fix
NO_PROPOSAL_CHOSEN Cipher suite mismatch Ensure both sides have matching proposals and esp_proposals
AUTHENTICATION_FAILED PSK mismatch or wrong certificate Verify PSK matches exactly. Check cert CN/SAN matches id
TS_UNACCEPTABLE Traffic selectors mismatch A's local_ts must equal B's remote_ts and vice versa
Tunnel up, no traffic flows Missing routes or nftables blocking Check ip route, ip xfrm policy, nftables forward rules
Frequent disconnects DPD timeout or NAT device dropping UDP state Reduce dpd_delay. Ensure NAT-T is active (UDP 4500)
Integrity failures (non-zero failed count) Path corruption, key mismatch, or injection attack Check ip -s xfrm state. Rekey the SA. Investigate the path.
Replay counter incrementing Packet reordering or replay attack Increase replay window or investigate network path
Cannot connect from behind NAT NAT-T not enabled Allow UDP 4500. strongSwan does NAT-T automatically

Related pages

← DNS Masterclass eBPF Masterclass →