Configure Docker to use ZFS

# Create a dedicated dataset for Docker
zfs create -o mountpoint=/var/lib/docker rpool/docker
zfs create rpool/docker/volumes

# Configure the storage driver
mkdir -p /etc/docker
cat > /etc/docker/daemon.json <<'EOF'
{
  "storage-driver": "zfs",
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  },
  "default-ulimits": {
    "nofile": { "Name": "nofile", "Hard": 65536, "Soft": 65536 }
  }
}
EOF

# Restart Docker
systemctl restart docker

# Verify
docker info | grep -A5 'Storage Driver'
# Storage Driver: zfs
#  Zpool: rpool
#  Zpool Health: ONLINE
#  Parent Dataset: rpool/docker

# Check ZFS datasets created by Docker
zfs list -r rpool/docker | head -20

What ZFS gives Docker

With the ZFS storage driver, every docker pull creates ZFS datasets for each image layer. Every docker run creates a ZFS clone for the container's writable layer. This means:

# Snapshot before pulling a new image
zfs snapshot rpool/docker@before-pull-$(date +%F)
docker pull nginx:latest
# Something wrong with the new image? Rollback.
docker stop $(docker ps -q)
zfs rollback rpool/docker@before-pull-2026-03-23

# See compression savings on container data
zfs get compressratio rpool/docker
# rpool/docker  compressratio  2.83x  -
# That's 2.83x compression on all image layers. Free disk space.

# Clone a volume for testing (instant, zero disk cost)
zfs snapshot rpool/docker/volumes/myapp-data@test
zfs clone rpool/docker/volumes/myapp-data@test rpool/docker/volumes/myapp-data-test

# Replicate Docker state to another host
syncoid -r rpool/docker root@node2:rpool/docker

Podman on ZFS

# Install Podman (already included in kldloadOS desktop/server profiles)
dnf install -y podman

# Rootless setup — runs as your user, no daemon, no root
podman info | grep -A3 graphDriver
# graphDriverName: zfs

# Run a container (same syntax as Docker)
podman run -d --name web -p 8080:80 nginx:alpine

# Podman Compose (drop-in for docker compose)
dnf install -y podman-compose
podman-compose up -d

# Generate systemd unit from a running container
podman generate systemd --name web --files --new
systemctl --user enable --now container-web.service

# Rootless means: no daemon to crash, no root to exploit,
# and the container runs as your UID with user namespaces

When to use Podman vs Docker

Use Podman when you want rootless containers, systemd integration, no long-running daemon, or you're running on a machine where Docker's daemon model is a liability (single-user servers, CI runners, embedded systems).

Use Docker when you need Docker Compose with full feature parity, Docker Swarm, or your team's tooling depends on the Docker API socket.

Use both — they coexist on kldloadOS. Same images, same registries, same OCI format.

Set up a local registry

# Create a ZFS dataset for the registry
zfs create -o compression=zstd rpool/srv/registry

# Run the registry
docker run -d --restart=always --name registry \
    -p 5000:5000 \
    -v /srv/registry:/var/lib/registry \
    registry:2

# Tag and push an image
docker tag myapp:latest localhost:5000/myapp:latest
docker push localhost:5000/myapp:latest

# Pull from the registry (from any host on the network)
docker pull 192.168.1.10:5000/myapp:latest

# Snapshot the registry before changes
ksnap /srv/registry

# Replicate the registry to another host
syncoid rpool/srv/registry root@node2:rpool/srv/registry

# Check compression savings
zfs get compressratio rpool/srv/registry
# Container images compress well — expect 2-4x with zstd

TLS and authentication

# Generate a self-signed cert (or use Let's Encrypt)
mkdir -p /srv/registry/certs /srv/registry/auth
openssl req -x509 -nodes -days 3650 -newkey rsa:4096 \
    -keyout /srv/registry/certs/registry.key \
    -out /srv/registry/certs/registry.crt \
    -subj "/CN=registry.local"

# Create htpasswd auth
dnf install -y httpd-tools
htpasswd -Bc /srv/registry/auth/htpasswd admin

# Run registry with TLS + auth
docker run -d --restart=always --name registry \
    -p 5000:5000 \
    -v /srv/registry/data:/var/lib/registry \
    -v /srv/registry/certs:/certs \
    -v /srv/registry/auth:/auth \
    -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/registry.crt \
    -e REGISTRY_HTTP_TLS_KEY=/certs/registry.key \
    -e REGISTRY_AUTH=htpasswd \
    -e REGISTRY_AUTH_HTPASSWD_REALM="kldload Registry" \
    -e REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd \
    registry:2

# Login from any host
docker login registry.local:5000

Web + Database + Cache

# docker-compose.yml — typical web application stack
services:
  web:
    image: nginx:1.27-alpine
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
      - ./html:/usr/share/nginx/html:ro
    depends_on:
      app:
        condition: service_healthy
    restart: unless-stopped

  app:
    image: node:22-alpine
    working_dir: /app
    volumes:
      - ./app:/app
    command: node server.js
    healthcheck:
      test: ["CMD", "wget", "-qO-", "http://localhost:3000/health"]
      interval: 10s
      timeout: 5s
      retries: 3
    environment:
      - DATABASE_URL=postgresql://app:secret@db:5432/myapp
      - REDIS_URL=redis://cache:6379
    restart: unless-stopped

  db:
    image: postgres:16-alpine
    volumes:
      - pgdata:/var/lib/postgresql/data
    environment:
      POSTGRES_DB: myapp
      POSTGRES_USER: app
      POSTGRES_PASSWORD: secret
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U app -d myapp"]
      interval: 10s
      timeout: 5s
      retries: 5
    restart: unless-stopped

  cache:
    image: redis:7-alpine
    command: redis-server --maxmemory 256mb --maxmemory-policy allkeys-lru
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      timeout: 5s
      retries: 3
    restart: unless-stopped

volumes:
  pgdata:
    driver: local

# Create a ZFS dataset for the postgres volume with tuned recordsize
zfs create -o recordsize=16k rpool/docker/volumes/pgdata

# Snapshot before deploying
ksnap /var/lib/docker/volumes/pgdata

# Deploy
docker compose up -d

Firecracker on kldloadOS

# Download Firecracker
ARCH=$(uname -m)
curl -L "https://github.com/firecracker-microvm/firecracker/releases/latest/download/firecracker-v1.9.1-${ARCH}.tgz" | \
    tar xz -C /usr/local/bin --strip-components=1

# Prepare rootfs on ZFS
zfs create rpool/srv/firecracker
# (copy or build your rootfs.ext4 and vmlinux kernel here)

# Snapshot the clean rootfs — rollback after each run
zfs snapshot rpool/srv/firecracker@clean

# Launch a microVM
firecracker --api-sock /tmp/fc.sock --config-file config.json

# After the workload completes, rollback to pristine state
zfs rollback rpool/srv/firecracker@clean
# Next microVM starts with a perfectly clean filesystem. Every time.

Resource limits in practice

# Memory limit: container is OOM-killed if it exceeds 512MB
docker run -d --memory=512m --memory-swap=512m myapp

# CPU limit: container gets at most 1.5 CPU cores
docker run -d --cpus=1.5 myapp

# CPU shares: relative weight (default 1024)
docker run -d --cpu-shares=512 myapp    # half priority
docker run -d --cpu-shares=2048 myapp   # double priority

# I/O limit: cap write throughput to 50MB/s
docker run -d --device-write-bps /dev/zd0:50mb myapp

# PID limit: prevent fork bombs
docker run -d --pids-limit=100 myapp

# In Compose:
services:
  app:
    image: myapp:latest
    deploy:
      resources:
        limits:
          cpus: '2.0'
          memory: 1G
        reservations:
          cpus: '0.5'
          memory: 256M

Security hardening checklist

# 1. Rootless containers (Podman does this by default)
podman run --user 1000:1000 myapp

# 2. Read-only root filesystem
docker run --read-only --tmpfs /tmp --tmpfs /run myapp

# 3. Drop ALL capabilities, add only what's needed
docker run --cap-drop=ALL --cap-add=NET_BIND_SERVICE myapp

# 4. No new privileges (prevent setuid binaries)
docker run --security-opt=no-new-privileges myapp

# 5. Seccomp profile (restrict syscalls)
docker run --security-opt seccomp=strict-profile.json myapp

# 6. AppArmor profile
docker run --security-opt apparmor=docker-custom myapp

# 7. Resource limits (prevent resource exhaustion)
docker run --memory=512m --cpus=1.0 --pids-limit=100 myapp

# 8. Non-root user in Dockerfile
# FROM alpine:3.19
# RUN adduser -D -u 1000 appuser
# USER appuser
# CMD ["/app/server"]

# 9. Scan images for vulnerabilities
trivy image myapp:latest

# 10. Never use --privileged unless you have a specific,
# documented reason. --privileged gives the container
# full access to the host. It defeats the purpose of
# containerization.