The ZFS Boot Chain — from power-on to login prompt.

Booting Linux from a ZFS root is the single hardest thing about running ZFS on Linux. It touches firmware, bootloaders, kernel modules, initramfs generators, systemd services, dataset properties, and pool import mechanics — and if any one of these is wrong, you get a blinking cursor instead of a login prompt. This page documents the complete chain, from the moment current hits the CPU to the moment you see a shell. Every step, what it does, why it matters, what breaks, and how to fix it.

The complete chain at a glance

Every ZFS-on-root Linux system follows this sequence. No exceptions.

UEFI firmware
  ↓ reads NVRAM boot entries
  ↓ loads EFI binary from ESP (FAT32 partition)
Bootloader (ZFSBootMenu / GRUB / systemd-boot)
  ↓ imports ZFS pool (or reads /boot on ext4)
  ↓ finds vmlinuz + initramfs
  ↓ loads them into memory (kexec or chainload)
Linux kernel
  ↓ decompresses, initializes hardware
  ↓ mounts initramfs as temporary root
  ↓ starts systemd (PID 1) inside initramfs
Initramfs (dracut / initramfs-tools / mkinitcpio)
  ↓ parses root=zfs:... from kernel command line
  ↓ modprobe zfs (loads zfs.ko, spl.ko)
  ↓ zpool import -N rpool
  ↓ mount -t zfs rpool/ROOT/myhost /sysroot
  ↓ switch_root /sysroot
Real systemd (PID 1 on the real root)
  ↓ zfs-import-cache.service (idempotent)
  ↓ zfs-mount.service (mounts /home, /var, etc.)
  ↓ zfs-zed.service, NetworkManager, sshd...
Login prompt

If you understand this diagram, you understand ZFS boot. The rest of this page is the details of each layer — and what happens when each one breaks.

Layer 1: UEFI firmware and the ESP

Every modern x86_64 system boots via UEFI (Unified Extensible Firmware Interface). The firmware initializes the CPU, memory controller, and PCIe bus, then looks for something to execute. It reads the NVRAM boot entries — an ordered list of EFI executables stored in the firmware's non-volatile memory — and loads the first one it finds.

The EFI binary lives on the ESP (EFI System Partition) — a small FAT32 partition, typically 512MB–1GB, at the start of the disk. The ESP is the only part of the boot chain that must be on a traditional filesystem. ZFS cannot serve as an ESP because UEFI firmware only understands FAT32. This is why every ZFS-on-root system has at least two partitions: the ESP (FAT32) and the ZFS partition.

# Typical disk layout for ZFS on root
/dev/vda1   512M  EFI System Partition (FAT32, mounted at /boot/efi)
/dev/vda2   rest  ZFS partition (Solaris / ZFS type, entire pool)

# View EFI boot entries
efibootmgr -v
BootOrder: 0001,0002
Boot0001* ZFSBootMenu     HD(1,GPT,...)/File(\EFI\zbm\BOOTX64.EFI)
Boot0002* ZFSBootMenu (Backup) HD(1,GPT,...)/File(\EFI\zbm\BOOTX64-BACKUP.EFI)

Layer 2: The bootloader — ZFSBootMenu vs GRUB vs systemd-boot

The bootloader's job is to find a Linux kernel and initramfs, load them into memory, and execute the kernel. For a ZFS-on-root system, this is harder than it sounds — the kernel and initramfs live inside the ZFS pool, so the bootloader needs to understand ZFS to read them.

ZFSBootMenu — the right answer

ZFSBootMenu is a standalone EFI binary built specifically for ZFS-on-root systems. It is a full Linux kernel + initramfs compiled into a single .EFI file. When the firmware loads it, it boots a tiny Linux environment that can import ZFS pools, enumerate boot environments, present a menu, and use kexec to jump directly into the selected kernel.

The key insight: ZFSBootMenu doesn't chainload GRUB. It doesn't read config files from /boot/grub/. It reads the kernel and initramfs directly from the ZFS dataset and uses kexec to execute them. This means: no GRUB configuration, no grub-mkconfig, no grub-install. The kernel and initramfs are the only things that matter.

# ZFSBootMenu reads these properties to find boot environments:
zfs get mountpoint rpool/ROOT/myhost
#  rpool/ROOT/myhost  mountpoint  /  local

# The kernel and initramfs live inside the BE:
ls /boot/
vmlinuz-5.14.0-687.el9.x86_64
initramfs-5.14.0-687.el9.x86_64.img

# ZFSBootMenu key bindings at the menu:
#   Enter    = boot selected BE
#   e        = edit kernel command line
#   s        = open a shell (full ZFS tools available)
#   d        = set default BE
#   p        = pool status

GRUB with ZFS — the legacy approach

Before ZFSBootMenu, GRUB was the only option for ZFS-on-root. GRUB has a built-in ZFS reader (not the kernel module — its own read-only implementation) that can read files from ZFS datasets. This works, but it's fragile:

# GRUB ZFS boot (legacy approach — not recommended)
grub-probe /
#  zfs

grub-probe --device /dev/vda2 --target=fs_label
#  rpool

# grub.cfg entry for ZFS root:
menuentry 'CentOS Stream 9' {
  search --no-floppy --set=root --label rpool
  linux  /@/boot/vmlinuz-5.14.0-687.el9.x86_64 root=ZFS=rpool/ROOT/myhost ro
  initrd /@/boot/initramfs-5.14.0-687.el9.x86_64.img
}

systemd-boot with ZFS — the "almost" solution

systemd-boot (formerly gummiboot) is a simple UEFI boot manager. It reads boot entries from the ESP itself — the kernel and initramfs must live on the ESP, not in the ZFS pool. This means you copy your kernel and initramfs to the FAT32 partition after every kernel update.

# systemd-boot with ZFS root (requires kernel + initramfs on ESP)
bootctl install

# /boot/efi/loader/entries/centos.conf
title   CentOS Stream 9
linux   /vmlinuz-5.14.0-687.el9.x86_64
initrd  /initramfs-5.14.0-687.el9.x86_64.img
options root=zfs:rpool/ROOT/myhost ro

# After every kernel update, copy new files to ESP:
cp /boot/vmlinuz-* /boot/efi/
cp /boot/initramfs-* /boot/efi/

systemd-boot is fast and simple, but it doesn't understand ZFS. It can't enumerate boot environments. It can't read datasets. It just loads whatever kernel is on the ESP. For ZFS-on-root with boot environments, it's a non-starter.

Comparison: bootloaders for ZFS on root

Phase 1: Build time — what the installer does

Before a system can boot from ZFS, the installer must build every piece of the chain and install it in the right place. This is what kldload automates across all 8 supported distros. Here is every step, in order.

# RPM distros (CentOS, RHEL, Rocky, Fedora)
dnf install kernel kernel-core kernel-modules kernel-devel

# Debian / Ubuntu
apt install linux-image-amd64 linux-headers-amd64

# Arch Linux
pacman -S linux linux-headers

# Alpine
apk add linux-lts linux-lts-dev

# RPM distros
dnf install zfs zfs-dkms dkms gcc make

# Debian / Ubuntu
apt install zfsutils-linux zfs-dkms

# Arch Linux
pacman -S zfs-dkms

# Alpine (kmod-zfs is precompiled for the LTS kernel)
apk add zfs zfs-lts

dkms build -m zfs -v 2.2.9 -k 5.14.0-687.el9.x86_64
dkms install -m zfs -v 2.2.9 -k 5.14.0-687.el9.x86_64
depmod -a 5.14.0-687.el9.x86_64

# Verify DKMS built successfully
dkms status
zfs/2.2.9, 5.14.0-687.el9.x86_64, x86_64: installed

# Verify the module is loadable
modinfo -n zfs
/usr/lib/modules/5.14.0-687.el9.x86_64/extra/zfs/zfs.ko

# RPM distros (CentOS, RHEL, Rocky, Fedora)
chroot /target dnf install -y zfs-dracut

# Debian / Ubuntu (included with zfs-initramfs)
chroot /target apt install -y zfs-initramfs

# Arch Linux (included with zfs-dkms AUR package)
# mkinitcpio is configured via /etc/mkinitcpio.conf HOOKS

# dracut (CentOS, RHEL, Rocky, Fedora)
dracut --force --add "zfs" --kver 5.14.0-687.el9.x86_64

# initramfs-tools (Debian, Ubuntu)
update-initramfs -c -k all

# mkinitcpio (Arch Linux)
mkinitcpio -P

# mkinitfs (Alpine)
mkinitfs -k 6.6.80-0-lts

# Generate a stable, unique hostid
zgenhostid -f

# Verify
hostid
007f0101

# The file is 4 bytes, binary
xxd /etc/hostid
00000000: 0101 7f00                                ....

# Download the ZFSBootMenu EFI binary
curl -sL -o /tmp/zfsbootmenu.EFI https://get.zfsbootmenu.org/efi

# Install to the ESP (primary + fallback + backup)
mkdir -p /boot/efi/EFI/zbm /boot/efi/EFI/BOOT
cp /tmp/zfsbootmenu.EFI /boot/efi/EFI/zbm/BOOTX64.EFI
cp /tmp/zfsbootmenu.EFI /boot/efi/EFI/zbm/BOOTX64-BACKUP.EFI
cp /tmp/zfsbootmenu.EFI /boot/efi/EFI/BOOT/BOOTX64.EFI

# Write startup.nsh for UEFI shell environments (exported images)
echo '\EFI\BOOT\BOOTX64.EFI' > /boot/efi/startup.nsh

# Register with UEFI firmware
efibootmgr -c -d /dev/vda -p 1 -L "ZFSBootMenu" -l '\EFI\zbm\BOOTX64.EFI'
efibootmgr -c -d /dev/vda -p 1 -L "ZFSBootMenu (Backup)" -l '\EFI\zbm\BOOTX64-BACKUP.EFI'

# /etc/fstab — only the ESP needs an entry. ZFS handles everything else.
UUID=ABCD-1234 /boot/efi vfat umask=0077 0 1

# Generate zpool.cache — tells ZFS which pools to import at boot
zpool set cachefile=/etc/zfs/zpool.cache rpool

systemctl enable zfs-import-cache.service
systemctl enable zfs-mount.service
systemctl enable zfs-zed.service
systemctl enable zfs.target
systemctl enable zfs-import.target

Phase 2: Boot time — power-on to login prompt

The root= kernel parameter

The kernel command line parameter root= tells the initramfs where to find the real root filesystem. For ZFS, there are several valid forms:

# View the kernel command line of the running system
cat /proc/cmdline
root=zfs:rpool/ROOT/myhost ro quiet

# ZFSBootMenu sets root= automatically based on the selected boot environment.
# You never edit this manually unless you're debugging.

ZFS dataset properties that affect boot

Three ZFS properties control how datasets interact with the boot process: mountpoint, canmount, and com.ubuntu.zsys:bootfs (Ubuntu-specific). Getting these wrong is the second most common cause of boot failures after missing initramfs modules.

mountpoint

The mountpoint property tells ZFS where to mount a dataset in the filesystem hierarchy. For boot environments, the root dataset must have mountpoint=/. Child datasets inherit their mountpoints relative to the parent:

# Typical kldload dataset layout
zfs list -o name,mountpoint,canmount
NAME                        MOUNTPOINT    CANMOUNT
rpool                       none          off
rpool/ROOT                  none          off
rpool/ROOT/myhost           /             noauto
rpool/ROOT/myhost/home      /home         on
rpool/ROOT/myhost/var       /var          on
rpool/ROOT/myhost/var/log   /var/log      on
rpool/data                  /srv          on

canmount

The canmount property has three values, and each one has a specific role in the boot chain:

Legacy mount vs ZFS mount

ZFS datasets can be mounted in two ways:

# ZFS mount (default) — ZFS handles mounting
zfs set mountpoint=/var/lib/docker rpool/docker
# Mounted automatically by zfs-mount.service

# Legacy mount — fstab handles mounting
zfs set mountpoint=legacy rpool/docker
# Add to /etc/fstab:
# rpool/docker  /var/lib/docker  zfs  defaults  0  0

For boot, the root dataset is always a ZFS mount (not legacy). The initramfs uses mount -t zfs rpool/ROOT/myhost /sysroot directly. Child datasets like /home and /var are also ZFS mounts, handled by zfs-mount.service. You only need legacy mounts for edge cases like Docker's storage driver, which expects fstab control.

Pool import at boot: cache vs scan

Before ZFS can mount any datasets, the pool must be imported. There are two mechanisms for this, and they behave very differently:

# View the current cachefile setting
zpool get cachefile rpool
NAME   PROPERTY   VALUE                    SOURCE
rpool  cachefile  /etc/zfs/zpool.cache     local

# Regenerate zpool.cache
zpool set cachefile=/etc/zfs/zpool.cache rpool

# Disable cache (force scan on every boot — not recommended)
zpool set cachefile=none rpool

# In an emergency, if cache is corrupt:
systemctl start zfs-import-scan.service
# Or manually:
zpool import -f rpool

Initramfs generators: dracut vs initramfs-tools vs mkinitcpio vs mkinitfs

The initramfs is the bridge between "kernel loaded" and "real root mounted." Every distro uses a different tool to build it, but they all produce the same thing: a compressed cpio archive containing a minimal Linux userspace with ZFS tools. Here's how each one works:

dracut (CentOS, RHEL, Rocky, Fedora)

dracut is a modular initramfs generator. Each "module" is a directory under /usr/lib/dracut/modules.d/ containing shell scripts. The ZFS module lives at 90zfs/ (the number is the priority — higher numbers run later).

# List what's in the ZFS dracut module
ls /usr/lib/dracut/modules.d/90zfs/
module-setup.sh   parse-zfs.sh   mount-zfs.sh   zfs-generator.sh   zfs-lib.sh

# Rebuild initramfs with ZFS support
dracut --force --add "zfs" --kver $(uname -r)

# Verify ZFS is included in the initramfs
lsinitrd /boot/initramfs-$(uname -r).img | grep zfs
drwxr-xr-x  usr/lib/modules/5.14.0-687.el9.x86_64/extra/zfs
-rw-r--r--  usr/lib/modules/5.14.0-687.el9.x86_64/extra/zfs/zfs.ko
-rwxr-xr-x  usr/sbin/zpool
-rwxr-xr-x  usr/sbin/zfs
-rwxr-xr-x  usr/lib/dracut/modules.d/90zfs/mount-zfs.sh

initramfs-tools (Debian, Ubuntu)

Debian and Ubuntu use initramfs-tools, which has a different module structure. ZFS support comes from the zfs-initramfs package, which installs hooks and scripts into /usr/share/initramfs-tools/.

# ZFS initramfs-tools files
ls /usr/share/initramfs-tools/hooks/zfs
ls /usr/share/initramfs-tools/scripts/zfs

# Rebuild initramfs
update-initramfs -c -k all
# Or update existing:
update-initramfs -u -k all

# Verify
lsinitramfs /boot/initrd.img-$(uname -r) | grep zfs

mkinitcpio (Arch Linux)

Arch uses mkinitcpio, configured via /etc/mkinitcpio.conf. ZFS support requires adding zfs to the HOOKS array:

# /etc/mkinitcpio.conf
HOOKS=(base udev autodetect modconf block zfs filesystems keyboard fsck)

# Rebuild all presets
mkinitcpio -P

# The zfs hook is provided by the zfs-utils package from the AUR or archzfs repo

mkinitfs (Alpine Linux)

Alpine uses mkinitfs, a minimal initramfs generator. ZFS support requires the zfs feature in /etc/mkinitfs/mkinitfs.conf:

# /etc/mkinitfs/mkinitfs.conf
features="ata base cdrom ext4 keymap kms mmc nvme scsi usb virtio zfs"

# Rebuild
mkinitfs -k $(uname -r)

Comparison table: initramfs generators

Boot environments

A boot environment (BE) is a ZFS dataset that contains a complete, bootable operating system. The concept comes from Solaris and illumos, where beadm has been managing boot environments for decades. On Linux with ZFSBootMenu, boot environments work the same way.

The idea: before a risky operation (kernel update, major package upgrade, config change), you snapshot or clone the current root dataset. If the update breaks something, you reboot, select the previous BE from the ZFSBootMenu menu, and you're back to a working system in seconds. No reinstall. No restore from backup. Just pick a different boot environment.

Dataset layout for boot environments

# Standard BE layout
rpool/ROOT                      mountpoint=none  canmount=off
rpool/ROOT/myhost               mountpoint=/     canmount=noauto   <-- active BE
rpool/ROOT/myhost@pre-upgrade   (snapshot of the active BE)
rpool/ROOT/myhost-backup        mountpoint=/     canmount=noauto   <-- cloned BE

# ZFSBootMenu sees both datasets with mountpoint=/ and shows them in the menu.
# The one marked as the default (org.zfsbootmenu:commandline property) boots automatically.

Creating and managing boot environments

# Snapshot before upgrade
zfs snapshot -r rpool/ROOT/myhost@pre-kernel-6.12

# Clone to a new BE (instant — no data copied, COW)
zfs clone rpool/ROOT/myhost@pre-kernel-6.12 rpool/ROOT/myhost-pre-6.12
zfs set mountpoint=/ rpool/ROOT/myhost-pre-6.12
zfs set canmount=noauto rpool/ROOT/myhost-pre-6.12

# Now upgrade the kernel on the active BE
dnf upgrade kernel

# If it breaks, reboot and select myhost-pre-6.12 from ZFSBootMenu

# Set the default BE (boots automatically without menu interaction)
zfs set org.zfsbootmenu:commandline="root=zfs:rpool/ROOT/myhost ro quiet" rpool/ROOT/myhost

# List all boot environments
zfs list -r -o name,mountpoint,canmount,used rpool/ROOT
NAME                          MOUNTPOINT  CANMOUNT  USED
rpool/ROOT                    none        off       12.4G
rpool/ROOT/myhost             /           noauto    12.1G
rpool/ROOT/myhost-pre-6.12   /           noauto    312K

BE management tools

The failure point

Kernel boots → initramfs runs → sees root=zfs:... →
NO MODULE CLAIMS IT → "FATAL: Don't know how to handle root=zfs:" →
dracut emergency shell (or kernel panic)

Kernel boots → initramfs runs → sees root=zfs:... →
parse-zfs.sh claims it → modprobe zfs → zpool import →
mount root → switch_root → boot completes

Common boot failures and how to fix them

Every ZFS boot failure falls into one of these categories. Learn to recognize the symptoms and you can fix any of them from a live USB in minutes.

# Fix: boot from live USB, import pool, chroot, reinstall ZFS initramfs
zpool import -f -R /mnt rpool
mount -t zfs rpool/ROOT/myhost /mnt
mount /dev/vda1 /mnt/boot/efi
mount --bind /dev /mnt/dev
mount --bind /proc /mnt/proc
mount --bind /sys /mnt/sys

chroot /mnt

# RPM distros:
dnf reinstall zfs-dracut
dracut --force --add "zfs" --kver $(ls /lib/modules/ | tail -1)

# Debian/Ubuntu:
apt install --reinstall zfs-initramfs
update-initramfs -c -k all

exit
umount -R /mnt

# Fix 1: force import (use cautiously — only if you know no other machine has this pool)
zpool import -f rpool

# Fix 2: regenerate hostid to match the pool's expectation
# Boot live USB, import the pool, read the hostid from the pool:
zdb -C rpool | grep hostid
#     hostid: 8323072
# Convert to 4-byte little-endian and write to /etc/hostid
printf '\x00\x01\x7f\x00' > /etc/hostid
# Then rebuild initramfs to include the new hostid

# Press 's' in ZFSBootMenu to open a shell, then:
zpool status
zfs list -o name,mountpoint,canmount rpool/ROOT

# If the pool isn't imported:
zpool import -f rpool

# If properties are wrong:
zfs set mountpoint=/ rpool/ROOT/myhost
zfs set canmount=noauto rpool/ROOT/myhost

# Exit the shell and ZFSBootMenu will re-scan

# Fix: add missing drivers to the initramfs
# For dracut, force inclusion of all storage drivers:
echo 'force_drivers+=" nvme nvme_core ahci sd_mod "' > /etc/dracut.conf.d/storage.conf
dracut --force --kver $(uname -r)

# For mkinitcpio, add modules:
# /etc/mkinitcpio.conf
MODULES=(nvme nvme_core)

# Boot the old kernel from ZFSBootMenu (select it from the menu)
# Then fix DKMS:

# Check DKMS status
dkms status
zfs/2.2.9, 5.14.0-687.el9.x86_64: installed
zfs/2.2.9, 5.14.0-700.el9.x86_64: built (FAILED)

# Rebuild manually
dkms remove zfs/2.2.9 -k 5.14.0-700.el9.x86_64
dkms build zfs/2.2.9 -k 5.14.0-700.el9.x86_64
dkms install zfs/2.2.9 -k 5.14.0-700.el9.x86_64
depmod -a 5.14.0-700.el9.x86_64

# Rebuild initramfs for the new kernel
dracut --force --add "zfs" --kver 5.14.0-700.el9.x86_64

# Quick fix: mount all datasets manually
zfs mount -a

# Permanent fix: enable the services
systemctl enable zfs-import-cache.service
systemctl enable zfs-mount.service
systemctl enable zfs.target

# Verify
systemctl status zfs-mount.service

Emergency shell recovery

When the initramfs can't mount root, you get dropped to an emergency shell (dracut rescue shell, initramfs-tools busybox shell, or ZFSBootMenu's shell). This is your lifeline. Here's how to use it.

dracut emergency shell

# You're in the dracut shell. First, figure out what's available:
ls /dev/disk/by-id/       # Can you see your disks?
modprobe zfs              # Can you load ZFS?
zpool import              # Can you see your pool?

# If zfs.ko is missing: the initramfs doesn't have ZFS. You need a live USB.
# If the pool is visible but won't import:
zpool import -f rpool
zpool status rpool

# If the pool imports, mount root manually:
mount -t zfs rpool/ROOT/myhost /sysroot
# Then tell dracut to continue:
exit    # dracut will attempt switch_root

ZFSBootMenu shell

# Press 's' at the ZFSBootMenu menu to open a recovery shell.
# This shell has FULL ZFS tools — zpool, zfs, zdb, everything.

# Useful recovery operations:
zpool status                     # Check pool health
zpool import -f rpool            # Force import
zfs list -r rpool/ROOT           # List all BEs
zfs rollback rpool/ROOT/myhost@last-good    # Rollback to snapshot

# Mount a BE and chroot into it:
mkdir -p /mnt
mount -t zfs rpool/ROOT/myhost /mnt
mount --bind /dev /mnt/dev
mount --bind /proc /mnt/proc
mount --bind /sys /mnt/sys
chroot /mnt /bin/bash

# Fix things inside the chroot (reinstall packages, rebuild initramfs, etc.)
# Then exit and reboot

Scenario: recovering from a failed kernel update

This is the most common recovery scenario. A kernel update broke something — DKMS failed, the initramfs is bad, or the new kernel doesn't work with your hardware. Here's the complete recovery procedure.

# Step 1: Reboot. At the ZFSBootMenu menu, select the previous kernel.
#         ZFSBootMenu shows all kernels in /boot — pick the old one.
#         If only one BE exists, press 'e' to edit the kernel command line
#         and change the kernel version.

# Step 2: Once booted on the old kernel, check DKMS status:
dkms status
# If the new kernel shows "FAILED", rebuild:
dkms build zfs/2.2.9 -k <new-kernel-version>
dkms install zfs/2.2.9 -k <new-kernel-version>

# Step 3: Rebuild the initramfs for the new kernel:
dracut --force --add "zfs" --kver <new-kernel-version>

# Step 4: Reboot and select the new kernel. Should work now.

# If the new kernel itself is broken (not just ZFS):
# Remove it and stick with the old one:
dnf remove kernel-5.14.0-700.el9
# Or on Debian:
apt remove linux-image-6.1.0-new

Scenario: repairing GRUB after pool changes (legacy systems)

If you're on a legacy system still using GRUB with ZFS, here's how to fix GRUB after pool changes (enabling features, renaming datasets, moving disks):

# Boot from live USB
zpool import -f -R /mnt rpool
mount -t zfs rpool/ROOT/myhost /mnt
mount /dev/vda1 /mnt/boot/efi
for d in dev proc sys; do mount --bind /$d /mnt/$d; done

chroot /mnt

# Verify grub-probe can see ZFS
grub-probe /
# Should output: zfs

# Regenerate GRUB config
grub-mkconfig -o /boot/grub/grub.cfg

# Reinstall GRUB to the ESP
grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=centos

exit
umount -R /mnt

# Better fix: migrate to ZFSBootMenu and never touch GRUB again.

Multi-boot with ZFS

ZFSBootMenu makes multi-boot with ZFS trivial. Each operating system is just a boot environment under rpool/ROOT/. ZFSBootMenu discovers them all automatically.

# Multiple distros on the same pool:
rpool/ROOT/centos-9        mountpoint=/  canmount=noauto
rpool/ROOT/debian-13       mountpoint=/  canmount=noauto
rpool/ROOT/ubuntu-24.04    mountpoint=/  canmount=noauto
rpool/ROOT/fedora-41       mountpoint=/  canmount=noauto

# Each has its own kernel, initramfs, and packages inside /boot
# ZFSBootMenu shows all four in the menu

# Shared data can live in a separate dataset:
rpool/data                 mountpoint=/shared   canmount=on
# Mount it from any BE after boot

The key requirement: each BE must have its own /boot with its own kernel and initramfs. The root dataset contains the full OS. Child datasets (/home, /var) can be per-BE or shared, depending on your needs.

How kldload sets up the boot chain (all 8 distros)

kldload's bootloader.sh handles the complete boot chain setup for every supported distro. Here's what it does, in order, and how it adapts per distro:

FreeBSD exception

FreeBSD is the one distro that doesn't use ZFSBootMenu. FreeBSD has native ZFS boot support built into its loader.efi. The FreeBSD boot loader understands ZFS pools, datasets, and boot environments out of the box — it was one of the first operating systems to support ZFS on root. kldload installs loader.efi to /EFI/BOOT/BOOTX64.EFI and skips ZFSBootMenu entirely.

/etc/fstab on a ZFS root system

On a traditional Linux system, /etc/fstab is the master list of everything that gets mounted. On a ZFS-on-root system, fstab is almost empty because ZFS handles its own mounting. Only non-ZFS filesystems need fstab entries:

# /etc/fstab — minimal ZFS root system
# ZFS datasets are mounted by zfs-mount.service, NOT by fstab.
# Only the ESP and swap (if any) need entries here.

UUID=ABCD-1234  /boot/efi  vfat  umask=0077  0  1
/dev/zvol/rpool/swap  none  swap  defaults  0  0

If you add ZFS datasets to fstab (with mountpoint=legacy), they'll be mounted by systemd's normal fstab processing. But for most use cases, let ZFS handle it — it's simpler and less error-prone.

Swap on ZFS

ZFS supports swap via zvols (ZFS block devices). A zvol looks like a regular block device to the kernel, so you can use it as swap space:

# Create a zvol for swap (4GB, no compression, no sync)
zfs create -V 4G -o compression=off -o sync=disabled \
  -o primarycache=metadata -o secondarycache=none \
  rpool/swap

# Format and enable
mkswap /dev/zvol/rpool/swap
swapon /dev/zvol/rpool/swap

# Add to /etc/fstab for boot persistence
echo "/dev/zvol/rpool/swap none swap defaults 0 0" >> /etc/fstab

Important properties for swap zvols: compression=off (compressing swap is pointless and wastes CPU), sync=disabled (swap doesn't need transaction group commits), and primarycache=metadata (don't waste ARC on swap blocks).

Secure Boot and the ZFS boot chain

Secure Boot verifies that every binary in the boot chain is signed by a trusted key. The standard Linux Secure Boot chain is: firmware → shim (signed by Microsoft) → GRUB (signed by distro) → kernel (signed by distro). ZFS complicates this because ZFSBootMenu and the DKMS-built zfs.ko are not signed by any distro.

# Signing zfs.ko with a MOK key (manual process):
# 1. Generate a MOK keypair
openssl req -new -x509 -newkey rsa:2048 -keyout /root/mok.key \
  -outform DER -out /root/mok.der -nodes -days 36500 \
  -subj "/CN=ZFS DKMS Signing Key/"

# 2. Enroll the public key
mokutil --import /root/mok.der
# (requires reboot and physical console access to approve)

# 3. Sign the module after each DKMS build
/usr/src/kernels/$(uname -r)/scripts/sign-file sha256 \
  /root/mok.key /root/mok.der \
  /usr/lib/modules/$(uname -r)/extra/zfs/zfs.ko

Distro-specific boot chain notes

Boot chain verification checklist

After installing a ZFS-on-root system (or recovering from a boot failure), run through this checklist to verify every link in the chain is solid:

# 1. ZFS module loads
modprobe zfs && echo "OK: zfs.ko loaded"

# 2. Pool is imported and healthy
zpool status rpool | head -5

# 3. Root dataset has correct properties
zfs get mountpoint,canmount rpool/ROOT/$(hostname)
# Should be: mountpoint=/ canmount=noauto

# 4. zpool.cache exists
ls -la /etc/zfs/zpool.cache

# 5. hostid exists and matches
hostid
zdb -C rpool | grep hostid

# 6. Initramfs contains ZFS
lsinitrd /boot/initramfs-$(uname -r).img 2>/dev/null | grep -c zfs || \
  lsinitramfs /boot/initrd.img-$(uname -r) 2>/dev/null | grep -c zfs
# Should be > 0

# 7. ZFS services are enabled
systemctl is-enabled zfs-import-cache.service
systemctl is-enabled zfs-mount.service
systemctl is-enabled zfs-zed.service

# 8. ESP is mounted and ZFSBootMenu is present
mountpoint /boot/efi && ls /boot/efi/EFI/zbm/BOOTX64.EFI

# 9. EFI boot entries exist
efibootmgr | grep -i zfsbootmenu

# 10. fstab has ESP entry
grep '/boot/efi' /etc/fstab