kldload — AI for KVM & Virtual Machine Management

Build Your Own

AI for KVM & Virtual Machine Management — a local model that knows your hypervisor.

Generic LLMs know that virsh exists. This model knows your running VMs, your storage pools, your network definitions, your CPU topology, and your memory allocation. It reads virsh list before every answer. It generates virt-install commands from natural language. It diagnoses boot failures by reading domain XML and libvirt logs. It knows that your VM disks live on ZFS zvols and uses that for snapshots, clones, and migration.

1. The KVM Modelfile

This system prompt encodes KVM, libvirt, QEMU, virt-install, virsh, domain XML, storage pools, networking, performance tuning, and the ZFS-backed storage layer.

Complete KVM expert Modelfile

# /srv/ollama/Modelfile.kvm-expert
FROM llama3.1:8b

SYSTEM """
You are a KVM/libvirt/QEMU virtualization expert for this kldload-based infrastructure.
You give precise commands, reference actual VM names from context, and always recommend
ZFS snapshots (ksnap) before destructive operations on VM disks.

=== VM LIFECYCLE ===
List running VMs:       virsh list
List all VMs:           virsh list --all
Start VM:               virsh start VMNAME
Shutdown VM:            virsh shutdown VMNAME (graceful, via ACPI)
Force off:              virsh destroy VMNAME (like pulling the power plug)
Reboot:                 virsh reboot VMNAME
Suspend/resume:         virsh suspend VMNAME / virsh resume VMNAME
Delete VM:              virsh undefine VMNAME --remove-all-storage (WARNING: destroys disk)
Delete VM keep disk:    virsh undefine VMNAME (keeps zvol/image)
Console:                virsh console VMNAME (serial) or virt-viewer VMNAME (graphical)
VM info:                virsh dominfo VMNAME
VM XML:                 virsh dumpxml VMNAME

=== VIRT-INSTALL ===
Basic VM:               virt-install --name test --ram 4096 --vcpus 4 \
                          --disk path=/dev/zvol/rpool/vm/test,bus=virtio \
                          --cdrom /srv/iso/centos.iso --os-variant centos-stream9 \
                          --network bridge=br0,model=virtio --graphics vnc --noautoconsole
Q35 machine:            --machine q35 (modern chipset, PCIe, better for GPU passthrough)
UEFI boot:              --boot uefi (requires OVMF firmware)
Serial console:         --extra-args "console=ttyS0,115200" --graphics none
Cloud-init:             --cloud-init root-password-generate=on,ssh-key=/root/.ssh/id_ed25519.pub
Import existing disk:   --import --disk path=/srv/vm/disk.qcow2 (skip installer)
CPU host passthrough:   --cpu host (expose real CPU features to guest)

=== STORAGE ===
ZFS zvol for VMs:       zfs create -V 40G -o volblocksize=64k rpool/vm/VMNAME
List storage pools:     virsh pool-list --all
Pool info:              virsh pool-info default
List volumes:           virsh vol-list POOL
Resize disk:            virsh blockresize VMNAME /dev/zvol/rpool/vm/VMNAME 60G
Snapshot zvol:          zfs snapshot rpool/vm/VMNAME@before-upgrade
Clone zvol:             zfs clone rpool/vm/VMNAME@snap rpool/vm/VMNAME-clone
Rollback zvol:          zfs rollback rpool/vm/VMNAME@before-upgrade (VM must be off)
ZFS send VM:            zfs send rpool/vm/VMNAME@snap | ssh node2 zfs recv rpool/vm/VMNAME

Qcow2 on ZFS:           Store qcow2 on a dataset (not zvol): rpool/srv/vm/VMNAME.qcow2
                          Dataset gives compression + snapshots on the qcow2 file.
Zvol vs dataset:        Zvol = block device, direct. Dataset + qcow2 = file, flexible (snapshots of snapshots).
                          Zvol is faster. Dataset + qcow2 supports qcow2 snapshots too.

=== NETWORKING ===
List networks:          virsh net-list --all
Network info:           virsh net-info default / virsh net-dumpxml default
Create bridge:          nmcli con add type bridge ifname br0
                        nmcli con add type bridge-slave ifname eth0 master br0
Macvtap (simple):       --network type=direct,source=eth0,model=virtio (guest gets LAN IP)
NAT (default):          --network network=default (192.168.122.0/24, NAT to host)
Isolated network:       virsh net-define isolated.xml && virsh net-start isolated
WireGuard in VM:        Pass through /dev/net/tun, or use host WireGuard + bridge

=== CPU & MEMORY TUNING ===
CPU pinning:            virsh vcpupin VMNAME 0 2 (pin vCPU 0 to host CPU 2)
                        virsh vcpupin VMNAME 1 3 (pin vCPU 1 to host CPU 3)
NUMA topology:          virsh numatune VMNAME --mode strict --nodeset 0
Huge pages:             echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
                        Add  to domain XML
Hot-add vCPU:           virsh setvcpus VMNAME 8 --live (if maxvcpus allows)
Hot-add memory:         virsh setmem VMNAME 8G --live (if maxmemory allows)
I/O scheduler:          Use none/noop for zvol-backed VMs (host ZFS handles scheduling)
Virtio:                 ALWAYS use virtio for disk (bus=virtio) and network (model=virtio)
                          Virtio is 2-5x faster than emulated IDE/e1000
Cache mode:             cache=none for zvol (ZFS handles caching via ARC)
                          cache=writeback for qcow2 on dataset (qcow2 needs write coalescing)
IO mode:                io=native for zvol, io=threads for qcow2

=== VM TEMPLATES & GOLDEN IMAGES ===
Create template:        1. Install base VM  2. Customize  3. sysprep  4. Snapshot zvol
                        virt-sysprep -d VMNAME (removes SSH keys, hostname, machine-id)
Clone from template:    zfs clone rpool/vm/template@gold rpool/vm/new-vm
                        virt-clone --original template --name new-vm --auto-clone
Linked clone (fast):    ZFS clone is instant. New VM shares template blocks until divergence.
Full clone:             zfs send rpool/vm/template@gold | zfs recv rpool/vm/new-vm

=== FIRECRACKER MICROVMS ===
What is it:             Lightweight VMM by AWS. Boots a Linux kernel in <125ms.
                        No BIOS, no device model, minimal attack surface.
When to use:            Short-lived workloads, serverless functions, CI runners, sandboxing.
                        Use KVM for full OS VMs. Use Firecracker for microVMs.
Install:                curl -L https://github.com/firecracker-microvm/firecracker/releases/latest/download/firecracker-v*-x86_64.tgz | tar xz
Launch:                 firecracker --api-sock /tmp/fc.sock
Configure via API:      curl --unix-socket /tmp/fc.sock -X PUT http://localhost/boot-source \
                          -d '{"kernel_image_path":"/srv/fc/vmlinux","boot_args":"console=ttyS0"}'
Root drive:             curl --unix-socket /tmp/fc.sock -X PUT http://localhost/drives/rootfs \
                          -d '{"drive_id":"rootfs","path_on_host":"/srv/fc/rootfs.ext4","is_root_device":true}'
ZFS-backed rootfs:      Store rootfs images on ZFS dataset. Snapshot before launch. Clone for parallel runs.

=== TROUBLESHOOTING ===
VM won't start:         virsh start VMNAME 2>&1 (read the error)
                        journalctl -u libvirtd --since '5 min ago'
                        Check: disk exists? network exists? enough RAM? CPU features?
Permission denied:      Check /dev/zvol permissions. qemu user needs read/write.
                        chown qemu:qemu /dev/zvol/rpool/vm/VMNAME
                        Or: Add  to domain XML (less secure)
Slow VM:                Check virtio (not IDE/e1000). Check cache mode. Check CPU pinning.
                        virsh domstats VMNAME (CPU, memory, I/O stats)
                        Check ARC: zvol I/O goes through ARC. Low hit rate = slow VMs.
Migration:              Live: virsh migrate --live VMNAME qemu+ssh://node2/system
                        Cold: virsh shutdown VMNAME && zfs send zvol | ssh node2 zfs recv
                        Shared storage: live migration just moves memory state

=== PHILOSOPHY ===
Snapshot before everything. VMs are not cattle until you can rebuild them in minutes.
Virtio is not optional. Emulated hardware is for compatibility testing, not production.
CPU pinning matters for latency-sensitive workloads. Don't pin everything.
Huge pages help VMs with large memory. Don't allocate more huge pages than you need.
ZFS zvols give you block-level snapshots of VM disks. Use them.
"""

PARAMETER temperature 0.3
PARAMETER num_ctx 16384

# Build the KVM expert model
ollama create kvm-expert -f /srv/ollama/Modelfile.kvm-expert

# Verify it
ollama run kvm-expert "Create a VM with 4 cores, 8GB RAM, and a 40GB ZFS zvol"

virsh has 140+ subcommands. virt-install has 80+ flags. Nobody memorizes all of them. This model did, and it also reads your VM list before answering.

2. Live context script

The Modelfile gives the AI KVM expertise. The context script gives it your hypervisor state. Every query includes running VMs, resource allocation, storage pools, networks, and host capacity.

kai-kvm — query with live hypervisor state

#!/bin/bash
# /usr/local/bin/kai-kvm — query the KVM AI with live hypervisor context

build_kvm_context() {
    echo "=== LIVE KVM STATE ($(date -Iseconds)) ==="

    echo -e "\n--- Running VMs ---"
    virsh list 2>/dev/null

    echo -e "\n--- All VMs ---"
    virsh list --all 2>/dev/null

    echo -e "\n--- VM details ---"
    for vm in $(virsh list --name 2>/dev/null); do
        [ -z "$vm" ] && continue
        echo "== $vm =="
        virsh dominfo "$vm" 2>/dev/null
        echo "  Disks:"
        virsh domblklist "$vm" 2>/dev/null | grep -v '^$\|^---\|Target'
        echo "  NICs:"
        virsh domiflist "$vm" 2>/dev/null | grep -v '^$\|^---\|Interface'
        echo ""
    done

    echo -e "\n--- Storage pools ---"
    virsh pool-list --all 2>/dev/null

    echo -e "\n--- Networks ---"
    virsh net-list --all 2>/dev/null

    echo -e "\n--- Host CPU ---"
    lscpu 2>/dev/null | grep -E 'Model name|Socket|Core|Thread|CPU\(s\)|NUMA'

    echo -e "\n--- Host memory ---"
    free -h 2>/dev/null

    echo -e "\n--- VM zvols ---"
    zfs list -r -o name,used,avail,volsize,volblocksize rpool/vm 2>/dev/null

    echo -e "\n--- VM zvol snapshots ---"
    zfs list -t snapshot -r rpool/vm -o name,used,creation 2>/dev/null | tail -20

    echo -e "\n--- CPU allocation ---"
    echo "Host CPUs: $(nproc)"
    echo "Allocated vCPUs:"
    for vm in $(virsh list --name 2>/dev/null); do
        [ -z "$vm" ] && continue
        vcpus=$(virsh dominfo "$vm" 2>/dev/null | awk '/CPU\(s\)/{print $2}')
        echo "  $vm: $vcpus vCPUs"
    done

    echo -e "\n--- Memory allocation ---"
    echo "Host RAM: $(free -h | awk '/^Mem/{print $2}')"
    echo "Allocated:"
    for vm in $(virsh list --name 2>/dev/null); do
        [ -z "$vm" ] && continue
        mem=$(virsh dominfo "$vm" 2>/dev/null | awk '/Max memory/{print $3, $4}')
        echo "  $vm: $mem"
    done

    echo -e "\n--- Libvirt errors (last 10 min) ---"
    journalctl -u libvirtd --since '10 min ago' --no-pager -p err 2>/dev/null | tail -10
}

QUESTION="$*"
if [ -z "$QUESTION" ]; then
    echo "Usage: kai-kvm <question>"
    echo ""
    echo "Examples:"
    echo "  kai-kvm 'create a VM with 4 cores and 8GB RAM'"
    echo "  kai-kvm 'why won't my VM start?'"
    echo "  kai-kvm 'resize the disk on web-server to 60GB'"
    echo "  kai-kvm 'set up CPU pinning for my database VM'"
    echo "  kai-kvm 'migrate db-server to node2'"
    echo "  kai-kvm 'which VMs are overprovisioned?'"
    exit 1
fi

CONTEXT=$(build_kvm_context)

echo -e "${CONTEXT}\n\n=== QUESTION ===\n${QUESTION}" | ollama run kvm-expert

You don't ask a mechanic to diagnose your car without letting them look under the hood. This script lifts the hood — virsh list, dominfo, storage, networks, CPU topology — before the AI says a word.

3. Natural language to virt-install

Describe the VM you want in plain English. The AI generates the exact virt-install command with the right flags, the right storage path, the right network, and the right CPU model. It knows your host capacity and won't overcommit without telling you.

VM creation examples

# "I need a CentOS VM for a database with 8 cores and 16GB RAM"
kai-kvm "create a CentOS Stream 9 VM for PostgreSQL. \
8 cores, 16GB RAM, 100GB zvol. Use q35, UEFI, virtio, CPU host passthrough. \
Bridge network on br0. recordsize 16k on the zvol."

# AI generates:
# zfs create -V 100G -o volblocksize=16k rpool/vm/db-server
# virt-install --name db-server --ram 16384 --vcpus 8 \
#   --machine q35 --cpu host --boot uefi \
#   --disk path=/dev/zvol/rpool/vm/db-server,bus=virtio,cache=none,io=native \
#   --cdrom /srv/iso/CentOS-Stream-9-latest-x86_64-dvd1.iso \
#   --os-variant centos-stream9 --network bridge=br0,model=virtio \
#   --graphics vnc --noautoconsole

# "Give me a lightweight test VM"
kai-kvm "create a small test VM — 1 core, 1GB RAM, 10GB disk, NAT network"

# "Clone my template for a new web server"
kai-kvm "clone the centos-template VM to create web-server-3"

virt-install is a 200-line man page. The AI reads it so you don't have to. Describe what you want, get the command, review it, run it.

4. ZFS-backed VMs

On kldloadOS, VM disks are ZFS zvols. That means every VM disk gets snapshots, clones, compression, checksums, and send/recv. Snapshot a VM before upgrading the kernel. Clone a VM for testing in zero time. Replicate a VM to another host with syncoid.

ZFS + KVM workflows

# Snapshot a VM before a risky upgrade
virsh shutdown db-server
zfs snapshot rpool/vm/db-server@pre-kernel-upgrade
virsh start db-server
# ... upgrade goes wrong ...
virsh destroy db-server
zfs rollback rpool/vm/db-server@pre-kernel-upgrade
virsh start db-server
# Back to exactly where you were. Seconds, not hours.

# Clone a VM for testing (instant, shares blocks with original)
zfs snapshot rpool/vm/db-server@test-clone
zfs clone rpool/vm/db-server@test-clone rpool/vm/db-server-test
virt-clone --original db-server --name db-server-test \
    --file /dev/zvol/rpool/vm/db-server-test --preserve-data
virsh start db-server-test

# Migrate a VM to another host via ZFS send
virsh shutdown web-server
zfs snapshot rpool/vm/web-server@migrate
zfs send rpool/vm/web-server@migrate | ssh node2 zfs recv rpool/vm/web-server
virsh dumpxml web-server | ssh node2 'virsh define /dev/stdin'
ssh node2 virsh start web-server

# Continuous replication (live VM, incremental sync)
syncoid rpool/vm/web-server root@node2:rpool/vm/web-server

A qcow2 snapshot is a chain of deltas that gets fragile over time. A ZFS snapshot is a pointer in a copy-on-write tree that never fragments and never breaks. Same operation, different foundation.

5. Performance tuning

The AI knows your CPU topology, your NUMA layout, and your memory allocation. It recommends CPU pinning for latency-sensitive VMs, huge pages for memory-intensive workloads, and the right I/O settings for ZFS-backed zvols.

CPU pinning

The AI reads lscpu, identifies your physical cores vs hyperthreads, and generates virsh vcpupin commands that pin vCPUs to physical cores on the same NUMA node as the VM's memory. No cross-socket traffic.

kai-kvm "pin my database VM's 8 vCPUs to physical cores, same NUMA node"

Huge pages

For VMs with large memory allocations, huge pages eliminate TLB misses. The AI calculates how many 2MB pages you need, shows you the sysctl command to allocate them, and the domain XML snippet to enable them.

kai-kvm "set up huge pages for my 16GB database VM"

I/O tuning for ZFS zvols

The AI sets cache=none because ZFS ARC handles caching, io=native for direct I/O to the zvol, and volblocksize=64k to match the guest filesystem. It also checks your ARC hit rate to make sure the zvol I/O is actually getting cached.

kai-kvm "optimize I/O settings for my VMs on ZFS zvols"

Overcommit detection

The AI adds up all allocated vCPUs and memory across your VMs, compares to host capacity, and flags overcommit. It recommends which VMs to rebalance and whether you need to migrate workloads to another node.

kai-kvm "am I overcommitting CPU or memory?"

6. VM templates and golden images

Build a base VM once. Sysprep it. Snapshot it. Clone it instantly for every new deployment. The AI helps design the golden image and automates the clone-and-customize workflow.

Template workflow

# 1. Create and install the base VM
zfs create -V 20G -o volblocksize=64k rpool/vm/centos9-template
virt-install --name centos9-template --ram 2048 --vcpus 2 \
    --disk path=/dev/zvol/rpool/vm/centos9-template,bus=virtio \
    --cdrom /srv/iso/CentOS-Stream-9-latest-x86_64-dvd1.iso \
    --os-variant centos-stream9 --network bridge=br0,model=virtio \
    --machine q35 --cpu host --boot uefi --graphics vnc --noautoconsole

# 2. Customize: install packages, configure, harden
# ... (install your standard packages, SSH keys, monitoring agents) ...

# 3. Sysprep: remove unique identifiers
virt-sysprep -d centos9-template \
    --operations defaults,-ssh-userdir \
    --firstboot-command 'systemd-machine-id-setup'

# 4. Snapshot the template (this is your golden image)
zfs snapshot rpool/vm/centos9-template@gold-v1

# 5. Clone for new VMs (instant, zero initial disk cost)
zfs clone rpool/vm/centos9-template@gold-v1 rpool/vm/web-server-4
virt-clone --original centos9-template --name web-server-4 \
    --file /dev/zvol/rpool/vm/web-server-4 --preserve-data
virsh start web-server-4

A golden image is a cookie cutter. ZFS clone is the dough that magically doesn't get used up — every clone shares the template's blocks until the clone diverges. Ten VMs from one template, and the template's 20 GB is stored once.

7. Firecracker microVMs

Not every workload needs a full VM. Firecracker boots a Linux kernel in under 125 milliseconds with minimal memory overhead. Use it for CI runners, serverless functions, disposable sandboxes, and any workload that starts fast and dies fast. The AI manages both KVM VMs and Firecracker microVMs side by side.

Firecracker on kldloadOS

# Download Firecracker
ARCH=$(uname -m)
curl -L "https://github.com/firecracker-microvm/firecracker/releases/latest/download/firecracker-v1.9.1-${ARCH}.tgz" | \
    tar xz -C /usr/local/bin --strip-components=1

# Prepare a rootfs on ZFS (snapshot before each run)
zfs create rpool/srv/firecracker
cp /srv/fc/rootfs.ext4 /srv/firecracker/rootfs.ext4
zfs snapshot rpool/srv/firecracker@clean

# Launch a microVM
firecracker --api-sock /tmp/fc.sock --config-file /srv/fc/config.json

# config.json
{
  "boot-source": {
    "kernel_image_path": "/srv/fc/vmlinux",
    "boot_args": "console=ttyS0 reboot=k panic=1 pci=off"
  },
  "drives": [{
    "drive_id": "rootfs",
    "path_on_host": "/srv/firecracker/rootfs.ext4",
    "is_root_device": true,
    "is_read_only": false
  }],
  "machine-config": {
    "vcpu_count": 2,
    "mem_size_mib": 512
  }
}

# After each run, rollback to clean state
zfs rollback rpool/srv/firecracker@clean

KVM VMs are houses. Firecracker microVMs are tents. A house takes minutes to build and lasts years. A tent goes up in seconds and you pack it when you leave. Both have their place. Use the right one.

KVM gives you hardware-level isolation. ZFS gives you instant snapshots, clones, and replication of VM disks. The AI ties them together — it knows your VMs, knows your storage, knows your CPU topology, and generates the exact commands you need. It doesn't replace your understanding of virtualization. It amplifies it.

Snapshot before you upgrade. Clone before you test. Pin your CPUs to the right cores. Use virtio for everything. And when you need something that boots in 125 milliseconds, Firecracker is right there on the same ZFS pool.

← Build Overview ↑ Top