← All courses

🖥️ Virtualization for Data

Turning hardware you control into the machines your data work needs — KVM, libvirt and system containers on one Debian box; XCP-ng and Xen Orchestra's backup story; Proxmox with the cluster filesystem that goes read-only when quorum drops; and TrueNAS as the storage under all of it. With the two sentences nobody says out loud: a snapshot is not a backup, and a VM restarted on another host is still a database that crashed.

No lessons published yet — this is the roadmap, so you can see exactly what the course will cover before it lands.

Before you start: Data Ops — Series 1. You need the terminal, ssh and a text editor on a remote box, and you need to have already met containers once — Series 1 · 37 is Docker as a tool, and this course explains what it was standing on. Series 1 · 46 rents a machine by clicking; this course is the other answer to the same question.

Before you start: Storage for Data — Series 1. Every platform here makes you choose a disk layout on the day you install it, and that choice is the hardest one to take back. Series 1 · 15 is RAID and Series 1 · 18 is LVM, both of which appear in the installer of every product in this course. Series 2 is ZFS, which Series 4 of this course assumes completely.

Short on time? The 36 episodes marked core are the shortest path to being productive at work. Start there and pick up the rest whenever a job asks for them.

Series 1 · Containers and Virtual Machines on One Linux Box

namespaces, KVM and LXC — what a "machine" is when there is only one machine

  1. 01

    Two Ways to Get More Machines — a Second Kernel, or a Fence Around Processes

    coreintermediatecoming soon
  2. 02

    Namespaces and cgroups — Building a Container by Hand, and Why free Lies

    coreintermediatecoming soon
  3. 03

    KVM, QEMU and libvirt — Three Layers People Say as One Word

    coreintermediatecoming soon
  4. 04

    virt-install and virsh — a Cloud Image, a Machine, and an Undo Button

    coreintermediatecoming soon
  5. 05

    System Containers — LXC, Incus, and Why This Is Not Docker

    coreintermediatecoming soon
  6. 06

    incus launch — Profiles, Snapshots, and One Database per Developer

    coreintermediatecoming soon
  7. 07

    Bridges and NAT — the Address Your VM Did Not Get

    coreintermediatecoming soon
  8. 08

    Where the Virtual Disk Lives — qcow2, LVM, zvols, and the Bind-Mount Trap

    intermediatecoming soon
  9. 09

    Unprivileged Containers — the Boundary You Are Actually Trusting

    intermediatecoming soon
  10. 10

    Templates and cloud-init — Stop Installing Debian by Hand

    coreintermediatecoming soon
  11. 11

    virtio and cache=writeback — the Setting That Can Lose a Commit

    coreintermediatecoming soon
  12. 12

    Project — a Template, a Database VM, and a Revert That Actually Works

    intermediatecoming soon

Series 2 · XCP-ng and Xen Orchestra

a type-1 hypervisor, a pool, and the backup story that sells it

  1. 01

    XCP-ng — Xen, a Fork of XenServer, and the Licence Renewal That Sells It

    coreintermediatecoming soon
  2. 02

    Installing a Host — and Why dom0 Is Not Somewhere You Put Things

    coreintermediatecoming soon
  3. 03

    Xen Orchestra — XO Lite, From Sources, or XOA, and Where It Must Not Live

    coreintermediatecoming soon
  4. 04

    Pools, Hosts and the Master — Why Two Servers Are Not Yet a Cluster

    coreintermediatecoming soon
  5. 05

    SRs, VDIs and the Coalesce Queue — the Way This Platform Fills a Disk

    coreintermediatecoming soon
  6. 06

    PIFs, VIFs and Bonds — and the Storage Network You Should Have Separated

    intermediatecoming soon
  7. 07

    Templates, Guest Tools and xe — Why a VM Without the Tools Cannot Be Shut Down

    coreintermediatecoming soon
  8. 08

    Live Migration and Maintenance Mode — Patching a Host at Eleven in the Morning

    coreintermediatecoming soon
  9. 09

    Delta Backups and Health Checks — the Restore That Tests Itself

    coreintermediatecoming soon
  10. 10

    Patching a Pool — the Order, the Rolling Upgrade, and the Single-Host Tax

    intermediatecoming soon
  11. 11

    SMlog First — Reading a Stuck SR, a VM That Will Not Start, and a Lost Host

    intermediatecoming soon
  12. 12

    Do You Need a Hypervisor Platform — and What the Second Host Is Really For

    coreintermediatecoming soon
  13. 13

    Project — a Host, a Backup Job, and a Restore You Timed

    intermediatecoming soon

Series 3 · Proxmox VE

Series 1's tools, with a cluster, a web UI and a backup server

  1. 01

    Proxmox VE — Debian, KVM and LXC, With a Cluster Around Them

    coreintermediatecoming soon
  2. 02

    Installing It — the Disk Layout You Choose Before You Know Anything

    coreintermediatecoming soon
  3. 03

    qm, pct and /etc/pve — the CLI, and the Directory That Goes Read-Only

    coreintermediatecoming soon
  4. 04

    Templates, Linked Clones and cloud-init — a Database per Developer in Seconds

    coreintermediatecoming soon
  5. 05

    Storage Types and Content Types — the Dropdown Your Storage Is Missing From

    coreintermediatecoming soon
  6. 06

    Corosync, Quorum and the QDevice — Why Two Nodes Is the Worst Number

    coreintermediatecoming soon
  7. 07

    HA and Fencing — the VM Comes Back, and the Database Still Crashed

    coreintermediatecoming soon
  8. 08

    The Ceph Button — Ten Minutes to Install, and What You Just Agreed To

    intermediatecoming soon
  9. 09

    vzdump — Crash-Consistent Is Not Consistent, and the Agent That Helps

    coreintermediatecoming soon
  10. 10

    Proxmox Backup Server — Deduplication, Verify Jobs, and One File Back

    coreintermediatecoming soon
  11. 11

    Roles, API Tokens and the Firewall — Least Privilege for the Script That Checks Backups

    intermediatecoming soon
  12. 12

    Major Upgrades — the Checklist Script, the Node Order, and the Missing Rollback

    intermediatecoming soon
  13. 13

    Do You Need Proxmox — One Box, Three Boxes, or Somebody Else's

    coreintermediatecoming soon
  14. 14

    Project — the Node a Small Data Team Runs, and Two Kinds of Restore

    intermediatecoming soon

Series 4 · TrueNAS

the storage box under the hypervisors — and the appliance rule

  1. 01

    TrueNAS — ZFS as an Appliance, and Why You Do Not Edit the Config Files

    coreintermediatecoming soon
  2. 02

    Installing It — the Mirrored Boot Device, and No RAID Card Under ZFS

    coreintermediatecoming soon
  3. 03

    Pools and Datasets in the UI — One Dataset per Thing You Will Ever Treat Differently

    intermediatecoming soon
  4. 04

    SMB, NFS and ACLs — Why the Pipeline Cannot Read What the Analyst Dropped

    coreintermediatecoming soon
  5. 05

    NFS and iSCSI as Hypervisor Storage — sync=disabled and the Commit That Vanished

    coreintermediatecoming soon
  6. 06

    Snapshot Tasks, Replication and Cloud Sync — and What Ransomware Replicates

    coreintermediatecoming soon
  7. 07

    Scrubs, SMART and the Alert You Never Tested — Replacing a Disk in the UI

    coreintermediatecoming soon
  8. 08

    Apps and Instances — What the NAS Can Run, and Why It Mostly Should Not

    intermediatecoming soon
  9. 09

    Boot Environments — the Rollback That Is Just a Reboot, and the Flags You Must Not Set

    intermediatecoming soon
  10. 10

    Do You Need TrueNAS — or Debian, ZFS and Twenty Minutes of Samba

    coreintermediatecoming soon
  11. 11

    Choosing — One Box, a Pool, an Appliance, or Somebody Else's Datacentre

    coreintermediatecoming soon
  12. 12

    Project — a NAS, a Hypervisor, and Three Failures You Caused on Purpose

    intermediatecoming soon

Not here, on purpose

  • Building and running application containers, and writing a Dockerfile — covered in Data Ops, Series 1 · 37, which is Docker as a tool a data person uses. In this course, Series 1 · 02 explains the kernel features underneath it and Series 1 · 10 covers the different thing a system container is
  • Orchestrating containers across a cluster — covered in Kubernetes for Data Workloads. This course stops at the machines; what schedules pods onto them is that track's subject
  • ZFS itself — pools, vdevs, RAIDZ, scrub, resilver, send and receive — covered in Storage for Data, Series 2, in ten episodes. In this course, Series 4 · 05 covers only what the TrueNAS interface calls those things and which decisions it makes for you
  • Ceph as a storage system — covered in Storage for Data, Series 4. In this course, Series 3 · 17 covers only the fact that Proxmox will build one for you from a web form, and what you have taken on when it does
  • Renting a virtual machine or a bucket from a provider — covered in Data Ops, Series 1 · 46 and Data Ops, Series 1 · 47. This course is the alternative, and it is honest about when renting is the better answer
  • Creating virtual machines from code, with a provider and a state file — covered in Terraform & Pulumi. In this course, Series 3 · 25 covers the API and the token that such a provider would authenticate with, and stops there
  • Operating PostgreSQL — backups, replication, upgrades and tuning — covered in Data Ops, Series 2. This course puts a database on a virtual disk and protects the machine; what happens inside the database is that course's subject and the two must not be confused
  • VMware vSphere, Hyper-V and the commercial platforms — not covered. This track is virtualization a data team can obtain and run without a purchase order, and the two products a reader is most often migrating away from are named as context in Series 2 · 00 rather than taught
  • Desktop virtualization — VirtualBox, VMware Workstation and running a VM on your laptop — not covered as a subject, though Series 1 · 00 says plainly where it fits. This course is about machines that other people depend on