Anders Tall

Build a Proxmox home lab inside your Proxmox (nested PVE)

6 min read

Your homelab is also your production. The moment you test something on it, you risk the thing you actually depend on.

The fix is a lab inside the lab: a Proxmox VE node running as a VM on your existing Proxmox host. Snapshot it, break it, roll it back. Point Terraform and Ansible at it and destroy things in a loop — all of it isolated from the household services you actually depend on.

This is the whole series in one place: five short videos plus the written depth behind them. Watch them in the playlist, or read on.

Playlist: Proxmox Lab

Contents

Why a lab inside the lab?

Most home labs grow in the opposite order. You start with one box, add VMs and containers, and within a year it runs the services your household actually relies on — DNS, files, maybe a reverse proxy in front of everything. It is production now, just without a change window.

That leaves two bad options whenever you want to learn something destructive — upgrade a node, try a cluster, rebuild a reverse proxy:

A Proxmox node that is itself a VM on your Proxmox host removes the trade-off:

Start with one node. A three-node cluster is a later chapter; you need it for quorum, HA, live migration and Ceph, but it triples the cost and complexity.

Everything here lives inside the nested lab; the host underneath is left alone.

Install Proxmox VE in a VM

The first video installs a second Proxmox VE inside the lab. A guest wired to a serial console (--serial0 socket) can be followed as plain text with qm terminal <vmid> — which turns a graphical install into something you can watch line by line.

The catch: the unattended installer’s GRUB entry runs proxmox-start-auto-installer without console=ttyS0,115200, so the serial console stays empty. The fix is to patch the installer ISO’s /boot/grub/grub.cfg (add console=ttyS0,115200 to the Automated entry) and rebuild the ISO — once, beforehand.

qm create 100 --name pve-l2 --memory 4096 --cores 2 \
  --cpu host --machine q35 --bios ovmf --scsihw virtio-scsi-single \
  --efidisk0 local-lvm:1,efitype=4m,pre-enrolled-keys=0 \
  --scsi0 local-lvm:8 \
  --ide2 local:iso/pve-l2-serial.iso,media=cdrom --boot order='scsi0;ide2' \
  --net0 virtio=BC:24:11:AA:BB:20,bridge=vmbr0 --serial0 socket --vga std --ostype l26
qm start 100
qm terminal 100          # the install runs live; leave with Ctrl+O

First login to the new node

The install finished — does it work? The verifiable end state is not “installed”, it is “answers on ssh and on its web UI”. We hop through the lab node into the fresh Proxmox, check the version, hit port 8006, and then log in to the same node in the web UI as root@pam.

ssh root@10.10.10.10          # into the lab node
ssh root@10.10.10.20          # on into the new node (key-based, no prompt)
pveversion
curl -sk -o /dev/null -w '%{http_code}' https://10.10.10.20:8006   # -> 200

-k is needed because Proxmox ships a self-signed certificate.

Snapshots: break it on purpose, roll it back

This is the payoff of having a lab at all. Freeze a known-good state, wreck the machine on purpose, roll it back — and lose nothing.

qm snapshot 100 base                 # freeze a known-good state
qm listsnapshot 100

ssh root@10.10.10.20 'echo broken > /root/oops.txt && ls -l /root/oops.txt'

qm stop 100
qm rollback 100 base                 # restore the frozen state
qm start 100

ssh root@10.10.10.20 'ls /root/oops.txt'   # gone

A snapshot is not a backup. A snapshot is a freeze-frame living on the same storage as the VM — for experiments and instant resets. A backup is a copy elsewhere; lose the storage and you lose the VM and every snapshot on it. Snapshots are for the loop of “try the scary thing, undo it”. They are not your 3-2-1.

A VM as code with Terraform

A VM you clicked into existence is undocumented and unreproducible. Connect Terraform to Proxmox’s API with an API token, describe a VM in HCL, and let terraform apply create it — then destroy it. The whole point: infrastructure you can throw away and rebuild exactly. And it all happens against the lab, where a runaway apply can only hurt a machine you were going to delete anyway.

cd series/proxmox-lab/05-terraform-first-vm/terraform
terraform init
terraform plan
terraform apply -auto-approve
ssh root@10.10.10.10 'qm list'      # it exists on the lab node

terraform destroy -auto-approve
ssh root@10.10.10.10 'qm list'      # it is gone

The provider is bpg/proxmox, authenticated with a Proxmox API token — no password in the config. The token secret is shown once; read it from the environment and never hard-code it.

A template with cloud-init

Building a VM by hand every time is slow and inconsistent. A template captures the base system once; with cloud-init the clone boots already configured — no console touched. This is the piece everything else builds on.

qm create 9000 --name tmpl-debian --memory 2048 --cores 2 --cpu host --net0 virtio,bridge=vmbr0

qm importdisk 9000 /var/lib/vz/template/iso/debian-13-genericcloud-amd64.qcow2 local-lvm >/dev/null 2>&1 \
  && qm set 9000 --scsihw virtio-scsi-single --scsi0 local-lvm:vm-9000-disk-0 \
       --ide2 local-lvm:cloudinit --boot order=scsi0

qm set 9000 --ciuser debian --sshkeys /root/.ssh/id_ed25519.pub \
  --ipconfig0 ip=10.10.10.30/24,gw=10.10.10.1
qm template 9000

qm clone 9000 101 --name cloud-demo
qm start 101
ssh debian@10.10.10.30 'hostname; ip -brief address'   # cloud-demo, 10.10.10.30

The clone comes up with the identity cloud-init gave it — user, key, hostname, IP — with no console. qm clone without --full makes a linked clone: near-instant, sharing the template’s base disk. Throw the clone away; the template stays.

Pitfalls

This is the finished series. If you want to build along, the playlist collects all five videos in order.

#proxmox #nested virtualization #homelab #terraform #cloud-init #devops #linux