I recently upgraded my RaspberryPi 4, which is running Home Assistant and a bunch of other services, to an x86 mini-PC (a Lenovo ThinkCenter M920). To ease the transition, I wanted to leave the RaspberryPi’s OS running as a virtual machine inside the x86 host. This way I can port services over to the x86 machine gradually, but already disconnect the RPi hardware and reuse it for some other project.

QEMU can emulate a generic ARM board using the versatilepb machine type, or it can more faithfully emulate the RaspberryPi with one of the raspi machines. The versatilepb system is limited to 1 CPU and 256 MB RAM so this was not going to cut it. The raspi models do not support the RaspberryPi 4, but do go up to the RaspberryPi 3B with 4 CPUs and 1 GB memory so I tried that one first.

QEMU’s raspi3b machine type

Porting a physical RaspberryPi to a QEMU raspi3b is pretty straightforward, there are many tutorials on the internet but I ended up with the following QEMU command line.

qemu-system-aarch64 \
    -display none \
    -machine raspi3b \
    -cpu cortex-a72 \
    -dtb /rpi/boot/bcm2710-rpi-3-b-plus.dtb \
    -m 1G -smp 4 -serial stdio \
    -kernel /rpi/boot/kernel8.img \
    -append "rw earlyprintk loglevel=8 console=ttyAMA0,115200 dwc_otg.lpm_enable=0 root=/dev/mmcblk0p1 rootdelay=1" \
    -sd /rpi/root.img \
    -device usb-net,netdev=net0 -netdev tap,id=net0,ifname=tap0,script=no,downscript=no

This assumes that you have copied the RPi’s SD card to a file /rpi/root.img on the host, as well as the kernel and relevant DTB file from /boot. Networking goes through an emulated USB Ethernet device, which is what the real Raspberry Pi 3 uses and as far as I could find the only device that works using QEMU’s raspi3b machine type. Since I wanted my emulated RPi to have full networking capabilities, I loosely followed this post to set up a TAP device bridged to the host’s network adapter:

ip tuntap add name tap0 mode tap
ip link set up dev tap0
ip link set tap0 master br0

Since I was already running Home Assistant using Docker on my x86 machine, I also got bitten by this issue which prevented my emulated RPi from communicating with the outside. This was fixed by adding this to my startup script:

sysctl net.bridge.bridge-nf-call-iptables=0

While this setup was fully functional, the speed of the emulated USB Ethernet device was quite terrible: usually less than 1 Mbit/s (I was used to my physical RaspberryPi 4 having a Gigabit Ethernet connection), as well as CPU usage being very high on the host even when the RPi wasn’t doing very much. Luckily, QEMU has a better solution.

QEMU virt generic virtual platform

The virt machine type is a generic ARM virtual platform, unlike the raspi models it isn’t locked to a particular configuration (number of CPUs, memory, devices, etc.). It also doesn’t really model most devices such as disks and network controllers directly, but relies on the guest system’s kernel to use special virtualization-specific calls. This means a suitable kernel must be used, but it also means IO is much faster since the emulated kernel isn’t going through all the motions it usually does to write to device registers, nor does it have to run the entire USB stack to talk to an USB Ethernet controller. Instead, most of the work is done by the host, making things much more efficient.

The kernel you get with Raspbian unfortunately does not support these virtualized disk and network devices. However the Debian ARM distro has a suitable kernel, and since Raspbian is based on Debian it’s pretty easy to install it. So while my RPi was still running in QEMU’s raspi3b mode, I ran the following:

wget http://security.debian.org/debian-security/pool/updates/main/l/linux/linux-image-5.10.0-21-armmp-lpae_5.10.162-1_armhf.deb
sudo dpkg --install linux-image-5.10.0-21-armmp-lpae_5.10.162-1_armhf.deb

There are many kernels to choose from in the Debian package repository, so to maximize the probability of success you should pick one that is pretty close to the one that the RPi is already running (mine was on 5.10.103). My Raspbian was still 32-bit — even though it last ran on a Raspberry Pi 4, it was first installed on a Raspberry Pi B in 2014 — so I picked the 32-bit kernel, although the arm64 one should work similarly.

Running the dpkg --install command inside Raspbian is an important step: it doesn’t just extract the kernel (which we could have obtained by extracting the .deb manually on the x86 host); it also installs the kernel modules in /lib/modules/5.10.0-21-armmp-lpae and it builds the initial ramdisk containing all the required device drivers, including the modules needed to access the virtualized disk and network adapter. We can now copy the kernel and initrd from /boot (vmlinuz-5.10.0-21-armmp-lpae and initrd.img-5.10.0-21-armmp-lpae) to the host machine, and run the VM using a command similar to this one:

qemu-system-arm \
    -nographic \
    -machine virt \
    -cpu cortex-a7 \
    -m 2G -smp 4 \
    -drive file=/rpi/root.img,format=raw,id=hd,if=none,media=disk \
    -device virtio-scsi-device -device scsi-hd,drive=hd \
    -device virtio-net-device,netdev=net0 \
    -netdev tap,id=net0,ifname=tap0,script=no,downscript=no \
    -kernel /rpi/vmlinuz-5.10.0-21-armmp-lpae \
    -initrd /rpi/initrd.img-5.10.0-21-armmp-lpae \
    -append 'root=/dev/sda1 panic=1 console=ttyAMA0,115200'

While functionally pretty much the same as using the raspi3b machine type, CPU utilization on the host has been noticeably lower, and network speeds have gone up to 250 Mbit/s. So everything is now pretty much as fast as it did when ran on a real Raspberry Pi, while the actual RPi hardware will be move on to do other things.