libvirt user networking or: secure setup of libvirt-based server VMs

I know, the introduction says

there is no much point in publishing the 999th blog on Linux / ZigBee / HomeAssistant / whatever

and yet already the first post will be at least somewhat HomeAssistant related. But I think that this is a poorly-covered detail on the internet, so it might be worth a read.

A couple of weeks ago, HomeAssistant deprecated the core and supervised installation methods, requiring a migration to a VM- or container-based setup .

As much as I liked running HomeAssistant as a Python package in a virtual environment with no overhead of a VM or docker, I have no other choice than migrating my HomeAssistant instances one after another. As I don’t like docker, a haos-VM with libvirt/qemu was the obvious choice. Additionally, it supports add-ons and one-click-updates in contrast to the container.

HomeAssistant recommends to bridge the network connection into the VM. However, I don’t want to allow unrestricted network access to the VM, so I prefer a NAT connection. systemd-nspawn, for example, which I usually use for containers, supports forwarding ports over a NATted connection out of the box . But as it turns out, forwarding ports over a NATted connection into a libvirt/qemu-based VM is not as trivial as I expected it to be.

Apparently, there is no libvirt-based mechanism for this problem. There is, however, user networking in QEMU, so there is at least a solution for QEMU-backed VMs. However, I couldn’t find much (if any) documentation on how to do this reliably when running VMs from QEMU. I don’t recall my sources, as some weeks passed since I did this for the first time right after the core deprecation in May, but I have done this on multiple HomeAssistant installations by now, and I want to share my solution. (And document it for my future self to avoid copying XML from one host to another;))

Configuration

Long story short, to expose the HomeAssistant port on my VM host, I add the following QEMU parameters to my libvirt XML domain (e.g. via virsh edit <domain> or using the XML-tab in virt-manager):

<qemu:commandline>
	<qemu:arg value="-netdev"/>
	<qemu:arg value="user,id=mynet.0,net=10.252.10.0/24,hostfwd=tcp::8123-:8123"/>
	<qemu:arg value="-device"/>
	<qemu:arg value="e1000,netdev=mynet.0"/>
</qemu:commandline>

The IP address space can be modified of course, I chose one that doesn’t collide with the default libvirt namespace (192.168.122.0/24 in my case). It is also possible to specify a host IP address to bind the listener to, consult the QEMU documentation for details. To allow qemu:commandline in the XML file, the XML namespace has to be adapted to. Change

<domain type="kvm">

to

<domain xmlns:qemu="http://libvirt.org/schemas/domain/qemu/1.0" type="kvm">

at the top of the XML file.

Important: These two changes need to be done at the same time, because libvirt will not only remove the qemu:commandline when the XML namespace is not set, it will also unset the XML namespace if it is not required by the configuration. This confused me a lot initially, I always thought my syntax of modifying the XML namespace was wrong.

Afterwards, stop and restart the Virtual Machine, and check the open ports on your host:

user@host:~$ ss -tlpn
State     Recv-Q    Send-Q        Local Address:Port         Peer Address:Port    Process    
LISTEN    0         1                   0.0.0.0:8123              0.0.0.0:*                  

Resolving PCI collisions

A second network adapter will be added to the VM, the first one probably can be removed, but I left mine set to the NATted default network of libvirt. As the additional network interface will use a default PCI bus and slot, a collision with one of the libvirt PCI devices may occur. In this case, libvirt will show an error message like

PCI: slot 2 function 0 not available for virtio-vga, in use by e1000,id=(null)

when trying to start the VM. I didn’t find a way to change the PCI slot used by the second network adapter, so I usually go and increment the PCI slot id of my affected libvirt devices by 10. So in this case,

<video>
  <model type="virtio" heads="1" primary="yes"/>
  <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x0"/>
</video>

becomes

<video>
  <model type="virtio" heads="1" primary="yes"/>
  <address type="pci" domain="0x0000" bus="0x00" slot="0x12" function="0x0"/>
</video>

repeat this for any other devices that may collide, e.g. the default network interface. In some case, I had to change the PCI slot of multiple devices in my libvirt configuration.

I hope this helps in setting up secure servers in libvirtd, like HomeAssistant in this case :)