OpenWrt in QEMU

QEMU is an open-source processor emulator and virtualizer. This document describes how to run OpenWrt as a guest operating system in QEMU. If you are looking to use OpenWrt as a QEMU host, see Running QEMU guests on OpenWrt.

It is mixed descriptions from Windows and Linux, so please read through all of it before starting.

Choosing different emulation settings can affect performance greatly.

Example: 30s iperf-s@openwrt (QEMU running on host) to host

ne2k_pci:0.0-31.3 sec  14.6 MBytes  3.92 Mbits/sec
pcnet: 0.0-30.0 sec  2.38 GBytes   682 Mbits/sec
e1000: 0.0-30.0 sec  6.23 GBytes  1.79 Gbits/sec
vmxnet3: 0.0-30.0 sec  8.67 GBytes  2.48 Gbits/sec
virtio-net-pci:  0.0-30.0 sec  44.6 GBytes  12.8 Gbits/sec

• Trunk: test kernel image with rootfs
• Trunk: use SD card with rootfs, NFS rootfs, NBD rootfs
• Trunk: no sound, pcibus, USB emulation in QEMU possible?

QEMU is compatible with a wide range of operating systems. For most users, the easiest way to install it is through your system's package manager.

Most major Linux distributions (including Debian, Ubuntu, openSUSE, and Fedora) include QEMU in their official repositories.

Full Installation: Install the complete suite to emulate all supported architectures.

# Debian
sudo apt-get install qemu

# Fedora
sudo dnf install qemu

Targeted Architecture: If you only need to run or debug specific virtual machines (e.g., x86_64), you can save disk space by installing only that target:

# Debian
sudo apt-get install qemu-system-x86_64

The most straightforward way to install QEMU on macOS is using Homebrew or MacPorts.

# homebrew
homebrew install qemu

# MacPorts
sudo port install qemu

On Apple Silicon (M1/M2/M3/M4/M5) Macs, QEMU supports the native HVF (Hypervisor.framework) acceleration backend for near-native performance when running ARM-based guest OSes.

Install manually by downloading the latest 64-bit installer from the official QEMU download page, or use a package manager:

# Winget
winget install -e --id SoftwareFreedomConservancy.QEMU

# Chocolatey
choco install qemu

To enable hardware acceleration on Windows, ensure WHPX (Windows Hypervisor Platform) or HAXM is enabled in your system settings.

When running OpenWrt in QEMU, you can either emulate a foreign architecture (e.g., running ARM code on an x86_64 host) or virtualize a matching architecture with hardware acceleration (e.g., running x86_64 on x86_64 with KVM).

• Native (Accelerated): Uses KVM (Linux), HVF (macOS), or WHPX (Windows) for near-native performance. Requires the host and guest architectures to match.
• Emulated: QEMU translates every instruction, which is significantly slower but allows running any supported architecture on any host.

For the best experience, use an accelerated setup whenever possible.

Linux hosts can use KVM for full hardware acceleration. The libvirt management framework can also be used to manage QEMU guests more easily. Non-accelerated emulation (without KVM) is also possible but slower.

qemu-system-x86_64 -enable-kvm .. # other QEMU options

On Apple Silicon Macs (M1+), QEMU can use HVF for native virtualization of ARM guests. This provides near-native performance. You must use an armsr/armv8 OpenWrt image for this to work.

qemu-system-aarch64 -accel hvf .. # other QEMU options

OpenWrt provides several types of images. Understanding these is key to choosing the right boot method.

Open https://downloads.openwrt.org/ and find desired OpenWrt release and architecture. For Arm64 use armsr target.

Simplest method for testing. Runs entirely in RAM; changes are lost on reboot.

# x86_64
qemu-system-x86_64 -M q35 -m 512 -nographic \
  -kernel openwrt-x86-64-generic-initramfs-kernel.bin

# armv8
qemu-system-aarch64 -M virt-8.2 -m 512 -nographic \
  -kernel openwrt-armsr-armv8-generic-initramfs-kernel.bin

Boot a kernel and point it to a separate rootfs image via VirtIO block device.

# x86_64
qemu-system-x86_64 -M q35 -m 512 -nographic \
  -kernel openwrt-x86-64-generic-kernel.bin \
  -drive file=openwrt-x86-64-generic-ext4-rootfs.img,format=raw,if=virtio \
  -append "root=/dev/vda console=ttyS0 rootwait"

# armv8
qemu-system-aarch64 -M virt-8.2 -m 512 -nographic \
  -kernel openwrt-armsr-armv8-generic-kernel.bin \
  -drive file=openwrt-armsr-armv8-generic-ext4-rootfs.img,format=raw,if=virtio \
  -append "root=/dev/vda rootwait"

Straightforward persistent method. Use a combined image as a drive.

# x86_64 boot using BIOS
qemu-system-x86_64 -M q35 -m 512 -nographic \
  -drive file=openwrt-x86-64-generic-ext4-combined.img,format=raw,if=virtio

# x86_64 boot using UEFI
qemu-system-x86_64 -M q35 -m 512 -nographic \
  -drive file=openwrt-x86-64-generic-ext4-combined-efi.img,format=raw,if=virtio

# armv8 boot using UEFI (change to your actual path to UEFI firmware image supplied with QEMU)
qemu-system-aarch64 -M virt-8.2 -m 512 -nographic \
  -drive file=openwrt-armsr-armv8-generic-ext4-combined-efi.img,format=raw,if=virtio \
  -bios /opt/homebrew/share/qemu/edk2-aarch64-code.fd

Use a local directory on the host as the guest's root filesystem via 9p driver.

# x86_64
mkdir -p root-fs && tar xvz -C root-fs -f openwrt-x86-64-rootfs.tar.gz

qemu-system-x86_64 -M q35 -m 512 -nographic \
  -kernel openwrt-x86-64-generic-kernel.bin \
  -fsdev local,id=rootdev,path=./root-fs/,security_model=none \
  -device virtio-9p-pci,fsdev=rootdev,mount_tag=/dev/root \
  -append "rootflags=trans=virtio,version=9p2000.L,cache=loose rootfstype=9p"

# armv8
mkdir -p root-fs && tar xvz -C root-fs -f openwrt-armsr-armv8-rootfs.tar.gz

qemu-system-aarch64 -M virt-8.2 -m 512 -nographic \
  -kernel openwrt-armsr-armv8-generic-kernel.bin \
  -fsdev local,id=rootdev,path=./root-fs/,security_model=none \
  -device virtio-9p-pci,fsdev=rootdev,mount_tag=/dev/root \
  -append "rootflags=trans=virtio,version=9p2000.L,cache=loose rootfstype=9p" \
  -bios /opt/homebrew/share/qemu/edk2-aarch64-code.fd

By default, QEMU's user-mode network stack provides NAT and DHCP/DNS to the guest, so the guest can access the internet by default.

QEMU offers several networking options. The modern approach uses -netdev and -device:

-device virtio-net-pci,netdev=net0 \
-netdev user,id=net0,net=192.168.1.0/24

To provide two interfaces (LAN and WAN) for a more router-like setup:

# LAN_IP=192.168.1.0/24
# WAN_IP=10.1.1.0/24

-netdev user,id=lan,net=${LAN}> \
-device virtio-net-pci,netdev=lan \
-netdev user,id=wan,net=${WAN} \
-device virtio-net-pci,netdev=wan

In OpenWrt, eth0 will be enslaved as a bridge port of br-lan and eth1 will be WAN.

-netdev user,id=net0,net=192.168.1.0/24,hostfwd=tcp:127.0.0.1:2222-192.168.1.1:22,hostfwd=tcp:127.0.0.1:8888-192.168.1.1:80 \
-device virtio-net-pci,netdev=net0

This creates a user-mode network with port forwarding for SSH and LuCI.

Access:

• SSH: ssh root@127.0.0.1 -p 2222
• LuCI: http://127.0.0.1:8888

Keep in mind that you need to install LuCI from a command line: apk update && apk add luci

If for some reason you want to connect to OpenWrt via WAN adapter - add firewall rules to allow SSH:

uci -q delete firewall.ssh
uci set firewall.ssh="rule"
uci set firewall.ssh.name="Allow-SSH"
uci set firewall.ssh.src="wan"
uci set firewall.ssh.dest_port="22"
uci set firewall.ssh.proto="tcp"
uci set firewall.ssh.target="ACCEPT"
uci commit firewall
/etc/init.d/firewall restart

The x86_64 target is well-supported. Use the q35 machine type for modern PCIe emulation.

Boot combined image with ext4, using KVM:

qemu-system-x86_64 \
  -enable-kvm \
  -M q35 -m 512 -smp 2 -nographic \
  -drive file=openwrt-x86-64-generic-ext4-combined.img,format=raw,if=virtio \
  -netdev user,id=net0,net=192.168.1.0/24,hostfwd=tcp:127.0.0.1:2222-192.168.1.1:22,hostfwd=tcp:127.0.0.1:8888-192.168.1.1:80 \
  -device virtio-net-pci,netdev=net0 \
  -netdev user,id=net1,net=10.1.1.0/24 \
  -device virtio-net-pci,netdev=net1

For OpenWrt 23 and newer, use the armsr target (replaces armvirt). The armv8 subtarget is for 64-bit ARM.

Boot combined image with ext4, using Apple Hypervisor.framework (hvf)

qemu-system-aarch64 \
  -accel hvf \
  -M virt-8.2 -m 512 -smp 2 -nographic \
  -drive file=openwrt-armsr-armv8-generic-ext4-combined-efi.img,format=raw,if=virtio \
  -netdev user,id=net0,net=192.168.1.0/24,hostfwd=tcp:127.0.0.1:2222-192.168.1.1:22,hostfwd=tcp:127.0.0.1:8888-192.168.1.1:80 \
  -device virtio-net-pci,netdev=net0 \
  -netdev user,id=net1,net=10.1.1.0/24 \
  -device virtio-net-pci,netdev=net1 \
  -bios /opt/homebrew/share/qemu/edk2-aarch64-code.fd

The malta target emulates a MIPS development board. It supports 32-bit and 64-bit, big and little-endian variants.

qemu-system-mipsel -kernel bin/targets/malta/le/openwrt-malta-le-vmlinux-initramfs.elf \
  -nographic -m 256

qemu-system-mips -kernel bin/targets/malta/be/openwrt-malta-be-vmlinux-initramfs.elf \
  -nographic -m 256

qemu-system-mips64el -kernel bin/targets/malta/le64/openwrt-malta-le64-vmlinux-initramfs.elf \
  -cpu MIPS64R2-generic -nographic -m 256

qemu-system-mips64 -kernel bin/targets/malta/be64/openwrt-malta-be64-vmlinux-initramfs.elf \
  -cpu MIPS64R2-generic -nographic -m 256

RISC-V support in OpenWrt uses the virt machine with an embedded boot flow: OpenSBI → U-Boot → FIT image from NOR flash.

Default boot (auto-detect sysupgrade.bin via U-Boot with FIT on pflash):

qemu-system-riscv64 -nographic -machine virt \
  -smp 2 -m 512 \
  -kernel u-boot.bin \
  -drive file=sysupgrade.bin,format=raw,if=pflash,unit=1

Explicit rootfs with initramfs kernel (legacy mode):

qemu-system-riscv64 -nographic -machine virt \
  -smp 2 -m 512 \
  -kernel openwrt-riscv64-generic-initramfs-kernel.bin \
  -drive file=sdcard.img,format=raw,if=virtio \
  -append "root=/dev/vda2 ro console=ttyS0"

Fallback (no firmware images found): boot initramfs kernel directly via OpenSBI:

qemu-system-riscv64 -nographic -machine virt \
  -smp 2 -m 512 \
  -kernel openwrt-riscv64-generic-initramfs-kernel.bin

For SquashFS images, OpenWrt initializes the writable rootfs_data overlay on first boot using the unused space following the read-only root filesystem. Depending on the available space and configuration, this area is formatted as F2FS or ext4; F2FS is selected automatically when more than 100 MiB is available, unless overridden by the kernel command line.

Perform hard factory reset if F2FS is corrupted or failed to set up properly. The exact expanding method depends on your virtualization system.

If you are using KVM/QEMU virtualization.

Increase base image size to 512M:

gzip -dk openwrt-x86-64-generic-squashfs-combined-efi.img.gz
qemu-img resize -f raw openwrt-x86-64-generic-squashfs-combined-efi.img 512M

Loop mount the base image so it can be modified:

loop_device=$(sudo losetup -f)
sudo losetup $loop_device openwrt-x86-64-generic-squashfs-combined-efi.img

Fix the GPT partition and increase the root partition size to 100% (512M):

echo -e "OK\nFix" | sudo parted ---pretend-input-tty "$loop_device" print
sudo parted "$loop_device" resizepart 2 100%
sudo parted "$loop_device" print

Remove the loop mount device

sudo losetup -d $loop_device

If you are using KVM/QEMU with libvirt management framework. Import the image as-is and stop the first boot on the GRUB screen. Then expand the block device online as follows and continue the boot.

virsh blockresize openwrt vda 300M
virsh vol-resize openwrt 300M default

If you are using other virtualization systems like Proxmox, VMWare ESXi/Workstation, VirtualBox, XenServer/XCP-ng, and any other self-respecting virtualization software it is also possible to expand the drive but you will have to look at its own documentation for guidance.

(taken from mailing list post by Philip Prindeville)

On my Centos 7.4 KVM host, I did:

To provision 10 VFs per NIC:

cat << EOF > /etc/modprobe.d/sr-iov.conf
# for SR-IOV support
options igb max_vfs=10
EOF

This will take effect after the next reboot. Alternatively by unloading and reloading the IGB module.

Create XML files for each NIC you want to support virtualization on:

# cat << EOF > /tmp/hostdev-net0.xml
<network>
<name>hostdev-net0</name>
<uuid>$(uuidgen)</uuid>
<forward mode='hostdev' managed='yes'>
<pf dev='eno1'/>
</forward>
</network>
EOF

cat << EOF > /tmp/hostdev-net1.xml
<network>
<name>hostdev-net1</name>
<uuid>$(uuidgen)</uuid>
<forward mode='hostdev' managed='yes'>
<pf dev='eno2'/>
</forward>
</network>
EOF

virsh net-destroy default
virsh net-define /tmp/hostdev-net0.xml
virsh net-autostart hostdev-net0
virsh net-define /tmp/hostdev-net1.xml
virsh net-autostart hostdev-net1

To create the pool of VF interfaces.

Then to add interfaces to VMs, I did:

# cat << EOF > /tmp/new-interface-0.1.xml
<interface type='network'>
<mac address='52:54:00:0d:84:f4'/>
<source network='hostdev-net0'/>
<address type='pci' domain='0x0000' bus='0x07' slot='0x10' function='0x0'/>
</interface>
EOF

# Where the ‘0d:84:f4’ is 3 unique bytes
dd status=none bs=1 count=3 if=/dev/urandom | hexdump -e '/1 "%x"\n'

virsh attach-device my-machine-1 /tmp/new-interface-0.1.xml