GPU Passthrough Crashes on UnRaid

Fixing RTX 3090 VFIO Passthrough Crashes on Unraid: QEMU, IOMMU and Physical Address Width

I recently ran into a particularly awkward GPU passthrough problem on my Unraid server. The machine is based on an AMD Threadripper 3960X with 128 GB RAM and an NVIDIA RTX 3090, with the 3090 passed through to a Windows 11 VM.

GPU passthrough initially appeared to work. Windows booted, the NVIDIA driver loaded, the desktop was stable, and even 4K video playback worked normally. However, as soon as the GPU was placed under substantial 3D load, it disappeared from the host and the VM failed.

Symptoms

Running Unigine Superposition at 1080p Medium caused the display to go black almost immediately. The Unraid/QEMU logs then contained errors along these lines:

vfio: Unable to power on device, stuck in D3 vfio_container_dma_map(..., 0x383000000000, 0x10000000, ...) = -14 (Bad address) 0000:4d:00.0: PCI peer-to-peer transactions on BARs are not supported. error: kvm run failed Bad address

Afterwards the GPU would often be inaccessible to VFIO, with failed D0/D3 power transitions, failed resets and PCIe link recovery attempts. A complete host power cycle was sometimes required before the card could be passed through again.