Cloud and Self-Hosting

GPU Passthrough in Proxmox VE

Proxmox GPU passthrough step by step for PVE 8 and 9: IOMMU, vfio-pci binding, VM settings for NVIDIA, AMD and Intel, verification commands and common fixes.

By Amjid Ali · · Updated · 7 min read

In this article
  1. Quick steps
  2. Requirements
  3. Step 1: enable IOMMU in the BIOS
  4. Step 2: set the kernel command line
  5. Step 3: load the VFIO modules
  6. Step 4: reboot and verify IOMMU
  7. Step 5: check the IOMMU groups
  8. Step 6: find the GPU IDs and bind it to vfio-pci
  9. Step 7: verify the GPU is bound to vfio-pci
  10. Step 8: add the GPU to a VM
  11. Step 9: install drivers and verify in the guest
  12. Intel iGPU passthrough
  13. Troubleshooting
  14. Related guides

Prefer to watch? This video walks through it step by step. Prefer to read? The full guide is below.

To set up GPU passthrough in Proxmox VE, enable VT-d (Intel) or AMD-Vi/IOMMU (AMD) in the BIOS, turn on IOMMU passthrough mode with iommu=pt, load the vfio modules, bind the GPU to vfio-pci so the host leaves it alone, reboot, and add the card to a q35 + OVMF VM as a PCI device with All Functions and PCI-Express ticked. On Proxmox VE 8.2 and later (kernel 6.8+), intel_iommu=on is no longer needed because IOMMU is on by default.

I use passthrough in my homelab and have done it with an AMD FirePro W5100 on a Xeon box, which I wrote up as a worked example on Proxmox VE 9. This guide is the general method for any NVIDIA, AMD or Intel GPU, checked against the current Proxmox documentation.

Quick steps

  1. BIOS: enable VT-d / AMD-Vi (IOMMU), set it to Enabled not Auto, and disable CSM if you can.
  2. Kernel command line: add iommu=pt (and intel_iommu=on only on kernels older than 6.8).
  3. Modules: vfio, vfio_iommu_type1, vfio_pci in /etc/modules-load.d/vfio.conf.
  4. Bind the GPU: options vfio-pci ids=<gpu-id>,<audio-id> and blacklist or softdep the host driver.
  5. update-initramfs -u -k all, reboot, and check lspci -nnk shows vfio-pci.
  6. VM: machine q35, BIOS OVMF, add PCI Device, tick All Functions, PCI-Express and, if it drives a display, Primary GPU.

Requirements

  • A CPU and motherboard with IOMMU and interrupt remapping: Intel VT-d or AMD-Vi. Most current desktop and server platforms have it; server boards usually behave better.
  • A GPU that the host does not need. If it is your only GPU, you lose the host's local console once it is bound to vfio (SSH and the web UI still work).
  • For OVMF (UEFI) VMs, a GPU with a UEFI-capable ROM. Most cards from the last ten years have one; otherwise use SeaBIOS.
  • Proxmox VE 8 or 9 with shell access. Commands below are run as root on the host.

Step 1: enable IOMMU in the BIOS

Look for VT-d or Intel Virtualization Technology for Directed I/O on Intel, and IOMMU or AMD-Vi on AMD (often under Advanced, Chipset or NBIO). Set it to Enabled, not Auto. While you are there:

  • enable Above 4G Decoding;
  • turn off Resizable BAR / Smart Access Memory if you hit problems with an AMD card (the wiki mentions it can cause trouble);
  • enable ACS if the board offers it, which gives cleaner IOMMU groups.

Step 2: set the kernel command line

First find out which boot loader your node uses:

proxmox-boot-tool status

If it reports systemd-boot (typical for ZFS root on UEFI) you edit /etc/kernel/cmdline; otherwise (GRUB, typical for ext4/LVM installs) you edit /etc/default/grub.

GRUB:

nano /etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="quiet iommu=pt"
update-grub

systemd-boot: /etc/kernel/cmdline is a single line. Append iommu=pt to the end of it, for example:

root=ZFS=rpool/ROOT/pve-1 boot=zfs iommu=pt
proxmox-boot-tool refresh

On Intel with a kernel older than 6.8 (Proxmox VE 8.1 or earlier), also add intel_iommu=on. Check your kernel with uname -r. AMD never needs amd_iommu=on; IOMMU is enabled by default.

Older guides, including the first version of this one, also added pcie_acs_override=downstream,multifunction nofb nomodeset video=vesafb:off,efifb:off by default. Do not start with those. The ACS override is a last resort with real security trade-offs, and the framebuffer options are only for specific errors covered in troubleshooting below.

Step 3: load the VFIO modules

cat > /etc/modules-load.d/vfio.conf <<'EOF'
vfio
vfio_iommu_type1
vfio_pci
EOF

vfio_virqfd is no longer a separate module on current kernels (it was merged into vfio in kernel 6.2), so you can leave it out; older guides that list it are from the Proxmox VE 7 era.

Step 4: reboot and verify IOMMU

update-initramfs -u -k all
reboot

After the reboot:

dmesg | grep -e DMAR -e IOMMU -e AMD-Vi

You want lines like DMAR: IOMMU enabled (Intel) or AMD-Vi: Interrupt remapping enabled (AMD). Check interrupt remapping:

dmesg | grep remapping
DMAR-IR: Enabled IRQ remapping in x2apic mode

Check the modules loaded:

lsmod | grep vfio

Step 5: check the IOMMU groups

The GPU must be in its own IOMMU group (sharing it with its own HDMI audio function or its PCIe root port is fine):

pvesh get /nodes/$(hostname)/hardware/pci --pci-class-blacklist ""

Look at the iommugroup column for your GPU. If it shares a group with, say, a SATA controller or NIC the host needs, try another PCIe slot or enable ACS in the BIOS. Only as a last resort add pcie_acs_override=downstream to the kernel command line, knowing it weakens isolation between devices.

Step 6: find the GPU IDs and bind it to vfio-pci

lspci -nn | grep -E "VGA|3D|Audio"

Example output from an NVIDIA card:

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP108 [GeForce GT 1030] [10de:1d01] (rev a1)
01:00.1 Audio device [0403]: NVIDIA Corporation GP108 High Definition Audio Controller [10de:0fb8] (rev a1)

The IDs in the last brackets (10de:1d01 and 10de:0fb8) are what you need. Use the IDs from your own output, for both the VGA and the audio function:

echo "options vfio-pci ids=10de:1d01,10de:0fb8" > /etc/modprobe.d/vfio.conf

Proxmox tries to take the device away from the host automatically when a VM starts, but binding it at boot is more reliable for GPUs. Next stop the host driver grabbing the card first. Either load vfio-pci before the GPU driver with a soft dependency (my preferred way, it only affects ordering):

echo "softdep nouveau pre: vfio-pci" >> /etc/modprobe.d/nouveau.conf
echo "softdep nvidia pre: vfio-pci" >> /etc/modprobe.d/nvidia.conf

or blacklist the driver completely, which is simpler if the host never uses that GPU family:

# NVIDIA
echo "blacklist nouveau" >> /etc/modprobe.d/blacklist.conf
echo "blacklist nvidia*" >> /etc/modprobe.d/blacklist.conf
# AMD
echo "blacklist amdgpu" >> /etc/modprobe.d/blacklist.conf
echo "blacklist radeon" >> /etc/modprobe.d/blacklist.conf

Do not blacklist i915 if the host uses its Intel iGPU, for example for an LXC Plex or Jellyfin container. On AMD use amdgpu/radeon in the softdep lines instead.

If you will use OVMF, you can also disable VGA arbitration, as the admin guide suggests:

echo "options vfio-pci ids=10de:1d01,10de:0fb8 disable_vga=1" > /etc/modprobe.d/vfio.conf

Apply and reboot:

update-initramfs -u -k all
reboot

Step 7: verify the GPU is bound to vfio-pci

lspci -nnk -s 01:00
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP108 [GeForce GT 1030] [10de:1d01] (rev a1)
        Kernel driver in use: vfio-pci
        Kernel modules: nvidiafb, nouveau
01:00.1 Audio device [0403]: NVIDIA Corporation GP108 High Definition Audio Controller [10de:0fb8] (rev a1)
        Kernel driver in use: vfio-pci
        Kernel modules: snd_hda_intel

Kernel driver in use: vfio-pci on both functions means the host is ready. If it still shows nouveau, nvidia or amdgpu, check the IDs in vfio.conf, the softdep or blacklist, and that you ran update-initramfs.

Step 8: add the GPU to a VM

Create the VM with Machine: q35 and BIOS: OVMF (UEFI) with an EFI disk. The admin guide says q35, OVMF and PCIe give the best compatibility. Install the OS first with the default display, then:

  1. VM, Hardware > Add > PCI Device.
  2. Choose Raw Device and select the GPU (0000:01:00), or a Mapped Device if you created one under Datacenter > Resource Mappings (useful in a cluster).
  3. Tick All Functions so the HDMI audio comes along.
  4. Under Advanced, tick PCI-Express. Tick Primary GPU only if the card should be the VM's main display, and leave ROM-Bar on.
  5. Click Add.

The CLI equivalent:

qm set 101 --machine q35 --bios ovmf
qm set 101 --hostpci0 0000:01:00,pcie=1,x-vga=1

Leave out x-vga=1 for a compute-only card (CUDA, Ollama, transcoding). With x-vga on, the virtual display is ignored and the noVNC console goes blank, which is expected: Proxmox cannot show a passed-through GPU in noVNC or SPICE. Plug in a monitor or HDMI dummy plug, or use RDP, VNC, Parsec or Sunshine inside the guest.

Set CPU type: host for best performance, and turn Ballooning off: memory of a VM with a PCI device must be fully allocated, so ballooning does not help.

Step 9: install drivers and verify in the guest

Linux guest with NVIDIA (Ubuntu example):

lspci | grep -i nvidia
sudo ubuntu-drivers install
sudo reboot
nvidia-smi

nvidia-smi should show your card, driver version and memory. From there you can run CUDA workloads or set up Ollama on Ubuntu Server for local LLMs.

Windows guest: install the normal NVIDIA or AMD driver from the vendor. The GPU should show in Device Manager without a warning. The old NVIDIA Code 43 problem with consumer cards in VMs is gone with driver 465 and newer, according to the Proxmox wiki.

AMD on Linux: the amdgpu driver is in the kernel; check with lspci -k inside the guest and dmesg | grep amdgpu.

Intel iGPU passthrough

For an Intel integrated GPU, pass the iGPU at 00:02.0 the same way. Older GVT-g (splitting the iGPU into virtual GPUs) only supports up to 10th generation CPUs. Newer Intel GPUs (some 11th gen and later, Arc) support SR-IOV with additional drivers, which is outside this guide. If you only need Quick Sync for Plex or Jellyfin, an LXC container with the /dev/dri device passed in is simpler and lets the host keep the iGPU.

Troubleshooting

No IOMMU detected or the PCI Device dialog says "No IOMMU detected, please activate it". VT-d/AMD-Vi is off in the BIOS, or the command line change did not apply. Check cat /proc/cmdline shows your options, and that you ran update-grub or proxmox-boot-tool refresh for the right boot loader.

VM will not start with group X is not viable, please ensure all devices within the iommu_group are bound to their vfio bus driver. Another device in the same IOMMU group is still in use by the host. Pass the whole group, move the card to another slot, or as a last resort use the ACS override.

BAR 3: can't reserve [mem ...] in dmesg. The host's framebuffer is using the card. Add video=efifb:off to the kernel command line (and on older kernels video=vesafb:off), refresh the boot loader, reboot.

Windows shows Code 12, or large-VRAM cards fail with OVMF. The VM does not have enough MMIO space. The wiki's fix:

qm set 101 --cpu host
qm set 101 --args '-fw_cfg name=opt/ovmf/X-PciMmio64Mb,string=65536'

Black screen but the VM runs. Make sure the monitor is on the passed-through card, try ticking or unticking Primary GPU, and set the virtual display to none once drivers are installed.

AMD GPU works once, then fails until a host reboot. That is the AMD reset bug. Install the vendor-reset module, which the Proxmox wiki points to.

Host freezes or VM fails with many ignored rdmsr messages. Set options kvm ignore_msrs=1 report_ignored_msrs=0 in /etc/modprobe.d/kvm.conf, update the initramfs and reboot. If dmesg | grep remapping shows no interrupt remapping, the wiki's workaround is options vfio_iommu_type1 allow_unsafe_interrupts=1, but it can make the system unstable, so only use it if needed.

It broke after a Proxmox upgrade. Some passthrough setups failed with newer kernels. Boot the previous kernel and pin it with proxmox-boot-tool kernel pin <version> while you investigate, and check that your kernel command line and /etc/modprobe.d files survived. My Proxmox VE 8 to 9 upgrade guide lists the known issues.

For every option mentioned here, the reference is the PCI(e) passthrough chapter of the Proxmox VE Administration Guide and the PCI Passthrough wiki page.

Frequently asked questions

How do I enable GPU passthrough in Proxmox?+

Enable VT-d or AMD-Vi in the BIOS, add iommu=pt to the kernel command line (plus intel_iommu=on on kernels older than 6.8), load the vfio, vfio_iommu_type1 and vfio_pci modules, bind the GPU to vfio-pci by its vendor and device IDs, update the initramfs and reboot. Then add the GPU to a q35 VM with OVMF under Hardware, Add, PCI Device, with All Functions and PCI-Express ticked.

Do I still need intel_iommu=on in Proxmox VE 8 and 9?+

No, not on current kernels. The Proxmox admin guide says IOMMU is on by default for AMD CPUs, and for Intel CPUs on kernel 6.8 or newer, which Proxmox VE 8.2 and later use. You still need VT-d enabled in the BIOS, and iommu=pt is optional but recommended.

How do I check that IOMMU is working in Proxmox?+

Run dmesg | grep -e DMAR -e IOMMU -e AMD-Vi and look for a line saying IOMMU or interrupt remapping is enabled. Then list the IOMMU groups with pvesh get /nodes/<node>/hardware/pci --pci-class-blacklist "" and make sure the GPU is not grouped with other devices you need on the host.

Can I see a passed-through GPU in the Proxmox noVNC console?+

No. Proxmox cannot show the passed-through GPU's frame buffer in noVNC or SPICE. Connect a monitor (or an HDMI dummy plug) to the card, or use remote desktop inside the guest. For compute only, such as CUDA or Ollama, no display is needed.

Can I share one GPU between several VMs or containers?+

Full passthrough gives the whole GPU to one running VM. To share, use LXC containers that bind the host's GPU device nodes, NVIDIA vGPU on supported data centre cards, or SR-IOV on GPUs that support it, such as some recent Intel GPUs.

Why does my AMD GPU fail after a VM reboot?+

Many older AMD cards (Polaris, Vega and some Navi) do not reset cleanly, the so-called AMD reset bug. The VM works once and then fails to start until the host reboots. The vendor-reset kernel module linked from the Proxmox wiki fixes this for many cards.

Sources

  1. Proxmox VE Administration Guide, PCI(e) Passthrough, Proxmox Server Solutions
  2. PCI Passthrough (wiki), Proxmox Server Solutions