Virtualised Workstation: Local Cloud-Gaming

My setup is fairly typical amongst homelabbers in that I have my server cluster, high performance workstation PC and a laptop. The workstation is equipped with a Threadripper 2990WX, an RTX 3090 and 128GB RAM. This is mostly used for gaming, CAD and 3D design and the occasional GPU heavy tasks such as running AI locally.

For the past year or so I've been experimenting with game streaming using Moonlight, allowing me to leverage my hardware on any display, including TVs and projectors around the house. While I've mostly moved away from gaming, family members took advantage of the on-demand gaming experience.

I recently considered integrating my main server with the workstation to streamline my infrastructure, allowing it to serve both as a server and a gaming/CAD PC. My initial attempt a few years ago faced challenges with virtualization latency and network bottlenecks, leading to suboptimal performance. Now I am giving it another chance, with recent advancements in Proxmox’s support for GPU pass-through and my plan is to give the system a 10Gig NIC and a dedicated 1TB SSD.

Note: Anticheat systems in online games do not play well with virtualized environments, potentially blocking access to certain titles.

The Goal

Have a dedicated, high-powered Windows VM that is always available for GPU-heavy workloads such as gaming and 3D design. The VM will have direct access to the graphics card and will be accessible from any device on the network, with a low-latency streaming client.

High-level steps:

• Create a Windows VM in Proxmox.
• Pass the entire GPU to the VM.
• Use a dummy HDMI plug to trick the GPU into thinking a screen is attached.
• Install Sunshine and Moonlight to stream to any client.

Prepare the Storage

I want to back up this VM without including all the non-relevant data. To achieve this, I will create the VM on one disk that contains the system and assign a second disk specifically for other content, such as games. This second disk will not be part of the backup since it is not critical. In case of recovery, I will restore the main VM and attach a new empty drive, then reinstall the games if necessary.

Enable Virtualisation Support

Before we begin, ensure all virtualisation options are enabled within the BIOS of the machine. For Intel CPUs, enable everything in the BIOS that is related to VT-x, and for AMD CPUs, look for the keywords SVM and AMD-V and enable all settings. Enable IOMMU, if applicable.

Install and Configure Proxmox

I found Nvidia drivers not to be reliable during the installation, so installed Proxmox with console nomodeset. After installation, access the node's shell.

Next, configure the Proxmox host to successfully pass through the PCIe device.

Update the GRUB bootloader:

nano /etc/default/grub

GRUB_CMDLINE_LINUX_DEFAULT="quiet iommu=pt initcall_blacklist=sysfb_init"

update-grub

Then, activate IOMMU for systemd:

nano /etc/kernel/cmdline

root=ZFS=rpool/ROOT/pve-1 boot=zfs quiet iommu=pt

pve-efiboot-tool refresh

Add VFIO modules:

echo "vfio" >> /etc/modules
echo "vfio_iommu_type1" >> /etc/modules
echo "vfio_pci" >> /etc/modules

update-initramfs -u -k all

Additional performance improvements:

echo "options kvm ignore_msrs=1 report_ignored_msrs=0" > /etc/modprobe.d/kvm.conf

Identify the GPU:

Find out the vendor and device IDs of the GPU in order to blacklist them from the host using them and later on pass them through to the VM.

lspci -nn | grep 'NVIDIA'

The output will look similar to this, indicating the IDs, in my case 10de:2204 and 10de:1aef, the first being the GPU, the second the audio controller.

43:00.0 VGA compatible controller [0300]: NVIDIA Corporation GA102 [GeForce RTX 3090] [10de:2204] (rev a1)
43:00.1 Audio device [0403]: NVIDIA Corporation GA102 High Definition Audio Controller [10de:1aef] (rev a1)

Take a note of both IDs as they are required in the next step.

Blacklist drivers:

To isolate the GPU from the host, the host must be told not to use the devices and drivers for the GPU:

echo "options vfio-pci ids=10de:2204,10de:1aef" > /etc/modprobe.d/vfio.conf

Then, blacklist the Nvidia drivers with:

echo "blacklist nouveau" >> /etc/modprobe.d/blacklist.conf
echo "blacklist nvidia" >> /etc/modprobe.d/blacklist.conf
echo "blacklist nvidiafb" >> /etc/modprobe.d/blacklist.conf
echo "blacklist nvidia_drm" >> /etc/modprobe.d/blacklist.conf

The configuration is now complete. Go ahead and reboot the host to apply the changes.

reboot

Verify it worked by running

dmesg | grep -E "DMAR|IOMMU"

dmesg | grep -i vfio

dmesg | grep 'remapping'

Build the VM

Next, build the Windows 11 virtual machine that the GPU will be assigned to.

I used the following settings:

• Memory: turn off ballooning
• Processors:
  • Type: host
  • Enable NUMA
• BIOS: OVMF (UEFI)
• Display: Default
• Machine: q35, kernel version 6.2
• Enable TPM, Enable EFI disk, Enable QEMU Quest Agent
• SCSI Controller: VirtIO SCSI Single
• Attach other (secondary, third) data drives if needed.
• In addition to the Windows 11 ISO, attach the VirtIO ISO to the VM as we need drivers from it in order to install the system.

Boot up the VM. During target disk selection, the VM will not see the hard drive attached without the drivers. Browse the VirtIO disk, navigate to Windows 11 and load the drivers. Install Windows 11 as usual.

After the system installation is complete, install the VirtIO drivers from the root of the VirtIO disk, then open the QEMU Guest folder and install the QEMU Guest Agent.

At this point, the GPU is not yet passed to the device, because passing the GPU to the VM will cause noVNC to stop working within Proxmox. So, the next step is to install your RDP server or enable RDP on the Windows 11 VM. I use MeshCentral, but even the built-in RDP will work as long as it is enabled and set to accept connections.

GPU Passthrough

Now that the system is installed, the last step is to pass through the GPU. To do this, shut down the virtual machine.

Connect the Headless HDMI / Dummy HDMI plug

Plug in a dummy HDMI plug to the GPU as by itself the GPU won't function. Alternatively, you can plug in a real monitor, but as my goal is to have a headless server, I went with a dummy HDMI to trick the GPU into thinking there is an actual monitor connected to it. I tried software alternatives to this, but ended up getting a dummy HDMI plug anyway, as it was more reliable and allows me to set the resolution easily.

Disable the default display

Navigate to the VM's settings in Proxmox > Hardware > Display and set Graphic card to none.

Connect the PCIe card

On the same page, click Add > PCI device > Raw Device and select the GPU's device ID you identified earlier above. Select "All Functions", "ROM-Bar", "PCI-Express" and "Primary GPU".

Drivers, Sunshine and Moonlight Setup

Next, power up the VM. Proxmox's built-in noVNC will no longer be able to connect to it as we are now using the GPU, so wait for the system to boot up and connect to it using your remote solution.

Once connected, install the latest Nvidia drivers,

As you may notice, navigating the VM through RDP is not the best experience, and it by no means feels like using bare metal. This is where Sunshine and Moonlight come in. Sunshine is an open-source, self-hosted implementation of Nvidia's GameStream protocol. It allows you to stream your desktop from a PC with an Nvidia GPU to a variety of devices, providing a smoother and more responsive experience than traditional remote desktop protocols.

To set up Sunshine:

1. Download Sunshine: You can find the latest release on the Sunshine GitHub page.
2. Install Sunshine: Follow the installation instructions provided in the README file of the Sunshine GitHub repository.
3. Configure Sunshine:
   • Open Sunshine and configure it according to your network setup.
   • Ensure your firewall settings allow traffic through the ports used by Sunshine (default is 47984).
   • Set up a secure password for accessing the Sunshine server.
4. Start Sunshine: Launch Sunshine and ensure it is running in the background. You might want to set it to start automatically with your system for convenience.

Now, to connect to your VM using Moonlight:

Moonlight is the client used on your client device to connect to the VM running Sunshine.

1. Download Moonlight: Moonlight is available on multiple platforms including Windows, macOS, Linux, Android, and iOS. Download and install Moonlight from Moonlight's official website.
2. Pair Moonlight with Sunshine:
   • Open Moonlight and it should automatically detect your Sunshine server if both devices are on the same network.
   • Select your server and follow the prompts to pair Moonlight with Sunshine. You will need to navigate to Sunshine's web portal to verify the PIN and allow connections to the VM.
3. Start Streaming:
   • Once paired, you can select your VM from the list of available devices in Moonlight.
   • Click on your VM to start streaming. You should now have a much smoother and more responsive experience, similar to using a bare metal PC.

Note: To achieve the best experience with the lowest latency, add a NIC to the server dedicated to this VM and use a wired network on the client to connect to it.

Conclusion

Today, we can experience high-performance gaming on any screen, without the need for a physically connected, high-end machine. It feels like having an invisible gaming rig that can be accessed from any display around the home, with zero cables.