I do all my development on a Linux box with no GPU. My RTX 5070 lives across the room in a Windows gaming machine. Rather than dual-booting or doing native Windows dev (no), I turned the gaming box into a headless Linux GPU server I can SSH into, and it still plays games.

The setup

Diagram: Linux dev box connecting over ssh to the Windows gaming PC on port 2345 via netsh portproxy, which relays into WSL2 Ubuntu 24.04 with full GPU passthrough

WSL2 gives the GPU to Linux via NVIDIA’s driver passthrough. nvidia-smi works inside Ubuntu with zero driver installs on the Linux side. The trick is making it reachable and keeping it alive:

  1. sshd inside WSL on port 2345, key auth only.
  2. netsh interface portproxy on Windows relays 2345 from the LAN into the WSL VM’s NAT IP, plus a firewall rule.
  3. An ssh gpu alias in ~/.ssh/config on the dev box. That’s the whole interface.

One idempotent PowerShell script does all of it, and re-running it after a reboot repoints the portproxy at WSL’s new NAT IP.

The gotchas that actually cost me time

  • WSL idle shutdown is the killer. The VM stops ~60 seconds after the last wsl.exe exits, taking sshd with it, causing endless mysterious “connection reset”. vmIdleTimeout=-1 in .wslconfig did not work for me. The fix: a scheduled task that runs wsl -- sleep infinity at logon. Crude, bulletproof.
  • Ubuntu 24.04 ignores Port 2345 in sshd_config because of systemd socket activation. systemctl disable --now ssh.socket && systemctl enable --now ssh.service.
  • Don’t try networkingMode=mirrored if you run Docker/Podman on the Windows side. It nuked WSL networking entirely for me. Plain NAT + portproxy is boring and works.
  • nvidia-smi “not found” over non-interactive SSH. WSL’s /usr/lib/wsl/lib isn’t in PATH for non-login shells. Use the full path.
  • Virtualization off in BIOS (HCS_E_HYPERV_NOT_INSTALLED). On AMD boards that’s SVM Mode, buried under Tweaker → Advanced CPU Settings.

What it’s for

Blackwell cards need CUDA 12.8+, so PyTorch has to be the cu128 build. With that pinned, the workflow from the Linux box is just:

scp -r screenshots gpu:~/
ssh gpu
cd ~/ksp_clip && source .venv/bin/activate
python index.py ~/screenshots
python search.py "rocket on launchpad at night"

That’s CLIP-based free-text search over my own images, running on the 5070, driven entirely from the machine I actually work on. When I’m done, the gaming box is still just a gaming box. Nothing about this setup gets in the way of that.

Best of both worlds: Windows keeps the games, Linux gets the CUDA.