https://stephenkrings.com/tags/gpu/ Recent content in Gpu on Stephen Krings Hugo en-us Tue, 14 Jul 2026 19:58:00 -0600 https://stephenkrings.com/posts/hugging-face-image-generation-rtx-5070-cpu-offload/ Tue, 14 Jul 2026 19:58:00 -0600 https://stephenkrings.com/posts/hugging-face-image-generation-rtx-5070-cpu-offload/ <p>Goal: wiring Hugging Face models into my local llm (Ollama) app, testing prompt-following, and seeing how far I can push a 12GB RTX 5070 before the system starts leaning hard on the CPU.</p> <p>The short version: it works, but only because the stack is allowed to spill parts of the model back to system memory.</p> <h2 id="the-current-setup">The current setup</h2> <p>The image UI lives on my web server, but the GPU does not. The request path looks like this:</p> https://stephenkrings.com/posts/rtx-5070-linux-gpu-server/ Tue, 14 Jul 2026 01:05:00 -0600 https://stephenkrings.com/posts/rtx-5070-linux-gpu-server/ <p>I do all my development on a Linux box with no GPU. My RTX 5070 lives across the room in a Windows gaming machine. Rather than dual-booting or doing native Windows dev (no), I turned the gaming box into a headless Linux GPU server I can SSH into, and it still plays games.</p> <h2 id="the-setup">The setup</h2> <p><img alt="Diagram: Linux dev box connecting over ssh to the Windows gaming PC on port 2345 via netsh portproxy, which relays into WSL2 Ubuntu 24.04 with full GPU passthrough" loading="lazy" src="https://stephenkrings.com/images/gpu-setup-diagram.png"></p>