Making Hugging Face Image Generation Fit on a 12GB RTX 5070
Goal: wiring Hugging Face models into my local llm (Ollama) app, testing prompt-following, and seeing how far I can push a 12GB RTX 5070 before the system starts leaning hard on the CPU. The short version: it works, but only because the stack is allowed to spill parts of the model back to system memory. The current setup The image UI lives on my web server, but the GPU does not. The request path looks like this: ...