Local AI runtime recommendations — platform: windows
Beginner pick: LM Studio. Best GUI on Windows, auto-detects CUDA/Vulkan backends.
Power-user pick: Ollama (CUDA). Scriptable server; CUDA path is fastest on NVIDIA.
Power-user command: ollama run llama3.1:8b
Runtimes:
name: LM Studio
what: GUI, CUDA/Vulkan/ROCm
status: stable
name: Ollama
what: CLI/server
status: stable
name: llama.cpp
what: Reference engine
status: stable
Gotcha: AMD GPUs run via Vulkan/ROCm at roughly half CUDA throughput. NVIDIA is the smooth path on Windows.
Ceiling: 12GB VRAM runs up to ~14B at Q4; 24GB runs 32B at Q4.