Local AI runtime recommendations — platform: mac
Beginner pick: LM Studio. Polished GUI, ships MLX on Apple Silicon, one-click model downloads.
Power-user pick: mlx-lm. Apple's MLX framework, usually the fastest on Apple Silicon for the same quant.
Power-user command: pip install mlx-lm && mlx_lm.generate --model mlx-community/Llama-3.1-8B-Instruct-4bit
Runtimes:
name: MLX / mlx-lm
what: Apple-native inference, usually fastest on M-series
status: stable
name: LM Studio
what: GUI app, MLX + GGUF backends
status: stable
name: Ollama
what: CLI/server, simple ollama run UX
status: stable
name: llama.cpp (Metal)
what: Reference cross-platform engine
status: stable
name: Rapid-MLX
what: MLX inference server with an OpenAI-compatible API, prompt cache and tool calling; brew or pip install
status: stable
Gotcha: vLLM is NOT a Mac tool, it is a CUDA/Linux serving engine. Unified memory is not a fixed VRAM slice; ~70% is usable for weights.
Ceiling: M-series with 16GB runs up to ~8B comfortably; 64GB+ runs 70B at Q4.