switchboard-router-4b-GGUF

The 4.28 GB build of switchboardllm/switchboard-router-4b: a 4B router that reads a prompt and answers with one line of JSON, the cheapest model tier that can handle it. One Q8_0 file for Ollama and llama.cpp on Windows, macOS and Linux. No account or API key needed.

Want something lighter? switchboard-router-0.5b-GGUF is 994 MB and runs on any laptop CPU. For a server with a 48 GB GPU, switchboard-router-32b is the most careful size.

Run it

ollama run hf.co/switchboardllm/switchboard-router-4b-GGUF "Write a haiku about rain"

{"tier":"small","domain":"writing"}

Or download the files and use the included Modelfile:

hf download switchboardllm/switchboard-router-4b-GGUF --local-dir switchboard-4b
cd switchboard-4b
ollama create switchboard-4b -f Modelfile
ollama run switchboard-4b "Prove there are infinitely many primes of the form 4k+3"

With llama.cpp, as an OpenAI-compatible server:

llama-server -hf switchboardllm/switchboard-router-4b-GGUF --jinja --temp 0 -n 24 --port 8080

What it answers

Field	Values
tier	small, mid or frontier: the cheapest class of model expected to answer the prompt well
domain	code, math, writing or general: a hint, from keyword labels

If the output isn't valid JSON, send the prompt to frontier. A failed routing call should cost money, never quality.

Use it in front of your LLM calls

pip install switchboard-router

from switchboard_router import Router

router = Router(backend="ollama", backend_model="hf.co/switchboardllm/switchboard-router-4b-GGUF")
router.pick_model(prompt)   # falls back to frontier if anything goes wrong

The file

File	switchboard-router-4b-Q8_0.gguf, 4.28 GB
Quantization	Q8_0 (8-bit)
Full precision size	8.04 GB in BF16
Memory	about 5 GB of RAM or VRAM
Speed	about 190 ms per routing decision on an RTX 5070 Ti with Ollama. Expect seconds per decision on a CPU.

Results

Does this file route like the full model? Measured on this exact file with Ollama at temperature 0, against the BF16 model, on 347 held-out prompts (2026-10-13):

	Same decision as BF16
300 random held-out prompts	293 of 300
47 borderline prompts (picked to cover all three tiers)	46 of 47
Same domain	100.0%
Valid JSON	347 of 347

The disagreements are borderline prompts. The BF16 model itself changes 5 of 347 decisions between batched and one-at-a-time inference. We also tested Q4_K_M (2.50 GB): only 302 of 347 matched, so it is not in this repo.

How good is the routing? BF16 Switchboard 4B on 10,000 held-out prompts:

	4B	0.5B (v0.1)
Cost saved vs. always using Claude Opus 5.5	67.0%	64.5%
Hard prompts sent to frontier	63.2%	59.1%
Sent too low (needed frontier, sent cheaper)	5.0%	5.5%
Invalid JSON	0	0

Method, confidence intervals and training details are in the full model card.

Limitations

• It still misroutes some prompts: 5.0% of held-out prompts were sent to a cheaper tier than they needed.
• Labels come from 2023 models and a GPT-4 judge, not human preferences. Mostly English.
• It reads one prompt at a time, not the whole conversation.
• domain comes from keyword labels and is only a hint.
• Switchboard is an independent project and is not affiliated with Anthropic, Alibaba, Robinhood, Surplus or the Switchboard oracle network.