simplixity-router-2b

Qwen3.5-2B fine-tuned as the router of Simplixity, a self-hosted research assistant. The router reads a question, in English or German and with the chat so far, and fills Simplixity's Classification schema as JSON. It decides:

• whether to search, and with which queries
• how much research the question needs
• the shape of the answer
• how recent the sources must be
• which sources to search

This is v1, distilled from Claude Haiku 5.5 with thinking turned on. It is meant to run locally as Q4_K_M, from simplixity-router-2b-GGUF.

Use with Ollama

On Linux and Windows, Ollama cannot import safetensors. Download the GGUF file from simplixity-router-2b-GGUF and follow the instructions there.

On macOS, from a download of this repo:

ollama create simplixity-router:2b-v1-q4_k_m -f Modelfile --quantize q4_K_M

In Simplixity, choose simplixity-router:2b-v1-q4_k_m as the router model with provider Ollama.

Also set a title model. Without one, Simplixity names new chats with the router, and this model was not trained to write titles. simplixity-title-0.8b is made for that job.

PARAMETER num_ctx 8192 in the Modelfile matters. Simplixity sets no context length, and a router prompt with a chat history runs past Ollama's default.

Prompt

The model was trained on Simplixity's own classifier prompt, built by the app's code (app/agent/prompts.classify), and should get exactly that prompt. Simplixity asks for strict JSON with the hardened Classification schema, at most 500 tokens, with thinking off.

• training/eval/system_prompt.txt holds one example of the system prompt.
• training/eval/schema.json holds the schema.

The prompt varies with the date, the depth (2 or 3 search queries), the sources (Auto, where the router picks them, or a fixed selection), a library catalog, attached images, a linked page and the chat history. Training covered all of these.

Results

Testset

105 hand-labelled questions (52 English, 53 German), run in Ollama 0.40 with temperature 0 and num_gpu 99. "Route" means that search yes or no, complexity and answer shape are all right.

	Route	Complexity	Shape	Recency	Sources	Median latency (RTX 3070 Ti)
Qwen3.5-2B Q4_K_M, untuned	38%	71%	59%	91%	81%	1.24 s
simplixity-router-2b v1 Q4_K_M	96%	98%	98%	98%	97%	0.53 s
Claude Haiku 5.5, thinking	99%	99%	100%	99%	100%	3.0 s (OpenRouter)

v1 always writes valid JSON with the allowed number of queries. The untuned model wrote facet names into answer_shape and set recency to "year" almost everywhere. v1 does neither.

Agreement with the teacher

Held-out questions, measured as route agreement with Haiku. To show how much of the gap is noise, Haiku labelled the same questions a second time, and that run is compared with the first.

Dev set	Haiku vs Haiku	v1 vs Haiku
Synthetic, 150 questions	91%	66%
Real questions from the author's own instance, 69	79%	75%

Almost every disagreement is between neighbouring levels:

• complexity: standard vs research, simple vs standard
• shape: no shape vs explainer, steps or profile

v2_plan.md describes the next run, which goes after this gap.

Limits

• Gray-zone questions. On questions where Haiku itself is unsure, v1 often decides differently.
• Repetition loops. On 1 of 219 dev questions, v1 fell into a repetition loop and Ollama aborted the answer with "token repeat limit reached". Simplixity treats a failed classification as "search the question as it is", so the run still goes on.
• No titles. The model writes the router's JSON only, not titles.

Training

• Data: 2,803 synthetic questions for training and 150 for validation, half German and half English.
  • Haiku wrote them from 37 categories, in different writing styles and with follow-ups, images, linked pages and library chats.
  • Questions too close to the testset were dropped (difflib ratio above 0.85).
  • Each question was turned into the exact prompt the app would send, with sampled settings, and labelled by Haiku 5.5 with a 1,024-token thinking budget.
  • Labels that break a rule of the app or the prompt were dropped, not repaired: 23 in all.
  • Fields the app ignores in a given case were emptied: place outside weather questions, lenses without Auto sources and image_query without a wanted image.
  • Everything is in training/data/ and was built by the scripts in training/, in this order: gen_questions.py, build_prompts.py, label.py, make_dataset.py.
• Not published: 145 real questions from the author's own Simplixity instance, shown twice each, were also part of the training. They and the 69 real dev questions stay private. training/extract_real.py shows how they were taken out of the database.
• Method:
  • QLoRA: the base model in NF4 with double quantization and bf16 compute, and fp32 adapters.
  • LoRA rank 16, alpha 320 and dropout 0, on the 186 linear layers of the language model.
  • Loss on the JSON answer only.
  • Batch 1 with gradient accumulation 8, constant learning rate 2e-5.
  • The vision tower was taken off the GPU during training, because the router never sees an image.
  • Code: training/train.py and training/run_train.sh.
  • This model is the checkpoint at step 350 (0.9 epochs), where the eval loss was lowest. After that the run overfit: by step 700 it rated simple questions as standard more often.
• Export: the LoRA deltas were added to the original bf16 weights, so the vision tower and the mtp.* tensors are unchanged (training/export_ckpt.py). GGUF: llama.cpp's convert_hf_to_gguf.py, then llama-quantize to Q4_K_M (training/to_ollama.sh).
• Evaluation:
  • Testset and baselines: training/eval/make_testset.py, eval.py and report.py.
  • Agreement with the teacher: eval_dev.py.
  • Stored results: results-*.jsonl and dev-syn-*.jsonl.
  • The second Haiku labelling: training/data/relabel_syn_labels.jsonl.

Licence

Apache 2.0, like the base model.