slime2-cascade-router

Weights for the cascade-routing setup in Jianshu-She/llamacpp-routing — a small model generates locally, a trained gate hands individual steps to a larger model over an OpenAI-style API. Full explanation, options and results: that repo's tools/server/README-routing.md.

Files

• slime2-f16.gguf — the small model (F16 GGUF).
• slime2-chatml.jinja — its chat template (llama.cpp's own minja parser rejects the model's bundled template; pass this one explicitly with --chat-template-file).
• router/router.rml2 — the trained gate, one general-purpose gate covering five domains.
• router/router.json — the gate's metadata: which model it was trained for (SHA256 fingerprint), validated default thresholds, per-domain AUC.
• router/router.ref — reference (input, p_route) pairs for checking a C++ port against the original PyTorch module.

Use

hf download Jianshu001/slime2-cascade-router --local-dir .
llama-server -m slime2-f16.gguf -ngl 99 -c 262144 --parallel 8 \
  --chat-template-file slime2-chatml.jinja \
  --router router/router.rml2 --router-upstream http://<large-model-host>:8000 \
  --router-threshold 0.7 --router-gate 0.8