javimosch/mtlm-7m-router2s384

mtlm-7m-router

A 7M-parameter Llama-architecture model trained in pure machin/MFL (see mtlm) — a tiny dispatcher/router: it classifies each request into a tool call, a specialist-assistant handoff, an escalation to a bigger model, or plain chat. Sibling of javimosch/mtlm-7m-tools with the same architecture and tool vocabulary, plus an escalate route.

Routes it can emit

calculator, get_weather, get_time, web_search, wikipedia, read_file, write_file, http_get, send_email, translate, run_shell, convert_units, set_reminder, save_note, and escalate — emitted for requests beyond a dispatcher's pay grade (long-form writing, code, document analysis, planning, professional advice).

Held-out eval (140 probes, v4 spec)

name_acc 1.000 · call_ok 0.957 · value_acc 0.968 · escalate dispatch 5/5

Typed decision heads (Jev-style primitives)

The same frozen trunk also serves typed decisions without generation: three tiny linear heads (mhd1 format, 50 KB each) read the hidden state at the last prompt position in a single forward pass (15 ms on CPU). Served by anvil via ANVIL_HEAD / ANVIL_NOUL / ANVIL_SCORE:

endpoint	head	returns	holdout
POST /v1/decide {state, options?}	m7router1.head — 16 routes	{choice, confidence, probabilities}	99.1% (800 stem-diverse probes), 94.9% on handwritten natural probes, ECE 0.017
POST /v1/noul {state}	m7router1.noul.head — yes/no	{answer, probability} (fixed question: "should this escalate?")	99.6%
POST /v1/score {state}	m7router1.score.head — ordinal 1..4	{grade, confidence, probabilities} (fixed rubric: request complexity)	97.4%, adjacent-tier errors only
POST /v1/assess {state}	all three heads, one forward pass	{decision, noul, score}	multi-question per state, ~15ms

options on /v1/decide restricts the route set and renormalizes — a constrained choice is representable by construction, malformed output is not. Confidence is temperature-scaled softmax (T fitted on held-out). Heads are swappable per deployment: a customer's route table is a ~50 KB artifact trained from a synthetic spec, not a model fine-tune.

Recommended dispatch pattern (reference: tools/router_demo.py --decide in mtlm): decide → escalate on low confidence, explicit escalate, noul p(yes) ≥ 0.5, or head-vs-generation route disagreement; only agreed tool routes execute.

Generation path

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("javimosch/mtlm-7m-router")
model = AutoModelForCausalLM.from_pretrained("javimosch/mtlm-7m-router")
ids = tok.apply_chat_template([
    {"role": "system", "content": "You are a helpful assistant. You can call tools."},
    {"role": "user", "content": "write a 500-word essay about the French Revolution"},
], add_generation_prompt=True, return_tensors="pt", return_dict=True)
out = model.generate(**ids, max_new_tokens=80, do_sample=False)
print(tok.decode(out[0][ids["input_ids"].shape[1]:]))
# {"tool_call": {"name": "escalate", "arguments": {"request": "...", "reason": "long-form writing"}}}

Architecture

Llama-compatible: dim 288, 6 layers, 6 heads (head_dim 48), SwiGLU hidden 768, vocab 4096, context 256, RMSNorm eps 1e-5, RoPE theta 10000, tied embeddings. ~8.3M params, fp32 safetensors (33 MB). Designed to sit in front of a router shim that maps tool_call names to executors — see tools/router_demo.py in the mtlm repo for a reference dispatcher.

Limits

7M params: argument values can be sloppy (digit bleed on long numbers, occasional operator slips). Validate arguments platform-side before executing. Not a chat model — chat answers are TinyStories-grade.