Five-Lane Request Router (ModernBERT-large)

A fine-tuned ModernBERT-large classifier that decides what kind of answer a request needs, before any larger model runs:

chat · image · search · audio · video

It was trained to take over the routing step from a 9B generative model (an internal GPTQ build of a Qwen3.5-9B derivative on vLLM, thinking disabled) in a personal AI assistant, and runs as a TensorRT FP16 engine on an NVIDIA Jetson Orin Nano. As of release it is integrated but not yet serving live traffic; the 9B still routes production.

Results (one sealed blind test)

Both routers were run on the same 60 held-out cases, three times each (180 calls). The cases were written separately from the training data, sealed before training, leak-checked against it, and used once.

	9B generative router	this model
correct routes	175 / 180	180 / 180
chat messages sent to image/audio/video	0	0
image/audio/video/search requests sent to chat	2 / 120	0 / 120
empty requests ("make me one") sent to chat	9 / 9	9 / 9
p50 / p95 latency, client-side	771 / 1,269 ms	42 / 45.6 ms

Read these carefully. This is one sealed set of 60 cases, not a production soak. The model is deterministic, so the three repeats agree; the fair reading is 60 of 60 distinct cases. For intuition only: if 60 cases were independent random draws from real traffic, zero errors would still be consistent with an error rate of up to about 5%. These cases were written to probe the lane boundaries, not sampled from traffic, so that figure is an illustration, not an estimate of the live error rate. Latency is measured end to end from a client on the same network, including tokenization, for each router as deployed: this model as a TensorRT engine on a Jetson Orin Nano, the 9B on an Intel Arc GPU server. The hardware differs, so this compares routing calls in one setup, not model speed. The PyTorch weights here will run at different speeds on your hardware.

An earlier candidate trained on less data scored 174/180 on a different sealed set and failed: it sent one chat request that mentioned drawing to image, on all three repeats. This model was trained after 253 contrast rows were added for that failure shape, and was then tested once on a fresh sealed set.

Usage

from transformers import pipeline

router = pipeline("text-classification", model="ericmey/five-lane-router-modernbert-large", top_k=None)
router.tokenizer.truncation_side = "left"      # keep the END of long conversations

scores = router("user: can you draw me a lighthouse at dusk?", truncation=True, max_length=512)[0]  # all five lanes, best first
print(scores[0]["label"], scores[0]["score"])   # e.g. image 0.99...

Input format matters. The model was trained on the last up-to-8 conversation turns, one per line, formatted role: text for user and assistant turns only, with the request being routed on the last line:

user: what should i wear saturday?
assistant: Maybe the blue shirt you liked?
user: yeah i guess. show me

Truncate from the left at 512 tokens, or long conversations lose the request itself.

Training


base	`answerdotai/ModernBERT-large`
data	`ericmey/five-lane-router`: 2,334 train rows, 19 dev rows. The public files contain the same rows, in the same order, with identical `id`, `text` and `label` values as the files the model was trained on; internal audit columns were removed for release. Trained-file SHA-256: train `82ae7c96cc7b2717…`, dev `6eb8be247dd71f5a…`. Public-file SHA-256: `train.jsonl` `8b8d563e78f9abdb…`, `validation.jsonl` `314c49daa7e6dde3…`.
label order	`chat, image, search, audio, video`
max length	512 tokens, truncated from the left
epochs	3
batch	8, gradient accumulation 4 (effective 32)
learning rate	3e-5, AdamW (weight decay 0.01), 10% linear warmup then linear decay
loss	cross-entropy with inverse-square-root class weights, normalized to mean 1
precision	bf16 autocast on CUDA
seed	42
hardware	one NVIDIA RTX 5090

code/train_router.py is a portable version of the training script (CUDA, Apple MPS or CPU). code/train_modernbert.py is the exact script that produced these weights. code/evaluate_router.py runs a gated evaluation, and code/export_onnx.py exports to ONNX with a PyTorch-parity check. The full walkthrough is in the accompanying blog post: Train Your Own Request Router.

Limitations

Narrow by design. It picks a lane; it does not write a reply or a brief. In the intended deployment, a larger model still writes the short description that image/voice/video/search stages need, on those turns only.
One domain. Synthetic, English, informal requests to a companion-style assistant. Expect degradation on other registers. Fine-tune on your own lanes and data.
Lane boundaries are product decisions, encoded in the labels (see the dataset card). If your product treats "show me my cat" differently, this model will be wrong for you in exactly that place.
Fail safe. Wire it with a fallback: if the classifier errors, times out, or returns an unexpected label, route that turn the old way.