aisquared/bolt-cls-router

Bolt CLS Router

Bolt CLS Router is a sequence classifier based on the ModernBERT-Large architecture, specifically tuned for routing prompts to the appropriate downstream handlers.

Model Overview

Bolt CLS Router is trained specifically to predict which downstream handler pipeline a given prompt should be routed to. It is a multi-class routing decision rather than a binary safety flag.

Key design goals:

• High precision across imbalanced routing classes
• Long-context support so routing decisions can account for full prompts
• A direct top-label routing decision, with no threshold tuning step

Model Details

• Type: Text classifier (multi-class, single-label)
• Architecture: ModernBERT-Large (ModernBertForSequenceClassification)
• Base model: modernbert-large
• Task: Text Classification / Routing
• Max input length: 8,192 tokens
• Loss function: Weighted Focal Loss (to handle class imbalance)

Training Data

Trained on aisquared routing dataset of roughly 100k examples built specifically to teach the model position-invariant routing — i.e. to key off route semantics rather than memorizing which option letter or slot a route tends to appear in.

How it's built:

1. Source rows come from an internal routing dataset.
2. Merged with LLM-generated synthetic rows — an LLM is prompted once per route to write realistic user prompts that would trigger that route, without ever naming the route, specifically to backfill routes the source dataset under-covers.
3. Each source row is expanded into multiple candidate-conditioned variants (default 8 per row): a shuffled subset of 2–10 of the 18 catalog routes is shown as options, one of 5 routing modes (quality, cost_optimized, low_latency, balanced, description_based) determines the target label among "acceptable" candidates, and the prompt phrasing/option-labeling style (letters, numbered letters, or random tokens) is randomized. This shuffling and re-labeling is exactly what makes the dataset — and the resulting model — position-invariant: the model can't learn to associate a route with a fixed slot, letter, or token, since those are re-randomized on every row.
4. Rows are stratified-split (default 10% test) and pushed as JSONL/CSV in both a classification-shaped and pairwise-shaped format.

Key features: rendered prompt (conversation + routing-mode instructions + candidate options) → label (the index of the correct candidate).

The dataset totals ~100k examples across the 18 routes. Exact per-route label distribution and the synthetic/source split ratio are not published in this card.

Preprocessing: Tokenized using the ModernBERT tokenizer with truncation at 8k tokens.

Training

Trained with AI Squared's internal classifier training pipeline, shared across all four Bolt CLS classifiers.

• Learning rate: 2e-5
• Epochs: 4.0
• Weight decay: 0.01
• Scheduler: Cosine
• Warmup ratio: 0.06
• Loss function: Weighted Focal Loss (γ = 2.0 default) — for this multiclass router, weighting is inverse-class-frequency only (the extra positive-class multiplier the binary sieve classifiers get doesn't apply here, since there's no single "positive" class in a routing task).
• Effective batch size: per-device batch size 1 × 32 gradient-accumulation steps = 32 (default; long 8k-token sequences don't fit multiple per batch)
• Max sequence length / seed: 8,192 tokens / seed 42 (defaults)

Unlike the binary sieve classifiers, there's no post-training threshold search here — label selection is direct argmax over the candidate options (see Threshold Guidance below).

Hardware: Nvidia H200

Evaluation

Numbers below are from an internal routing evaluation benchmark plus a synthetic unseen-route stress test:

Metric	Value
Strict accuracy (matched the single gold route)	89.8%
Lenient accuracy (matched any route in the acceptable-alternatives set)	94.2%
Probe strict accuracy (hand-written sanity cases)	81.1%
Probe lenient accuracy	86.8%

Intended Use

• Primary use case: Routing incoming user queries to specific model pipelines.
• Input: Raw text prompts.
• Output: A class label representing the optimal route.

Deployment Notes (Hugging Face Inference Endpoints)

If you're deploying this model on HF Inference Endpoints:

1. The two frontmatter lines above are required. Without pipeline_tag and library_name set, HF cannot identify this repo's task/architecture, which blocks every specialized engine (vLLM, TEI, TGI, SGLang) from even appearing as selectable options — you'll only see "Default," along with a "no handler.py found" warning. Adding these two lines is what unlocks the full engine list on the deployment screen.
2. Manually select vLLM as the Inference Engine on the deployment screen — it is not selected by default, and the default recommendation may pick a different engine that doesn't correctly support this architecture's classification head. This has to be set explicitly every time you create a new endpoint; it is not remembered from prior deployments and cannot currently be pinned via any repo file.
3. No handler.py or requirements.txt is needed once vLLM is selected — vLLM reads config.json directly.

By design: id2label / label2id in config.json are left as generic placeholders (LABEL_0...LABEL_9) rather than hardcoded to specific route names. This is intentional, not an outstanding gap — it keeps the router configurable, so each deployment can map the numeric labels to its own set of downstream routes instead of being locked to a fixed route list baked into the card.

Usage

from transformers import pipeline

router = pipeline("text-classification", model="aisquared/bolt-cls-route-mb-large-v3")
result = router("Your prompt here")
print(result)

Limitations

• Not threshold-based, so the guardrails available on the binary sieve classifiers (e.g. tuning a low sieve threshold for high recall) don't apply here — a misrouted prompt goes straight to the wrong handler.
• id2label/label2id are left as generic placeholders by design (see Deployment Notes) — deployers must map them to their own route set; don't assume the numeric labels correspond to any fixed, universal route naming.

License

Bolt CLS Router is released under the AI Squared Community License.