lapis-sal-421M

The larger teacher checkpoint for bilingual tool routing, soft-label generation and further fine-tuning.

Give the router a message and a menu of tools. It returns the best matching tool ID, or abstains with None.

Quick start · Model family · Tools · Training · Evaluation · Limitations

Model family

Release	Format	Download	Best for
lapis-sal-150M	PyTorch FP32	~658 MB	Fine-tuning, research and conversion
lapis-sal-421M	PyTorch FP32	~1.69 GB	Teacher inference and soft-label generation
lapis-sal-150M-MLX	MLX FP16 / load-time INT8	~332 MB	Apple Silicon deployment

Download sizes include the tokenizer and runtime. Each repository contains one checkpoint or backend; you do not need the full family to run one model. The family links point to the released checkpoint and MLX repositories. The ONNX export is retained locally for evaluation and is not part of this Hub release.

Quick start

Use Python 3.12. Download this repository and run from the downloaded folder:

python -m pip install 'huggingface_hub==1.33.0'
hf download lapislabsai/lapis-sal-421M --local-dir lapis-sal-421M
cd lapis-sal-421M
python -m pip install -r requirements.txt

Run the following Python example from the downloaded folder. Replace the menu with the tools in your application:

from lapis_runtime import load_router

router = load_router('.', backend='torch', device='cpu', threads=4)
paths = {
    'WEATHER': ['Check weather conditions, forecasts and weather alerts'],
    'CALENDAR': ['Create, read or update calendar events and meetings'],
}
result = router.route('Will it rain in Madrid tomorrow?', paths)
print(result['match'], result['score'])

match contains your tool ID, or None for abstention. score is the selected route's calibrated score; probabilities contains the option scores, including the internal __none__ option. Evaluate calibration for your own menu and traffic.

For a CUDA deployment, pass device='cuda' to load_router. The checkpoint uses FP32 safetensors. This is a custom Laya DecisionModel; loading its encoder alone, AutoModelForSequenceClassification, or pipeline() will not reproduce routing. The bundled loader pins the validated Laya/PyTorch/Transformers stack. Accelerator performance beyond the reported measurements is not claimed.

Why Lapis SAL?

A bilingual assistant needs to decide which tool to use before it can act. Lapis SAL makes that decision a dedicated, local step: pass the user's message and the tools currently available, then receive a tool ID or an abstention.

The project pairs a larger teacher with a smaller distilled student so applications can choose between a research checkpoint and compact deployment exports. The aim is practical English/Spanish routing with explicit tool descriptions and a way to say that none of the tools fits.

What can you build?

• A personal assistant that routes requests to weather, calendars, reminders, email or search.
• An agent or conversational workflow that chooses among the tools available for a particular request.
• A local routing layer for desktop assistants, with CPU inference through ONNX or Apple Silicon inference through MLX.
• Teacher-based soft-label generation, student distillation or further task-specific fine-tuning with the PyTorch checkpoints.

The router selects a tool; your application supplies tool arguments, permissions and execution.

Tools and abstention

The adaptation and evaluation cover 13 tools plus NONE:

Tool	Intended route
CALCULATOR	Arithmetic and numerical calculations
CALENDAR	Calendar events, appointments and meetings
CLOCK	Time, timers, alarms and stopwatches
CODE	Programming questions and code assistance
CONTACTS	Contact lookup and contact management
HEALTH	Health, fitness and wellness information
LOCATION	Current location and location information
MAIL	Email search, reading and composition
MAPS	Maps, directions, places and navigation
MESSAGES	Instant messaging and text messages
REMINDERS	Reminders and personal tasks
WEATHER	Weather conditions, forecasts and alerts
WEB_SEARCH	Web information search
NONE	No appropriate tool or out-of-scope request

These are the evaluated tool families, rather than a fixed output vocabulary. You provide IDs, descriptions and optional example phrasings at runtime. The router also considers its internal out-of-scope option; new tools and descriptions need independent evaluation.

Architecture and size

Property	Value
Encoder	ModernBERT-large / Laya
Encoder layers / hidden size / attention heads	28 / 1,024 / 16
Intermediate size / vocabulary	2,624 / 50,368
Encoder parameters	394,781,696
Complete learned parameters	421,293,827
Temperature-buffer elements	3
Decision head	Two norm-first transformer layers, typed embeddings and per-option scorer
Input budget	512 tokens total; 384-token head budget
Release format	PyTorch FP32

The saved action head is retained for checkpoint compatibility; production tool routing reads the option logits. The teacher is named 421M from its measured complete parameter count.

Training

The Lapis adaptation workflow was configured in an environment with 2 × NVIDIA T4 GPUs, using FP16 mixed precision. The retained training implementation runs on a single device (world_size: 1); the release does not claim that both GPUs participated in distributed training.

1. Adapt the teacher: fine-tune the Laya / ModernBERT-large decision model on bilingual tool-routing episodes.
2. Create soft targets: run the teacher over training episodes and retain probabilities for the candidate tools and NONE.
3. Distill the student: train an Ettin-150M encoder with the Laya decision head, mixing teacher probabilities and gold labels with a teacher-target weight of 0.5.
4. Calibrate and export: fit temperature buckets on held-out training/development data, then export and check the student backends on development episodes.

Setting	Teacher	Student
Epochs	1	2
Micro batch / gradient accumulation	2 / 16	8 / 6
Encoder / head learning rate	2.5e-5 / 1e-4	5e-5 / 3e-4
Batch token budget	4,096	4,096
Prompt / total token limit / head budget	compact_fill / 512 / 384	Same
Teacher-target weight	Gold labels	0.5
Training seed	20260926	20260926

Dataset

The privately supplied Lapis bilingual tool-routing v4.1-refined source has 10,000 synthetic examples: 8,000 train, 1,000 validation and 1,000 final test. Deterministic preparation produces 100,000 training episodes and 5,000 development episodes, with seed 7, shuffled variable tool menus and nominal 5% gold-tool removal to exercise abstention. Only train/validation rows are expanded; the reserved test stays separate.

The source dataset and teacher soft labels are not redistributed. The author's private reproducibility repository retains preparation/training/export scripts, a dependency lock and source data. Original complete training logs and historical initialization revisions were not retained; exact reproduction of the original optimizer trajectory is not claimed. The released weights can be used for a new fine-tuning run, but original optimizer/scheduler state is not included.

Evaluation

The reserved final test contains 1,000 examples: 500 English and 500 Spanish. All 13 tools plus the internal NONE option are available, with shortlist disabled and threshold 0.625. Test examples and tool descriptions were preserved. Development episodes were used for numerical checks and latency measurements.

Backend	Correct / total	Accuracy	English	Spanish
PyTorch FP32	960 / 1000	96.00%	99.20%	92.80%

Macro F1 for PyTorch FP32: 96.48%. Class recall is the fraction of examples of that class routed correctly.

Per-class precision, recall and F1

Class	Test examples	Precision	Recall	F1
CALCULATOR	60	100.00%	100.00%	100.00%
CALENDAR	60	100.00%	90.00%	94.74%
CLOCK	60	100.00%	96.67%	98.31%
CODE	60	100.00%	100.00%	100.00%
CONTACTS	60	100.00%	95.00%	97.44%
HEALTH	60	100.00%	91.67%	95.65%
LOCATION	60	100.00%	100.00%	100.00%
MAIL	60	95.00%	95.00%	95.00%
MAPS	60	100.00%	98.33%	99.16%
MESSAGES	60	100.00%	91.67%	95.65%
NONE	220	88.00%	100.00%	93.62%
REMINDERS	60	88.33%	88.33%	88.33%
WEATHER	60	100.00%	86.67%	92.86%
WEB_SEARCH	60	100.00%	100.00%	100.00%

The historical 63.80% reference used different tool descriptions and is not a controlled same-prompt baseline. No causal improvement claim is based on that comparison. Detailed audit records and file checksums are retained in the private reproducibility repository. No raw evaluation messages are shipped.

Limitations

• Evaluation uses a small synthetic bilingual test. Its scores are not estimates for arbitrary real-world traffic; validate with your own held-out examples.
• Only English and Spanish were evaluated. Health or code route selection does not validate the correctness of health advice or generated code.
• Overlapping descriptions, implicit intent, long conversations and ambiguous messages can produce confident routing errors. Menus/history share a 512-token budget.
• Thresholds and scores depend on the menu. Keep threshold tuning and model selection separate from your final test.
• The model scores options; it does not generate text, extract tool arguments, execute actions or verify user authorization.
• Apple MLX measurements cover macOS Apple Silicon. iPhone/iOS, other languages and unreported accelerators have not been validated.
• This custom DecisionModel requires the included runtime. Standard Transformers pipelines and hosted inference providers are not advertised as supported.

License and acknowledgements

The release is distributed under Apache-2.0, retaining applicable upstream notices. Ettin's MIT notice is included under third_party/. The model license does not grant rights to the private source dataset.

Built on Laya DecisionModel, the Laya intent-router implementation, ModernBERT-large, and Ettin-encoder-150M. See LICENSE and NOTICE.md for attribution.

Keywords

semantic routing, tool routing, intent classification, dynamic tool selection, abstention, out-of-scope detection, bilingual English Spanish, knowledge distillation, local inference, ModernBERT, Ettin, Laya