Otron/snap-1

Snap 1

Fast, local, typed decisions. Give Snap a state (text or JSON) and a few typed questions — yes/no, pick-one, or a level — and it answers all of them in a single forward pass, with probabilities. It never generates text, so there is nothing to parse and nothing to hallucinate. It runs on your machine: no API key, no network, nothing leaves the device.

Otron/snap-1 · 421M parameters · English · up to 4,096 tokens per pass

Quick start

# pip install "laya==0.3.4" torch transformers safetensors huggingface_hub
# then download snap.py from this repo
from snap import Snap

snap = Snap("Otron/snap-1")
snap.decide(
    state={"message": "my card got charged twice for the same order"},
    questions={
        "topic": {"type": "choice", "instructions": "What is the message about?",
                  "criteria": {"billing": "Payments, charges, refunds",
                               "shipping": "Delivery and tracking",
                               "other": "Anything else"}},
        "urgent": {"type": "noul", "instructions": "The customer needs a reply today"},
        "mood": {"type": "score", "instructions": "How upset is the customer?",
                 "criteria": ["calm", "annoyed", "angry", "furious"]},
    },
)
# {"topic":  {"choice": "billing", "confidence": 0.997, "probabilities": {...}},
#  "urgent": {"noul": 0.42, "confidence": 0.58},
#  "mood":   {"score": 2, "confidence": 0.41, "probabilities": {"0": 0.05, "1": 0.38, "2": 0.41, "3": 0.15}}}

Question types:

type	answers	criteria
noul	probability the statement in instructions is true	—
choice	one key	dict of key → description (or a list of keys)
score	one ordered level (0-based)	list of level descriptions, lowest first

Descriptions matter: Snap reads every option's text, so options it has never seen work as well as familiar ones. Option lists can be long (hundreds of options in one question).

Core ML (macOS 14+)

coreml/Snap.mlpackage is the same model converted for Core ML (fp16 weights, 819 MB). It takes one sequence (input_ids, tok_type) and returns a score per token; coreml/snap.json holds the token ids, limits and confidence temperatures needed to build the input and read the answers, and coreml/tokenizer.json is the tokenizer. The input format is exactly the one snap.py builds (Snap._block / decide). Run it on .cpuAndGPU: the Neural Engine recompiles for every new input length.

Apps that download the model on demand can use coreml/manifest.json: every Core ML file with its size and SHA-256, at URLs pinned to the commit they were published in.

Speed

Measured on an Apple M5 (32 GB), one decision at a time, model already loaded:

	PyTorch (MPS, fp16)	Core ML (GPU, fp16)
short decision — one sentence, 3 questions (109 tokens)	25 ms	16 ms
long decision — a spoken request routed over ~100 options plus context (1.6k tokens median, up to 3.7k)	214 ms median · 545 ms p90	191 ms median · 557 ms p90
load	18.9 s	5.8 s first time (compiles), then faster

Cost grows with the total length of the state and the options, not with the number of questions: every question about the same state shares one pass when they fit together.

Results

Accuracy against each dataset's own labels, on 500 examples per dataset from the official evaluation split (never trained on). Questions are phrased as plain typed decisions with the labels' names and descriptions as options; no task-specific heads or prompts.

dataset (split)	majority class	Laya (base)	Snap 1
Banking77 (test) — 77 intents	0.028	0.464	0.812
CLINC150 plus (test) — 151 intents incl. out-of-scope	0.196	0.514	0.838
ContractNLI (test) — whole NDAs	0.488	0.488	0.712
MNLI (validation matched)	0.356	0.872	0.870
ANLI (test r1–r3)	0.366	0.506	0.520
BoolQ (validation) — accuracy · AUROC	0.626	0.850 · 0.913	0.860 · 0.933
Civil Comments (test) — AUROC	0.5	0.944	0.923

Laya is scored on the same examples in its own input format, with its 512-token limit and with Snap's 4,096; the better of the two is shown (with its own 192-token option budget it cannot fit Banking77's or CLINC150's option lists at all). Snap's gains are where the fine-tuning aimed — many-option routing and long documents; it is level on reasoning-style NLI and slightly behind the base model at ranking toxicity.

Civil Comments is reported as AUROC: its label (≥50% of raters said toxic) is narrower than a plain "is this rude or disrespectful" question, so raw accuracy mostly measures the cut-off.

Unseen tools. On held-out synthetic decisions for 300 invented tools (spoken requests routed to each tool's own actions, none seen in training), Snap picks the reference action 69.8% of the time, with expected calibration error 0.025.

Calibration. Confidences are temperature-calibrated per question type and option count; across all held-out decisions the expected calibration error is 0.026.

JevBench (public half)

JevBench is an independent benchmark for typed-decision models. We ran Snap 1 on its 231 public items with the benchmark's own harness and scorer (commit 609a664). The Laya and Jev columns are the per-item results JevBench publishes for the same items (release v1.3.0). This is a self-run of the public half, not an official board entry: the board also scores sealed items, calibration, speed and cost. Snap 1 was not trained on any JevBench item.

tier (items)	Laya (base)	Snap 1	Jev 1.13
easy (48)	0.958	1.000	1.000
standard (72)	0.694	0.806	0.986
hard (111)	0.351	0.342	0.730
all (231)	0.584	0.623	0.866
median latency per decision	—	24 ms on-device (M5)	665 ms over the network

The hard tier (long insurance policies, contracts and rule sets with multi-step reasoning, written by frontier models) is where a 421M encoder falls behind; none of those items exceed Snap's 4,096- token window. Jev's latency is as JevBench measured it, over the network; Snap's runs locally.

How it was made

• Base: Laya by ConvAI Innovations, a typed- decision model on ModernBERT-large.
• Training: full-parameter fine-tuning (all weights) on the data below.
• Data: training splits of Banking77, CLINC150, MNLI, ANLI, BoolQ, Civil Comments, ContractNLI and QuALITY, plus ~30k synthetic decisions for 300 invented tools generated with GPT-6 Luna, and a small amount of private data that is not released. Examples were labelled with TypeSafe's Jev, combined with each dataset's own labels where they exist.
• Layout: Snap packs all questions about one state into a single sequence (each question's options are [MASK] markers that the model scores), instead of one sequence per question. This is what makes many questions cost one pass. extra.safetensors holds the extra token-type embedding this layout uses; snap.py implements it.

Limitations

• Snap learned from model-generated labels and inherits their mistakes. It is weakest on hard multi-step reasoning (ANLI; JevBench's hard tier) and less sure of itself on tools it has never seen.
• English only, and 4,096 tokens per pass (state + questions); longer states are truncated.
• Text in the state can steer the answer (prompt injection). Treat decisions on untrusted input as advice, and keep a human or a hard rule in the loop for anything consequential.
• Not for safety-critical, medical, legal or financial decisions.

License and attribution

Apache 2.0 (see LICENSE and NOTICE). Built on Laya (Apache 2.0) and ModernBERT (Apache 2.0). Trained on public datasets under their own licenses: ANLI is CC BY-NC 4.0 and BoolQ CC BY-SA 3.0; the datasets themselves are not redistributed here.

Not affiliated with or endorsed by TypeSafe or ConvAI Innovations.