mvbalaji/od1-nano-v2

OD-1 Nano v2 (0.8B)

OD-1 is an open System One decision model: give it a state (text or JSON) and typed questions (choice, noul = yes/no, score = ordinal) and it returns every answer with full probability distributions (plus an explicit NOT_ANSWERABLE outcome), in the Jev / TypeSafe /v1/systemone shape.

Qwen3.5-0.8B with OD-1 typed heads, early exits and a contrastive shortlist head, retrained with the zero-shot fix (fewer synthetic NOT_ANSWERABLE golds, teacher-verified structured JSON data, checkpoints selected on an out-of-domain validation set) and a fitted NOT_ANSWERABLE threshold.

When to use it

Fast general decisions over text or JSON, zero-shot. Replaces od1-nano (v1) for anything unfamiliar.

Results on held-out test sets

Same fixed samples for every system (≤ 2,000 decisions per set, seed 0; typed-decisions: all 2,000 test decisions). Baselines answer each question as one choice over its options (2–24 options; n/a = more options than their interface allows). Jev 1.13 = jev-1.13.0, measured 2026-09-25. Accuracy:

test set	this model	Jev 1.13	Laya	Laya typed-decisions	Tev1-4B	CLM-8B
typed-decisions	0.558 (n=2000)	0.741	0.353	0.737	0.690	0.393
AG News	0.761 (n=2000)	0.882	0.924	0.922	0.886	0.354
Emotion	0.554 (n=2000)	0.595	0.597	0.603	0.583	0.281
SST-5	0.406 (n=2000)	0.579	0.341	0.463	0.533	0.273
Banking77	0.501 (n=2000)	n/a	n/a	n/a	n/a	n/a
MASSIVE-en	0.633 (n=2000)	n/a	n/a	n/a	n/a	n/a
BFCL native	0.935 (n=1252)	0.975	0.679	0.847	0.954	0.694
BFCL 24 options	0.928 (n=1909)	0.969	0.600	0.741	0.950	0.625
BFCL 100 options	0.835 (n=1909)	n/a	n/a	n/a	n/a	n/a
BFCL 1,000 options	0.425 (n=1909)	n/a	n/a	n/a	n/a	n/a
BFCL irrelevance	0.583 (n=1101)	0.702	0.788	0.390	0.701	0.661
Adversarial (ours)	0.825 (n=2000)	0.832	0.619	0.628	0.793	0.449

On the typed-decisions test split, scored with the benchmark card's formulas (KL from the soft gold and Brier verified against the card's reference rows): accuracy 0.558, KL 0.545, Brier 0.259 (generalist, zero-shot). For reference the card lists TypeSafe Jev 1.13 at 0.727 / 1.442 / 0.148 and meraGPT Decider 1 at 0.768 / 0.096 / 0.052 (both generalists).

Speed (one H100, bf16, batch-1 requests, CUDA graphs)

state / questions	this model (p50 ms)	Laya typed-decisions, all questions in one call (p50 ms)
short (≤64 tokens) / 1	5.8	15.9
short (≤64 tokens) / 5	11.2	17.9
short (≤64 tokens) / 20	34.1	21.4
medium (200–400 tokens) / 1	8.6	16.9
medium (200–400 tokens) / 5	24.9	17.7
medium (200–400 tokens) / 20	106.8	41.4

Limitations

• On unfamiliar tasks accuracy is well below the best closed systems (see the table); NOT_ANSWERABLE is only returned above the fitted threshold in serving.json.

• English only. Baselines were called through a choice-wrapped interface (Laya's typed-decisions checkpoint also natively), which may understate them. No frontier-LLM reference was run.

• Scores on typed-decisions measure agreement with its labelling teacher (see the dataset card).

Usage

pip install torch "transformers>=5.17.0" huggingface_hub

from huggingface_hub import snapshot_download
path = snapshot_download('mvbalaji/od1-nano-v2')
import sys; sys.path.insert(0, path)
from od1.model import OD1Model
m = OD1Model.load(path, dtype='bf16', device='cuda')   # applies the fitted serving.json
m.enable_cuda_graphs()
answers = m.answer(state, {'route': {'type': 'choice', 'instructions': 'Which team?', 'criteria': {'billing': 'payments', 'tech': 'bugs'}}})

Smoke test of example.py for this checkpoint: {"route": {"answer": "billing", "confidence": 0.988}, "refund": {"answer": 0.994507760126595, "confidence": 0.994}, "urgency": {"answer": 3, "confidence": 0.524}}

Requirements and speed

The Qwen3.5 backbone mixes full attention with gated delta-rule (linear attention) layers. Install the fast kernels, otherwise transformers silently falls back to a slow pure-PyTorch implementation of those layers and a one-question request takes hundreds of milliseconds instead of a few:

pip install "transformers>=5.17.0" flash-linear-attention==0.5.2
pip install causal-conv1d==1.7.0 --no-build-isolation   # compiles against your torch/CUDA (needs nvcc)

Check: import fla, causal_conv1d must succeed; when they are missing, transformers logs "falling back to its reference PyTorch implementation". Then enable CUDA graphs and warm up before timing:

m.enable_cuda_graphs()
for _ in range(5):                       # warm-up: kernel autotuning + graph capture per input shape
    m.answer(state, questions)
torch.cuda.synchronize(); t0 = time.perf_counter()
answers = m.answer(state, questions)
torch.cuda.synchronize(); print((time.perf_counter() - t0) * 1000, "ms")

Tested with torch 2.14.0+cu130, transformers 5.17.0, flash-linear-attention 0.5.2, causal-conv1d 1.7.0 on one NVIDIA H100 80GB HBM3 (bf16). Expected on that setup for a short one-question request: about 35 ms without CUDA graphs and about 6 ms with them (see the speed table); other GPUs differ. CPU inference works but is slow.

Training data and licenses

Openly licensed datasets (see sources.csv: dataset, revision, license, split, counts), project-generated synthetic policy checks and teacher-verified structured JSON workflows (written by Qwen3.5-27B-FP8, which is never a compared system, without access to any benchmark data), with teacher distillation labels. Held-out evaluation sets were never trained on (13-gram deduplication against every test set).

Citation

Paper in preparation (OpenDecide-1).