typical-medium

Typical is an open System-One decision model family, built by OzLabs.

Give it a state — a support ticket, a document, a form, a log line, any text — a question with its rubric, and a label set defined at runtime, and it returns a calibrated probability over those labels plus an explicit abstain option. One forward pass, no generated text to parse, about 57 ms per decision on an H100. Every decision is one of three typed primitives: Choice (categorical), Score (ordinal), Noul (yes/no, exactly order-invariant).

typical-medium is Release 1 at 4B (checkpoint tm1b): the same recipe as typical-small — dedicated Score and Noul heads, the hard curriculum, 1,024-token states — on a larger backbone. It is the knee of capability per millisecond in the release ladder: +11 points of JevBench standard and +11 points of MMLU-Pro among-K over typical-small, at 1.25x the per-decision latency.

Why it's different

• The decision is read directly from the model's own candidate-conditioned computation. The state and candidates are encoded once; the answer is read off that computation instead of being sampled one token at a time.
• The state is KV-cached, so asking many questions about one state is cheap — each additional question is a short suffix against a cache that's already built.
• Latency is flat up to ~64 options (57 ms at K = 2, 58 ms at K = 32); 96 ms at K = 256 — the state dominates until the rendered option list does.
• Abstention is a first-class output, not a parsed refusal: every decision carries a calibrated P(∅) alongside the label probabilities.
• Probabilities, not verbalized confidence. No "I'm 80% sure" string to parse — an actual calibrated number.

Model family

	typical-small-preview	typical-small	typical-medium
Backbone	Qwen3-1.7B-Base	Qwen3-1.7B-Base	Qwen3-4B-Base
Tap layer	20 / 28	20 / 28	26 / 36
Per-decision latency, K=2 (one H100)	45 ms	45 ms	57 ms
Peak memory, K=2	6.4 GB	6.4 GB	14.6 GB
JevBench standard / easy / hard	.750 / 1.00 / .387	.694 / 1.00 / .432	.806 / 1.00 / .423
MMLU-Pro among-K	.330	.343	.458
Held-out workflow noul / score	.699 / .498	.715 / .520	.811 / .528
Calibration (typed-decisions NLL)	2.06	1.25	1.18

Full ablations and matched-baseline tables: REPORT.md §3af.

Results

	typical-medium (tm1b)
SNLI / MNLI / BoolQ / ANLI	.909 / .861 / .843 / .544
CLINC-150 / TREC-fine / HWU64 / 20NG	.847 / .414 / .769 / .588
Held-out workflow: noul / score / style	.811 / .528 / .859
External floor: PagerDuty (floor .792)	.838
JevBench standard / easy / hard (Brier std / hard; ECE std)	.806 / 1.00 / .423 (.30 / .77; .09)

JevBench standard, same 231 public ids, next to the open field:

jeff (GLiFormer 400M)	Laya (ModernBERT)	typical-medium	open-alternative-jev (Qwen3.5-4B)	system-one-open (Gemma E2B LoRA)
.750	.694	.806	.833	.931

Public-subset per-item comparison on the 231 public ids; not a ranked leaderboard entry.

Quickstart

huggingface-cli download OzLabs/typical-medium --include "inference/*" --local-dir typical-medium
pip install -r typical-medium/inference/requirements.txt

import sys; sys.path.insert(0, "typical-medium/inference")
from typical import Typical

m = Typical.from_pretrained("OzLabs/typical-medium", device="auto")

ticket = ("Customer reports repeated 500 errors on checkout after the last deploy; "
          "asking for a refund.")
m.choice(ticket, "Which team should handle this?", ["billing", "infra", "support", "security"])
m.score(ticket, "How severe is this ticket?", ["1", "2", "3", "4"])
m.noul(ticket, "Does this ticket violate the refund-abuse policy (3+ refunds in 30 days)?")

Verified: this exact sequence (download, install, run) works on CPU (checked against typical-small's identical inference package; typical-medium uses the same code path on a larger backbone).

choice/score return {label: probability, ...} plus p_null; noul returns P(yes). decide mirrors the internal PCDMDecider.decide exactly, including the runtime block (latency, p_null, truncation flags) — see inference/README.md for the full API and inference/example.py for a runnable script.

Training recipe

• Data mix E .40 / K .15 / W .35 / U .10 of every batch (evidence, knowledge-MCQ, rubric- conditioned workflow, soft-target uncertainty), plus 15–20% null augmentation on W rows (--null_aug W:0.20) so the model learns to abstain, not just choose.
• 1,024-token decision states, so policy documents and multi-hop states are in-distribution.
• Typed heads: Score is K-way Choice trained with ordinal-smoothed targets (τ = 0.7, zero head code); Noul is a dedicated Bernoulli head, routed per row — only rows whose candidates are exactly {yes, no} use it, every other 2-way set scores as ordinary Choice.
• 12,000 steps, effective batch 64 (bs 64, grad_accum 4), LoRA rank 16 on the top 8 layers of a 26-of-36-layer truncated backbone.
• Held-out by construction: one rubric style and three whole domains of the hard curriculum (data_wh), plus temporal/numeric/probability/trade-off composition (level 7), are eval-only — never trained on.

Data sources and licenses:

Source	Role	License
SNLI, MultiNLI, ANLI, BoolQ, MMLU-aux, ARC, OpenBookQA, CommonsenseQA, SciQ, QASC, LogiQA2, AQuA-RAT, MedMCQA	evidence + knowledge-MCQ	permissive research licenses (CC-BY-SA / MIT / Apache-2.0 family)
cua-ai/cua-s1-forms	workflow (GUI-form synth)	MIT
dwidlee/systemone-lite-general	workflow (rule-labeled typed decisions)	MIT
MagaBitmex/jev-4b-distill-data	workflow (programmatic gold)	Apache-2.0
in-repo rubric-conditioned generator (data_wf, data_wf_long)	workflow	generated, no external license
in-repo rule-engine generator (data_wh, DecisionMix v2)	hard curriculum	generated, no external license
Zhengping/UNLI (validation split)	soft-target uncertainty	MIT
metaeval/ambient (ambiguous rows, trained on)	soft-target uncertainty	license undeclared on its HF card — flagged, not asserted
metaeval/chaos-mnli-ambiguity (eval-only, never trained on)	soft-target uncertainty	license undeclared on its HF card — flagged, not asserted

Evaluation-only, never trained on: JevBench (public subset), typed-decisions, PagerDuty, tree-choice, jevlogs (Loghub research license, caveated), Mind2Web.

Limitations

• Hard tier is .423 (chance .336, SE ≈ .045 at n = 111) — the least reliable tier; the open field's best is .73. The long-policy slice specifically is .21 — the weakest single family.
• TREC-fine regresses 10 points vs. ladder_4b, the same backbone trained without typed heads or DecisionMix v2 (K = 50 fine-grained topics) — the one clear regression at this size.
• Held-out rubric styles and Noul accuracy give back a few points vs. that same untyped 4B baseline, in exchange for the calibration and external-floor gains above.
• Inference code is included in this repo; training code release to follow.

License

Apache-2.0 (backbone Qwen/Qwen3-4B-Base is Apache-2.0; this release's weights are a LoRA adapter plus small Choice/Score/Noul heads over that backbone). Training-data licenses above; JevBench numbers are a public-subset run against fstandhartinger/jevbench v1.2.1, not a submitted or ranked leaderboard entry.

Files in this repository

• best.pt — this checkpoint (tm1b).
• results.json / eval_wf_full.json / probe_results.json / jevbench_summary.json — full eval suite, post-hoc held-out/external eval, probe results, JevBench run summary.
• inference/ — the self-contained Typical inference package (see Quickstart above).
• MANIFEST.json — checksum + provenance record for this release.

Lineage

typical-small-preview (Release 0) → typical-small (Release 1 at 1.7B) → typical-medium (Release 1 at 4B, this repo).

Coming: typical-large — a 14B ladder checkpoint scores JevBench .875/.468 (standard/hard), but probability calibration at that size is not yet fixed (§3ab). Trained, not released.

Typical is built by OzLabs.