Tiny-Jev-1.7B

A 1.7B "System One" decision model: structured state in, typed probabilistic decisions out.

Not affiliated with TypeSafe AI. Tiny-Jev is an independent, open-weights model inspired by the System One idea and the Jev decision API published by TypeSafe AI. It is not built, endorsed, or supported by them, shares no code or weights with their models, and "Jev" in the name refers only to the interface style it reproduces (Choice / Score / Noul with calibrated probabilities).

Tiny-Jev-1.7B is the larger sibling of Tiny-Jev (0.6B). It does not generate text. It reads a state (a message, a ticket, a JSON record, a transcript, a log, a diff), a question written the way a developer would write it in code, and a fixed set of options — and returns a probability distribution over those options in a single forward pass, with a confidence you can gate on.

primitive	returns	example
Choice	the best option + a distribution over all of them	Which team should handle this? → billing / technical / sales
Score	a position on an ordered scale (expected value, can land between levels)	How frustrated is the customer? → calm … hostile
Noul	a probability that a statement about the state is true	The customer explicitly requests a refund → 0.97

Use it as the smart if in a pipeline: routing, triage, filtering before an expensive context window, guard-railing another model's output, moderation, tagging at volume, reading logs / schedules / access rules / inventory, and sub-100 ms decisions in a request handler.

Which one to pick. The 1.7B model is more accurate everywhere and much better at reading structured state (logs, schedules, access rules, board positions); the 0.6B model is about twice as fast and fits anywhere. On a consumer GPU the 1.7B model answers in ~10–40 ms per question; on an Apple M-series laptop ~60–150 ms.

Quick start

from transformers import AutoModel, AutoTokenizer

tok = AutoTokenizer.from_pretrained("lostargon/Tiny-Jev-1.7B")
model = AutoModel.from_pretrained("lostargon/Tiny-Jev-1.7B", trust_remote_code=True).eval()   # .to("cuda") / .to("mps")

state = {"message": "My card was charged twice and nobody answers the phone.", "plan": "pro"}

model.choice(tok, state, "Which team should handle this",
             {"billing": "Payment or subscription issues", "technical": "Bugs or integration problems", "sales": "Plans and pricing"})
# {'choice': 'billing', 'probabilities': {...}, 'confidence': 0.99}

model.noul(tok, state, "The customer explicitly requests a refund")
model.score(tok, state, "How frustrated is the customer", ["calm", "annoyed", "angry"])

# Fan-out: several questions over one state in a single batched call
model.decide(tok, state, [
    {"kind": "choice", "instructions": "Which team should handle this", "criteria": ["billing", "technical", "sales"]},
    {"kind": "noul",   "instructions": "The customer is reporting a bug"},
    {"kind": "score",  "instructions": "Urgency", "criteria": ["low", "normal", "high", "urgent"]},
])

One safetensors file holds the decoder stack and the decision head; trust_remote_code=True loads the small modeling_tiny_jev.py shipped in this repo (only torch and transformers needed).

How it works

Each option is rendered as its own line after the state and the question, followed by a marker token. The hidden state at every marker goes through a linear head to a scalar; a softmax over the option scalars is the answer. Noul is a two-option Choice (no / yes); Score is a Choice over ordered levels whose expected value is reported. Training uses soft cross-entropy against probability-vector targets, followed by temperature calibration on a held-out split. The answer space is the option list you pass in, so the output can never be malformed.

What it was trained on

Fine-tuned from Qwen3-1.7B (LoRA, merged into the weights) on synthetic datasets of typed decisions — about 160 000 training questions over states whose labels come from explicit rules or are computed by code:

• Support and operations — ticket routing, refund / cancellation / escalation intents, severity and churn, email triage with multi-ask messages, IT helpdesk priority and assignment, invoice and security-incident decisions, workflow policies applied to facts.
• Guard-railing models and agents — grounding, policy compliance and failure type of an assistant's answer; safety, reversibility and secret exposure of an agent's proposed shell / file / API action.
• Business text — moderation, reviews, product listings, contract clauses against a playbook, résumé screening against a job posting, fraud decisions on transactions, search intent, code-review verdicts.
• Voice and dialogue — turn-taking, addressee, voicemail detection, answer type and next action from noisy ASR transcripts.
• Structured reading (procedurally generated, exact labels) — log analysis, calendars across time zones, inventory, access control with roles and wildcards, state machines with guards, dependency graphs, rankings with tie rules, format validation (checksums, dates), pricing with discounts, tax and currency; plus board-game positions.
• Robustness sets — contrastive negation and scoping, prompt-injection attempts embedded in the state, and non-English states with English questions.

All hand-written data was audited by blind re-labelling; every procedural generator was reviewed for label bugs and position leaks before training.

Evaluation

Accuracy on held-out items; ECE is expected calibration error (lower is better). acc@0.9 is accuracy on answers given with confidence ≥ 0.9 and cov@0.9 the share of items that clear that bar.

Held-out splits of the training distribution

split	items	accuracy	ECE	acc@0.9	cov@0.9
test (same task families)	22 110	97.3	0.003	99.8	93 %
OOD (held-out template and dialogue families)	48 021	94.7	0.033	96.8	94 %
structured reading, test	10 101	86.0	0.007	98.7	62 %
hand-written business domains, test	179	73.7	0.141	85.5	61 %
two fully held-out domains (code review, search intent)	830	60.5	0.209	73.8	50 %

Public classification benchmarks

Item lists follow the open open-system-one protocol (2 500 items per dataset, fixed seed). The model was additionally fitted on 2 000 items per dataset that are disjoint from the evaluation items; the reference column is a zero-shot decision API, so read it as "1.7B + a small fit set" versus "frontier API, no fit set".

dataset	options	accuracy	ECE	acc@0.9	cov@0.9	reference (zero-shot)
SST-2	2	92.7	0.019	97.1	82 %	91.6
AG News	4	90.6	0.038	97.2	61 %	88.6
Emotion	6	82.0	0.013	96.6	48 %	59.0
BANKING77	77	83.5	0.057	99.1	47 %	77.8
CLINC150 (+ out-of-scope)†	151	66.5	0.030	96.3	29 %	—

† not in the fit set: transfer from the other intent tasks only.

Zero-shot multiple choice (no training on these tasks)

500 items per task, options passed as a Choice question.

task	Tiny-Jev-1.7B	Tiny-Jev 0.6B
MMLU	47.4	37.4
WinoGrande	56.6	52.6
ARC-Easy	90.2	72.8
ARC-Challenge	73.8	51.2
Chess — the one legal move among four (synthetic positions)	50.2	38.8
GSM8K — correct answer among 4 numeric options	54.4	41.0
GSM8K — among 10 numeric options	33.0	23.0

These measure general knowledge and multi-step arithmetic, which a model this size only partly has; they are shown to mark where its limits are, not as its purpose.

Limitations

• Not a text generator. It cannot write, summarise, or explain its answer. Extract candidates with code or another model; use Tiny-Jev to pick.
• Reads questions literally. Negations and scoping words are taken at face value — phrase questions the way you would phrase a unit test.
• Arithmetic is weak. Multi-step word problems stay near 50 % with four options; compute exact things in code and ask the model about the result.
• New rule systems. Domains whose rules it has never seen are handled at ~60 %, with confidence that correctly drops; add a few hundred examples of your own to close the gap.
• Unarmoured state. Content inside the state can try to argue for its own label. The model was trained against this, but treat user-controlled text with the usual care.
• Context up to 4 096 tokens per question; the state is truncated (head and tail kept) when it does not fit alongside the options.

Model details

Base	Qwen3-1.7B (decoder stack; the language-model head is not used)
Added	one marker token, a 1-dimensional decision head, temperature
Parameters	1.72 B
Precision	bfloat16 weights
Training	LoRA r=32 on all linear layers + head, soft cross-entropy, one epoch, post-hoc temperature calibration
License	Apache-2.0

If you use Tiny-Jev, a link back is appreciated. Issues and results on your own data are welcome in the community tab.