Yev-Decision-1

A Qwen3.5-4B decision model, trained without Jev.

A local System One decision model. Give it a state, which can be text, images or both, plus named questions. It returns a typed probability distribution for each one: a Choice between options, a Score on an ordered scale, or a Noul (yes or no). It generates no text and samples nothing, and nothing leaves the machine. The call shape mirrors TypeSafe's SDK for its hosted jev-1.13.0, but no Jev output was used anywhere in its training or calibration: its labels come from people, from code, and from open-weight teachers under MIT or Apache-2.0.

This repository is a yev export, not a standalone transformers model. It holds:

• LoRA adapters, rank 32;
• the decision heads and the three temperatures;
• the tokenizer and processor files;
• metadata.json, which records provenance;
• sources.json, every training source with its licence and where that licence was checked.

The backbone, Qwen/Qwen3.5-4B-Base, is downloaded at a pinned revision on first load. It runs with the yev package.

This is the first release (2026-10-21).

Use

uv add "yev[inference] @ git+https://huggingface.co/HallD/yev"

from yev import Yev, Choice, Image, Noul, Score

with Yev("HallD/Yev-Decision-1") as client:
    response = client.system_one(
        state="I was charged twice. Please fix this ASAP.",
        questions={
            "billing": Noul(instructions="Is this ticket about billing?"),
            "tone": Choice(instructions="What is the customer's tone?",
                           criteria={"calm": None, "frustrated": None, "angry": None}),
            "urgency": Score(instructions="How urgent is this ticket?",
                             criteria=["can wait", "this week", "today"]),
        },
    )
    chart = client.system_one(
        state=["Quarterly sales:", Image("chart.png")],
        questions={"growing": Noul(instructions="Do sales rise every quarter?")},
    )

Each request's questions are scored independently, as separate sequences.

• A state and its question read up to 4,096 tokens, and each picture is scaled to at most 256 image tokens.
• For longer text, windows="mean" scores overlapping windows and averages them.
• windows="max" (Noul only) answers from the window most likely to be Yes, such as the email in an inbox that holds an instruction.
• On a CUDA GPU, quantize4bit=True loads the backbone in NF4. On an earlier build it cost 0.8 points on jev-bench, mostly MMLU and BoolQ, and about a point on VizWiz and the people-labelled pictures; it has not been measured on this one.

Results

Clean teacher-check bundle: 5,382 held-out questions with no Jev answers, each model using the temperatures it ships with. yev 4B is HallD/yev-qwen3.5-4b-v2, the same backbone trained in part on Jev's labels.

	This model	yev 4B
CFPB complaints (3,000): product area against people's labels	89.2%	87.8%
jev-bench (12 human-labelled datasets, 2,382 questions): accuracy	83.5%	84.2%
jev-bench: expected calibration error	0.028	0.071
jev-bench: TVD to annotators' vote shares	0.097	0.085

With temperatures refitted the same way for both, jev-bench calibration error is 0.016 against 0.048, and TVD to annotators 0.105 against 0.074.

Strongest	Weakest
SMS spam 97.3%, FEVER 95.5%, civil comments 94.5%, StrategyQA with evidence 93.5%, BoolQ 92.5%, MNLI 92.0%	HelpSteer2 helpfulness 44.5%, StrategyQA closed-book 70.0%, HelpSteer2 verbosity 70.0%, MMLU 71.5%

Image questions: dev pictures from the training sets' own dev splits. Accuracy is the share of questions where the people's most common answer comes top.

	This model	yev 4B
CLEVR (1,000)	98.3%	97.7%
TextVQA (1,000)	84.0%	83.5%
VizWiz (956)	81.4%	77.5%
Pictures of text labelled by people (515): picture / same question as text	89.3% / 89.7%	90.3% / 90.9%
Calibration error on those pictures	0.023	0.016

Definitions: 228 questions whose answer follows from a definition the question gives, so the right answer is known. They cover thresholds ("a parcel is a heavy parcel if it weighs more than 25 kg"), lists of conditions, and features under a definition that can run against the usual meaning, asked mirrored, in the question, in the true description only, and inverted. Flips ask about the same state under two thresholds whose answers differ; both must be right.

	This model	yev 4B
All 228: accuracy	0.95	0.93
Threshold flips (20 pairs): both right	1.00	0.90
Thresholds	1.00	0.97
Features	0.83	0.85
Conditions	1.00	1.00
Inverted questions	0.96	1.00

Consistency: the average violation over about 150 dev questions per test, where 0 is perfectly consistent.

• Option order: reordering a Choice's options.
• Binary: the same yes/no question asked as a Noul and as a two-option Choice.
• Irrelevant option: dropping one option.
• Definition swap: a Noul asked with its two descriptions exchanged, under neutral instructions.

	This model	yev 4B
Option order (TVD)	0.000	0.000
Binary: yes/no against two-option Choice (P(yes) gap)	0.024	0.023
Irrelevant option dropped (TVD)	0.023	0.021
Definition swap (P(true) + P(true after swap), gap from 1)	0.043	0.066

Prompt-injection screening and speed have not been measured for this model.

How it was trained

The model is a frozen Qwen3.5-4B-Base with its language-model head removed. On top sit:

• LoRA on the attention and Gated DeltaNet projections;
• a bilinear head over pooled spans, which scores Choice options in sorted order, so their order cannot change the answer;
• a readout of the last token of the question and of each option, added to the pooled spans, which tells apart options that differ by a word ("is a taco" against "is not a taco").

It was trained from the base model with RLCD's objective (Plackett–Luce plus a Brier term):

Run	Data	Updates
Stage A	2.25 million questions from licensed human-labelled datasets, code-labelled definitions and open-teacher logits	71,002
Vision	CLEVR, TextVQA and VizWiz pictures, rendered pictures of text and a text replay (137,072 questions); blended 70/30 with Stage A	6,984
Hard	31,784 questions, about half of them long-document, multi-hop, numeric and judgement questions written and answered by Qwen3.8-27B and DeepSeek-V4.1-Flash, the rest replayed from Stage A's sources; blended 70/30 with its start	1,111
Kev	684,459 answer distributions from Kev-27B plus 768,333 questions replayed from Stage A's data	43,513
Sharp	35,821 questions with one certain answer: both hard sets' questions (the second written and answered by Qwen3.8-27B, with answer-judging questions on GSM8K and ARC-Challenge), each target its checked answer, computed by code or agreed by two blind solves, and the replayed people- and code-labelled questions that have one answer. It starts from 40% of the Kev run blended with 60% of its start	2,194

This export is an exact blend: 50% of the Sharp run's last update and 50% of its start. Of blends of 50%, 70% and 100%, it stayed best calibrated across the teacher-check bundle and the pictures.

Temperatures were fitted on 1,066 dev questions whose answer was checked rather than voted on: the hard sets' questions and the code-labelled definitions. Choice 1.091, Noul 1.414, Score 1.414. Earlier builds fitted them to annotators' vote shares, which left the model less sure than it is right on questions that have one answer; the Sharp run is there for the same reason, since Kev-27B's answer distributions taught it to hedge.

Data and licences

Every source is in sources.json. The rules for what it could train on:

• Licences that allow commercial use: CC0, public domain, MIT, Apache-2.0, BSD, AFL, ODC-BY, CC BY and CC BY-SA.
• Labels from people, from code, or from open-weight teachers under MIT or Apache-2.0. Never from Jev or from a closed model.
• Left out: non-commercial, research-only or unlicensed sets, platform terms (Twitter, Yelp, Yahoo, Quora, Amazon reviews), GPL, mixed sets that cannot be split, unverified sets, and COCO images.

The teachers:

• Qwen/Qwen3.5-35B-A3B-Base (Apache-2.0): logits on question-bank questions;
• Qwen/Qwen3.8-27B (Apache-2.0) and deepseek-ai/DeepSeek-V4.1-Flash (MIT): the hard questions, among them Qwen3.8-27B's answers to GSM8K (MIT) and ARC-Challenge (CC BY-SA 4.0) training questions, graded by code against the datasets' own answers;
• jaredpalmer/kev-27b at af0e6d5 (Apache-2.0): answer distributions over four option orders. Its card says no Jev output was used in its training, and that its consumer-complaint labels were filtered by closed-model judges.

Some source texts were written by models and labelled by people under their datasets' licences: WANLI (GPT-3, CC BY 4.0), hh-rlhf (MIT) and HelpSteer2 and 3 (CC BY 4.0). The pictures are CLEVR (CC BY 4.0), TextVQA (CC BY 4.0, with Open Images pictures listed as CC BY 2.0) and VizWiz (CC BY 4.0). The base model is Apache-2.0.

The weights are released under Apache-2.0. The CC BY and CC BY-SA sources ask for attribution, which sources.json gives. Check each source's terms before redistributing any of the data.

Limitations

• Annotator agreement: its distributions sit 0.097 TVD from annotators' vote shares on jev-bench, against 0.085 for yev 4B (and 0.098 for an earlier build without the Kev and Sharp runs).
• MMLU is 71.5%, against 74.5% for that earlier build.
• Definitions that run against a word's usual meaning are followed 83% of the time, against 85% for yev 4B.
• Fine-grained ratings are weak: HelpSteer2 helpfulness 44.5%, verbosity 70.0%.
• Image accuracy is measured on these multiple-choice questions, not with each benchmark's official metric.
• Prompt injection: it isn't a prompt-injection guard, and this model hasn't been measured on those sets.
• Validate on your own questions before relying on a threshold.