Verdin-4B

Verdin-4B is a System One decision model. You give it a state (text or JSON) and a set of typed questions — choice (pick one of named options), noul (yes / no), score (an ordered scale). It returns a probability for every option of every question. It does not generate text and there is nothing to parse. All questions about one state share a single prefill pass.

• Model: Qwen/Qwen3.5-4B (instruct, revision 851bf6e8) + a LoRA fine-tune (r = 16, merged) + a pointer head that reads the option-boundary tokens. 4.21 B parameters served, bf16. The 8 full-attention layers read the state bidirectionally within blocks of 8,192 tokens (trained that way; see Training recipe).
• Engine: Verdin, our inference engine for this model family, shipped in this repo (Apache-2.0). It runs on NVIDIA GPUs (CUDA, bf16), on the CPU and on Apple silicon (MLX), all with the state-bidirectional attention of this checkpoint (on an M5 Max, MLX picks the same option as the CUDA engine on 99.74 % of 8,129 evaluation questions; see verdin/README-mac.md). It speaks the TypeSafe-style POST /v1/systemone wire format.
• License: weights CC-BY-NC-4.0 (non-commercial; see License); Verdin code Apache-2.0.

Results

Decision Index 0.3, public index (all 37 panel benchmarks, chance-corrected, coverage-adjusted; self-run with the official kit on the hash-verified 0.3 suite: 140,620 requests, all answered, none refused; the kit scores 140,178 of them):

	Decision Index 0.3 public	raw	breadth
Verdin-4B	58.29	68.86	55.64

area	skill (0–100)	raw accuracy
Knowledge & Reasoning	35.3	49.9
Language Understanding	66.1	76.2
Retrieval & Classification	64.3	73.6
Tools & Automation	85.6	87.4
Arts & Human Taste	35.5	55.5

The previous revision of this repo (f35e9c18, the same recipe and data with causal attention) scored 56.64 / 67.64 / 53.81; see Revisions.

The board's headline Full score is 20 % this public index and 80 % private tests that only the maintainers run, so the public number above is not the Full score. Several public panel benchmarks have public train splits that we trained on (declared below, with the overlap checks we ran). Expect the public index to overstate performance on new domains.

Other evaluations (accuracy on held-out sets; the same weights run through the kev reference implementation, bf16 backbone):

suite	accuracy
JevBench public (223 requests)	0.821
typed-decisions test (2,000; its train split was not used)	0.706
kev decision-v7 test (1,264)	0.863
kev hard-v1 development	0.729
kev documents-v1 development	0.828
kev devtools-v1 development	0.727

Quickstart

hf download victorbennett/Verdin-4B --local-dir Verdin-4B && cd Verdin-4B
pip install -r requirements-cuda.txt        # NVIDIA;  Apple silicon: requirements-mac.txt and --backend mlx
python -m verdin.server --model . --port 8000

curl -s localhost:8000/v1/systemone -H 'content-type: application/json' -d '{
  "state": {"ticket": "Charged twice for order 1182, please fix today"},
  "questions": {
    "team":   {"type": "choice", "instructions": "Route the ticket.", "criteria": {"billing": "charges, refunds", "tech": null}},
    "urgent": {"type": "noul",   "instructions": "Is it urgent?"},
    "anger":  {"type": "score",  "instructions": "How angry is the customer?", "criteria": ["calm", "annoyed", "furious"]}}}'

from verdin import Verdin
v = Verdin.from_pretrained(".")          # CUDA if available, else CPU
v.decide("The mug arrived broken.", {"damaged": {"type": "noul", "instructions": "Was anything damaged?"}})

Requests are never truncated: a state over 65,536 tokens (or a request over the device's memory) is refused with 422 too many tokens: .... More in verdin/README.md and, for Macs, verdin/README-mac.md.

Training recipe

The training code is not released; this is the full recipe.

• Framework: kev (Apache-2.0) with our patches. Each question is one causal row [state] [question] [options] [decide] on top of the shared state. A pointer head (dim 256) scores each option's end token against the decide token. Rendering is schema-first: a short block listing every question and its options (≤ 512 tokens) comes before the state, so the state is read knowing what will be asked. Date facts: when the state holds two or more absolute dates, a date_facts field states the day difference of each pair ("August 3, 2026 is 12 days after July 22, 2026.").
• State-bidirectional attention: in the 8 full-attention layers, a state token at position i attends to a state token at position j when j ≤ i or both lie in the same block of 8,192 positions (i // 8192 = j // 8192). A state of up to 8,192 tokens (schema block included) is therefore read fully bidirectionally; longer states are bidirectional within each block and causal across blocks. Question, option and decide tokens stay causal, and so do the 24 Gated DeltaNet layers. The model was trained with this mask from the first step; it is not a causal model run with a different mask.
• Adapter: LoRA r = 16, alpha = 32, on every linear projection of the backbone (attention q/k/v/o, Gated DeltaNet in_proj_qkv / in_proj_z / in_proj_a / in_proj_b / out_proj, MLP gate/up/down). The backbone is frozen in bf16. The LoRA and the head are trained.
• Optimisation: lr 5e-5, 2 epochs, batch 1 request × gradient accumulation 8 = 24,804 optimizer steps, seed 0 (data-parallel over 4 GPUs; the 8 requests of a step are split across them). Token caps during training: state 14,336 / branch 14,848 / packed 18,432 (no request was truncated or dropped). No rationale loss.
• Calibration: one temperature (T = 2.462) fitted after training on the calibration split of kev's decision-v7 set.
• Compute: 4× RTX 3090 (24 GB), 48.3 h, peak 20.9 GiB per GPU. 239 M forward tokens.
• Export: LoRA merged in fp32, then cast to bf16 (max |Δp| 0.0011 vs the unmerged fp32 model on probe requests, 0 argmax flips).

Training data

Verdin-4B was trained for 2 epochs on one mix of 99,213 records holding 270,905 questions. Each record is a state (text or structured data) plus one or more questions about it. By type, there are 216,931 choice questions (pick one option), 36,856 noul questions (true / false) and 17,118 score questions (a point on an ordinal scale). The checks we ran against Decision Index 0.3 and JevBench are described under Decontamination.

Sources

The mix has 12 families and 162 source tags. Where an upstream source has train / dev / test splits, we used train and dev data only. The only exceptions are listed under "Removed before training".

#	family	records	questions	upstream (revision)	licence	how the labels were made
F1	Decision Index skill data I: tool calls, contract NLI, RAG hallucination, sarcasm, entity sentiment	20,480	25,740	liminghao1630/API-Bank@12e8158b, Team-ACE/ToolACE@6bda777c, nvidia/When2Call@0582f774, stanfordnlp/contract-nli@eced6528, ParticleMedia/RAGTruth@c103204b, iabufarha/iSarcasmEval@dfc708b5 (En, Ar), pyRis/SEntFiN@eba43610	MIT; Apache-2.0 (ToolACE); CC-BY-4.0 (When2Call, ContractNLI)	Gold labels from each dataset, put into the Decision Index request format by code. No model involved.
F2	Decision Index skill data II: stance, intent, product search, commonsense, aspect sentiment, captions, phishing	18,166	124,317	emilyallaway/zero-shot-stance@e7c47751 (VAST), clinc/oos-eval@828f8093, amazon-science/esci-data@7916cdf6, Rowan/hellaswag@218ec52e, allenai/winogrande@01e74176, NUSTM/ACOS@45d179a3, jmhessel/newyorker_caption_contest@d81cbab7, Nazario phishing corpus (monkey.org, listing of 2026-02-20), SpamAssassin public corpus (20030228)	CC-BY-4.0 (New Yorker, Nazario); CC-BY-3.0 (CLINC150); Apache-2.0 (ESCI); MIT (HellaSwag); CC-BY with no version (WinoGrande); not stated (VAST, ACOS, SpamAssassin)	Gold labels. For phishing, the class is the corpus an email came from, and two signal questions are computed by code. No model involved. ACOS alone gives 101,523 questions.
F3	Decision Index skill data III: claim verification, clinical-trial NLI, maths, an appliance simulator, code execution	10,398	38,316	hover-nlp/hover@39b84697 (+ HoVer Wikipedia DB), ai-systems/Task-2-SemEval-2024@7f32fa6c (NLI4CT), openai/gsm8k@740312ad; our code generators for an appliance simulator and CRUXEval-style programs (no data from either benchmark)	CC-BY-SA-4.0 (HoVer); MIT (GSM8K); not stated (NLI4CT, SemEval-2024 Task 2 terms); ours (generated)	Gold labels for HoVer, NLI4CT and GSM8K. GSM8K distractors come from the benchmark's own answer-group code. Generated items are labelled by code: the simulator is re-run, or the program is executed in a sandbox. No model involved.
F4	Classic classification sets (1,000 train items each)	10,000	14,522	fancyzhx/ag_news@eb185aad, SetFit/amazon_reviews_multi_en@ec73b665, legacy-datasets/banking77@f5412156, google/boolq@35b264d0, fancyzhx/dbpedia_14@9abd46cf, stanfordnlp/imdb@e6281661, nyu-mll/multi_nli@da70db2a, SetFit/sst5@e51bdcd8, CogComp/trec@65752bf5, Yelp/yelp_review_full@c1f9ee93	Not recorded in our data; upstream terms apply (see below)	Gold labels are stored. For 9 of the 10 sets, though, training uses a Qwen3.8-Flash-Next answer (see Labels). BANKING77 trains on gold.
F5	kev policy and composition items	2,576	2,644	kev decision-v7 training partition (github.com/jaredpalmer/kev, commit not pinned)	Apache-2.0	Labels are computed by kev's code. Training uses a Qwen3.8-Flash-Next answer that matches the code label on 98.1% and 100% of the two sets.
F6	kev date and unknowable items	1,425	1,541	kev night2 generator (seed night2-20260920)	Apache-2.0	Labels are computed by code. Items that cannot be answered get a uniform target.
F7	Uncertainty items: unknowable, conflicting, counterfactual, buried evidence, missing option	3,140	3,570	Our generator over kev's contrastive families. 780 records embed train texts from F4 sets (same revisions).	Ours, plus the terms of the embedded sets	Labels are computed by code, with uniform targets for unknowable and conflicting items. No model labels.
F8	World-first decisions and document QA	7,202	14,789	World-first (4,908 records) is our generator: labels are computed by code from a hidden world, the world is rendered to text by Qwen3.8-Flash-Next / Qwen3.8-27B, and a record is kept only when a blind re-answer matched. Document QA (2,294 records, 1,866 en / 428 zh) uses real web pages from HuggingFaceFW/fineweb (sample-10BT), HuggingFaceFW/fineweb-edu (sample-10BT) and HuggingFaceFW/fineweb-2 (cmn_Hani).	Ours; FineWeb is ODC-By and the page text belongs to the site owners; model-output terms of the Qwen / GLM models	World-first: code labels. Soft targets from Qwen3.8-Flash-Next are used only where its top answer equals the code label. Document QA: a Qwen model wrote the question and answer. An item is kept only when two blind Qwen answers and one GLM-5.3-Flash answer agreed, and its target is the vote distribution.
F9	Safety and security sets as policy-conditioned decisions	9,663	20,470	PKU-Alignment/BeaverTails (round0 30k train), nvidia/Aegis-AI-Content-Safety-Dataset-2.0 (train), nvidia/Nemotron-Safety-Guard-Dataset-v3 (zh train), neuralchemy/Prompt-injection-dataset (core, train), thu-coai/cold (train), toxigen/toxigen-data (annotated train), mmathys/openai-moderation-api-evaluation, NVD CVE feeds (fkie-cad/nvd-json-data-feeds release v2026.09.26). Fetched from main; revisions not pinned.	CC-BY-NC-4.0 (BeaverTails); CC-BY-4.0 (Aegis, Nemotron); Apache-2.0 (neuralchemy, COLD); MIT (ToxiGen, OpenAI moderation eval); public domain (NVD)	Each dataset's own label, mapped by code through a written moderation or security policy. No model labels.
F10	Native Japanese / Chinese NLI and paraphrase	5,999	7,998	zenless-lab/jamp, hpprc/jsick, C-MTEB/LCQMC, C-MTEB/BQ (train; revisions not pinned)	Not recorded in our data	Gold labels. JSICK also gives a gold relatedness score.
F11	Chinese / Japanese translations	5,664	7,998	Train splits of MNLI (1,080), stanfordnlp/snli (798), alisawuffles/WANLI (800) and facebook/anli r1-r3 (797), plus 2,189 F8 records. All were translated by Qwen3.8-27B.	Inherited: ANLI is CC-BY-NC-4.0; SNLI / WANLI / MNLI not recorded; F8 ours	Inherited from the source item.
F12	Score questions	4,500	9,000	Our score generator (90 template pairs, 2 questions per record)	Ours	Computed by code on banded scales. No model involved.
	total	99,213	270,905

Models used to make data

We used only open-weight models, all run on our own hardware:

• Qwen3.8-Flash-Next rendered world-first records into text and gave the blind re-answers and votes. It produced the soft targets on world-first items and the thinking-mode answers used as targets for F4 and F5. It also wrote document QA questions. Finally, it rewrote 4,345 true / false questions into claim and negated-claim pairs (8,690 questions in 4,232 records).
• Qwen3.8-27B rendered part of the world-first and document QA records. It also translated all 5,664 F11 records.
• GLM-5.3-Flash gave one agreement vote on each document QA question.

5,641 F8 questions store a rationale written by Qwen. It is not a training target.

No data, labels or text in the mix were generated by closed models. We checked this on the final file:

• Every question-level teacher is a Qwen3.8-Flash-Next variant (12,986 questions).
• Every translator field reads Qwen3.8-27B (5,664 records).
• No record comes from a closed-model generator.
• A scan for closed-model names finds only upstream dataset names and ordinary words such as "octopus", "Jean-Claude" and "Capgemini".

Some public upstream datasets do contain closed-model text that they ship themselves. We did not generate this text:

• RAGTruth: 1,377 of the 4,000 responses come from GPT-3.5-turbo-0613 (645) or GPT-4-0613 (732), as recorded per item. The rest come from Llama-2 and Mistral.
• ToxiGen (564 records): GPT-3 generated the statements upstream.
• WANLI (800 translated records): GPT-3 generated the examples upstream, and humans then revised them.
• API-Bank (2,000 records): ChatGPT generated the training dialogues upstream.
• When2Call (4,000) and ToolACE (1,200): an LLM synthesised these upstream. The generator model is not recorded in our data.

Labels

• Classic sets (F4) train on Qwen's answers, not on gold. The loss uses a question's soft target whenever one exists. For 9 of the 10 classic sets (all but BANKING77), that target is a one-hot on Qwen3.8-Flash-Next's thinking-mode answer (13,522 questions). Agreement with the gold label ranges from 58.3% to 99.0%:

  • SST-5 58.3%, Amazon 60.7%, Yelp 81.1%, MNLI 86.6%
  • AG News 91.1%, BoolQ 92.3%, TREC 92.4%, IMDB 94.8%, DBpedia-14 99.0%

  On these sets, the model learns Qwen's answer. The kev items (F5, 2,644 questions) work the same way, with 98.1% and 100% agreement with the code label.

• Generated sets are labelled by code. This covers the F3 generators, F6, F7, the world-first part of F8 and F12.

• Document QA uses the writing model's answer. An item is kept only when three blind answers from two models agreed.

• Unknowable items have a uniform target over all options (975 questions). Conflicting-evidence items have a uniform target over the two answers the evidence supports (351 questions).

• Everything else uses the dataset's own gold label, mapped by code. This covers F1, F2, the public parts of F3, F9 and F10.

Licences and non-commercial data

• Non-commercial:
  • BeaverTails (CC-BY-NC-4.0, 2,603 records).
  • The ANLI translations (CC-BY-NC-4.0, 797 records).
  • RAGTruth contexts: 1,340 QA items are built on MS MARCO, which has non-commercial terms. 1,348 data-to-text items are built on the Yelp Open Dataset, which forbids redistribution. The other 1,312 items are CNN/DailyMail summaries.
• ShareAlike: HoVer (CC-BY-SA-4.0, 1,900 records).
• Licence not stated upstream: ACOS (4,872), VAST (1,900), SpamAssassin ham (950) and NLI4CT (946, SemEval-2024 Task 2 terms), 8,668 records in all. WinoGrande gives CC-BY with no version.
• Not recorded in our data:
  • The ten classic sets, the four ja / zh pools, SNLI, WANLI and MNLI. Their upstream terms apply. Some of those terms are research-only or ShareAlike, for example the Yelp and Amazon review terms.
  • Document QA quotes web pages from FineWeb (ODC-By); the page text belongs to the site owners.
  • Outputs of Qwen3.8-Flash-Next, Qwen3.8-27B and GLM-5.3-Flash fall under those models' terms.

Because of the non-commercial data and the unstated licences, we release the weights under CC-BY-NC-4.0.

Decontamination

• Decision Index skill data (F1-F3). These are public train / dev splits of benchmarks that Decision Index tests on their test splits (declared in the table above), plus our generators. When each set was built, it was checked against every Decision Index row of the current suite (0.2 for F1 and F2; 0.3 for F3, whose new rows are 513 HLE items and rebuilt GSM8K options) with a generic gate (shared word 13-grams, near-duplicate state, question or option set) and with item-level tests per benchmark: no HoVer claim that shares an evidence document with a Decision Index claim, no NLI4CT trial section that Decision Index uses, GSM8K question text against every Decision Index GSM8K question, generated functions against all 800 CRUXEval functions, ContractNLI contracts with more than 10 % 13-gram overlap. Flagged records were dropped. Where a benchmark's fixed instruction and short option lists make the generic gate fire on every item (CLINC150, GSM8K, CRUXEval, standard contract clauses), the item-level test replaced it.
• JevBench. Against the 231 public items: no exact normalized state, no instruction and no shared 13-word sequence.

Removed before training

Before training, we removed 6,340 records (9,651 questions) from the mix:

• 1,980 records that a closed model wrote and labelled.
• 929 records from an LGPL-3.0 phishing-email set (zefang-liu/phishing-email-dataset).
• 1,900 records from the part of the cfcolor colour-rating pool that Decision Index 0.3 does not ask about.
• 1,531 safety records from upstream validation or test splits: civil_comments validation 1,128, neuralchemy validation 113 and test 94, COLD test 196.

The safety records that remain come from upstream train splits, with two exceptions: NVD, which is CVE feeds with no splits, and the OpenAI moderation evaluation set, which has only one release.

Decision Index submission settings

• Kit: decision-index 0.3 at commit 9eb2dbe2, edition 0.3, suite hash-verified (rows, added rows, rebuilt GSM8K, exclusions, subsets). Engine http against one Verdin server.
• Server: python -m verdin.server --model . --port 8907 on one RTX 5090 32 GB. CUDA, bf16 (checkpoint dtype). Render settings and the attention block (architecture.state_bidir = 8192) come from verdin_config.json (schema-first, date facts, caps 65,536 / 66,560 / 131,072). Engine defaults otherwise (branch GDR recurrent, prefill chunk 8192 = one attention block, max sweep tokens 16,384, prefix cache 4 × ≥ 384 tokens, dynamic batching).
• Client: the kit's own runner in 8 parallel shards (rows split by blake2b(run_id) mod 8 through the runner's keep filter), results concatenated, scored with decision_index score. The latencies in the results include queueing behind 7 other clients: median 211 ms, p95 1.63 s (the causal revision: 214 ms, 1.60 s). Single-client median latency of this engine on an RTX 5090 was 75–95 ms on a 400-request mix of our evaluation suites (measured on the causal revision; the bidirectional mask changes only which keys a state token attends to).

Limitations

• English first. Chinese and Japanese are covered by part of the data; other languages are weaker.
• States longer than 8,192 tokens are read bidirectionally only within each 8,192-token block.
• Probabilities are calibrated with one temperature on one calibration set. Recalibrate on your own data before thresholding.
• Option order can change answers on long option lists, as with every model of this kind.
• Not for safety-critical or high-stakes decisions without human review.
• The public Decision Index number is inflated by training on public train splits of panel benchmarks (declared above). The private parts of the Full score are the honest measure.

Revisions

revision	attention	Decision Index 0.3 public	results
this one	state-bidirectional, blocks of 8,192	58.29	results at d7eef62c
f35e9c18	causal	56.64	results at 4e93b446

Both revisions use the same training mix, recipe, seed and render settings; only the attention mask of the full-attention layers differs (and the training hardware: 4× RTX 3090 here, one RTX 5090 for f35e9c18). A checkpoint with architecture.state_bidir needs the Verdin code of this revision (format_version 2); older Verdin code refuses it instead of running it causally.

License

• Weights: CC-BY-NC-4.0. Part of the training data is licensed for non-commercial use only (for example BeaverTails, CC-BY-NC-4.0), so the fine-tuned weights are released for non-commercial use.
• Base model: Qwen/Qwen3.5-4B, Apache-2.0 (LICENSE.base).
• Verdin code (verdin/): Apache-2.0 (verdin/LICENSE, verdin/NOTICE). It ports parts of kev (Apache-2.0).

Acknowledgements

The Qwen team for Qwen3.5. Jared Palmer for kev. The Decision Index maintainers for the kit and the board. The authors of every dataset listed above.