Verdin-4B-v1.1

Verdin-4B-v1.1 is a System One decision model. You give it a state (text or JSON) and a set of typed questions — choice (pick one of named options), noul (yes / no), score (an ordered scale). It returns a probability for every option of every question. It does not generate text and there is nothing to parse. All questions about one state share a single prefill pass.

• Model: Qwen/Qwen3.5-4B (instruct, revision 851bf6e8) + a LoRA fine-tune merged into the weights + a pointer head that reads the option-boundary tokens. The fine-tune is a uniform soup of three seeds: three LoRA runs (r = 16, seeds 0, 1 and 2) of one recipe on one training mix, averaged exactly (the mean of their weight deltas, merged in fp32), with the three pointer heads averaged. 4.21 B parameters served, bf16. The 8 full-attention layers read the state bidirectionally within blocks of 8,192 tokens (trained that way; see Training recipe).
• What is new compared with Verdin-4B: a much broader training mix (179,197 records instead of 99,213): Verdin-4B's kind of data, plus records from 560 tasksource task sources, plus about 170 new task families converted from human-labelled public data (Hugging Face datasets, government open-data portals and other public sources, in several countries and languages). One epoch per seed, a three-seed soup, and a temperature fitted on held-out data. The architecture, the engine and the wire format are unchanged.
• Engine: Verdin, our inference engine for this model family, shipped in this repo (Apache-2.0; the same code as Verdin-4B at revision 406a3f7d). It runs on NVIDIA GPUs (CUDA, bf16), on the CPU and on Apple silicon (MLX), all with the state-bidirectional attention of this checkpoint (MLX-vs-CUDA agreement was measured on Verdin-4B: the same option on 99.74 % of 8,129 questions; this checkpoint uses the same code path). It speaks the TypeSafe-style POST /v1/systemone wire format.
• License: weights CC-BY-NC-4.0 (non-commercial; see License); Verdin code Apache-2.0.

Results

Decision Index 0.3 public index, the public run that Decision Index 0.4 requires with every submission (all 37 panel benchmarks, chance-corrected, coverage-adjusted; self-run with the official kit on the hash-verified 0.3 suite: 140,620 requests, all answered in one pass, none refused; the kit scores 140,178 of them):

	Decision Index 0.3 public	raw	breadth
Verdin-4B-v1.1	60.96	70.34	59.77
Verdin-4B (406a3f7d, for reference)	58.29	68.86	55.64

Full rows and scores: runs/Verdin-4B-v1.1/ in victorbennett/decision-index-results at c5c68d83. The run was recorded under the model's working label Verdin-4B-v2 (the model field in environment.json and in each result row); it is the same weights as victorbennett/Verdin-4B-v1.1, renamed before release.

area	skill (0–100)	raw accuracy	Verdin-4B skill / raw
Knowledge & Reasoning	43.7	55.5	35.3 / 49.9
Language Understanding	63.3	73.1	66.1 / 76.2
Retrieval & Classification	66.9	75.0	64.3 / 73.6
Tools & Automation	81.0	82.8	85.6 / 87.4
Arts & Human Taste	51.1	69.4	35.5 / 55.5

The board's ranking (Decision Index 0.4) is the Full score: 60 % private tests of the same skills as the public benchmarks and 40 % private decision tasks from new domains, both run by the maintainers. The public index above does not count toward it. Several public panel benchmarks have public train splits that we trained on (declared below), so expect the public index to overstate performance on new tasks.

Held-out proxies for the private parts. We cannot run the private tests, so we track them with our own held-out sets. Their sources are excluded from every training mix; we did use them to compare training recipes, so they are not untouched by model selection:

• proxy-O2 (new domains): 7,542 items in 44 task families that are not in the training mix, scored like the Decision Index (chance-corrected, 0–100). On four 4B models from the 2026-10-20 board it gave the same order as the board's new-domain part.
• O2-shift: the same 44 sets with the wording and layout of every item rewritten (same answers). It shows how much of a score survives surface changes.
• proxy-S (same skills): 32 sets, at least one per public benchmark skill: 28 sibling public datasets (for example SVAMP for GSM8K, HWU64 for CLINC150) and 4 code-generated tool-use sets, never the benchmarks' own rows, combined with the Decision Index area weights.

	proxy-O2	O2-shift	proxy-S
Verdin-4B-v1.1 (three-seed soup)	52.96	51.29	71.54
seed 0 alone	52.03	50.90	71.27
seed 1 alone	52.98	50.81	70.22
seed 2 alone	51.53	50.32	70.88
Verdin-4B (406a3f7d)	47.21	46.01	68.95

All rows were evaluated the same way (kev reference implementation, raw logits). The proxies are ours, not the board's; differences of about one point between single runs are within seed noise.

Other evaluations (accuracy on held-out sets; the same weights run through the kev reference implementation, fp32):

suite	Verdin-4B-v1.1	Verdin-4B
JevBench public (223 requests)	0.830	0.821
typed-decisions test (2,000; its train split was not used)	0.748	0.706
kev decision-v7 test (1,264)	0.873	0.863
kev hard-v1 development (1,021)	0.759	0.729
kev documents-v1 development (914)	0.877	0.828
kev devtools-v1 development (1,074)	0.736	0.727

Quickstart

hf download victorbennett/Verdin-4B-v1.1 --local-dir Verdin-4B-v1.1 && cd Verdin-4B-v1.1
pip install -r requirements-cuda.txt        # NVIDIA;  Apple silicon: requirements-mac.txt and --backend mlx
python -m verdin.server --model . --port 8000

curl -s localhost:8000/v1/systemone -H 'content-type: application/json' -d '{
  "state": {"ticket": "Charged twice for order 1182, please fix today"},
  "questions": {
    "team":   {"type": "choice", "instructions": "Route the ticket.", "criteria": {"billing": "charges, refunds", "tech": null}},
    "urgent": {"type": "noul",   "instructions": "Is it urgent?"},
    "anger":  {"type": "score",  "instructions": "How angry is the customer?", "criteria": ["calm", "annoyed", "furious"]}}}'

from verdin import Verdin
v = Verdin.from_pretrained(".")          # CUDA if available, else CPU
v.decide("The mug arrived broken.", {"damaged": {"type": "noul", "instructions": "Was anything damaged?"}})

Requests are never truncated: a state over 65,536 tokens (or a request over the device's memory) is refused with 422 too many tokens: .... On 24 GB cards, serve long requests with --prefix-cache 0 --attn-chunk-mb 1024 --max-batch-tokens 16384 (the settings of our Decision Index run; they only lower peak memory). More in verdin/README.md and, for Macs, verdin/README-mac.md.

Training recipe

The training code is not released; this is the full recipe.

• Framework: kev (Apache-2.0) with our patches. Each question is one causal row [state] [question] [options] [decide] on top of the shared state. A pointer head (dim 256) scores each option's end token against the decide token. Rendering is schema-first: a short block listing every question and its options (≤ 512 tokens) comes before the state, so the state is read knowing what will be asked. Date facts: when the state holds two or more absolute dates, a date_facts field states the day difference of each pair.
• State-bidirectional attention: in the 8 full-attention layers, a state token at position i attends to a state token at position j when j ≤ i or both lie in the same block of 8,192 positions. Question, option and decide tokens stay causal, and so do the 24 Gated DeltaNet layers. Every seed was trained with this mask from the first step.
• Adapter (per seed): LoRA r = 16, alpha = 32, on every linear projection of the backbone (attention q/k/v/o, Gated DeltaNet in_proj_qkv / in_proj_z / in_proj_a / in_proj_b / out_proj, MLP gate/up/down). The backbone is frozen in bf16; the LoRA and the head are trained.
• Optimisation (per seed): lr 5e-5, weight decay 0.01, 1 epoch, batch 1 request × gradient accumulation 8 = 22,400 optimizer steps; seeds 0, 1 and 2 (data-parallel; seed 0 on 2× RTX 5090, seeds 1 and 2 on 4× RTX 3090 each). Token caps during training: state 14,336 / branch 14,848 / packed 18,432, padded rows ≤ 24,576 (no request was truncated, rejected or skipped). About 180 M forward tokens per seed. No rationale loss.
• Seed soup: the three adapters are concatenated along the rank (A = [A₀; A₁; A₂], B = [B₀, B₁, B₂], alpha kept), which is exactly the mean of the three weight deltas (scale alpha / 48 = one third of alpha / 16); no SVD or truncation. The pointer heads are averaged element-wise in fp32. verdin_config.json therefore records a rank-48 adapter with scale 0.667. On our proxies the soup scores 0.6–0.8 points above the mean of its three seeds (proxy-O2 +0.78, O2-shift +0.61, proxy-S +0.75).
• Calibration: one temperature, T = 0.310, fitted after averaging on the soup's raw logits by minimum NLL on a probe of 17,421 questions (632 task sources): records of our data pool that were not used for training and are not part of any reported evaluation, most of them from sources that are in the training mix (ECE 1.5 % in a two-fold cross-fit by source). The fit on the calibration split of kev's decision-v7 set gave T = 0.406. On our out-of-domain proxies the shipped T = 0.310 is over-confident (ECE 5.9 % on proxy-O2, 7.2 % on O2-shift; with T = 0.406: 1.7 % and 2.2 %), so set head.temperature in verdin_config.json to about 0.41 if you need calibrated probabilities on new domains. Answers and accuracy do not depend on T; the soup's raw logits are flat, so some T well below 1 is needed.
• Compute: seed 0 18.6 h on 2× RTX 5090 (32 GB); seed 1 31.2 h and seed 2 30.7 h on 4× RTX 3090 (24 GB); peak 21.1 GiB per GPU.
• Export: the soup adapter merged in fp32, then cast to bf16 (max |Δp| 0.0007 vs the unmerged fp32 soup on probe requests, 0 argmax flips).
• Not used: a full fine-tune of the backbone on the same mix scored clearly lower on our proxies (proxy-O2 48.16, O2-shift 46.73) than the LoRA seeds, so this release stays a LoRA model. No distillation targets from large teacher models are used beyond those listed under Training data.

Training data

Verdin-4B-v1.1 was trained on one mix of 179,197 records holding 306,594 questions. Each record is a state (text or structured data) plus one or more questions about it. By type, there are 202,582 choice questions (pick one option), 65,314 noul questions (true / false) and 38,698 score questions (a point on an ordinal scale). 58,973 questions carry a soft target. What we checked against Decision Index and our own evaluation sets is described under Decontamination.

The mix joins eight files and shuffles them once (seed 0):

part	records	questions
Base mix: the Verdin-4B mix after the changes listed below, plus 62,193 tasksource records	153,894	278,446
Human-labelled public datasets (bulk v2)	6,016	7,296
Human-labelled public datasets (pilot, permissive licences only)	1,170	1,330
Non-HF public sources (ext v1)	3,840	4,608
Other countries' official open data (xcountry v2)	6,016	6,144
Chinese-language data (zh v2)	3,328	3,584
Decision Index same-skill data (diskills v6)	4,178	4,374
Board games (xiangqi, Go)	755	812
total	179,197	306,594

The base mix grew out of Verdin-4B's 99,213-record mix in these steps:

1. We removed 131 records (130 ids; one id appears twice) that a whole-mix re-check flagged against Decision Index rows (see Decontamination).
2. We added 6,000 operational-decision records (F15).
3. We added a fourth set of Decision Index skill data: ANLI 2,820, BANKING77 1,831, ChessBench 2,375 and CLadder 2,375 records. The Habermas Machine part of that set was left out (see "Removed before training").
4. We capped every Decision Index skill source from the first four sets at 1,000 records (seeded sample). This took those sources from 58,445 to 25,846 records.
5. We added 9,817 code-labelled reasoning and chord-reading records (F14), for 91,701 records.
6. We added 62,193 records from 560 tasksource sources, at most 128 per source (F16), for 153,894 records.

Sources

The mix has 22 families and 902 source tags. As for Verdin-4B, the base parts (F1-F12) use upstream train and dev data only. The new parts (F13-F22) use no upstream test, dev or validation rows. Where an upstream split exists, they use train rows only. The one gap is 4,643 tasksource records (370 sources) whose upstream split is not recorded; all of them come from the train split of the tasksource collection.

#	family	records	questions	upstream (revision)	licence	how the labels were made
F1	Decision Index skill data I: tool calls, contract NLI, RAG hallucination, sarcasm, entity sentiment (1,000 per source)	8,000	11,296	liminghao1630/API-Bank@12e8158b, Team-ACE/ToolACE@6bda777c, nvidia/When2Call@0582f774, stanfordnlp/contract-nli@eced6528, ParticleMedia/RAGTruth@c103204b, iabufarha/iSarcasmEval@dfc708b5 (En, Ar), pyRis/SEntFiN@eba43610	MIT; Apache-2.0 (ToolACE); CC-BY-4.0 (When2Call, ContractNLI)	Gold labels from each dataset, put into the Decision Index request format by code. No model involved.
F2	Decision Index skill data II: stance, intent, product search, commonsense, aspect sentiment, captions, phishing	8,900	38,264	emilyallaway/zero-shot-stance@e7c47751 (VAST), clinc/oos-eval@828f8093, amazon-science/esci-data@7916cdf6, Rowan/hellaswag@218ec52e, allenai/winogrande@01e74176, NUSTM/ACOS@45d179a3, jmhessel/newyorker_caption_contest@d81cbab7, Nazario phishing corpus (monkey.org, listing of 2026-02-20), SpamAssassin public corpus (20030228)	CC-BY-4.0 (New Yorker, Nazario); CC-BY-3.0 (CLINC150); Apache-2.0 (ESCI); MIT (HellaSwag); CC-BY with no version (WinoGrande); not stated (VAST, ACOS, SpamAssassin)	Gold labels. For phishing, the class is the corpus an email came from, and two signal questions are computed by code. No model involved. ACOS alone gives 20,864 questions.
F3	Decision Index skill data III: claim verification, clinical-trial NLI, maths, an appliance simulator, code execution	4,946	12,353	hover-nlp/hover@39b84697 (+ HoVer Wikipedia DB), ai-systems/Task-2-SemEval-2024@7f32fa6c (NLI4CT), openai/gsm8k@740312ad; our code generators for an appliance simulator and CRUXEval-style programs (no data from either benchmark)	CC-BY-SA-4.0 (HoVer); MIT (GSM8K); not stated (NLI4CT, SemEval-2024 Task 2 terms); ours (generated)	Gold labels for HoVer, NLI4CT and GSM8K. GSM8K distractors come from the benchmark's own answer-group code. Generated items are labelled by code: the simulator is re-run, or the program is executed in a sandbox. No model involved.
F4	Classic classification sets (about 1,000 train items each)	9,967	14,488	fancyzhx/ag_news@eb185aad, SetFit/amazon_reviews_multi_en@ec73b665, legacy-datasets/banking77@f5412156, google/boolq@35b264d0, fancyzhx/dbpedia_14@9abd46cf, stanfordnlp/imdb@e6281661, nyu-mll/multi_nli@da70db2a, SetFit/sst5@e51bdcd8, CogComp/trec@65752bf5, Yelp/yelp_review_full@c1f9ee93	Not recorded in our data; upstream terms apply (see below)	Gold labels are stored. For 9 of the 10 sets, though, training uses a Qwen3.8-Flash-Next answer (see Labels). BANKING77 trains on gold.
F5	kev policy and composition items	2,576	2,644	kev decision-v7 training partition (github.com/jaredpalmer/kev, commit not pinned)	Apache-2.0	Labels are computed by kev's code. Training uses a Qwen3.8-Flash-Next answer that matches the code label on 98.1% and 100% of the two sets.
F6	kev date and unknowable items	1,425	1,541	kev night2 generator (seed night2-20260920)	Apache-2.0	Labels are computed by code. Items that cannot be answered get a uniform target.
F7	Uncertainty items: unknowable, conflicting, counterfactual, buried evidence, missing option	3,137	3,566	Our generator over kev's contrastive families. Some records embed train texts from F4 sets (same revisions).	Ours, plus the terms of the embedded sets	Labels are computed by code, with uniform targets for unknowable and conflicting items. No model labels.
F8	World-first decisions and document QA	7,169	14,745	World-first (4,902 records) is our generator: labels are computed by code from a hidden world, the world is rendered to text by Qwen3.8-Flash-Next / Qwen3.8-27B, and a record is kept only when a blind re-answer matched. Document QA (2,267 records) uses real web pages from HuggingFaceFW/fineweb (sample-10BT), HuggingFaceFW/fineweb-edu (sample-10BT) and HuggingFaceFW/fineweb-2 (cmn_Hani).	Ours; FineWeb is ODC-By and the page text belongs to the site owners; model-output terms of the Qwen / GLM models	World-first: code labels. Soft targets from Qwen3.8-Flash-Next are used only where its top answer equals the code label. Document QA: a Qwen model wrote the question and answer. An item is kept only when two blind Qwen answers and one GLM-5.3-Flash answer agreed, and its target is the vote distribution.
F9	Safety and security sets as policy-conditioned decisions	9,662	20,468	PKU-Alignment/BeaverTails (round0 30k train), nvidia/Aegis-AI-Content-Safety-Dataset-2.0 (train), nvidia/Nemotron-Safety-Guard-Dataset-v3 (zh train), neuralchemy/Prompt-injection-dataset (core, train), thu-coai/cold (train), toxigen/toxigen-data (annotated train), mmathys/openai-moderation-api-evaluation, NVD CVE feeds (fkie-cad/nvd-json-data-feeds release v2026.09.26). Fetched from main; revisions not pinned.	CC-BY-NC-4.0 (BeaverTails); CC-BY-4.0 (Aegis, Nemotron); Apache-2.0 (neuralchemy, COLD); MIT (ToxiGen, OpenAI moderation eval); public domain (NVD)	Each dataset's own label, mapped by code through a written moderation or security policy. No model labels.
F10	Native Japanese / Chinese NLI and paraphrase	5,999	7,998	zenless-lab/jamp, hpprc/jsick, C-MTEB/LCQMC, C-MTEB/BQ (train; revisions not pinned)	Not recorded in our data	Gold labels. JSICK also gives a gold relatedness score.
F11	Chinese / Japanese translations	5,603	7,905	Train splits of MNLI (1,080), stanfordnlp/snli (798), alisawuffles/WANLI (800) and facebook/anli r1-r3 (785), plus 2,140 F8 records. All were translated by Qwen3.8-27B.	Inherited: ANLI is CC-BY-NC-4.0; SNLI / WANLI / MNLI not recorded; F8 ours	Inherited from the source item.
F12	Score questions	4,500	9,000	Our score generator (90 template pairs, 2 questions per record)	Ours	Computed by code on banded scales. No model involved.
F13	Decision Index skill data IV: NLI, intent, chess, causal ladder (1,000 per source)	4,000	4,000	facebook/anli@8e4813d8 (train r1-r3), PolyAI-LDN/task-specific-datasets@57ec275d (BANKING77 train.csv), google-deepmind/searchless_chess@90ae0e6b (train behavioural-cloning data), our CLadder-style generator	CC-BY-NC-4.0 (ANLI); CC-BY-4.0 (BANKING77, searchless_chess data; Lichess positions CC0); generated (CLadder semantics, MIT)	Gold labels for ANLI and BANKING77; ChessBench best moves come from the dataset's Stockfish annotations; CLadder-style answers are computed by code. No model involved.
F14	Decision Index skill data V: BBH-, MuSR- and SATA-style reasoning, chord reading	9,817	24,991	Our generators only (BBH-style 3,540, MuSR-style 1,468, SATA-style 1,811, chord 2,998). No BBH, MuSR, SATA-Bench or POP909 items are used.	Ours (generated)	All labels are computed by code, and an independent re-solver from the rendered text agreed on every checked task group. For MuSR-style records, Qwen3.8-Flash-Next wrote the story from code fact lists; a story is kept only when a code check and an independent Qwen read-back agree with the code label.
F15	Operational decisions: tickets, invoices, alerts, agent runs, changes, shipments, expenses, data access	6,000	30,000	Our generator (sampled factors rendered by code templates; 718 records have a free-text slot paraphrased by Qwen3.8-Flash-Next and fact-checked)	Ours	label is the answer the factors imply where they settle it, else the teacher's argmax. Training uses the soft target, which is Qwen3.8-Flash-Next's averaged answer distribution (logprob readout) on all 30,000 questions. A record is dropped when that answer contradicts a factor-implied answer.
F16	tasksource typed decisions: 560 sources (NLI, sentiment, topic, intent, QA as choice, preference, safety, discourse, multilingual sets)	62,193	75,187	tasksource/tasksource-jev-typed-decisions@f7a6cce9 (default config, train split), at most 128 records per source; each record keeps its upstream dataset and revision	Per source: commercial-use 34,179 records, unspecified 23,034, non-commercial 4,278, research-only 702; share-alike or copyleft 9,500	Each dataset's own gold label. Where the dataset gives annotator votes or interpolated ratings, they are kept as a soft target (3,585 questions). No model labels from us.
F17	Human-labelled public datasets: 30 HF Hub datasets and 17 non-HF public sources (bulk v2), plus 15 HF datasets (pilot)	7,186	8,626	45 HF datasets with pinned revisions; non-HF: openFDA recall and device data, MSHA, OSHA, NOAA Storm Events, FRA (2 sets), Census NAICS, DOJ press releases, Texas HHSC child-care inspections, UK Parliament written questions, UK Charity Commission register, OSDG (Zenodo), bioRxiv, UCI Druglib	Permissive, no share-alike: CC-BY-4.0, Apache-2.0, CC0, MIT, US federal public domain, CC-BY-3.0, AFL-3.0, UK OGL v3, UK Open Parliament Licence v3	The dataset's own human, expert, institutional or natural-signal label, taken from a fixed column and mapped by deterministic code. DeepSeek Flash wrote the question wording and option descriptions from the dataset card (see Labels).
F18	Non-HF public sources (ext v1)	3,840	4,608	30 families: 15 UCI Machine Learning Repository datasets; 8 government or public-body datasets (US Census occupation coding, CPSC recalls, NHTSA recalls, U.S. Fire Administration fatalities, O*NET; UK National Lottery Heritage Fund grants, UK OPSS product safety; Canada access-to-information requests); 1 Inter-American Development Bank projects set; 6 research corpora (ABCD, CaSiNo x2, PersuasionForGood x2, MFRC). Each record stores its source URLs and the sha256 of the prepared table.	CC-BY-4.0 (20 families), US federal public domain (4), UK OGL v3 (2), Apache-2.0 (2), MIT (1), OGL-Canada 2.0 (1)	As F17: the recorded outcome, official code or human annotation from the source; DeepSeek Flash wrote the question wording and option descriptions.
F19	Other countries' official open data (xcountry v2)	6,016	6,144	47 datasets from 24 jurisdictions (AT, AU, BR, CA, DE, DK, EU, FI, FR, IE, IL, IT, JP, KR, MA, MX, NL, NO, PT, SE, UK, Scotland, ZA, and UN OCHA) in 15 languages plus one multilingual set: parliaments (15: written questions to ministries, bill subjects, press releases, petitions), statistics offices' classification indexes (10), regulators and safety boards (14: recalls, accident and occurrence reports, device grades), charity and NPO registers (5), other government (3). Text stays in the original language; no machine translation.	Government open licences (OGL-UK, OGL-Canada, Etalab 2.0, NLOD 2.0, PDL-JP 1.0, data.gov.il and others), CC-BY-4.0 (11), CC-BY-3.0 AU / ZA, CC0; all allow commercial use, none share-alike	The category or outcome recorded by the publishing institution. DeepSeek Flash wrote the question wording and option descriptions.
F20	Chinese-language data (zh v2)	3,328	3,584	26 datasets: 11 from mainland China (research datasets on GitHub, HF and Zenodo: restaurant and film reviews, Q&A boards, Weibo, health questions, video partitions, paper disciplines, search relevance, Xiaohongshu notes), 6 from Hong Kong (5 DATA.GOV.HK government datasets, OpenRice reviews), 9 classical-literature attribution tasks (chinese-poetry, Werneror/Poetry, PoetryMTEB). No Taiwan data, government or otherwise.	MIT (12), DATA.GOV.HK terms (5), Apache-2.0 (4), CC-BY-4.0 (3), CC0 (2)	The dataset's own human or natural-signal label (reviewer stars, the board or partition the author chose, the collection a poem belongs to), or the category recorded by the HK government. DeepSeek Flash wrote the question wording and option descriptions in Chinese.
F21	Decision Index same-skill data VI: original-vs-perturbed poems, group endorsement, caption humour, stance	4,178	4,374	Poems written for this set by Qwen3.8-Flash-Next (1,140); Pol.is open data compdemocracy/openData@3be5785c (949, 18 public deliberations); jmhessel/newyorker_caption_contest@d81cbab7 ranking task, train fold (949); stance texts written by Qwen3.8-Flash-Next (1,063) plus English Wikipedia Articles-for-Deletion comments from copenlu/wiki-stance@a2f4d1bf (77)	Generated (poems, stance texts); CC-BY-4.0 (Pol.is, New Yorker); CC-BY-SA-4.0 (wiki-stance)	Poems: the unaltered poem is the answer, perturbations are code edits checked with CMUdict. Pol.is: human agree / disagree votes. New Yorker: the contest's official or crowd winner. Stance: the writer's intended label (or the editor's own keep / delete vote), kept only when a blind DeepSeek answer agreed. No Decision Index item or request layout is copied.
F22	Board games: xiangqi and Go	755	812	Our own Fairy-Stockfish self-play xiangqi games (375 records) and KataGo policy-sampled Go games (380 records)	Generated by us; the engines (Fairy-Stockfish, GPL-3.0; KataGo, MIT, with its network licence) were used as tools only	Engine output under agreement thresholds (two search depths for xiangqi, root and child searches for Go) or our own rules code (legality, captures, checks). No language model involved.
	total	179,197	306,594

Models used to make data

Every model that wrote text, labels, soft targets, verification votes, question wordings or translations for this mix belongs to an open-weight model family. Qwen3.8 and GLM-5.3 ran on our own servers. DeepSeek Flash ran first through DeepSeek's API (deepseek-flash) and later as our own deployment of deepseek-v4.1-flash.

• Qwen3.8-Flash-Next:
  • It is the teacher behind 50,509 question targets: 13,493 F4 questions and 2,644 F5 questions (thinking-mode answers), 30,000 F15 questions (logprob distributions) and 4,372 world-first questions in F8.
  • It rendered world-first records into text and gave the blind re-answers and votes.
  • It wrote the document QA questions and the 1,468 MuSR-style stories in F14, and checked each story by reading it back.
  • It paraphrased the free-text slot of 718 F15 records.
  • It rewrote true / false questions into claim and negated-claim pairs in 4,230 records.
  • It wrote the 1,140 poems and 1,063 stance texts in F21.
  • It also read every converted dataset in F17-F20 as a release gate: a dataset was kept only when Qwen's answers agreed with the gold labels well above chance. These readouts are not stored as labels or targets.
• Qwen3.8-27B translated all 5,603 F11 records and rendered part of the world-first and document QA records.
• GLM-5.3-Flash gave one agreement vote on each of the 3,553 document QA questions (F8 and its F11 translations). It also gave an informational second read of 556 MuSR-style stories; that read was not a rule for keeping a story.
• DeepSeek Flash (thinking mode) wrote the question wordings and option descriptions for all 165 converted datasets in F17-F20: 151 datasets (18,578 records) through DeepSeek's API and 14 datasets (1,792 records) on our own deployment. It never chose the gold column or the label. DeepSeek's terms allow training other models on its outputs. deepseek-v4.1-flash was also the blind verifier for all 1,140 F21 stance records.

5,618 F8 questions store a rationale written by Qwen. It is not a training target.

What we checked about closed models. We used no closed model to write, label, verify, score or translate any record. We checked this on all eight files:

• Every question-level teacher is a Qwen3.8-Flash-Next variant (42,959 questions), and every record-level teacher is Qwen3.8-Flash-Next (7,468 records).
• Every translator field reads Qwen3.8-27B (5,603 records).
• Every model vote field names GLM-5.3-Flash (3,553 questions).
• Every question-wording author in F17-F20 is DeepSeek Flash (20,370 records). Each record's stamp matches the model recorded in its spec file.
• Every verifier field names deepseek-v4.1-flash, or Qwen for an informational re-label (F21).
• Every generator field names one of our own code generators.
• A scan of all metadata for closed-model and vendor names finds three kinds of hit:
  • upstream dataset names and repositories, such as openai/gsm8k and google-deepmind/searchless_chess;
  • the upstream model descriptions listed below;
  • ordinary words, such as "River tubing", "GitHub Copilot" as a stance topic and a "palm sleep pillow" recall.

The mix does contain closed-model text, and a few closed-model labels, that public upstream datasets ship themselves. We did not generate any of it:

• In the base mix (F1, F9, F11):
  • RAGTruth: 339 of the 1,000 responses come from GPT-3.5-turbo-0613 (156) or GPT-4-0613 (183), as recorded per item. The rest come from Llama-2 and Mistral.
  • ToxiGen (564 records): GPT-3 generated the statements upstream.
  • WANLI (800 translated records): GPT-3 generated the examples upstream, and humans then revised them.
  • API-Bank (1,000 records): ChatGPT generated the training dialogues upstream.
  • When2Call (1,000) and ToolACE (1,000): an LLM synthesised these upstream. The generator model is not recorded in our data.
• In F16, 26 tasksource sources (3,254 records) are flagged by our builder as holding closed-model text:
  • hh-rlhf (4 sources, 512 records): Anthropic's RLHF assistant model responses.
  • HelpSteer (5, 640): responses from an NVIDIA in-house 43B model.
  • HelpSteer2 (5, 640): Nemotron / Mixtral responses.
  • WildGuardMix (3, 384): GPT-4 generated prompts and responses.
  • prm800k_dpo (2, 256): GPT-4 solutions with human step labels.
  • UltraFeedback-paired (128): GPT-4 / GPT-3.5 and other completions. The "which response is better" label comes from GPT-4 ratings, so this is a closed-model label.
  • WANLI (128): GPT-3 pairs, human-revised.
  • toxigen-data (128): GPT-3 statements.
  • persuasion (128): arguments written by Anthropic's models (Anthropic/persuasion).
  • TuringBench (128): machine text including GPT-3.
  • prosocial-dialog (123): GPT-3 generated contexts.
  • RAGTruth grounding (59).
• Also in F16: 11 sources (1,408 records) hold open-model text (Alpaca-7B, GPT-2, GPT-J, Mistral Large 3, HelpSteer3's open-weight responses), and 6 sources (743 records) are partly synthetic with no generator stated (prompt-injection sets, dnd_style_intents, WebInstruct).

Labels

• Classic sets (F4) train on Qwen's answers, not on gold. The loss uses a question's soft target whenever one exists. For 9 of the 10 classic sets (all but BANKING77), that target is a one-hot on Qwen3.8-Flash-Next's thinking-mode answer (13,493 questions). Agreement with the gold label ranges from 58.3% to 99.0%:

  • SST-5 58.3%, Amazon 60.7%, Yelp 81.1%, MNLI 86.6%
  • AG News 91.1%, BoolQ 92.3%, TREC 92.7%, IMDB 94.8%, DBpedia-14 99.0%

  On these sets, the model learns Qwen's answer. The kev items (F5, 2,644 questions) work the same way, with 98.1% and 100% agreement with the code label.

• Operational decisions (F15) train on Qwen's soft distribution on all 30,000 questions. The factor-implied answer is defined for 64% of the questions; it is used to filter records, not as the target.

• Generated sets are labelled by code. This covers the F3 generators, F6, F7, the world-first part of F8, F12, the CLadder part of F13 and all of F14.

• Document QA uses the writing model's answer. An item is kept only when three blind answers from two models agreed, and its target is the vote distribution (3,553 questions).

• Unknowable items have a uniform target over all options (975 questions). Conflicting-evidence items have a uniform target over the two answers the evidence supports (351 questions).

• Converted public datasets (F17-F20) keep the dataset's own label. The gold column and any transform (integer bins, top-N classes, majority of annotators) are fixed by hand in code. DeepSeek Flash only wrote the question text and the option names and descriptions, from the dataset card, the columns and a few examples. Two more checks follow:

  • Each dataset passed a gate: Qwen, reading the converted records, had to beat chance by at least 0.05, or reach a balanced accuracy of at least 0.55 on imbalanced binaries.
  • At least 2 records per dataset were read against the source documentation before the dataset was accepted.

• tasksource (F16) keeps each dataset's gold label. Where annotator votes or interpolated ratings exist, they are kept as the target (3,585 questions).

• F21 and F22 are described in the table. Stance texts written by Qwen keep the writer's intended label only when a blind DeepSeek answer agreed.

• Everything else uses the dataset's own gold label, mapped by code. This covers F1, F2, the public parts of F3, F9, F10 and the public parts of F13.

Licences and non-commercial data

• Non-commercial (8,666 records with a non-commercial licence tag, plus the RAGTruth contexts):
  • BeaverTails (CC-BY-NC-4.0, 2,603 records in F9).
  • ANLI (CC-BY-NC-4.0): 1,000 records in F13 and 785 translated records in F11.
  • 38 tasksource sources (4,278 records), for example ANLI (384), BeaverTails (128), PKU-SafeRLHF (256), persuasion (128, CC-BY-NC-SA-4.0), ContractNLI (256, tagged CC-BY-NC-SA-4.0 by tasksource), QuAIL, SICK, WebLINX, TabMWP, XL-WiC and EmpatheticDialogues.
  • RAGTruth contexts (F1): 345 QA items are built on MS MARCO, which has non-commercial terms. 318 data-to-text items are built on the Yelp Open Dataset, which forbids redistribution. The other 337 items are CNN/DailyMail summaries.
• Research-only: 6 tasksource sources (702 records) whose terms are research-only or need a request form: ANTIQUE, CoNLL-2003, implicit-hate (stage 1), PreCo, RiddleSense and WikiQA.
• ShareAlike:
  • HoVer (CC-BY-SA-4.0, 1,000 records).
  • 9,500 tasksource records flagged share-alike or copyleft; this count includes the NC-SA sets above.
  • 77 Wikipedia comment records in F21 (CC-BY-SA-4.0).
  • Not recorded in our data, but tagged share-alike by tasksource for the same datasets: DBpedia-14 (CC-BY-SA-3.0, 996 records in F4) and SNLI (CC-BY-SA-4.0, 798 translated records in F11).
• Licence not stated upstream:
  • ACOS (1,000), VAST (1,000), SpamAssassin ham (950) and NLI4CT (946, SemEval-2024 Task 2 terms): 3,896 records in all. WinoGrande gives CC-BY with no version.
  • 23,034 tasksource records have no licence stated by the source.
• Not recorded in our data:
  • The ten classic sets, the four ja / zh pools, WANLI and MNLI. Their upstream terms apply, and some of them are research-only or ShareAlike, for example the Yelp and Amazon review terms.
  • Document QA quotes web pages from FineWeb (ODC-By); the page text belongs to the site owners.
• Platform terms (terms flag). 40 of the converted datasets (5,120 records) carry _meta.terms_flag. Their dataset licence is permissive, but the text comes from a platform or site whose own terms limit reuse:
  • F17, 17 datasets (2,176 records): reviews, app-store and forum posts, news articles, tweets and Reddit posts.
  • F18, MFRC (128): Reddit comments.
  • F19, 2 Japanese ministry databases (256): their subsite terms sit beside the government-wide PDL 1.0.
  • F20, 20 datasets (2,560): Dianping, OpenRice, Douban, Weibo, Bilibili, Q&A and health sites, web passages, Xiaohongshu, LegCo question text and classical-poetry web text.
• Model outputs. Outputs of Qwen3.8-Flash-Next, Qwen3.8-27B, GLM-5.3-Flash and DeepSeek Flash fall under those models' terms.
• Tools. The board-game labels come from Fairy-Stockfish (GPL-3.0) and KataGo (MIT; network under the KataGo Neural Network License), used as tools.

The new parts F17-F20 are permissive and not share-alike, but the mix as a whole still holds non-commercial, research-only and unstated-licence data. We therefore release the weights under CC-BY-NC-4.0, as for Verdin-4B.

Decontamination

The checks differ by part. No single check was run over the whole 179,197-record mix.

• Base parts carried over from Verdin-4B (F1-F12).

  • When each Decision Index skill set (F1-F3) was built, it was checked against every Decision Index row of the current suite (0.2 for F1 and F2; 0.3 for F3), with the generic gate and the item-level tests described in the Verdin-4B card.
  • The whole 99,213-record Verdin-4B mix was then re-checked against Decision Index 0.2 rows plus the 0.3 additions. The check used the same generic gate (shared word 13-grams, near-duplicate state, question or option set) with a data-frequency cap of 5. It flagged 7,596 records:
    • 7,466 were F1-F3 records matching their own benchmark's fixed instructions and option lists. The item-level tests at build time had already cleared these, and they stay.
    • The other 130 flagged ids (131 records) were removed for this mix. They were shared boilerplate, reused ANLI premises, Wikipedia passages shared with HoVer and short questions close to MMLU-Pro states. None of the 130 ids is in the mix.
  • Against the 231 public JevBench items, the Verdin-4B mix had no exact normalised state, no instruction and no shared 13-word sequence.

• F13 (Decision Index skill data IV). These records were checked against the same benchmark's Decision Index 0.3 rows:

  • ANLI premises equal to or sharing a 13-gram with a Decision Index premise were dropped (39,624 candidates), as were 3 near-duplicate hypotheses.
  • BANKING77 utterances that near-duplicate a Decision Index utterance were dropped (722), plus 70 close paraphrases found in a later full scan (word Jaccard of 0.8 or more).
  • Chess positions equal to a Decision Index position were dropped (1). CLadder items matching a CLadder-v1 file or a Decision Index row were dropped (1,702).
  • The generic gate then ran against Decision Index 0.2 rows, our local evaluation sets and the Decision Index 0.3 additions.

• F14. These records were checked against the same benchmark's Decision Index 0.3 rows: exact match, word 4-gram Jaccard of 0.75 or more, and story or paragraph 13-grams. This dropped 22 BBH-style items, plus 5 chord windows that share 30% or more of their relative note events with one Decision Index row. The generic gate then ran against Decision Index 0.2 rows and the 0.3 additions.

• F15. 0 records were flagged against our local evaluation sets, the Decision Index 0.2 rows, or the typed-decisions train and test files.

• F16 (tasksource). Every row of the 620 trainable sources (715,427 rows) was hashed against an index of 161 evaluation files (260,500 records). The index holds the Decision Index 0.3 public rows (158,028), our own evaluation and proxy sets, and every held-out file. Matching used exact normalised text units (state, state lines, instructions, options of 60 or more characters) plus word 13-grams, and a group with any flagged row was dropped whole. The check removed:

  • 13,939 rows on content hits;
  • 7,359 rows from upstream test / dev / validation splits;
  • 3,297 non-train rows of the 14 sources built from Decision Index public benchmarks (only train rows are kept);
  • all 131 excluded sources (listed under Removed before training).

• F17-F20 (converted datasets). Each dataset's pool (up to 600 rows in the pilot, 1,200 elsewhere) was hashed against the same index plus our later proxy sets, with the same units. Flagged rows were dropped: 46 (bulk), 16 (pilot), 360 (ext), 29 (xcountry) and 16 (zh). A dataset whose pool hit rate was above 5% was dropped. Three ext families above 5% (CaSiNo outcome, PersuasionForGood donation, ABCD) were cleared by hand, because all their hits were generic one-line phrases with no shared 13-gram.

• F21. Each source was tested against its own benchmark's Decision Index 0.3 rows:

  • poems sharing a line or a word 8-gram with BPoMP (13 dropped);
  • statements near-duplicating a Habermas Machine item (1 dropped);
  • contests or captions used by Decision Index's New Yorker rows (none used);
  • stance texts against all 3,006 VAST posts (0 hits).

  Two generic gates then ran. The first used Decision Index 0.2 rows, our local evaluations and the 0.3 additions. The second used the tasksource index plus our proxy sets. Together they dropped 1,045 New Yorker candidates and 2 stance records.

• F22. The same two generic gates ran and dropped nothing. None of our evaluation sets uses xiangqi or Go.

Removed before training

• Verdin-4B's removals still apply. No record in this mix comes from the closed-model-written gen_* set (1,980 records), the LGPL-3.0 phishing-email set, the cfcolor pool or the upstream validation / test safety splits.

• The 131 records with the 130 flagged ids were removed from the base, as described under Decontamination.

• Habermas Machine training data (1,628 records) was left out, because its candidate statements were written by the paper's closed Chinchilla-based model. Its skill is covered by Pol.is votes in F21 instead.

• Capped sources. Each Decision Index skill source of the first four sets was capped at 1,000 records, which leaves 32,599 earlier records unused. The tool-use set of the fifth set is not in this mix.

• tasksource sources excluded as a whole (131 of 751):

  • 5 under OpenAI terms: Chatbot Arena conversations, summarize-from-feedback, ToxicChat (2), WebGPT comparisons.
  • 46 that match our evaluation-only list.
  • 3 aggregates holding such sets.
  • 39 hold-outs of our proxy benchmark, plus 20 of their same-text or same-family siblings.
  • 18 typed-decisions sources.

  55 further records were dropped as too long.

• Pilot datasets dropped (11 datasets, 880 records): five non-commercial ones (four climatebert sets, CheckThat), two share-alike ones (Law Stack Exchange, medical abstracts) and four that share a source with our evaluation sets (CFPB complaints, CVE descriptions, BigCloneBench, Aegis 1.0).

• Taiwan data. All Taiwan sets built for the Chinese part (15) were deleted before any use. The Pol.is deliberations exclude vTaiwan.

• Kept out of this mix:

  • our final-training-only files: Lichess puzzles, court outcomes, BBQ;
  • the model-generated country-swap and Chinese official-scheme sets.

Decision Index submission settings

• Kit: decision-index 0.3 at commit 9eb2dbe2 (its runner and scorer are byte-identical to the 0.4 kit on main, 25c6ec7b), edition 0.3, suite hash-verified (rows, added rows, rebuilt GSM8K, exclusions, subsets). Engine http.
• Servers: 4 identical Verdin servers, one per NVIDIA GeForce RTX 3090 (24 GB), each python -m verdin.server --model . --port <port> --prefix-cache 0 --attn-chunk-mb 1024 --max-batch-tokens 16384 (CUDA, bf16 = the checkpoint dtype). The three flags only lower peak memory on 24 GB cards (no state cache across requests, attention in 1 GB chunks, smaller cross-request batches); the render settings and the attention block (architecture.state_bidir = 8192) come from verdin_config.json (schema-first, date facts, caps 65,536 / 66,560 / 131,072). Engine defaults otherwise (branch GDR recurrent, prefill chunk 8,192 = one attention block, dynamic batching).
• Client: the kit's own runner in 32 parallel shards (rows split by blake2b(run_id) mod 32 through the runner's keep filter); shard i sent its requests to server i mod 4, so every server had 8 clients in flight. The shard results were concatenated and scored with decision_index score. One client pass answered all 140,620 requests (no errors, no retries).
• Latency in the results includes queueing behind 7 other clients: median 694 ms, mean 1,535 ms, p80 2,066 ms (server-side compute: median 345 ms, mean 723 ms). One client, one request at a time, on one RTX 3090 (377 requests sampled across the suite): median 124 ms, mean 265 ms, p80 277 ms.

Limitations

• English first. Chinese and Japanese are covered by a larger part of the data; other languages appear in some of the new families but are weaker.
• States longer than 8,192 tokens are read bidirectionally only within each 8,192-token block.
• Probabilities use one temperature fitted on one probe of mostly in-distribution sources; they are over-confident on new domains (see Training recipe, Calibration). Recalibrate on your own data before thresholding.
• Option order can change answers on long option lists, as with every model of this kind.
• Our new-domain proxy still sits well below the same-skill proxy: decisions in unfamiliar domains are the weakest part.
• Not for safety-critical or high-stakes decisions without human review.
• The public Decision Index number is inflated by training on public train splits of panel benchmarks (declared above). The private parts of the Full score are the honest measure.

License

• Weights: CC-BY-NC-4.0. Part of the training data is licensed for non-commercial use only (see Licences), so the fine-tuned weights are released for non-commercial use.
• Base model: Qwen/Qwen3.5-4B, Apache-2.0 (LICENSE.base).
• Verdin code (verdin/): Apache-2.0 (verdin/LICENSE, verdin/NOTICE). It ports parts of kev (Apache-2.0).

Acknowledgements

The Qwen team for Qwen3.5. Jared Palmer for kev. The tasksource authors. The Decision Index maintainers for the kit and the board. The authors and publishers of every dataset and open-data portal listed above.