zils-mini-body-24: Tools- and Retrieval-weighted exam-type LoRA fine-tune of a uniform mix of four d3-mini house ingredients

A decision body for the Zils d3-mini soup pool (Bittensor subnet 127), in the d3-mini chassis layout (Qwen3.5-4B language and vision tensors, bf16). Its start is a uniform weight average, built with the official zils.soup.build, of four of the pool's five public house ingredients: vllm-sr/d3-mini @df99bf4c; perplexity-ai/pplx-decider-flash-v1.2 @9ff7cf98; torchcast-ai/torchcast-decision-4b @492bfb19; Maincode/Matilda-Decision-v1-Mini @39479a8a. Start body digest be40457ed0725abfe6a66d0e3cdb29bff6c206a71440d54382bbdca89c01a2fc (soup hash 20c02cae). Credit for the starting weights belongs to the authors of those models and their upstream works (see NOTICE). No other miner's body was merged; it is not a re-upload of any public model.

Method

• Training: LoRA rank 32 / alpha 64 / dropout 0.05 on the attention, Gated-DeltaNet AND MLP (gate / up / down) projections of all language layers, learning rate 5e-5, AdamW with weight decay 1.0 on the LoRA weights only, batch 8 with no accumulation, warm-up 50 steps then cosine, 2 epochs, max 2,048 tokens, seed 4353; loss per row = cross-entropy on the row's label + 1.0 x Brier + 0.3 x KL(start || this model) on the same row. The KL term holds the model near its own start and is computed live from the start weights with the adapter off; it uses no teacher, no extra pin rows and no other miner's outputs. Merged into bf16. No weight interpolation. This body is the final save (the only one read and shipped, fixed before training).
• Target: Data weighted toward Tools and Retrieval (87.5% of the rows together), using only rows that passed our checks. The start is the fixed house mix above, not a soup.

Training data (866 rows, file sha256 6d93327a51b57774f98e25625d21f4cd8be92109599862f4839b1709978613e8)

A Tools- and Retrieval-weighted exam-type set in the exam's 15 known families: 4.2% of the rows are Reasoning, 4.2% Language, 45.3% Retrieval, 42.3% Tools and 4.2% Arts. Category shares do NOT follow the exam weights. Every label is the dataset's own label (human annotators, or the dataset's own prover / generator), derived by code from the dataset record, or computed by our own generator, except the 18 rows written by local open models, whose answer is the writer model's own (see the table). No other miner's outputs were used. Selected with ft_emphasis.py v0 (08597f00) from a 2,819-row set frozen for this purpose (sha256 8dd29124985985ae4e502090c448bf4123d1d70750021df085df8608ed383f34). This set has fewer than 1,600 rows, so it trains at batch 8 with no accumulation (about 216 optimizer steps).

CC BY-SA 4.0 note. 366 of the rows are built from the Schema-Guided Dialogue dataset (Google Research, CC BY-SA 4.0, https://github.com/google-research-datasets/dstc8-schema-guided-dialogue) and 6 from SNLI (CC BY-SA 4.0). They were used for training only; this repository holds model weights only, no dataset text. Check those datasets' terms before you reuse this body.

Difficulty rule: a row enters only if this start puts probability 0.30-0.80 on its correct answer, measured on the start body itself before training (all 866 rows checked: lowest 0.301, highest 0.800). Rows were taken only from sources that passed our checks, and nothing was padded to reach a row count. No family exceeds 20% of the rows (largest 15.4%); our own item generators supply 7.3% of the rows.

category	choice	yes/no	score	total
reasoning	12	12	12	36
language	12	12	12	36
retrieval	127	133	132	392
tools	122	122	122	366
arts	12	12	12	36

source	rows	label	original licence	original release
20 Newsgroups (TRAIN split)	127	the newsgroup the post was sent to	no open licence is named by the original release; used under the original authors' terms	http://qwone.com/~jason/20Newsgroups/
Schema-Guided Dialogue (DSTC8, TRAIN split; real dialogues, sibling function options made by code)	122	code: the arguments are the dialogue's annotated call; a schema check on every option passes only for the gold sibling	CC BY-SA 4.0	https://github.com/google-research-datasets/dstc8-schema-guided-dialogue
Schema-Guided Dialogue (DSTC8, TRAIN split; required arguments filled)	87	code: required slots of the active intent that are in the human-annotated dialogue state and whose value occurs verbatim in the conversation so far	CC BY-SA 4.0	https://github.com/google-research-datasets/dstc8-schema-guided-dialogue
SQuAD 2.0 (TRAIN split, passage level)	81	dataset gold: the number of answerable question parts	CC BY-SA 4.0	https://rajpurkar.github.io/SQuAD-explorer/
QNLI (GLUE, TRAIN split; text from SQuAD v1.1)	71	human gold label (the sentence answers the question)	CC BY-SA 4.0 (SQuAD)	https://rajpurkar.github.io/SQuAD-explorer/
Schema-Guided Dialogue (DSTC8, TRAIN split; does the next turn make a call)	56	code: whether the annotated next system turn of the real dialogue makes a service call	CC BY-SA 4.0	https://github.com/google-research-datasets/dstc8-schema-guided-dialogue
Schema-Guided Dialogue (DSTC8, TRAIN split; real dialogues, function list edited by code)	42	code: true = the annotated next call serves the request and the function list is the schema's; false = one edit of the list so no listed function can honour it	CC BY-SA 4.0	https://github.com/google-research-datasets/dstc8-schema-guided-dialogue
Schema-Guided Dialogue (DSTC8, TRAIN split; required arguments filled, corrected or referred-back values)	35	code: as the row above, only turns where a value was corrected, carried from assistant text or a function response, or referred back to	CC BY-SA 4.0	https://github.com/google-research-datasets/dstc8-schema-guided-dialogue
SQuAD 2.0 (TRAIN split, sentence level)	33	derived by code from the human answer spans (the sentence holds the answer or not)	CC BY-SA 4.0	https://rajpurkar.github.io/SQuAD-explorer/
QNLI (GLUE, TRAIN split; text from SQuAD v1.1; second selection)	29	human gold label (the sentence answers the question)	CC BY-SA 4.0 (SQuAD)	https://rajpurkar.github.io/SQuAD-explorer/
Schema-Guided Dialogue (DSTC8, TRAIN split; real dialogues, requested option removed by code)	24	code: true = the annotated next call honours the slot the user just stated; false = that option was taken out of the listed functions	CC BY-SA 4.0	https://github.com/google-research-datasets/dstc8-schema-guided-dialogue
SST-5 (Stanford Sentiment Treebank, TRAIN split)	12	human 5-class sentiment label	no open licence is named by the original release; used under the original authors' terms	https://nlp.stanford.edu/sentiment/
Social IQa (TRAIN split)	12	human gold answer; rows an AI reviewer marked wrong or unclear were removed	CC BY 4.0	https://allenai.org/data/socialiqa
iSarcasmEval task A (TRAIN split)	12	the tweet author's own sarcasm label	MIT	https://github.com/iabufarha/iSarcasmEval
Items written on our own GPU by the local open model microsoft/phi-4 (MIT); a public TRAIN row served as a style example only	11	the writer model's own answer; kept only after a blind check by local open models and a sealed read by an AI reviewer	text written by us; no dataset text reused	https://huggingface.co/microsoft/phi-4
PIQA (TRAIN split)	9	human gold label (the shown method is the correct or the wrong solution of the goal); rows an AI reviewer marked wrong or unclear were removed	AFL-3.0	https://yonatanbisk.com/piqa/
Items written on our own GPU by the local open model mistralai/Mistral-Nemo-Instruct-2407 (Apache-2.0); a public TRAIN row served as a style example only	7	the writer model's own answer; kept only after a blind check by local open models and a sealed read by an AI reviewer	text written by us; no dataset text reused	https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407
RTE (GLUE, TRAIN split)	6	human gold label (entailment / not entailment)	no open licence is named by the original release; used under the original authors' terms (PASCAL RTE challenges)	https://gluebenchmark.com/
MultiNLI (GLUE, TRAIN split; yes/no form)	6	human gold label (entailment = yes, neutral / contradiction = no); only rows an AI reviewer read and found correct	OANC licence; fiction parts CC BY-SA 3.0 / CC BY 3.0	https://cims.nyu.edu/~sbowman/multinli/
STS Benchmark (TRAIN split, captions genre)	6	human 0-5 similarity mean mapped to the nearest of 5 levels; rows near a level edge dropped	no open licence is named by the original release; used under the original authors' terms (mixed sources; the mirror card says 'unknown')	https://huggingface.co/datasets/mteb/stsbenchmark-sts
SNLI (TRAIN split; rows where all 5 annotators agree)	4	human label given by all 5 annotators; only rows an AI reviewer read and found correct	CC BY-SA 4.0	https://nlp.stanford.edu/projects/snli/
MultiNLI (GLUE, TRAIN split; second selection)	4	human gold label (3-way); only rows an AI reviewer read and found correct	OANC licence; fiction parts CC BY-SA 3.0 / CC BY 3.0	https://cims.nyu.edu/~sbowman/multinli/
PIQA (TRAIN split; second selection)	3	human gold label (the shown method is the correct or the wrong solution of the goal); only rows an AI reviewer read and found correct	AFL-3.0	https://yonatanbisk.com/piqa/
MultiNLI (GLUE, TRAIN split)	2	human gold label (3-way); only rows an AI reviewer read and found correct	OANC licence; fiction parts CC BY-SA 3.0 / CC BY 3.0	https://cims.nyu.edu/~sbowman/multinli/
SNLI (TRAIN split)	2	human gold label; only rows an AI reviewer read and found correct	CC BY-SA 4.0	https://nlp.stanford.edu/projects/snli/
our code generator: retrieval score items on invented passages	51	re-derived by code from the rendered passage; one hand audit of a sample	our own code	-
our code generator: reasoning score items (ASDiv-style word problems)	12	computed by the generator from its item spec and re-counted independently	our own code	-

Licence note. 151 of the 866 rows come from 4 datasets whose original release names no open licence (20 Newsgroups (TRAIN split), SST-5 (Stanford Sentiment Treebank, TRAIN split), RTE (GLUE, TRAIN split), STS Benchmark (TRAIN split, captions genre)). We used their TRAIN splits for training only, under the original authors' terms; this repository contains model weights only, no dataset text. Check those datasets' terms before you reuse this body for anything they may restrict.

Filters on every row: no exam split file was used for training (every row comes from a TRAIN split, from our own generators, or was written locally), no Glaive data, no BIG-bench / MMLU. We ran our exam-split 8-gram guard on these exact 866 rows before commit: the only match was one function-description phrase in the schema text of one dialogue service (24 rows), and no conversation or other text matched. Our test-overlap rule found no row sharing text with our own test items. No two rows share the same text; 18 rows have their text and answer written by local open models (listed in the table above); no other row was written by an LLM item writer, no house or relay labels, no other miner's outputs as targets. Share-alike terms (CC BY-SA) apply to redistributing source text; this repository holds model weights only.

Checks we ran before commit (not an official score)

• Change from Qwen/Qwen3.5-4B (Zils relative change): 0.0182 (cap 0.15). Zils body digest 6a00dfc97ed8b4b18b0e8777b308971df5e1d0af73f0b7f5a26cab928aebd97d.
• Copy screen with the official soup_copies code (our seeds; the validator's seed is private): own share beyond the house bodies, the listed public models, the soup and the release 0.0499-0.0550 (house-mix line 0.05); largest cosine to any of them or to our other committed bodies 0.962 (line 0.999); largest share explained by earlier bodies 0.142 (line 0.30).
• Our own exam-style reads vs the start (same code, same items; not the Zils exam): on our own test (795 items, without 8 Language items we excluded; not the Zils exam), gain over the start: total +0.67; Reasoning +1.55, Language known part -0.30, Retrieval +0.36, Tools -0.37, Arts +2.05. Tools and Language fell slightly, and this body did not meet all of our own pre-set bars on that test.

Licence

Apache-2.0 (LICENSE). See NOTICE for the upstream works and their notices. Nothing here is legal advice; check the original datasets' terms for your use.