zils-mini-body-22: Reasoning- and Tools-weighted LoRA fine-tune of a uniform mix of four d3-mini house ingredients

A decision body for the Zils d3-mini soup pool (Bittensor subnet 127), in the d3-mini chassis layout (Qwen3.5-4B language and vision tensors, bf16). Its start is a uniform weight average, built with the official zils.soup.build, of four of the pool's five public house ingredients: vllm-sr/d3-mini @df99bf4c; perplexity-ai/pplx-decider-flash-v1.2 @9ff7cf98; torchcast-ai/torchcast-decision-4b @492bfb19; Maincode/Matilda-Decision-v1-Mini @39479a8a. Start body digest be40457ed0725abfe6a66d0e3cdb29bff6c206a71440d54382bbdca89c01a2fc (soup hash 20c02cae). Credit for the starting weights belongs to the authors of those models and their upstream works (see NOTICE). No other miner's body was merged; it is not a re-upload of any public model.

Method

• Training: LoRA rank 32 / alpha 64 / dropout 0.05 on the attention, Gated-DeltaNet AND MLP (gate / up / down) projections of all language layers, learning rate 5e-5, AdamW with weight decay 1.0 on the LoRA weights only, batch 8 x accumulation 2, warm-up 50 steps then cosine, 2 epochs, max 2,048 tokens, seed 4350; loss per row = cross-entropy on the row's label + 1.0 x Brier + 0.1 x KL(start || this model) on the same row. The KL term holds the model near its own start and is computed live from the start weights with the adapter off; it uses no teacher, no extra pin rows and no other miner's outputs. Merged into bf16. No weight interpolation. This body is the final save (the only one read and shipped, fixed before training).
• Target: Data weighted toward Reasoning (40% of the rows) and Tools (30%). The start is the fixed house mix above, not a soup.

Training data (2,001 rows, file sha256 1a55d23ed29d9d0f807c6d66ba8597b681afcedef07932e0b9c712ede2a03ecb)

A Reasoning- and Tools-weighted set: 40.1% of the rows are Reasoning, 29.9% Tools, 13.0% Language, 12.0% Retrieval and 5.0% Arts, so category shares do NOT follow the exam weights. Every label is the dataset's own label (human annotators, or the dataset's own prover / generator), derived by code from the dataset record, or computed by our own generator; no label was predicted by a model we ran, and no other miner's outputs were used. Selected with select_final.py v0.12 (86049220) from a 6,983-row set frozen for this start only (sha256 482e664f314ec7b40e6836147ae268dade2c41e0490245458e3cba2648155926); 2,001 rows after the category, family and generator limits.

CC BY-SA 4.0 note. 599 of the rows are built from the Schema-Guided Dialogue dataset (Google Research, CC BY-SA 4.0, https://github.com/google-research-datasets/dstc8-schema-guided-dialogue). They were used for training only; this repository holds model weights only, no dataset text. Check that dataset's terms before you reuse this body.

Difficulty rule: a row enters only if this start puts probability 0.30-0.80 on its correct answer, measured on the start body itself before training (all 2,001 rows checked: lowest 0.300, highest 0.800). Rows with proven labels come first, then rows from sources that passed our hand audit; nothing was padded. No family exceeds 20% of the rows (largest 17.0%); code-generator rows are at most 20% each and 47.0% together, under a limit set for this one feed.

category	choice	yes/no	score	total
reasoning	59	402	341	802
language	20	106	134	260
retrieval	114	12	114	240
tools	200	200	199	599
arts	33	34	33	100

source	rows	label	original licence	original release
RuleTaker (tasksource mirror)	239	the dataset's prover label under the stated closed-world assumption	Apache-2.0 (allenai/ruletaker repository)	https://github.com/allenai/ruletaker
logical-entailment (DeepMind; tasksource mirror)	222	the dataset's entailment label	Apache-2.0 (repository)	https://github.com/google-deepmind/logical-entailment-dataset
Schema-Guided Dialogue (DSTC8, TRAIN split; real dialogues, sibling function options made by code)	200	code: the arguments are the dialogue's annotated call; a schema check on every option passes only for the gold sibling	CC BY-SA 4.0	https://github.com/google-research-datasets/dstc8-schema-guided-dialogue
Schema-Guided Dialogue (DSTC8, TRAIN split; required arguments filled)	139	code: required slots of the active intent that are in the human-annotated dialogue state and whose value occurs verbatim in the conversation so far	CC BY-SA 4.0	https://github.com/google-research-datasets/dstc8-schema-guided-dialogue
Schema-Guided Dialogue (DSTC8, TRAIN split; real dialogues, function list edited by code)	88	code: true = the annotated next call serves the request and the function list is the schema's; false = one edit of the list so no listed function can honour it	CC BY-SA 4.0	https://github.com/google-research-datasets/dstc8-schema-guided-dialogue
Schema-Guided Dialogue (DSTC8, TRAIN split; real dialogues, requested option removed by code)	72	code: true = the annotated next call honours the slot the user just stated; false = that option was taken out of the listed functions	CC BY-SA 4.0	https://github.com/google-research-datasets/dstc8-schema-guided-dialogue
Schema-Guided Dialogue (DSTC8, TRAIN split; required arguments filled, corrected or referred-back values)	60	code: as the row above, only turns where a value was corrected, carried from assistant text or a function response, or referred back to	CC BY-SA 4.0	https://github.com/google-research-datasets/dstc8-schema-guided-dialogue
Schema-Guided Dialogue (DSTC8, TRAIN split; does the next turn make a call)	40	code: whether the annotated next system turn of the real dialogue makes a service call	CC BY-SA 4.0	https://github.com/google-research-datasets/dstc8-schema-guided-dialogue
our code generator: reasoning score items (ASDiv-style word problems)	341	computed by the generator from its item spec and re-counted independently	our own code	-
our code generator: language score items	117	computed by the generator from its item spec and re-counted independently	our own code	-
our code generator: retrieval choice items on invented passages (computed over all current records after later corrections)	114	re-derived from the rendered text by an independent parser	our own code	-
our code generator: language yes/no items (does one caption facet agree; pairs built to the facet spec)	106	set by the generator's facet spec (the pair is built to agree or differ); one hand audit of a sample	our own code	-
our code generator: retrieval score items on invented passages (with later corrections)	105	re-derived from the rendered text by an independent parser	our own code	-
our code generator: arts yes/no items (a post score rule stated in the item; label recounted by the generator)	34	computed by the generator's recount on the final post; one hand audit of a sample	our own code	-
our code generator: arts choice items (which listed word a flip rule moves; label re-checked on the final post)	33	set by the generator and re-checked on the final post; one hand audit of a sample	our own code	-
our code generator: arts score items	25	computed by the generator from its item spec and re-counted independently	our own code	-
our code generator: language choice items (the one agreeing caption facet; pairs built to the facet spec)	20	set by the generator's facet spec (the pair is built to agree on one facet); one hand audit of a sample	our own code	-
our code generator: language score items, captions reworded by a local open model (Mistral Nemo)	17	set by the generator's facet spec; every kept phrase re-checked by code on the reworded text	our own code	-
our code generator: retrieval yes/no items on invented passages (the place is named through a second fact)	11	re-derived by code from the rendered passage; one hand audit of a sample	our own code	-
our code generator: retrieval score items on invented passages	9	re-derived by code from the rendered passage; one hand audit of a sample	our own code	-
our code generator: arts score items, post text reworded by a local open model (Mistral Nemo)	8	computed by the generator from its item spec and re-counted independently	our own code	-
our code generator: retrieval yes/no items on invented passages	1	re-derived by code from the rendered passage; one hand audit of a sample	our own code	-

Filters on every row: no exam split file was used for training (every row comes from a TRAIN split or from our own generators), no Glaive data, no BIG-bench / MMLU, no real-event rows. We ran our exam-split 8-gram guard on these exact 2,001 rows before commit: the only match was one function-description phrase in the schema text of one dialogue service (35 rows), and no conversation or other text matched. Our test-overlap rule found 1 row sharing an 8-word run with one of our own test items, which affects only our own read, not the exam. No two rows share the same text; 25 rows have text worded by a local open model from code-made specs (label set by code and re-verified on the final text); no other row was written by an LLM item writer, no house or relay labels, no other miner's outputs as targets. Share-alike terms (CC BY-SA) apply to redistributing source text; this repository holds model weights only.

Checks we ran before commit (not an official score)

• Change from Qwen/Qwen3.5-4B (Zils relative change): 0.0182 (cap 0.15). Zils body digest 0bc923b4e9d2570305943f19ac1164652a211d05b5c15860185ec6ae01968577.
• Copy screen with the official soup_copies code (our seeds; the validator's seed is private): own share beyond the house bodies, the listed public models, the soup and the release 0.0572-0.0615 (house-mix line 0.05); largest cosine to any of them or to our other committed bodies 0.949 (line 0.999); largest share explained by earlier bodies 0.053 (line 0.30).
• Our own exam-style reads vs the start (same code, same items; not the Zils exam): on our own test (795 items, without 8 Language items we excluded; not the Zils exam), gain over the start: total +0.57; Reasoning -2.49, Language known part +7.23, Retrieval -0.49, Tools +2.49, Arts +0.01. Reasoning and Retrieval fell, and this body did not meet all of our own pre-set bars on that test.

Licence

Apache-2.0 (LICENSE). See NOTICE for the upstream works and their notices. Nothing here is legal advice; check the original datasets' terms for your use.