zils-mini-body-15: class-balanced human-label LoRA fine-tune of the current Zils MINI soup

A decision body for the Zils d3-mini soup pool (Bittensor subnet 127), in the d3-mini chassis layout (Qwen3.5-4B language and vision tensors, bf16). It is a fine-tune of the current MINI soup: the uniform weight average (official zils.soup.build: bf16-canonical, fp32 mean, chassis vision tower) of qhoenix/zils-mini-decision-v2 @ecb9cadb39a2f84cefd2f27a755ec12095b62933 and Charlie320/zils-mini-s1 @53eb3ffa768c1ad1eace7acbfdb3ebb9674cefe6 (soup body digest df87aaa9…), as Zils's mining guide allows ("You may fine-tune the current soup"). Credit for the starting weights belongs to the authors of those two bodies and their upstream works (see NOTICE). No other miner's body was merged; it is not a re-upload of any public model.

Method

• Training: LoRA rank 32 / alpha 64 / dropout 0.05 on the attention, Gated-DeltaNet AND MLP (gate / up / down) projections of all language layers (248 matrices), learning rate 5e-5, AdamW with weight decay 1.0 on the LoRA weights only, batch 8 x accumulation 2, warm-up 50 steps then cosine, 2 epochs, max 2,048 tokens, seed 4348; loss per row = cross-entropy on the row's label + 1.0 x Brier. No KL / distillation term, no reference or teacher probabilities. Merged into bf16. No weight interpolation. This body is the final save (no picking among saves).

Training data (1,103 rows, file sha256 f45e84f617d3d920b26b1325bb7bb4f3013b277751571c2f8ea9bea50840b36c)

A label-balanced set covering 7 of the exam's 15 "known" family cells (5 categories x choice / yes-no / score); the empty cells are shown as 0 below, so category shares do NOT follow the exam weights. Every label is the dataset's own label (human annotators, or the dataset's own prover / generator), derived by code from the dataset record, or computed by our own generator; no label was predicted by a model we ran, and no other miner's outputs were used. This set is disjoint from the rows used by our earlier sibling bodies (every id of their two halves excluded; tool balM_freeze.py v0.2 3458b9d9 on the same pool, then the score cells removed because they held only some levels, a label shortcut). Difficulty rule per cell: rows with a human label were taken in a fixed hash order from those on which the MINI soup puts probability 0.30-0.90 on the correct answer, then (because the sibling bodies had already used most such rows) from rows at or above 0.90 to fill each cell, so many rows here are ones the soup already answers well; code-labelled and proven rows hardest-first; 50% cap per source except where noted in the freeze receipt.

category	choice	yes/no	score	total
reasoning	215	214	0	429
language	214	212	0	426
retrieval	43	164	0	207
tools	0	0	0	0
arts	41	0	0	41

source	rows	label	original licence	original release
PIQA (TRAIN split)	214	human gold label: the shown method is the correct or the wrong solution of the goal	AFL-3.0	https://yonatanbisk.com/piqa/
MultiNLI (GLUE, TRAIN split)	212	human gold label (3-way; in yes/no rows entailment = yes, neutral / contradiction = no)	OANC licence; fiction parts CC BY-SA 3.0 / CC BY 3.0	https://cims.nyu.edu/~sbowman/multinli/
Social IQa (TRAIN split)	108	human gold answer	CC BY 4.0	https://allenai.org/data/socialiqa
SNLI (TRAIN split)	107	human gold label	CC BY-SA 4.0	https://nlp.stanford.edu/projects/snli/
RTE (GLUE, TRAIN split)	107	human gold label (entailment / not entailment)	no open licence is named by the original release; used under the original authors' terms (PASCAL RTE challenges)	https://gluebenchmark.com/
QNLI (GLUE, TRAIN split; text from SQuAD v1.1)	83	human gold label (the sentence answers the question)	CC BY-SA 4.0 (SQuAD)	https://rajpurkar.github.io/SQuAD-explorer/
RuleTaker (tasksource mirror)	82	the dataset's prover label under the stated closed-world assumption	Apache-2.0 (allenai/ruletaker repository)	https://github.com/allenai/ruletaker
SQuAD 2.0 (TRAIN split, sentence level)	81	derived by code from the human answer spans (the sentence holds the answer or not)	CC BY-SA 4.0	https://rajpurkar.github.io/SQuAD-explorer/
20 Newsgroups (SetFit mirror, TRAIN split)	43	the newsgroup the post was sent to	no open licence is named by the original release; used under the original authors' terms	http://qwone.com/~jason/20Newsgroups/
GoEmotions (simplified, TRAIN split)	41	human label (rows with exactly one of the 6 emotion names)	Apache-2.0 (google-research repository)	https://github.com/google-research/google-research/tree/master/goemotions
Corr2Cause (tasksource mirror)	25	the dataset's generator label	MIT (repository)	https://github.com/causalNLP/corr2cause

Licence note. 150 of the 1,103 rows come from 2 datasets whose original release names no open licence (RTE (GLUE, TRAIN split), 20 Newsgroups (SetFit mirror, TRAIN split)). We used their TRAIN splits for training only, under the original authors' terms; this repository contains model weights only, no dataset text. Check those datasets' terms before you reuse this body for anything they may restrict.

Filters on every row: no exam split files (the validation / test splits the Zils exam-authoring code reads were never opened), no Glaive data, no BIG-bench / MMLU, no real-event rows; our exam-split 8-gram guard and test-overlap rule (both directions) removed every hit; near-duplicates removed; no rows written by an LLM item writer, no house or relay labels, no other miner's outputs as targets. Share-alike terms (CC BY-SA) apply to redistributing source text; this repository holds model weights only.

Checks we ran before commit (not an official score)

• Change from Qwen/Qwen3.5-4B (Zils relative change): 0.01227 (cap 0.15). Zils body digest c1d7894633949b8aee853b9f8a20242652287e9a10216eebf9d4cfb3365542d4.
• Copy screen with the official soup_copies code (our seeds; the validator's seed is private): own share beyond the house bodies, the listed public models, the soup and the release 0.075-0.084 (house-mix line 0.05); largest cosine to any of them or to our other committed bodies 0.962 (line 0.999); largest share explained by earlier bodies 0.225 (line 0.30).
• Our own exam-style reads vs the start (same code, same items; not the Zils exam): predicted score +1.69 points overall; by category reasoning +1.5, language +5.0, retrieval -1.6, tools +0.1, arts +0.6. Retrieval fell.

Licence

Apache-2.0 (LICENSE). See NOTICE for the upstream works and their notices. Nothing here is legal advice; check the original datasets' terms for your use.