zils-mini-body-20: class-balanced human-label LoRA fine-tune of the current Zils MINI soup
A decision body for the Zils d3-mini soup pool (Bittensor subnet 127), in the d3-mini chassis layout (Qwen3.5-4B language and vision
tensors, bf16). It is a fine-tune of the current MINI soup: the uniform weight average (official zils.soup.build: bf16-canonical, fp32
mean, chassis vision tower) of qhoenix/zils-mini-decision-v2 @ecb9cadb39a2f84cefd2f27a755ec12095b62933 and Charlie320/zils-mini-s1
@53eb3ffa768c1ad1eace7acbfdb3ebb9674cefe6 (soup body digest df87aaa9…), as Zils's mining guide allows ("You may fine-tune the current
soup"). Credit for the starting weights belongs to the authors of those two bodies and their upstream works (see NOTICE). No other miner's
body was merged; it is not a re-upload of any public model.
Method
- Training: LoRA rank 32 / alpha 64 / dropout 0.05 on the attention, Gated-DeltaNet AND MLP (gate / up / down) projections of all language layers (248 matrices), learning rate 5e-5, AdamW with weight decay 1.0 on the LoRA weights only, batch 8 x accumulation 2, warm-up 50 steps then cosine, 2 epochs, max 2,048 tokens, seed 4349; loss per row = cross-entropy on the row's label + 1.0 x Brier. No KL / distillation term, no reference or teacher probabilities. Merged into bf16. No weight interpolation. This body is the final save (no picking among saves).
Training data (2,001 rows, file sha256 4224537e55dfcf57724fae9f3e8d30cc5abee60500fcb0316a9f19a24e0aa951)
A class-balanced set in the exam's 15 "known" families (5 categories x choice / yes-no / score; category shares follow the published exam weights where a cell had enough rows). Every label is the dataset's own label (human annotators, or the dataset's own prover / generator), derived by code from the dataset record, or computed by our own generator; no label was predicted by a model we ran, and no other miner's outputs were used. Half B of a 4,007-row set frozen with MINI-soup difficulty and the band rule below (two disjoint, label-balanced halves; tool balM_freeze.py v0.2 3458b9d9); the other half trained a separate body of ours. These rows were also used in an earlier attention-only LoRA run of ours that was not shipped; this body adds LoRA on the MLP projections. Difficulty rule per cell (to avoid the label-noise tail of a pure hardest-first pick): rows with a human label were taken in a fixed hash order from those on which the MINI soup puts probability 0.30-0.90 on the correct answer (rows at or above 0.90 only to fill a short cell, the hardest of them first); code-labelled and proven rows hardest-first; 50% cap per source except where noted in the freeze receipt.
| category | choice | yes/no | score | total |
|---|---|---|---|---|
| reasoning | 215 | 215 | 215 | 645 |
| language | 215 | 214 | 236 | 665 |
| retrieval | 167 | 166 | 0 | 333 |
| tools | 87 | 0 | 0 | 87 |
| arts | 71 | 75 | 125 | 271 |
| source | rows | label | original licence | original release |
|---|---|---|---|---|
| PIQA (TRAIN split) | 215 | human gold label: the shown method is the correct or the wrong solution of the goal | AFL-3.0 | https://yonatanbisk.com/piqa/ |
| MultiNLI (GLUE, TRAIN split) | 212 | human gold label (3-way; in yes/no rows entailment = yes, neutral / contradiction = no) | OANC licence; fiction parts CC BY-SA 3.0 / CC BY 3.0 | https://cims.nyu.edu/~sbowman/multinli/ |
| 20 Newsgroups (SetFit mirror, TRAIN split) | 167 | the newsgroup the post was sent to | no open licence is named by the original release; used under the original authors' terms | http://qwone.com/~jason/20Newsgroups/ |
| STS Benchmark (TRAIN split, captions genre) | 161 | human 0-5 similarity mean mapped to the nearest of 5 levels; rows near a level edge dropped | no open licence is named by the original release; used under the original authors' terms (mixed sources; the mirror card says 'unknown') | https://huggingface.co/datasets/mteb/stsbenchmark-sts |
| Social IQa (TRAIN split) | 133 | human gold answer | CC BY 4.0 | https://allenai.org/data/socialiqa |
| RTE (GLUE, TRAIN split) | 114 | human gold label (entailment / not entailment) | no open licence is named by the original release; used under the original authors' terms (PASCAL RTE challenges) | https://gluebenchmark.com/ |
| SNLI (TRAIN split) | 103 | human gold label | CC BY-SA 4.0 | https://nlp.stanford.edu/projects/snli/ |
| SQuAD 2.0 (TRAIN split, sentence level) | 91 | derived by code from the human answer spans (the sentence holds the answer or not) | CC BY-SA 4.0 | https://rajpurkar.github.io/SQuAD-explorer/ |
| SST-5 (Stanford Sentiment Treebank, TRAIN split) | 84 | human 5-class sentiment label | no open licence is named by the original release; used under the original authors' terms | https://nlp.stanford.edu/sentiment/ |
| iSarcasmEval task A (TRAIN split) | 75 | the tweet author's own sarcasm label | MIT | https://github.com/iabufarha/iSarcasmEval |
| QNLI (GLUE, TRAIN split; text from SQuAD v1.1) | 75 | human gold label (the sentence answers the question) | CC BY-SA 4.0 (SQuAD) | https://rajpurkar.github.io/SQuAD-explorer/ |
| RuleTaker (tasksource mirror) | 70 | the dataset's prover label under the stated closed-world assumption | Apache-2.0 (allenai/ruletaker repository) | https://github.com/allenai/ruletaker |
| ToolACE @6bda777c | 52 | read off the record: the real call is the answer; distractors break a required argument (code-checked) | Apache-2.0 | https://huggingface.co/datasets/Team-ACE/ToolACE |
| GoEmotions (simplified, TRAIN split) | 40 | human label (rows with exactly one of the 6 emotion names) | Apache-2.0 (google-research repository) | https://github.com/google-research/google-research/tree/master/goemotions |
| Hermes function-calling v1 @dae3e1d2 (func-calling file; the authors' Glaive-derived glaive-function-calling-5k file excluded) | 35 | read off the record: the real call is the answer; distractors break a required argument (code-checked) | Apache-2.0 | https://huggingface.co/datasets/NousResearch/hermes-function-calling-v1 |
| TweetEval emotion (SemEval-2018 Task 1, TRAIN split) | 31 | human emotion label | no open licence is named by the original release; used under the original authors' terms (TweetEval: 'restrictions may apply to individual tasks'; Twitter terms) | https://github.com/cardiffnlp/tweeteval |
| Corr2Cause (tasksource mirror) | 12 | the dataset's generator label | MIT (repository) | https://github.com/causalNLP/corr2cause |
| our code generator: reasoning score items (ASDiv-style word problems) | 215 | computed by the generator from its item spec and re-counted independently | our own code | - |
| our code generator: language score items | 75 | computed by the generator from its item spec and re-counted independently | our own code | - |
| our code generator: arts score items | 41 | computed by the generator from its item spec and re-counted independently | our own code | - |
Licence note. 557 of the 2,001 rows come from 5 datasets whose original release names no open licence (20 Newsgroups (SetFit mirror, TRAIN split), STS Benchmark (TRAIN split, captions genre), RTE (GLUE, TRAIN split), SST-5 (Stanford Sentiment Treebank, TRAIN split), TweetEval emotion (SemEval-2018 Task 1, TRAIN split)). We used their TRAIN splits for training only, under the original authors' terms; this repository contains model weights only, no dataset text. Check those datasets' terms before you reuse this body for anything they may restrict.
Filters on every row: no exam split files (the validation / test splits the Zils exam-authoring code reads were never opened), no Glaive data, no BIG-bench / MMLU, no real-event rows; our exam-split 8-gram guard and test-overlap rule (both directions) removed every hit; near-duplicates removed; no rows written by an LLM item writer, no house or relay labels, no other miner's outputs as targets. Share-alike terms (CC BY-SA) apply to redistributing source text; this repository holds model weights only.
Checks we ran before commit (not an official score)
- Change from Qwen/Qwen3.5-4B (Zils relative change): 0.01273 (cap 0.15). Zils body digest 898895cd2f5bc7c87e48a4b139bb3ca828ceef7c22ee603cb4b28e3829333a4e.
- Copy screen with the official
soup_copiescode (our seeds; the validator's seed is private): own share beyond the house bodies, the listed public models, the soup and the release 0.139-0.150 (house-mix line 0.05); largest cosine to any of them or to our other committed bodies 0.928 (line 0.999); largest share explained by earlier bodies 0.083 (line 0.30). - Our own exam-style reads vs the start (same code, same items; not the Zils exam): no score result is claimed: committed without our read (quick submit).
Licence
Apache-2.0 (LICENSE). See NOTICE for the upstream works and their notices. Nothing here is legal advice; check the original datasets' terms for your use.
- Downloads last month
- 14
Model tree for tensor-tailor/zils-mini-body-20
Base model
Qwen/Qwen3.5-4B-Base