zils-lite-body-6

Repository: tensor-tailor/zils-lite-body-6.

A decision body for the Zils d3-lite soup pool (Bittensor subnet 127). It is in the d3-lite chassis layout (Qwen3.5-0.8B language and vision tensors, bf16). It is a fine-tune of the d3-lite pool's 6 November 2026 soup, as Zils's mining guide allows ("You may fine-tune the current soup"). It is not a re-upload of any public model.

Modified weights

The weights in this repository are modified. They are derived from the uniform average of 1lock/zils-lite-astrid-g1 @ac979f3 (apache-2.0) and bmars8218/zils-lite-body-2 @60b596b (apache-2.0), then fine-tuned as described below. Zils body digest of these weights: 0738f9c489404471fbba1eae5292972746df26fa400d2721d3c4bec2517dce25.

Method

• Starting weights: the uniform fp32 average of the two bodies above, built with Zils's own soup.build on the vllm-sr/d3-lite chassis at temperature 1.0, exactly as a round builds a soup. These bodies were the 6 November 2026 soup (Zils body digest of the start: df5a2e1be25a3d23494dd7d0e57d645376e7c062288ff30bf6067dd19833ce2c).
• Training: LoRA, rank 16, alpha 32, dropout 0.05, on the language layers 16-23 only (the top 8 of 24). Learning rate 5e-5, a linear warm-up over the first 50 steps times a cosine decay over the epoch; batch 16; seed 4343; one epoch (7,854 steps), final weights only; rows longer than 1,024 tokens skipped (training: 110,394 rows used, 2,161 skipped; regularisation: 15,262 used, 156 skipped). The LoRA is merged into the weights and stored in bf16. Every other tensor is identical to the start.
• Loss: forward KL to each training row's target option probabilities, and KL (weight 2) to the start's own precomputed option probabilities on the regularisation rows. No cross-entropy on labels is used (every training row has soft targets).
• Relative change of the language layers from Qwen/Qwen3.5-0.8B: 0.026 (Zils cap 0.15; the start: 0.020).

Training data (112,555 rows before the length limit)

Every row carries a target probability distribution over its options. The rows' labels are not used in the training loss; they were used only to choose each row group's teacher and to balance the labels (below).

• Distillation (30,113 questions). The targets are the option probabilities of vllm-sr/d3-ultra (Apache-2.0) from zils-ai/zils-relay-answers (Apache-2.0), with the question text rebuilt locally from the public sources that card pins (as for our earlier bodies).
• Rows from our earlier mixes (29,167): exam-format practice rows from our own generators and public train splits. Their targets are teacher option probabilities: for 10,780 of them, the mean of bmars8218/zils-lite-body-2 @60b596b and Charlie320/zils-lite-k3b @71d4ffe (per category, only where a teacher scored above the start on our dev set); for the others, the best teacher per row group (below).
• Public train splits (53,275 rows), the datasets' own labels converted to the exam's question format, each targeted by the best teacher for its group (below). Train splits only. The public Zils exam-authoring code reads other splits of six of these datasets (the validation splits of MultiNLI, QNLI, PIQA, SQuAD v2.0 and COPA, and the test split of Amazon Polarity); no row of those split files is used (see Filters and overlap). Rows (licence of the original dataset, as its authors state it): QNLI 5,696 (CC BY-SA 4.0, via SQuAD v1.1); PIQA 5,638 (Academic Free License 3.0); MultiNLI 5,068 (OANC licence; fiction part CC BY-SA 3.0 / CC BY 3.0); Amazon Polarity 4,658 (Apache-2.0); CoQA 4,544 (only its Wikipedia, literature (Project Gutenberg) and CNN stories: CC BY-SA 4.0 / Apache-2.0 per story source); SQuAD v2.0 3,790 (CC BY-SA 4.0); BoolQ 3,218 (CC BY-SA 3.0); Cosmos QA 3,204 (CC BY 4.0, allenai); SciNLI 3,203 (CC BY-SA 4.0); QASC 3,203 (CC BY 4.0); FEVER 3,177 (CC BY-SA 3.0); MathQA 2,794 (Apache-2.0); Fig-QA 1,435 (MIT); ROPES 1,382 (CC BY 4.0); MegaVeridicality 1,142 (MIT); DROP 435 (CC BY-SA 4.0); COPA 400 (BSD-2-Clause); PUBHEALTH 288 (MIT).
• Teachers (their option probabilities only; no teacher weights are in this body): for each row group (one public source, or one family of earlier rows), the teacher with the lowest Brier score against the TRUE labels on a sample of up to 200 of that group's training rows: the starting soup itself, dasLOL/zils-lite-skills-v5 @2c99205, 1lock/zils-lite-astrid-g1 @ac979f3, bmars8218/zils-lite-body-2 @60b596b, and our own tensor-tailor/zils-lite-body-5 and tensor-tailor/zils-lite-body-4 @f97d84e. dasLOL/zils-lite-skills-v5's card states public train splits including some (20 Newsgroups, Sarcasm Headlines) that we do not use ourselves; only its option probabilities on our own rows were used.
• Label balance: in every converted yes/no family the true and false rows were balanced, a converted score family without every level was dropped, and a converted choice family was balanced when one answer dominated (GSM8K and SIQA rows were dropped by these rules).

Regularisation (15,418 questions before the length limit): public-dataset questions whose target is the start's own probabilities.

Share-alike and copyleft terms apply to redistributing source text. This repository contains model weights only, no training text.

Filters and overlap

• Every row was checked against the whole Glaive function-calling file (every raw row, 8-word runs over state and content: 0 rows kept) and against the split files of the 81 sources the public Zils exam-authoring code reads (state and content: 0 rows kept). Rows sharing a non-template 8-word run with the same answer with our dev, held-out or fresh-instance sets were removed.
• Only datasets whose ORIGINAL licence is open (as their authors state it) were used. Rows from RTE, emotion, 20 Newsgroups and FOLIO were removed under our rules. The five datasets held back for our unseen-domain tests (LogiQA, ReClor, WiC, WikiQA, SST-5) are not used.
• No real-event (dated outcome) rows.

Inherited through the starting weights

The starting bodies were trained by others; their own cards describe their data.

• 1lock/zils-lite-astrid-g1 (@ac979f3): its card states a full fine-tune on exam-style items written with the public Zils exam-authoring code from training-split rows and kept after a blind check; see its card.
• bmars8218/zils-lite-body-2 (@60b596b): its card states a LoRA (rank 16) fine-tune of the 5 November soup with the official trainer, on relay soft answers (zils-ai/zils-relay-answers), generated rows with computed labels following the public exam seed recipes, and the start's own probabilities as a regulariser; see its card.

Our own measurements (not the Zils exam)

On our dev set of the 15 current exam families (299 items, Zils category weights) this body scores 58.94 against 59.18 for its start (-0.25); on our held-out v4 set its top-choice accuracy is 0.522 against 0.511 for the start. It was submitted on the owner's decision; it did not meet our own pre-registered submit line (+1.0 over the start).

Licence

All upstream weights (Qwen/Qwen3.5-0.8B, vllm-sr/d3-lite and the bodies inside the starting soup) and the starting bodies themselves (1lock/zils-lite-astrid-g1: apache-2.0, bmars8218/zils-lite-body-2: apache-2.0) are released under the licences their cards state; credit to their authors. This repository is licensed under Apache 2.0; see LICENSE and NOTICE. Nothing here is legal advice; please check the original datasets' terms for your use.