zils-lite-body-4

Repository: tensor-tailor/zils-lite-body-4.

A decision body for the Zils d3-lite soup pool (Bittensor subnet 127). It is in the d3-lite chassis layout (Qwen3.5-0.8B language and vision tensors, bf16). It is a fine-tune of the d3-lite pool's 5 November 2026 soup, as Zils's mining guide allows ("You may fine-tune the current soup"). It is not a re-upload of any public model.

Modified weights

The weights in this repository are modified. They are derived from the uniform average of thermopylae-777/zils-lite-body-2 @618946d (our own earlier body) and dasLOL/zils-lite-skills-v5 @2c99205 (another miner's body, Apache-2.0), then fine-tuned as described below. The same notice is stored in the safetensors metadata (modification_notice).

Method

• Starting weights: the uniform fp32 average of the two bodies above, built with Zils's own soup.build on the vllm-sr/d3-lite chassis at temperature 1.0, exactly as a round builds a soup. These two bodies were the 5 November 2026 soup.
• Training: LoRA, rank 16, alpha 32, on the language layers 16-23 only (the top 8 of 24). Learning rate 1e-4, a linear warm-up over the first 50 steps times a cosine decay over the epoch; batch 16; seed 1107; rows longer than 1,024 tokens skipped (training: 78,914 rows used, 2,508 skipped; regularisation: 15,565 used, 293 skipped). The LoRA is merged into the weights and stored in bf16. Every other tensor is identical to the start.
• Loss: cross-entropy on the labelled rows; forward KL to the teacher's option probabilities on the distillation rows; KL to the start's own precomputed option probabilities on the regularisation rows (labels in the stored rows are ignored there).
• This body is the final weights at the end of the single epoch (5,905 steps). No checkpoint was chosen by score.
• Relative change of the language layers from Qwen/Qwen3.5-0.8B: 0.041 (Zils cap 0.15).

Training data (81,422 rows before the length limit)

• Distillation (17,324 questions). The targets are the option probabilities of vllm-sr/d3-ultra (Apache-2.0) from zils-ai/zils-relay-answers (Apache-2.0). The question text was rebuilt locally from the public sources that card pins: 14,783 via tasksource, 2,259 from Hugging Face datasets and 282 BFCL tool-selection rows. Largest sources: sem_eval_2010_task_8 2,512; multilingual NLI 1,784; twitter-financial-news-sentiment 1,314; babi_nli 1,206; ARC-Easy 637; HellaSwag 635; WinoGrande 630; ARC-Challenge 357; HelpSteer (three versions) 855.
  • Licences of the source texts, as the relay card records them (rows): MIT 3,366; CC0 3,347; CC BY 4.0 3,079; Apache-2.0 2,768; CC BY-SA 4.0 2,152; BSD 1,270; AFL-3.0 623; GPL 303; CC BY-SA 3.0 190; ODC-By 65; ODbL 6; other 155. Splits as the relay card records them; not re-checked for this card.
• Generated questions with computed answers (20,398), from our own generators: tool choice, no-tool and argument scoring, tool-call policy, coverage, counting, word forms, rule-book scoring and syllables.
• Syllables (11,737), our own generator over the CMU Pronouncing Dictionary (BSD-2-Clause-style licence).
• Word problems (12,000), our own generator, own work.
• Intent scope (6,246), our own generator, own work.
• Procedural decisions (13,717) from tasksource/tasksource-jev-typed-decisions @8df2e5c9 (Apache-2.0), train split, with the dataset's own labels.
• Regularisation (15,858 questions before the length limit): public-dataset questions whose target is the start's own probabilities.
• Share-alike and copyleft terms apply to redistributing source text. This repository contains model weights only, no training text.

Filters and overlap

• Every row was checked against the published Zils seed sets v1, v2 and v3 (v4 was not checked) and our dev and held-out sets: 0 exact matches on the normalised scenario + question, and 0 scenario matches with our held-out sets. 25 distillation rows that equal a regularisation or probe row were dropped.
• Not used: 20 Newsgroups, Sarcasm Headlines, PIQA, SVAMP, MMLU and VitaminC (sources or held-out sets of the public Zils exam-authoring code), and no real-event (dated outcome) rows at all.
• The generated rows reuse public Zils seed instruction templates with new instances; no seed item is used.

Inherited through the starting weights

The two starting bodies were trained by others and by us earlier; their own cards describe their data.

• thermopylae-777/zils-lite-body-2: trained from the uniform average of the three d3-lite house bodies (vllm-sr/d3-lite, Maincode/Matilda-Decision-v1-Edge, zils-ai/jiwo-0.8b-full) with relay distillation and generated questions; see its card.
• dasLOL/zils-lite-skills-v5: its card states six full-parameter fine-tunes from the same house average, averaged, on generated skills and public train splits including some (20 Newsgroups, Sarcasm Headlines) that we do not use ourselves. We did not choose or check that data; see its card.

Our own measurements (not the Zils exam)

On our held-out sets, scored with Zils's open-source scoring code (soup_grade.evaluate, 5 calibration secrets):

set	this body	the start (the 5 Nov soup)
dev_v2 (3,274 items, both blocks)	47.05	45.90
dev_gold (787 known-family items, known-block score)	55.21	52.88
COMBO (dev_gold + the new-block half of dev_v2)	50.15	48.46

These are our measurements, not the subnet's results; the live exam uses a private item bank.

Licence

All upstream weights (Qwen/Qwen3.5-0.8B, vllm-sr/d3-lite, Maincode/Matilda-Decision-v1-Edge, zils-ai/jiwo-0.8b-full and the two starting bodies) are Apache-2.0; credit to their authors. This repository is licensed under Apache 2.0; see LICENSE and NOTICE. Nothing here is legal advice; please check the original datasets' terms for your use.