zils-lite-body-3

A decision body for the Zils d3-lite soup pool (Bittensor subnet 127). It is in the d3-lite chassis layout (Qwen3.5-0.8B language and vision tensors, bf16). It is a merged-LoRA fine-tune of our earlier, unpublished body B2, which itself starts from the uniform average of the three d3-lite house bodies vllm-sr/d3-lite, Maincode/Matilda-Decision-v1-Edge and zils-ai/jiwo-0.8b-full. It is not a re-upload of any public model. B2 and our other published body, thermopylae-777/zils-lite-body-2 (B1), were trained independently from the same house average, with substantially overlapping training data and templates. B1's weights were not used for B2 or for this body.

Method

• Starting weights: B2 (not published separately). B2 = the uniform fp32 average of the three house bodies above, plus one epoch of merged LoRA (rank 16) on the "B2 data" listed at the end of this card.
• Training: LoRA, rank 16, alpha 32, dropout 0.05, learning rate 1e-4 (linear warm-up over 50 steps, then cosine decay over the epoch), batch 8 with gradient accumulation 2, seed 4343. Targets: q/k/v/o in the 6 full-attention layers, and in_proj_qkv/z/a/b and out_proj in the 18 Gated-DeltaNet layers (114 matrices). The LoRA is merged into the weights and stored in bf16. No other tensor was trained: every other tensor is identical to B2's.
• This body is the final weights at the end of the single epoch (3,339 steps, in a seeded random batch order).
• Loss:
  • cross-entropy on the labelled rows (dated outcome questions, GSM8K, generated rows);
  • forward KL to the teacher's option probabilities on the distillation rows;
  • KL to B2's own precomputed option probabilities on the regularisation rows. The stored rows carry labels, but the pin KL ignores them (no label cross-entropy). This keeps B2's behaviour where the new data adds no signal.
• Length limit: rows longer than 2,048 tokens were skipped, not truncated: 80 training rows and 2 regularisation rows. That leaves 38,601 training rows and 14,817 regularisation rows in the epoch.
• Relative change of the language layers from Qwen/Qwen3.5-0.8B: 0.044 (Zils cap 0.15).

Training data (38,681 rows before the length limit)

• Dated outcome questions (8,604). Asked 2026-10-08 to 2026-10-17. Every one resolved before 2026-10-20 06:00 UTC (the latest at 05:59:23); nothing resolved later was used. They are rendered in the public Zils Current/Outcomes item format with Zils's open-source code.
  • Wikipedia edit outcomes in 9 language editions, 4,952 rows: en 1,001, es 890, pt 550, fr 504, it 477, ru 445, de 413, ja 352, zh 320. The text is from Wikipedia, CC BY-SA 4.0, with attribution via each edit's page history.
  • Wikidata edit outcomes, 1,152 rows (CC0 1.0).
  • arXiv listing categories from arXiv metadata (title, abstract, categories), 2,500 rows (CC0 1.0): cs 1,215, physics 577, math 520, other 188. Thank you to arXiv for use of its open access interoperability.
  • Revert labels were re-checked over the 72 hours after each edit (MediaWiki revert tags plus a content-restore check). This matches the 2-3 day age at which Zils's collector reads them.
• Reasoning and NLI distillation (18,754 questions). The targets are the option probabilities of vllm-sr/d3-ultra (Apache-2.0) from zils-ai/zils-relay-answers (Apache-2.0). The question text was rebuilt locally from the public sources that card pins:
  • babi_nli 3,341; WinoGrande 3,187; ARC-Easy 2,035; HellaSwag 1,756; defeasible-nli 957; ARC-Challenge 938;
  • probability_words_nli 638; 400 each from snli, propsegment, lonli, FOL-nli, SpaceNLI, help-nli, resnli, monotonicity-entailment and UNLI;
  • 319 each from logical-entailment, numer_sense, ruletaker and temporal-nli; multilingual 260; nan-nli 249; ethics 162; syntactic-augmentation-nli 142; naturallogic 130; causal-news 83.
  • Splits: 625 of these rows come from validation splits (ARC-Easy 419, ARC-Challenge 206). Every other row comes from a train split (the tasksource rows, WinoGrande, HellaSwag and the other ARC rows).
• Tool questions (8,329).
  • 7,500 generated rows from our own generators, with computed answers: tool choice, no-tool and argument scoring, 1,500 each, plus tool-call policy 1,000 each.
  • 829 BFCL tool-selection distillation rows (Apache-2.0, from the relay set above). They come from BFCL's benchmark files, which have no train/test split.
• Word forms (994). Generated inflection questions, our own generator, with computed answers.
• Labelled reasoning (2,000). GSM8K train split (MIT).
• Licences of the distillation source texts (19,583 rows including BFCL), as the relay card records them:
  • Apache-2.0 5,903; CC BY-SA 4.0 3,622; MIT 3,579; BSD 3,341; CC BY 4.0 800;
  • Apache-2.0 with MIT 638; Apache-2.0 with CC BY-SA 4.0 638; CC with CC BY-SA 4.0 400; Apache-2.0 319;
  • CC BY-SA 4.0 260; CC0 83.
• Regularisation (14,819 questions; labels present in the stored rows but ignored by the pin KL, no label cross-entropy). Public-dataset questions from 102 sources: 14,536 via tasksource and 283 from Hugging Face datasets. Every row is in a source classified for commercial use. Licences as the source records give them: CC BY 4.0 4,905; Apache-2.0 4,721; MIT 3,715; CC0 1,161; ODC-By 215; Apache-2.0 with MIT 61; BSD 41. The target is B2's own probabilities.
• Share-alike terms (CC BY-SA) apply to redistributing source text. This repository contains model weights only, no training text.

What was removed before training, and why

The starting mix had 43,158 rows. We removed:

• 2,478 rows generated from PIQA, 80 SVAMP rows and 1,237 MMLU distillation rows: these datasets are sources of items in the public Zils exam-authoring code;
• 400 VitaminC distillation rows: a dataset the exam holds out;
• 282 dated outcome rows whose 72-hour label window reached past 2026-10-20 06:00 UTC.

We then checked every row against every published Zils seed case and our own dev sets: exact matches on the normalised scenario text, and on scenario + question + label. Both checks found 0 matches.

Overlap with Zils material

• No Zils dev item and no published Zils seed item was used in this training run.
• The generated known-family rows (tools, word forms) reuse public Zils seed instruction templates with new instances; no seed item is used.
• GSM8K train is a public split that the earliest published Zils seed families also rewrite questions from. We used the original GSM8K train rows, not the seed rewrites.

B2 data (inherited through the starting weights)

B2 was trained in an earlier run, before the data filter described above. Its data is listed as it was used.

B2 was trained from the uniform average of the three d3-lite house bodies on 102,640 training rows and 15,783 regularisation rows. It used one epoch of LoRA, rank 16, on the attention and Gated-DeltaNet projections, merged.

• Distillation (77,241 questions). The targets are the option probabilities of vllm-sr/d3-ultra (Apache-2.0) from zils-ai/zils-relay-answers (Apache-2.0); the question text was rebuilt locally from the public sources that card pins.
  • tasksource: 57,062 rows from 249 sources of tasksource/tasksource-jev-typed-decisions, all from train splits, in rows tasksource classifies for commercial use.
  • Benchmark sets: WinoGrande (Apache-2.0) 4,000; BANKING77 (CC BY 4.0) 4,000; iSarcasmEval-A (MIT) 3,215; ARC-Easy (CC BY-SA 4.0) 2,555; HellaSwag (MIT) 2,204; MMLU (MIT) 1,553; ARC-Challenge (CC BY-SA 4.0) 1,178; BFCL tool selection (Apache-2.0) 829; iSarcasmEval-C (MIT) 645.
  • 2,321 of these rows come from validation and dev splits: MMLU validation 1,304 and dev 249, ARC-Easy validation 514, ARC-Challenge validation 254. The BFCL rows come from its benchmark files. Every other row comes from a train split.
  • Licences of the rebuilt source texts, as the relay card records them (rows): CC BY 3.0/4.0 21,155; MIT 15,745; Apache-2.0 14,829; CC BY-SA 3.0/4.0 12,049; BSD 4,582; CC0 / public domain 4,256; AFL-3.0 2,010; GPL 1,855; ODC-By 400; CDLA-Permissive-1.0 / other 328; ODbL 32.
• Generated questions (20,399, cross-entropy) from our own generators (counting, coverage, tool choice and tool-call policy, word forms, syllables, rule-based scoring), with computed answers.
• Exam-format questions (5,000, cross-entropy) written in the Zils seed formats from the train splits of CLINC150 (CC BY 3.0, 2,500) and BANKING77 (CC BY 4.0, 2,500), with their gold labels.
• Regularisation (15,783 questions; labels present in the stored rows but ignored, no label cross-entropy): KL to precomputed reference probabilities on public-dataset questions from 44 families. For each family the reference is the starting average's probabilities, or a house body's where it calibrated better on held-out probe rows.
• Overlap with Zils material, as measured for B2:
  • the relay rows were filtered against every public Zils seed case and our dev sets (exact match and shared word 8-grams; 280 rows dropped);
  • one generated row (tools-score-policy) has the same scenario, question and label as a published seed fixture (tools-known-score-k6); it was found after training;
  • 18 iSarcasmEval rows (15 iSarcasmEval-A, 3 iSarcasmEval-C) reuse the original source items that 15 published Zils seed cases cite. The seed cases are structural rewrites of them, not the same text. These were found after training; exact-match and 8-gram filters do not catch such a shared source;
  • shared source splits with the exam's known families: BANKING77 train (4,000 distillation + 2,500 exam-format rows), CLINC150 train (2,500 exam-format rows), iSarcasmEval train (3,860 distillation rows), poem_sentiment train (400 distillation rows); and, as regularisation inputs (labels stored but ignored), GSM8K train (520), BoolQ train (400), When2Call train (365), GLUE STS-B train (1,040) and GLUE RTE/MNLI train (520).

Our own measurements (not the Zils exam)

On our own held-out sets, built from public sources and scored with Zils's open-source scoring code (soup_grade.evaluate, 5 calibration secrets):

set	this body	B2 (its start)
dev_v2 (3,274 items, both blocks)	43.98	44.21
dev_gold (787 known-family items, known-block score)	54.01	53.36

These are our measurements, not the subnet's results; the live exam uses a private item bank.

Licence

All upstream weights (Qwen/Qwen3.5-0.8B and the three house bodies) are Apache-2.0; credit to their authors. This repository is licensed under Apache 2.0; see LICENSE. Nothing here is legal advice; please check the original datasets' terms for your use.