zils-mini-body-1

Repository: tensor-tailor/zils-mini-body-1.

A decision body for the Zils d3-mini soup pool (Bittensor subnet 127). It is in the d3-mini chassis layout (Qwen3.5-4B language and vision tensors, bf16). It is an average of two things: the uniform average of three d3-mini house bodies, and a merged-LoRA fine-tune of that same average (half each, see "This body is"). It is not a re-upload of any public model and it does not use the weights of any other miner's body.

Method

• Starting weights: the uniform fp32 average, over all chassis tensors, of torchcast-ai/torchcast-decision-4b @492bfb19 (Apache-2.0), Maincode/Matilda-Decision-v1-Mini @39479a8a (Apache-2.0) and perplexity-ai/pplx-decider-flash-v1.2 @9ff7cf98 (Apache-2.0). The chassis is zils-ai/d3-mini-4.5b-v1 @b54402a9 (Apache-2.0); parent Qwen/Qwen3.5-4B @851bf6e8 (Apache-2.0).
• Training: LoRA, rank 16, alpha 32, dropout 0.05, learning rate 1e-4 (linear warm-up over 50 steps, then cosine decay over the epoch), batch 8 with gradient accumulation 2, seed 4343. Targets: the attention and Gated-DeltaNet projections of all 32 language layers (152 matrices). The LoRA is merged into the weights and stored in bf16. No other tensor was trained.
• This body is: the uniform fp32 average, over all chassis tensors, of six entries: the three starting bodies above once each and the fine-tuned body (final weights after the single epoch, 3,089 steps) three times. So it is half the starting average and half the fine-tune, stored in bf16. The ratio was declared before any score of the fine-tune was seen. This body was chosen from six declared candidates (the final weights, the 0.75 and 0.5 checkpoints, two average-back ratios, and one earlier no-training average) by a fixed rule written before any score was seen: first the pre-screen eligibility checks, then the largest mean gain on our two primary dev sets among those that passed the pre-screen. A separate soup-round entry check (see "Our own soup-round check" below) then predicted this body would not join the round's soup; it is published here as a deliberate free try, not because it passed that entry check. The training run was restarted once from the start, with the same data, settings and seed, after the first attempt ran out of memory at its half-epoch checkpoint; nothing from the first attempt was used.
• Loss: cross-entropy on the labelled rows (dated outcome questions, GSM8K, generated rows); forward KL to the teacher's option probabilities on the distillation rows; KL to the starting average's own precomputed probabilities on the regularisation rows (the stored rows carry labels, but the pin KL ignores them).
• Length limit: rows longer than 2,048 tokens were skipped, not truncated: 80 training rows and 2 regularisation rows. That leaves 34,607 training rows and 14,817 regularisation rows.
• Relative change of the language tensors from Qwen/Qwen3.5-4B, as Zils computes it: 0.0244 (Zils cap 0.15).

Training data (34,687 rows before the length limit)

• Dated outcome questions (8,604). Asked 2026-10-08 to 2026-10-17. Every one resolved before 2026-10-20 06:00 UTC (the latest at 05:59:23); nothing resolved later was used. They are rendered in the public Zils Current/Outcomes item format with Zils's open-source code.
  • Wikipedia edit outcomes in 9 language editions, 4,952 rows: en 1,001, es 890, pt 550, fr 504, it 477, ru 445, de 413, ja 352, zh 320. The text is from Wikipedia, CC BY-SA 4.0, with attribution via each edit's page history.
  • Wikidata edit outcomes, 1,152 rows (CC0 1.0).
  • arXiv listing categories from arXiv metadata (title, abstract, categories), 2,500 rows (CC0 1.0): cs 1,215, physics 577, math 520, other 188. Thank you to arXiv for use of its open access interoperability.
  • Revert labels were re-checked over the 72 hours after each edit (MediaWiki revert tags plus a content-restore check).
• Reasoning and NLI distillation (18,754 questions). The targets are the option probabilities of vllm-sr/d3-ultra (Apache-2.0) from zils-ai/zils-relay-answers (Apache-2.0). The question text was rebuilt locally from the public sources that card pins:
  • babi_nli 3,341; WinoGrande 3,187; ARC-Easy 2,035; HellaSwag 1,756; defeasible-nli 957; ARC-Challenge 938;
  • probability_words_nli 638; 400 each from snli, propsegment, lonli, FOL-nli, SpaceNLI, help-nli, resnli, monotonicity-entailment and UNLI;
  • 319 each from logical-entailment, numer_sense, ruletaker and temporal-nli; multilingual 260; nan-nli 249; ethics 162; syntactic-augmentation-nli 142; naturallogic 130; causal-news 83.
  • Splits: 625 of these rows come from validation splits (ARC-Easy 419, ARC-Challenge 206). Every other row comes from a train split.
• Tool questions (5,329). 4,500 generated rows from our own generators, with computed answers (tool choice, no-tool and argument scoring, 1,500 each); 829 BFCL tool-selection distillation rows (Apache-2.0, from the relay set above; BFCL's benchmark files have no train/test split).
• Labelled reasoning (2,000). GSM8K train split (MIT).
• Licences of the distillation source texts (19,583 rows including BFCL), as the relay card records them: Apache-2.0 5,903; CC BY-SA 4.0 3,622; MIT 3,579; BSD 3,341; CC BY 4.0 800; Apache-2.0 with MIT 638; Apache-2.0 with CC BY-SA 4.0 638; CC with CC BY-SA 4.0 400; Apache-2.0 319; CC BY-SA 4.0 260; CC0 83.
• Regularisation (14,819 questions; labels ignored). Public-dataset questions from 102 sources: 14,536 via tasksource and 283 from Hugging Face datasets. Every row is in a source classified for commercial use. Licences as the source records give them: CC BY 4.0 4,905; Apache-2.0 4,721; MIT 3,715; CC0 1,161; ODC-By 215; Apache-2.0 with MIT 61; BSD 41. The target is the starting average's own probabilities.
• Share-alike terms (CC BY-SA) apply to redistributing source text. This repository contains model weights only, no training text.

What was removed before training, and why

The mix these rows come from had 43,158 rows. Removed: 2,478 rows generated from PIQA, 80 SVAMP rows and 1,237 MMLU distillation rows (sources of items in the public Zils exam-authoring code); 400 VitaminC distillation rows (a dataset the exam holds out); 282 dated outcome rows whose 72-hour label window reached past 2026-10-20 06:00 UTC. For this body we also left out 3,000 tool-call policy rows and 994 word-form rows, which target families the mini exam recipe does not list. Every row was checked against the published Zils seed sets we hold locally (v1 to v3; v4 was not checked) and our own dev sets: exact matches on the normalised scenario text, and on scenario + question + label; both found 0 matches. The 2,000 GSM8K rows were also compared with the public seeds we hold (v1 to v3) and our devs by character similarity and word overlap (no pair at or above 85 similarity or 0.6 Jaccard).

Overlap with Zils material

• No item from our dev sets and no item from published Zils seed sets v1 to v3 was used in this training run (v4 was not checked).
• The generated tool rows reuse public Zils seed instruction templates with new instances; no seed item is used.
• GSM8K train is a public split that the earliest published Zils seed families also rewrite questions from. We used the original GSM8K train rows, not the seed rewrites.

Our own numbers (not Zils's exam; our own development sets, labelled as ours)

These are our own development sets, scored with Zils's open-source grader (mean of 5 secrets). They are not the Zils exam, and on them adding Matilda-Decision-v1-Mini looks about 1.5 points better than it did in the two published d3-mini rounds, where the real selection did not keep it. Read the gains below as an upper side, not a forecast.

	mini_combo50 (999 items)	mini_dev_gold (299)	dev_v2 (3,274)	dev_gold (787)
this body, alone	74.51	78.09	63.59	78.13
{torchcast, d3-mini} (the 11-05 soup)	71.83	76.02	61.38	77.23
{torchcast, d3-mini, this body}	73.27	77.37	62.59	78.15

mini_dev_gold covers the 15 known families of Zils's public exam-authoring code (k2/k3/k5). mini_combo50 adds 50 items per new-topic family from our dev_v2. dev_v2 and dev_gold are our full development sets (dev_gold and mini_dev_gold are scored on the known block only); 5 dev_v2 items are excluded.

Our own soup-round check (disclosed in full)

Before this body was committed, we ran Zils's own soup-round logic (greedy +0.5, leave-one-out seat, crown rule) on our development dumps against the field committed for this round. Our own shadow predicted this body would NOT join the soup: starting from the strongest single body in that field, adding this body gained only about +0.08 on mini_combo50 and about −1.96 on mini_dev_gold, both below the +0.5 the greedy step needs. It is a good model on its own (the solo numbers above), but next to that stronger body it added little. We are entering it anyway, as a deliberate choice, and we state the prediction here so the record is honest.

Licence, attribution and modifications

This repository is released under the Apache License 2.0 (see LICENSE). The weights are a modification — a uniform average plus a merged LoRA fine-tune — of these Apache-2.0 works, with thanks to their authors:

• torchcast-ai/torchcast-decision-4b @492bfb19, Maincode/Matilda-Decision-v1-Mini @39479a8a, perplexity-ai/pplx-decider-flash-v1.2 @9ff7cf98 (the three averaged base bodies);
• zils-ai/d3-mini-4.5b-v1 @b54402a9 (the chassis) and its parent Qwen/Qwen3.5-4B @851bf6e8.

As required under Apache-2.0 §4(b), the accompanying NOTICE file states that these weights are modified from the works above and carries their attributions. Dataset-source licences are listed in "Training data"; share-alike terms (CC BY-SA) apply to the source texts, which are not redistributed here (weights only).