zils-mini-body-3

Repository: tensor-tailor/zils-mini-body-3.

A decision body for the Zils d3-mini soup pool (Bittensor subnet 127), in the d3-mini chassis layout (Qwen3.5-4B language and vision tensors, bf16). It starts from the uniform average of two d3-mini house bodies and adds a merged-LoRA fine-tune on data aimed at the question families where our earlier body trailed the strongest bodies on our dev sets the most. It is not a re-upload of any public model and does not use the weights of any other miner's body.

This is a seat attempt; it did not meet our own pre-registered screen (see "A seat attempt below our own screen").

Method

• Starting weights: the uniform fp32 average, over all chassis tensors, of torchcast-ai/torchcast-decision-4b @492bfb19 (Apache-2.0) and vllm-sr/d3-mini @df99bf4c (Apache-2.0). The chassis is zils-ai/d3-mini-4.5b-v1 @b54402a9 (Apache-2.0); parent Qwen/Qwen3.5-4B @851bf6e8 (Apache-2.0). Our rule, fixed before training, takes the house members of the paying soup when the run starts; the 6 November soup held no house body, so the rule kept these two.
• Training: LoRA, rank 16, alpha 32, dropout 0.05, learning rate 1e-4 (linear warm-up over 50 steps, then cosine decay over the epoch), batch 8 with gradient accumulation 2, seed 4343. Targets: the attention and Gated-DeltaNet projections of all 32 language layers (152 matrices). The LoRA is merged into the weights and stored in bf16. No other tensor was trained.
• This body is: the checkpoint at 0.75 of the single epoch. Our frozen rule would have chosen among the three candidates (0.5, 0.75, final) that passed our pre-screen; none did. For this attempt, entered by the owner's decision after our numbers were known, the choice rule set for the attempt took the candidate with the highest smaller-of-the-two soup gains on our two primary dev sets. A candidate's soup gain on a dev set is the score of the uniform average of torchcast, d3-mini and the candidate, minus the score of the start (the uniform average of torchcast and d3-mini). This body's soup gains are +0.63 on mini_combo50 and +0.86 on mini_dev_gold (the final body +0.61 / +0.89, the 0.5 body +0.63 / +0.58).
• Loss: cross-entropy on the labelled rows (every training row carries a dataset label or a computed answer); KL to the starting average's own precomputed probabilities on the regularisation rows (their labels are ignored). There is no teacher-distillation term: this body uses no teacher-probability rows.
• Length limit: rows longer than 2,048 tokens were skipped, not truncated.
• Relative change of the language tensors from Qwen/Qwen3.5-4B, as Zils computes it: 0.018, within the Zils 0.15 cap.

Training data (34,687 input rows)

• Aimed rows (9,792). Before any training number, a fixed rule ranked the known question families of the mini recipe by how far our earlier body tensor-tailor/zils-mini-body-2 trailed the best of five strong rival bodies there, times the room that rival still leaves (score = trail × (1 − rival skill), on our dev_v2 set), and gave rows to the top five in proportion; a family that could not fill its share passed the rest down the list. Rows taken: arts known-choice 3,117; reasoning known no-label 3,018; retrieval known-score 1,682; arts known-score 602; reasoning known-choice 1,373. Of these, 1,633 are new questions from our own generators with computed answers (1,031 question-parts, 602 rule-book scoring), with fresh seeds; the other 8,159 are public-dataset items with their own hard labels.
• Natural rows (9,791). Public-dataset items with their own hard labels from the same known families, taken in a fixed hash order (not by score).
• Rows kept from our earlier bodies (15,104), the same rows as in tensor-tailor/zils-mini-body-1 and -body-2: dated outcome questions 8,604 (asked 2026-10-08 to 2026-10-17, every one resolved before 2026-10-20 06:00 UTC; Wikipedia edit outcomes in 9 language editions, CC BY-SA 4.0 via each edit's page history; Wikidata edit outcomes, CC0 1.0; arXiv listing categories, CC0 1.0), generated tool questions 4,500 (our own generators, computed answers) and GSM8K train rows 2,000 (MIT).
• None of the 19,583 aimed and natural rows is a row of tensor-tailor/zils-mini-body-2's new data, so this body's new data differs from that body's.
• Regularisation (14,819 questions; labels ignored). Public-dataset questions from commercial-use sources; the target is the starting average's own probabilities.

This repository contains model weights only, no training text. The sources and their licences are named above for attribution.

Fair-play checks

• Every new row (aimed and natural, generator rows included) was checked against our development sets and the published Zils seed sets v1–v3 (v4 is not in our inventory and was not checked): a row was dropped if it copied a dev item (equal normalised scenario and question), shared a non-template run of 8 words with a dev item of the same answer while sharing no template run with that same item, or had the same normalised scenario as a seed item. 551 rows were dropped by that rule and 4 as duplicates of our other training rows. 832 kept rows share only a format template with a test item. An independent check against a held-out block built by a teammate who did not write our generators found 0 matches.
• The 15,104 kept rows are deliberately reused from our earlier bodies and disclosed as such; they passed the same dev/seed checks when first built.
• Sources of items in the public Zils exam-authoring code, the exam's held-out datasets, the counting recipe, Misra/ Sarcasm and 20 Newsgroups were excluded. No teacher-distillation rows are used.

Our own numbers (not Zils's exam; our own development sets, labelled as ours)

Scored with Zils's open-source grader (mean of 5 secrets). Not the Zils exam.

	mini_combo50 (999)	mini_dev_gold (299)	dev_v2 (3,274)	dev_gold (787)
this body, alone	72.80	77.91	61.49	78.12
the start {torchcast, d3-mini}	71.83	76.02	61.38	77.23
our earlier body tensor-tailor/zils-mini-body-2, alone	75.07	79.58	62.86	78.94

A seat attempt below our own screen

Before training we fixed a pre-screen: a candidate's soup gain (defined under Method) had to be at least 0.8 on both mini_combo50 and mini_dev_gold. None of the three candidates met it: on mini_combo50 their soup gains were only +0.61 to +0.63. On our dev sets this body alone is also clearly weaker than our earlier body tensor-tailor/zils-mini-body-2 (table above) and than 1lock/zils-mini-astrid-m1 @b3a18304, the crown of the 6 November d3-mini round (76.3 on mini_combo50, 84.9 on mini_dev_gold, our measurement). Because the screen failed, we did not run our soup-round forecast for this body. It is entered by the owner's decision as a seat attempt; the graded round is the only test. Zils's official checks were run on this exact body before commit: the local commit check passed (Zils body digest ac4d59f3…); and against both the soup predicted for its round (astrid-m1 alone) and the start soup {torchcast, d3-mini}, validation passed, the copy check found no copy, the twin check found no twin among the 41 bodies checked, the fine-tune gate did not apply, the rate is 1.0 and the base residual 0.81. On our dev sets its nearest body is its own start (temperature-fitted Jensen–Shannon divergence 0.0084); it is 0.028 from astrid-m1 and 0.021 from zils-mini-body-2, all above the 0.005 line Zils treats as a twin.

Licence, attribution and modifications

Released under the Apache License 2.0 (see LICENSE). The weights are a modification — a uniform average plus a merged LoRA fine-tune — of the works named above; the accompanying NOTICE file states the modification and carries the upstream attributions (Apache-2.0 §4(b)). No weights of any other Bittensor miner's committed body were used.