W75-K: a fine-tune of zils-mini-body-2

A decision body for the Zils d3-mini soup pool (Bittensor subnet 127), in the d3-mini chassis layout (Qwen3.5-4B language and vision tensors, bf16). It is a merged-LoRA fine-tune of our own body tensor-tailor/zils-mini-body-2 @195a13b23f3069c12ffc311b14aae3a1e2af6a84 ("W75", Zils body digest defcfabe…, Apache-2.0), which it replaces on the same hotkey (UID 63). It is not a re-upload of any public model and it does not contain the weights of any other miner's body.

Method

Starting weights: W75 exactly as published at @195a13b2 (model.safetensors sha256 65c7ad83…, body digest defcfabe…).
Training: LoRA, rank 16, alpha 32, dropout 0.05, learning rate 5e-5 (linear warm-up over 50 steps, then cosine decay over the epoch), batch 8 with gradient accumulation 2, seed 4343, ONE epoch (3,849 optimizer steps). Targets: the attention and Gated-DeltaNet projections of all 32 language layers (152 matrices). The LoRA is merged into the weights and stored in bf16. No other tensor was trained.
This body is: the final weights of that epoch (step 3,849 of 3,849), named by both of our ordered save-selection rules (best mini_dev_gold among the saves before the first failing save of our forgetting gate; no save failed it).
Loss (distillation only): on every training row, KL(teacher ‖ student) to the precomputed answer probabilities of ONE public teacher body chosen per question family (table below); no label cross-entropy, no Brier term. Plus KL to W75's own precomputed probabilities on 12,448 regularisation rows, weight 2.0. Teachers contribute probabilities only; none of their weights is in this body.
Length limit and loaded dose: rows longer than 2,048 tokens would be skipped, not truncated; none was: 49,136 training rows and 12,448 regularisation rows were loaded (mean 247 tokens per row).
Relative change of the language tensors from Qwen/Qwen3.5-4B, as Zils computes it: 0.02943 (Zils cap 0.15); from W75: 0.00947.

Teachers (answer probabilities only)

teacher body (public, Apache-2.0)	families it teaches	rows
1lock/zils-mini-astrid-m1 @b3a18304	COPA, MNLI, QNLI, SciNLI, SQuAD 2.0, veridicality	11,812
qhoenix/zils-mini-decision-v2 @ecb9cadb	DROP, FEVER, PUBHEALTH, MathQA, QASC, ROPES	10,883
crabby-devil/zils-mini-decision-r16b @c9400212	PIQA, ToolACE (no-tool)	9,784
ANGExllL/zils-mini-senova-ft @5efc6f0c	BoolQ, CosmosQA, Fig-QA	8,533
W75 itself (tensor-tailor/zils-mini-body-2 @195a13b2)	Amazon polarity, CoQA	8,124

Per family the teacher is the candidate (astrid-m1 and four field bodies, plus W75) with the lowest mean Brier/RPS against the TRUE labels of that family's rows; for the four entailment-style families astrid-m1 was set as the teacher by our rule. The teacher choice uses training rows only; no dev, exam or seed item was scored to choose a teacher.

Training data (49,136 rows, 19 question families; weights only here, no training text is redistributed)

Train splits of public datasets converted into Zils question formats: PIQA 5,780 (AFL-3.0), CosmosQA 4,061 (CC BY 4.0), QASC 4,054 (CC BY 4.0), Amazon polarity 4,062 (Apache-2.0), CoQA 4,062 (only its Wikipedia / literature / CNN parts: CC BY-SA 4.0 / Apache-2.0), ToolACE 4,004 (Apache-2.0, Team-ACE/ToolACE @6bda777c), SQuAD 2.0 3,790 (CC BY-SA 4.0), FEVER 3,177 (CC BY-SA 3.0), BoolQ 3,037 (CC BY-SA 3.0), MathQA 2,785 (Apache-2.0), MNLI 2,160 (OANC + CC BY-SA 3.0 / CC BY 3.0 parts), QNLI 2,160 (CC BY-SA 4.0), SciNLI 2,160 (CC BY-SA 4.0), Fig-QA 1,435 (MIT), MegaVeridicality 1,142 (MIT), COPA 400 (BSD-2-Clause), ROPES 294 (CC BY 4.0), PUBHEALTH 288 (MIT), DROP 285 (CC BY-SA 4.0). Share-alike terms apply to redistributing the source text, not to these weights.

Label balance per family before training: yes/no families 50/50, score families uniform over their levels, choice families only with the option count of our dev family and a bounded answer-key share; entailment-style families capped together.
Removed before training: every row whose source text appears in a Zils exam split or the exam-authoring sources, any glaive-derived row, rows overlapping our development sets, the five datasets we hold out as unseen, and every dataset whose licence does not allow this use (e.g. RTE, CoLA, MRPC, QQP, IMDB, SciQ, Yelp). No dated real-world outcome item is used.
Regularisation (12,448 rows): public-dataset questions from commercial-use sources (our pin set, cleaned of exam-split, exam-source and glaive rows); target = W75's own probabilities.

Our own numbers (our own development sets, scored with Zils's open-source grader, mean of 5 secrets; not the Zils exam)

vs W75 (same GPU)	result
mini_dev_gold (299 items, current KNOWN families)	82.32 vs 79.67 for W75 (+2.64); astrid-m1 84.87
mini_combo50 (999 items)	76.05 vs 75.13 (+0.92)
our combined KNOWN + NEW estimate	+1.44 (KNOWN part +1.41, NEW part +1.48)
our held-out MINI block	+0.61
guard families (6)	4 at or above W75, two at -0.0002 and -0.0001
unseen-domain slice (500 rows of the new-block proxy from the 5 datasets we hold out as unseen)	-1.10 (worse than W75)
full unseen-domain test (226 items)	-0.24 (worse than W75, within our -0.5 line)
new-block proxy (2,100 items)	-1.17 (worse than W75)
language KNOWN category	79.5 -> 76.5 (-3.0, worse than W75)

While this body was already training (before any of its numbers were read) we fixed a line "unseen and new-block may not drop by more than 0.5"; this body misses it on the new-block proxy (-1.17) and on the unseen slice (-1.10); on the full unseen test it is -0.24, within the line, and the language category drops by 3.0. It is committed anyway by an explicit owner decision that weighed these drops against the gains above.

Twin and copy checks (disclosed in full)

Behavioural twin (temperature-fitted Jensen-Shannon divergence on our dev sets, minimum over the 44 bodies of the round pool): 0.00703 bits, against our own previous body W75 (0.00704 when both are run on the same GPU); every other pool body is further away. This is above the 0.005 line Zils uses for a twin; our own stricter internal line of 0.01 was set aside by an owner decision.
Weight-space copy check (Zils's sketch method, soup kept and soup moved): not a copy of any other hotkey's body (largest ratio 0.003, from ONE sketch seed of our own choosing; the validator's private seed is not known to us; our reviewers accepted the one-seed result: about 100x margin to Zils's 0.30 line). Against our own previous body W75 on the same hotkey (which Zils's copy rule does not compare): cosine 0.948, explained share 0.736, i.e. this body is a small change of W75.

Licence, attribution and modifications

Released under the Apache License 2.0 (see LICENSE). The weights are a modification (a merged LoRA fine-tune) of W75, which is itself a modification of the Apache-2.0 works listed in NOTICE; the NOTICE file states the modifications and carries the upstream attributions (Apache-2.0 §4(b)). The teacher bodies above (all Apache-2.0) supplied answer probabilities only. No weights of any other Bittensor miner's committed body were used.