BIG-M: a two-stage fine-tune of zils-mini-astrid-m1

A decision body for the Zils d3-mini soup pool (Bittensor subnet 127), in the d3-mini chassis layout (Qwen3.5-4B language and vision tensors, bf16). It is a fine-tune, in two stages, of the d3-mini soup body 1lock/zils-mini-astrid-m1 @b3a183048fc361f3cccc4bd8e0ad58ddc57dac9f (Apache-2.0, Zils body digest 1533c041…), as Zils's mining guide allows ("You may fine-tune the current soup"). Credit for the starting body belongs to 1lock. It is not a re-upload of any public model; beyond its declared start astrid-m1 (another miner's published soup body), no weights of any other miner's body were merged. It is committed on UID 179, a hotkey of ours with no earlier commitment.

Method

Stage 1 (our "M7c-T2"; its result was never published): from astrid-m1 @b3a18304 unchanged; LoRA rank 16, alpha 32, dropout 0.05 on the attention and Gated-DeltaNet projections of all 32 language layers (152 matrices), lr 5e-5 (warm-up 50 steps, cosine), batch 8 x accumulation 2, seed 4343, one epoch over 49,136 rows; loss = KL to the precomputed answer probabilities of one public teacher body per question family (table below) + KL to astrid-m1's own probabilities on 12,448 regularisation rows (weight 2.0); merged into bf16. Its final weights ("Dfin", body digest dd2083b6…) are the start of stage 2.
Stage 2 (this body): from Dfin; LoRA rank 128, alpha 256, dropout 0.05 on the same 152 projections PLUS the MLPs of all 32 layers (248 matrices), lr 5e-5 (warm-up 50, cosine), batch 8 x accumulation 2, seed 4343, TWO epochs over 8,931 items + 2,977 regularisation rows (1,489 optimizer steps); loss per item = cross-entropy on the item's key + 1.0 x Brier + 0.1 x KL to Dfin (computed live with the LoRA switched off, i.e. the exact start); regularisation rows = KL (weight 1.0) to Dfin's own precomputed probabilities on public questions. Merged into bf16. No row was skipped (limit 2,048 tokens; mean 176 tokens).
This body is: the checkpoint at 0.75 of the two-epoch stage-2 run (step 1,117 of 1,489), chosen by the rule set for this submission: the best of the four saves (0.25 / 0.50 / 0.75 / final) on our combined KNOWN + NEW estimate, ties to the earlier save.
Relative change of the language tensors from Qwen/Qwen3.5-4B, as Zils computes it: 0.03015 (from Dfin: 0.02177) (Zils cap 0.15).

Stage-1 teachers (answer probabilities only; their weights are not in this body)

teacher body (public, Apache-2.0)	families	rows
1lock/zils-mini-astrid-m1 @b3a18304	COPA, MNLI, QNLI, SciNLI, SQuAD 2.0, veridicality	11,812
qhoenix/zils-mini-decision-v2 @ecb9cadb	DROP, FEVER, PUBHEALTH, MathQA, QASC, ROPES	10,883
crabby-devil/zils-mini-decision-r16b @c9400212	PIQA, ToolACE (no-tool)	9,784
ANGExllL/zils-mini-senova-ft @5efc6f0c	BoolQ, CosmosQA, Fig-QA	8,533
tensor-tailor/zils-mini-body-2 @195a13b2 (ours)	Amazon polarity, CoQA	8,124

Stage-1 data (49,136 rows, train splits converted to Zils formats): PIQA 5,780 (AFL-3.0), CosmosQA 4,061 (CC BY 4.0), QASC 4,054 (CC BY 4.0), Amazon polarity 4,062 (Apache-2.0), CoQA 4,062 (Wikipedia / literature / CNN parts only), ToolACE 4,004 (Apache-2.0), SQuAD 2.0 3,790 (CC BY-SA 4.0), FEVER 3,177 (CC BY-SA 3.0), BoolQ 3,037 (CC BY-SA 3.0), MathQA 2,785 (Apache-2.0), MNLI / QNLI / SciNLI 2,160 each, Fig-QA 1,435 (MIT), MegaVeridicality 1,142 (MIT), COPA 400 (BSD-2-Clause), ROPES 294 (CC BY 4.0), PUBHEALTH 288 (MIT), DROP 285 (CC BY-SA 4.0).

Stage-2 data (8,931 items, 8 question families; weights only here, no item text is redistributed)

family	rows	kind	source (licence)
language yes/no (NLI)	2,364	written	MNLI train (OANC; fiction parts CC BY-SA 3.0 / CC BY 3.0)
reasoning yes/no (k3)	1,506	written	PIQA train (AFL-3.0)
retrieval yes/no (k2)	1,430	gold	QNLI train (CC BY-SA 4.0)
language 3-way choice (NLI)	1,257	gold	MNLI train (OANC; fiction parts CC BY-SA 3.0 / CC BY 3.0)
veridicality yes/no	754	written	MegaVeridicality (MIT)
sentiment yes/no	734	written	Amazon polarity train (Apache-2.0)
BoolQ yes/no	498	written	BoolQ train (CC BY-SA 3.0)
COPA choice	388	gold	COPA (BSD-2-Clause)

"written" = exam-format items written from a public TRAIN-split row with the Zils exam-authoring code and a local Qwen3-30B-A3B-Instruct writer, kept only if a blind answer reproduced the key; "gold" = the dataset's own human label, converted to the exam format. Every item passed, before training: removal of rows taken from the exam's own split files (the train splits of the same datasets are kept), of glaive-derived data and of our tests (by source row), 8-gram overlap with the exam splits, near-duplicates and flagged seeds; label balance per family; a check that every training row is byte-identical to a reviewed row; and a manual audit of 20 items per family by a second reviewer (a family with 2 confirmed defects in 20 was removed whole). Only 8 families survived those gates, so this stage trains a narrow set of question types.

Our own numbers (our development sets, Zils's open-source grader, mean of 5 secrets; NOT the Zils exam; all RECORD ONLY)

measure (same GPU)	this body	Dfin (stage-2 start)	astrid-m1
our combined KNOWN + NEW estimate	80.69	80.40 (+0.30)
- KNOWN part / NEW part	79.20 / 82.93	80.46 / 80.29
mini_dev_gold (299 items, current KNOWN families)	76.32	84.24 (-7.92)	84.87
- by category: reasoning / language / retrieval / tools / arts	53.86 / 81.41 / 80.59 / 92.16 / 83.70	65.78 / 89.16 / 91.05 / 94.91 / 86.10
mini_combo50 (999 items)	71.10	76.67 (-5.57)	76.33
our held-out MINI block	92.82	92.77 (+0.06)
guard families (6): retrieval score / language yes-no / reasoning choice	-0.32 / -0.14 / -0.06 vs Dfin
unseen-domain slice of the new-block proxy (500) / full unseen test (226) / new-block proxy (2,100)	-2.11 / -0.76 / -1.79 vs Dfin
NEW-domain test sets N1/N2 (mean)	+2.64 vs Dfin

The stage-2 data covers only 8 question families; on the families it does not see, this body is clearly worse than its start (for example reasoning on mini_dev_gold 53.86 vs 65.78), while it is better on the NEW-domain sets and on the held-out block it is level.

This body is submitted by an explicit owner decision to ship BIG-M whatever these numbers show; none of our pre-registered gates (forgetting, generalization, gain, twin, copy) was used to stop it. The numbers above are disclosed in full, including every drop.

Twin and copy checks (record only)

Behavioural twin (temperature-fitted Jensen-Shannon divergence on our dev sets) against astrid-m1: 0.0238 bits, above the 0.005 line Zils uses for a twin and above our own 0.01 line.
Weight-space copy check (Zils's sketch method, record only, did not gate this submission), on ONE public sketch seed of our own choosing (the validator's private seed is not known to us) and our cached set of 39 earlier bodies from other hotkeys (including our own two earlier UID 63 bodies): with the current soup (astrid-m1) kept, the copy ratio is 0.0054; in a no-soup stress test, 0.0707; Zils's copy line is 0.30. The largest cosine is 0.633 (against astrid-m1). This is not a guarantee of how the official check, with its private seed and the round's own body set, will judge it.

Licence, attribution and modifications

Released under the Apache License 2.0 (see LICENSE). The weights are a modification (two merged LoRA fine-tunes) of astrid-m1, itself a modification of the Apache-2.0 works listed in NOTICE; the NOTICE file states the modifications and carries the upstream attributions (Apache-2.0 §4(b)). The stage-1 teacher bodies (all Apache-2.0) supplied answer probabilities only. No weights of any other Bittensor miner's committed body were used apart from the published soup body astrid-m1, which is the declared start.