zils-mini-body-17: formal / spatial / temporal reasoning LoRA fine-tune of the current Zils MINI soup
A decision body for the Zils d3-mini soup pool (Bittensor subnet 127), in the d3-mini chassis layout (Qwen3.5-4B language and vision
tensors, bf16). It is a fine-tune of the current MINI soup: the uniform weight average (official zils.soup.build: bf16-canonical, fp32
mean, chassis vision tower) of qhoenix/zils-mini-decision-v2 @ecb9cadb39a2f84cefd2f27a755ec12095b62933 and Charlie320/zils-mini-s1
@53eb3ffa768c1ad1eace7acbfdb3ebb9674cefe6 (soup body digest df87aaa9โฆ), as Zils's mining guide allows ("You may fine-tune the current
soup"). Credit for the starting weights belongs to the authors of those two bodies and their upstream works (see NOTICE). No other miner's
body was merged; it is not a re-upload of any public model.
Method
- Training: LoRA rank 32 / alpha 64 / dropout 0.05 on the attention and Gated-DeltaNet projections of all language layers (152 matrices), learning rate 6e-5, AdamW with weight decay 1.0 on the LoRA weights only, batch 8 x accumulation 2, warm-up 50 steps then cosine, 2 epoch(s), max 2,048 tokens, seed 4343; loss per row = cross-entropy on the row's label + 1.0 x Brier. No KL / distillation term, no reference or teacher probabilities. Merged into bf16. This body is the final save.
- Data:
- 4,315 relay rows from 16 formal / spatial / temporal reasoning public task sources (tasksource collection of public TRAIN splits). Their labels are HOUSE labels: the argmax answers of a Zils house teacher model in the archived relay pool (attributed to d3-ultra; exact teacher revision unknown). They are house hard targets, not gold labels; Zils allows house targets.
- 405 anchor rows with gold labels (code-generated score items, GoEmotions, MNLI / PIQA / GLUE train rows).
- No other miner's outputs were used as targets. No rows from our item-writer runs. Every row passed our exam-split 8-gram guard (81 exam sources, 0 hits); no Glaive data; no template tools items.
Checks we ran before commit (not an official score)
- Change from Qwen/Qwen3.5-4B (Zils relative change): 0.01309 (cap 0.15). Zils body digest a1d6d6fded624a2479c6a3074d175d2677c8b5cd3bdccae071d50f8d5dbaa591.
- Copy screen with the official
soup_copiescode (our seeds; the validator's seed is private): own share beyond the house bodies, the listed public models, the soup and the release 0.184-0.197 (house-mix line 0.05); largest cosine to any of them or to our sibling bodies 0.908 (line 0.999).
Licence
Apache-2.0 (LICENSE). See NOTICE for the upstream works and their notices.
- Downloads last month
- 15
Model tree for tensor-tailor/zils-mini-body-17
Base model
Qwen/Qwen3.5-4B-Base