Whittle dev: how a 27B was taken apart and what it cost

The developer side of the Whittle family. Every shape we built, what it scored, and the measurements that decided each step. If you just want a model to run, use Qwen3.8-Whittle-tri-14.7B.

The ladder

Fifteen shapes, all measured against the same 80-probe knowledge atlas and the same 39-prompt generation battery.

shape	params	battery	how it was made
Whittle-16B v1 heal	16.8B	36/39	44L cut, 25% width prune, 3h QLoRA
48L cut	20.8B	35/39	16 layers dropped by block pricing, no training
restored 18.3B	18.3B	34/39	44L trunk with blocks 8-11 spliced back at full width
v40 identity-guided	16.8B	31/39	40 layers chosen by measured identity, no training
tri repaired	14.7B	31/39	64 layers distilled into 32, then repaired
44L cut	19.2B	28/39	contiguous bands removed
w50 uniform width	18.3B	27/39	every MLP halved, all layers kept
tri raw, pre-heal	14.7B	12/39	the distillation before any repair

Findings that changed what we built

Knowledge is not where the general lens says it is. A four-prompt lens priced blocks 8-11 as nearly free. Domain-targeted probes showed those blocks carry web-dev and rare-identifier retrieval: restoring them alone scored above the intact model. Price per domain before dropping anything.

Stopping is a measurable organ. P(end-of-turn) at a finished answer falls from 0.53 in the intact model to 0.05 in the worst cuts. That number is looping, seen from the inside. Restoring blocks 8-11 took it back to 0.55.

Attention alone computes nothing. Keep the 16 attention stations, delete every recurrent layer, and the output is flat noise. The recurrent tissue does the work; attention routes it.

Placement beats amount. Three cuts at identical size and identical selection rule score 0.648, 0.544 and 0.379 on the atlas. Only the attention spacing differs.

Absorbing beats deleting. Distilling a full-attention station into its two recurrent neighbours costs 0.0045 relative error; deleting it costs 0.035.

Sequential distillation compounds, and a global objective fixes it. Per-stage error rose 70x across 24 chained stages and the assembled model scored 12/39. Ninety minutes of global fine-tuning took it to 31/39. The knowledge was mis-coordinated, not destroyed: forced-choice recognition went 0/7 to 5/7.

Width damage is dormant, depth damage is erased. Halve every MLP and the model still recognises answers it can no longer recall (6/7, 7/7). Cut layers and the knowledge is simply gone.

Traps worth inheriting

Synthetic drills that state a rule in prose teach the phrasing, not the skill: our repaired model recites "multiplication binds before addition" and still answers 30 for 10 + 2 * 5. Replay slices amplify whatever format they contain. peft writes a local base path into adapter metadata that HF rejects. modules_to_save norms are trained but carried by neither the LoRA path nor a naive merge. llama.cpp cannot apply LoRA to this architecture at all: the fused GDN projection reshape has no low-rank equivalent, so merging is mandatory.

Artifacts

Recipes, adapters and mixes live in the model repo under recipe/, heal-lora/, repair-lora/ and training/. The relay, assembler, distillation proof of concept, norm patcher and trainers are all there and all runnable on two 12GB cards.

Collections

Models you can run has the two variants with published, tested weights. Research and dev has everything else, including variants whose weights are not uploaded yet.