wiiiktor/chopper_minicpm3

Chopper: MiniCPM3-4B, 20% width slice

A chopper inside MiniCPM3-4B: 8 of its 40 attention heads and 960 of its 6,400 MLP neurons in every layer, plus a LoRA (r32) and a KEEP head, 44.9M trainable parameters. It cuts 16 retrieved passages down to the fragments that answer the question. Code: github.com/wiiiktor/chopper_demo (scripts/mla/).

Why MiniCPM3: it uses MLA (multi-head latent attention). Every head rebuilds its keys and values from one shared latent per token, so a slice of a few heads reads the full model's whole cache of the conversation, at no extra cost. The passages are read twice (<passages> The same passages again: <passages>) and KEEP is read from the second copy, so every token has seen all the passages.

file	what
permutation.safetensors	per layer layerN.heads [40] and layerN.mlp_neurons [6400]: the order, most important first. The slice is the leading 8 heads and 960 neurons. Applied at load to the unmodified openbmb/MiniCPM3-4B
chopper_lora.safetensors	LoRA r32 on the slice (q_a, q_b, kv_a, kv_b, o, gate, up, down per layer)
keep_head.safetensors	Linear(2560→1), per-token p(KEEP)
chopper_config.json	slice size, training args, decode
keeper.pt, perms_minicpm3.pt	the same, as scripts/mla/mla_keeper_train.py --eval-only and mla_speed.py read them

Trained for one epoch on all 54,000 rows of wiiiktor/chopper_demo mixed (benchmark + Wikipedia chains), on one RTX 3090 (27 h, 11.7 GiB). A second epoch is in training.

Validation

Decoded at p > 0.8 (gaps ≤ 5 tokens merged, runs < 4 dropped); 100 questions per set (200 for Wikipedia 2-hop):

set	character F1	answer in kept	text kept	for comparison, mmBERT-small: F1 / answer in kept
MuSiQue	.52	.47	3.9%	.52 / .47
HotpotQA	.58	.80	5.4%	.61 / .81
TriviaQA	.35	.69	4.0%	.33 / .74
Wikipedia 2-hop	.55	.90	6.0%	.60 / .90
Wikipedia 1-hop	.65	.96	3.0%	.69 / .94

It ties mmBERT-small (0.14B encoder, wiiiktor/chopper_mmbert). Thresholds 0.6–0.85 trade answer recall (.51 / .84 / .76 at 0.6) against F1 (.56 / .61 / .38 at 0.85).

Both choppers are only fine-tuned. The universal chopper is mmBERT, a general-purpose encoder, fine-tuned for chopping; the integrated chopper is a slice of MiniCPM3-4B, a model pretrained for chat, fine-tuned with a LoRA. Neither was trained for chopping from the start. Trained from scratch for this purpose, both should be better, and the integrated one has the larger potential: it reads the whole conversation straight from the model's own cache, which a separate encoder never sees.

Speed

One read = the question (from the cache) + 16 passages (~4,600 tokens) read twice, RTX 3090, 40 questions:

read path	median
as trained (LoRA as separate matmuls)	437 ms
LoRA merged into the slice weights	394 ms
merged + torch.compile	277 ms

Each time includes the full model's prefill of the question (87 ms). The merged and compiled paths agree with the trained one on 99.97% of the keep decisions.

python scripts/mla/mla_keeper_train.py <data dir> perms_minicpm3.pt <out> --frac 0.2 --eval-only keeper.pt
python scripts/mla/mla_speed.py perms_minicpm3.pt keeper.pt <contexts.jsonl> --n 40

License

These files are a derivative of the MiniCPM3-4B weights (the slice runs 0.76B of MiniCPM3's own parameters, with a LoRA and a KEEP head on top), so they are released under the same terms as those weights: the MiniCPM Model License. Use for academic research is free. Commercial use is also free, but OpenBMB / ModelBest require registration through their questionnaire first (see the MiniCPM3-4B model card). The code in github.com/wiiiktor/chopper_demo is separate; MiniCPM3's own modelling code is Apache 2.0.