wiiiktor/chopper_mmbert

Chopper: mmBERT

Cuts retrieved passages down to the fragments that answer the question. Two encoders, trained the same way, once, on all 54,000 rows of wiiiktor/chopper_demo mixed (benchmark chains + Wikipedia chains, one epoch). Code: github.com/wiiiktor/chopper_demo.

folder	what
mmbert-base/	mmBERT-base chopper (0.31B): encoder/ (a normal Transformers model) + heads.safetensors (KEEP + ANSWER/SEARCH)
mmbert-small/	the size-matched mmBERT-small chopper (0.14B), same layout

These are the 🟧 universal chopper. The 🟪 integrated chopper is wiiiktor/chopper_minicpm3 (a 20% slice inside MiniCPM3-4B).

Both choppers are only fine-tuned. The universal chopper is mmBERT, a general-purpose encoder, fine-tuned for chopping; the integrated chopper is a slice of MiniCPM3-4B, a model pretrained for chat, fine-tuned with a LoRA. Neither was trained for chopping from the start. Trained from scratch for this purpose, both should be better, and the integrated one has the larger potential: it reads the whole conversation straight from the model's own cache, which a separate encoder never sees.

Validation

Decoded at p > 0.8 (gaps ≤ 5 tokens merged, runs < 4 dropped); character F1 against the Qwen3-14B teacher's fragments / share of questions whose answer is in the kept text; 100 questions per set:

chopper	MuSiQue	HotpotQA	TriviaQA	Wikipedia 2-hop	Wikipedia 1-hop
mmBERT-base	.55 / .48	.64 / .80	.39 / .80	.62 / .92	.72 / .91
mmBERT-small	.53 / .47	.61 / .81	.33 / .74	.60 / .90	.68 / .94

Both keep 3–5% of the passage text.

Works with any model

The mmBERT choppers are independent of the answering model: plain text in, plain text out, so their fragments can go to any LLM.

Input: one string, up to 8,192 mmBERT tokens: the question (or the end of the conversation), then the passages.

Question: <question> [SEP] <passage 1> [SEP] <passage 2> [SEP] … [SEP] <passage 16>

Output:

• per token, p(KEEP) from keep.* in heads.safetensors. Keep tokens with p > 0.8, merge runs separated by up to 5 tokens, drop runs shorter than 4. Each kept run maps back to characters of the original passage: [passage, char_start, char_end, text]. Typically ~3 fragments of ~35 tokens are left out of ~3,000.
• one decision, ANSWER or SEARCH, from decision.* on the first token. (In practice it never predicts SEARCH; leave that decision to the answering model.)

How the main model uses it: only the fragments enter its context, as short notes before the question:

Notes:
- <fragment 1>
- <fragment 2>

Question: <question>
Answer:

Minimal use

import torch
from safetensors.torch import load_file
from transformers import AutoModel, AutoTokenizer
from huggingface_hub import snapshot_download

d = snapshot_download("wiiiktor/chopper_mmbert", allow_patterns=["mmbert-base/*"]) + "/mmbert-base"
tok, enc = AutoTokenizer.from_pretrained(f"{d}/encoder"), AutoModel.from_pretrained(f"{d}/encoder").eval()
H = load_file(f"{d}/heads.safetensors")
text = "Question: " + question + "".join(f" {tok.sep_token} {p}" for p in passages)
x = tok(text, return_tensors="pt", return_offsets_mapping=True, truncation=True, max_length=8192)
off = x.pop("offset_mapping")[0]
with torch.no_grad():
    h = enc(**x).last_hidden_state[0].float()
p_keep = torch.sigmoid(h @ H["keep.weight"].float().T + H["keep.bias"].float())[:, 0]
# kept characters of `text`: off[i] for every token i with p_keep[i] > 0.8 (then merge gaps ≤ 5, drop runs < 4)

The full decoding is in scripts/chopper/mm_train.py; a complete retrieve → chop → answer loop over your own documents is operator-for-llm-with-mmbert.py in the GitHub repo.