kordou/MinerU-Popo-NF4

MinerU-Popo — mixed-precision NF4

A 4-bit (NF4, bitsandbytes) quantisation of DreamEternal/MinerU-Popo, made to run the model on a 12 GB consumer GPU.

No fine-tuning, no architecture change, no re-training. Only the numeric precision of part of the weights differs from the source.

	Source	This repo
Weights on disk	17.75 GB (F32)	4.31 GB
Effective precision	32 bits/param	7.78 bits/param
Peak VRAM to load	~8.9 GB at bf16	4.33 GB
Parameters	4,437,815,808	unchanged

Why mixed precision, not flat 4-bit

MinerU-Popo is a Qwen3-VL-4B fine-tune whose entire job is reading rendered page images. A bare BitsAndBytesConfig(load_in_4bit=True) quantises every nn.Linear, which for this architecture includes all 104 Linear modules of the 24-block vision tower — the worst possible place to lose precision for this task.

So the vision tower, the mergers and lm_head are held at bfloat16, and only the text tower is quantised — following the recipe Unsloth publishes for this architecture, plus the eight text-tower layers their Dynamic build protects.

Of the 357 Linear modules: 227 are NF4, 130 stay bf16 — including all 104 vision modules.

A trap worth documenting

transformers matches skip patterns with re.match (quantizers/quantizers_utils.py), which is anchored at the start of the module path — it is not a substring test. A bare "visual" therefore does not protect model.visual.blocks.0.attn.qkv. Measured on this model, a bare-class-name skip list left 356 of 357 Linear modules quantised, all 104 vision modules among them. The patterns below use full anchored prefixes.

Recipe

BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
    llm_int8_skip_modules=[
        "lm_head",
        "model.visual",                               # whole vision tower
        "model.language_model.layers.0.mlp",
        "model.language_model.layers.6.mlp",
        "model.language_model.layers.9.mlp",
        "model.language_model.layers.11.self_attn",
        "model.language_model.layers.12.mlp",
        "model.language_model.layers.13.mlp",
        "model.language_model.layers.16.mlp",
        "model.language_model.layers.35.mlp",
    ],
)

Built with transformers 5.16.1, bitsandbytes 0.50.2, torch 2.8.0+cu128 on an RTX 4090.

Usage

The checkpoint self-describes via quantization_config in config.json, so it loads through the ordinary call with no quantisation arguments. bitsandbytes and a CUDA GPU are required.

from transformers import Qwen3VLForConditionalGeneration, AutoProcessor

model = Qwen3VLForConditionalGeneration.from_pretrained(
    "kordou/MinerU-Popo-NF4", dtype="bfloat16", device_map="auto",
)
processor = AutoProcessor.from_pretrained("kordou/MinerU-Popo-NF4")

Validation — and its one known deviation

Validated by running the real MinerU-Popo pipeline (label_normalization → run_inference → get_json_tree, upstream commit 97d5601) on real MinerU output, and comparing the resulting document tree against the unquantised model on identical inputs.

All runs use greedy decoding, and determinism was confirmed by running every model twice: each produced byte-identical output both times.

Document	Result vs unquantised
Transformer paper (11 pp.)	Byte-identical document tree
Nature-style paper w/ Reporting Summary (25 pp.)	28/28 titles identical; 21/28 depths identical; raw model output 99.995% identical

The known deviation: on the second document, seven headings inside the appended "Reporting Summary" form are assigned depth 3 instead of 4. No heading is lost, and none is invented. The entire main body matches exactly.

For contrast, the variant that protects only the vision tower (without the eight extra text layers) additionally promotes a non-heading — "level": -1 becomes "level": 2 on "Online content" — inserting a spurious node. That variant is not what is published here.

Reproduction-error note

Per-layer NF4 reconstruction error across all 252 text-tower matrices is essentially flat — worst 10.43%, best 9.22%, mean 9.29%. There is no "sensitive layer" signal in plain weight error to exploit, so a wider error-ranked skip list was measured and rejected rather than guessed at.

Deliberate change: greedy decoding

generation_config.json here sets do_sample: false. The source repo ships do_sample: true, temperature: 0.7, top_k: 20, top_p: 0.8, which makes document structure non-reproducible between runs — the same page can yield a different tree each time.

This was changed so that the validation above measures quantisation error rather than sampling noise, and it is published in the state it was validated in. To restore upstream behaviour, override generation_config at load time.

Credit

All credit for the model belongs to the MinerU-Popo authors at OpenDataLab. This repo only changes numeric precision. Licensing follows the base model and the upstream repository.