RASMUS/MiniCPM-V-4.7-1B-ONNX

MiniCPM-V 4.7 1B — ONNX for WebGPU

An ONNX conversion of openbmb/MiniCPM-V-4.7-1B (revision 6fde9ecee3e59c4b0ce555a9c774e179d46d19cc), made for direct ONNX Runtime Web sessions on WebGPU in chatxpu.ai. The weights and the licence (Apache-2.0) are those of the original model.

This is not a Transformers.js model. Transformers.js has no MiniCPM-V model type, and the decoder uses onnxruntime-genai cache names. You need your own pipeline: image slicing and the prompt layout, pixel preprocessing, canvas M-RoPE positions, and the decode loop.

Files

File	Size	Inputs → outputs
onnx/embed_tokens.onnx (+ _data)	485 MiB, fp16	input_ids [b, s] → inputs_embeds [b, s, 1024] fp16
onnx/vision_encoder.onnx (+ _data)	1046 MiB, fp16 weights, fp32 math	pixel_values [1, 3, H, W] fp32, position_ids [h·w] int64 → image_embeds [h·w/16, 1024]
onnx/decoder_model_int8.onnx (+ _data)	827 MiB, int8 weights	inputs_embeds fp16, attention_mask, position_ids [3, b, s], cache → logits [b, 1, 248094], cache

• Vision: one session call per crop (the thumbnail, then each slice). position_ids are the nearest-bucket patch ids of the 70×70 position table. Pixels are bicubic-resized (antialias) and mapped to (v/255 − 0.5)/0.5.
• Decoder: built with onnxruntime-genai 0.15.2 (-e webgpu -p int8) as a Qwen3.5 text model from the MiniCPM text_config. Then all 187 MatMulNBits nodes were set to accuracy_level=0 (see below). Cache: past_key_values.N.key/value for the 6 full-attention layers, past_key_values.N.conv_state and .recurrent_state for the 18 linear-attention layers. The conv state cannot be rewound.
• Tokenizer, chat template and processor configs are copied from the original repository.

Measured accuracy and speed

All numbers: one RTX 4080, Chrome 155, ORT Web 1.29.0, 2026-10-16.

• Whole chain on WebGPU (embed + vision + decoder), teacher-forced against Hugging Face fp32 on one real image: KL mean 0.00076, max 0.0028, top-1 15/15.
• Why accuracy_level=0: with level 4 (the builder's value) the same chain gave KL mean 0.0092 and top-1 14/15 on WebGPU. Level 0 removed most of the int8 MatMul error and costs no decode time.
• Vision fp16 weights with fp32 math: relative error 8e-6 against the fp32 graph. Full fp16 math gave 0.20 and is not used.
• Speed: decode about 45–47 ms per token; prefill 0.18 ms per prompt token at 1024 tokens; vision 54–59 ms per 448×448 crop.

These are measurements on one GPU, not a guarantee for other hardware.