Mike0021/MiniCPM5-2B-ONNX

MiniCPM5-2B ONNX for browser WebGPU

Independently converted from the pinned official checkpoint. Try the browser demo; its repository contains conversion scripts and validation evidence.

Our conversion

The included q4f16 model was converted directly from the official OpenBMB MiniCPM5-2B BF16 checkpoint, not from community-converted weights.

• Source revision: abe115e887989b14f05e64a3b260648329324c3f.
• Official weight SHA-256, verified before conversion: 14fb8e7f0a18d53d1f239773758bf581cee7e456a4523a54622c3a245b64402c.
• ONNX Runtime GenAI 0.15.2; symmetric int4, block size 32; FP16 activations and KV cache.
• Fused QKV, GQA, RoPE, and normalization operations.
• Last-position logits selected before the output projection.
• Lossless splitting of the 510 MiB embedding allocation into four 127.5 MiB tensors.
• Seven external-data shards; corrected tool-history serialization and Jinja compatibility.

The package manifest contains file sizes and SHA-256 hashes.

Validation

Six prompts with prefill and cached decoding give 41 tested prediction positions. These are scoped conversion checks, not a broad quality evaluation.

Comparison	Top-token agreement	Mean KL divergence
FP16 ONNX vs FP16 PyTorch	41/41	0.00000815
Int4 ONNX vs official BF16 PyTorch	36/41	0.125916
Packaged int4 vs our original int4 export	41/41; all logits bit-identical	0
Browser WebGPU vs native packaged ONNX	41/41	0.002517

Local Chromium tests on an AMD Radeon 8060S covered streaming, stopping, thinking mode, XML tool-call generation, and one retrieval fixture at 3,888 prompt tokens. Inference also passed with the GPU device capped at a 128 MiB storage binding limit and 256 MiB buffer limit. This desktop allocation test does not emulate a phone's memory pressure or thermals.

Warm local benchmark medians were about 35–42 decode tokens/s. At 1,890 prompt tokens, first-token latency improved about 9.5% against our own identical-weight control with all-position logits and unsplit embeddings. Timings depend on device, browser, prompt, and thermal state. See raw validation summaries.

Four-bit quantization has measurable quality loss. The base model also makes mistakes: one Chinese arithmetic prompt produced an incorrect answer in both the official BF16 model and the conversion. This Space is an inference demo, not an autonomous agent or a guarantee of accurate answers. Tool calls are XML; the demo does not execute them.