3hs4n/MiniCPM-V-4.7-1B-mlx-8bit-compact

MiniCPM-V 4.7 1B: 8-bit compact

Community native MLX conversion of OpenBMB MiniCPM-V-4.7-1B. Architecture, tokenizer, chat template and visual processing modes are retained.

This repository provides the 8-bit compact weights. Comparison tables include other profiles evaluated with the same setup. Runtime corrections are required; follow the setup instructions before loading the model.

For a smaller download with faster text decoding in the local benchmark, see the mixed 4-bit and 8-bit compact model.

Language MLP and full-attention projections use affine 8-bit quantization with groups of 64. Embeddings and the output head use affine 8-bit quantization with groups of 32. Vision weights, visual mergers and linear-attention weights remain at their original precision. Unchanged tensors were verified individually by SHA-256. Details are in conversion.json.

Package size

Profile	Full package, decimal GB
Community 4-bit baseline	2.69
8-bit compact	2.40
Mixed 4-bit and 8-bit compact	2.24

The 8-bit compact weights occupy 2,375,477,210 bytes; the mixed compact weights occupy 2,221,336,424 bytes. These packages can be smaller than the community 4-bit baseline because they compress the embeddings and output head, which the baseline retains in BF16. The table includes model metadata and tokenizer files; sizes are rounded and do not measure runtime memory or speed.

Runtime compatibility

Tested through Unsloth Studio's MLX inference backend with Unsloth 2026.10.5, MLX 0.32.3, Transformers 5.5.0 and a locally repaired MLX-VLM 0.7.4 runtime. That runtime includes MiniCPM-V 4.7 support from upstream PR 2499.

This model also requires two runtime corrections: preserving full-prompt positions when resuming a cached text suffix, and selecting a floating vision dtype when embeddings use packed integer storage. The bundled helper supports exact MLX-VLM versions 0.7.4 and 0.7.8. For 0.7.4 it adds the architecture backport and both fixes; for 0.7.8 it applies the two-file runtime-fixes.patch. It verifies source hashes, backs up affected files and refuses unfamiliar or partially patched runtimes. Runtime versions and patch hashes are in runtime-compatibility.json and runtime-patch-manifest.json.

For an existing Unsloth installation, close the app and use its own backend Python interpreter. Set UNSLOTH_PYTHON locally to that executable, then run:

hf download 3hs4n/MiniCPM-V-4.7-1B-mlx-8bit-compact \
  --local-dir ./MiniCPM-V-4.7-1B-mlx-8bit-compact

"$UNSLOTH_PYTHON" ./MiniCPM-V-4.7-1B-mlx-8bit-compact/apply_runtime_patches.py --check
"$UNSLOTH_PYTHON" ./MiniCPM-V-4.7-1B-mlx-8bit-compact/apply_runtime_patches.py --apply

Restart Unsloth afterward. This path does not upgrade its dependencies. RUNTIME.md also provides fresh standalone installation commands, backup details and troubleshooting. The fresh 0.7.8 path has structural patch verification; the app, correctness and speed checks below used the repaired 0.7.4 stack.

In Unsloth, add the model folder through the local model selector and select this model. For a comparison of the model's own capabilities, turn off Search, Tools and Skills.

Platform scope

This conversion was verified in Unsloth on an Apple Silicon Mac and has not been tested on iPhone. As of 2026-10-13, the official Swift MLX VLM registry lacks minicpmv4_7 support. An iPhone app needs explicit support for this architecture and its vision processor before device memory use and latency can be measured. OpenBMB's official phone deployment project uses a separate GGUF and llama.cpp-omni path; it does not validate this MLX package on iPhone.

Tool-free correctness comparison

All four profiles passed 12 of the same 15 fixed fixtures. The 8-bit compact profile matched the BF16-source reference's pass/fail pattern; the mixed profile matched the existing 4-bit conversion's pattern. Unsloth casts floating weights and quantization scales to FP16 during loading, including those in the BF16-source reference; these checks compare the profiles under that runtime. All five OCR and image checks passed in each profile, including 4x and 16x image modes. Arithmetic and text reasoning had failures in the reference models too. The thinking puzzle did not complete within the shared 512-token output limit.

These are small correctness checks, not a comprehensive intelligence benchmark or a guarantee of unchanged accuracy. Text answers used a 128-token limit, with thinking disabled except for the thinking puzzle. Web search, tools and skills were absent. Exact prompts and raw answers are in comparison.json.

Unsloth GUI smoke check

Both compact profiles were selected through Unsloth Studio's local model selector, loaded and generated with Search, Tools and Skills off. A separate shared prompt used SKU AZ9-04, quantity 7 and unit price 12.50. The 8-bit compact profile returned the correct total, 87.50; the mixed compact profile returned 85.00, which was incorrect. This single GUI check is separate from the 15-fixture comparison above.

Mac text benchmark

Measured on 2026-10-13 UTC using an Apple M1 Max with 64 GiB unified memory, Unsloth 2026.10.5, patched MLX-VLM 0.7.4, MLX 0.32.3, MLX-LM 0.31.3 and Transformers 5.5.0. Observed conditions stayed on AC power with the screen locked and raw powermode 0. Low Power Mode was unreported.

Median decode tokens per second, with speedup versus the existing community 4-bit baseline in parentheses:

Case	Community 4-bit baseline	8-bit compact	Mixed 4-bit and 8-bit compact
Code	199.6 (1.000x)	208.0 (1.042x)	227.7 (1.141x)
Chat	201.5 (1.000x)	208.2 (1.033x)	227.3 (1.128x)
Explanation	201.3 (1.000x)	208.0 (1.033x)	227.5 (1.130x)

Thinking, Search, Tools and Skills were off. Each case and profile had one discarded warmup and three rotating repeats, generating 256 tokens each with prompt-cache reuse disabled. Every load gate was below 4; the largest measured decode-rate deviation from its case/profile median was 1.32%. The accepted receipt is in benchmark.json. Unstable measurements were excluded.

These results apply to cold text decoding in this local run. This benchmark does not establish vision speed, comparative prefill throughput, cached multi-turn performance or per-model RAM requirements. The speed results do not establish an intelligence guarantee.

Precision and image detail

Quantization is lossy. More demanding reasoning, languages, images and long contexts require further evaluation. The default image mode is 16x; 4x retains finer visual detail and increases visual-token work. Keeping vision weights unchanged preserves their stored values, while quantized language weights can still affect answers about images.

The upstream Apache 2.0 license and model limitations continue to apply. This conversion does not add training, distillation or changes to the source model's alignment.