Instructions to use mlx-community/MiniCPM-V-4.7-35B-A3B-5bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mlx-community/MiniCPM-V-4.7-35B-A3B-5bit with MLX:

# Make sure mlx-vlm is installed
# pip install --upgrade mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

# Load the model
model, processor = load("mlx-community/MiniCPM-V-4.7-35B-A3B-5bit")
config = load_config("mlx-community/MiniCPM-V-4.7-35B-A3B-5bit")

# Prepare input
image = ["http://images.cocodataset.org/val2017/000000039769.jpg"]
prompt = "Describe this image."

# Apply chat template
formatted_prompt = apply_chat_template(
    processor, config, prompt, num_images=1
)

# Generate output
output = generate(model, processor, formatted_prompt, image)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use mlx-community/MiniCPM-V-4.7-35B-A3B-5bit with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "mlx-community/MiniCPM-V-4.7-35B-A3B-5bit"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "mlx-community/MiniCPM-V-4.7-35B-A3B-5bit"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent

How to use mlx-community/MiniCPM-V-4.7-35B-A3B-5bit with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "mlx-community/MiniCPM-V-4.7-35B-A3B-5bit"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default mlx-community/MiniCPM-V-4.7-35B-A3B-5bit

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use mlx-community/MiniCPM-V-4.7-35B-A3B-5bit with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "mlx-community/MiniCPM-V-4.7-35B-A3B-5bit"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "mlx-community/MiniCPM-V-4.7-35B-A3B-5bit" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

MiniCPM-V-4.7-35B-A3B-5bit

Converted from openbmb/MiniCPM-V-4.7-35B-A3B at revision 17e651151c161dce119cd0ac57570f60da8a0b40 with MLX-VLM main (e928101a144a3f11a27c7ef8d71b6d889b2d250c, version 0.7.8). Support was added in PR #2499.

5BIT weight quantization (affine, group size 64). This is mixed precision: vision modules, visual mergers, embeddings, output heads, and linear attention remain BF16; recurrent A_log tensors retain their source precision. The 35B-A3B expert routers use 8-bit affine quantization.

See the original model card for the architecture, intended uses, and limitations. This conversion retains the source model's apache-2.0 license.

Usage

Install MLX-VLM from main with MiniCPM-V 4.7 support:

pip install -U git+https://github.com/Blaizzy/mlx-vlm.git@main
mlx_vlm.generate --model mlx-community/MiniCPM-V-4.7-35B-A3B-5bit --image path/to/image.jpg --prompt "Describe this image." --max-tokens 200 --temperature 0

The native NumPy/Pillow processor requires neither PyTorch nor remote code. It supports text, images, multiple images, and video. The default visual downsampling is 16x; pass --processor-kwargs '{"downsample_mode": "4x"}' for finer detail.

Reproduce conversion

mlx_vlm.convert --hf-path openbmb/MiniCPM-V-4.7-35B-A3B --revision 17e651151c161dce119cd0ac57570f60da8a0b40 --mlx-path MiniCPM-V-4.7-35B-A3B-5bit --dtype bfloat16 -q --q-bits 5 --q-group-size 64 --q-mode affine

For the exact converter version, install MLX-VLM at commit e928101a144a3f11a27c7ef8d71b6d889b2d250c. Conversion options, per-module quantization overrides, and weight SHA-256 hashes are recorded in conversion.json.

Validation

The saved checkpoint passed strict reloading, quantization configuration checks, greedy text generation, and cat-image generation in both 16x and 4x downsampling modes on JarvisLabs RTX PRO 6000 Blackwell (cuda) with MLX 0.32.3. See validation.json for the outputs and environment. These are smoke checks, not a quality benchmark. Floating-point 4-bit formats can differ in output quality from affine quantization.

Downloads last month: 307

Safetensors

Model size

35B params

Tensor type

BF16

U32

MLX

Hardware compatibility

5-bit

Model tree for mlx-community/MiniCPM-V-4.7-35B-A3B-5bit

Base model

openbmb/MiniCPM-V-4.7-35B-A3B

Quantized

(74)

this model