MiniCPM5-2B-OptiQ-5bpw

This is openbmb/MiniCPM5-2B quantized to mixed-precision OptiQ, target 5.0 bits/weight — actual allocation landed at 5.34 bpw pre-conversion (block-aware floor/run-guard protections took precedence over the exact budget) and 6.124 bpw post-conversion once MLX's per-group scale/bias overhead is included (calibration-driven KL-sensitivity bit allocation, candidates 4/8-bit) for use with MLX on Apple Silicon.

• Size on disk: ~1.8 GB
• Architecture: LlamaForCausalLM (standard, no custom code required)
• Quantized with: mlx-optiq (optiq convert --target-bpw 5.0)

Other quantizations of this model

• MXFP4
• MXFP8

Use with mlx-lm

pip install mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("sahilchachra/MiniCPM5-2B-OptiQ-5bpw")
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Hello, who are you?"}],
    add_generation_prompt=True, tokenize=False,
)
print(generate(model, tokenizer, prompt=prompt, max_tokens=200))

Verification

Smoke-tested via mlx_lm.load + mlx_lm.generate — produces coherent, on-topic completions matching the base model's expected chat-reasoning style.

Run in LM Studio

Verified working in LM Studio (loads via the MLX engine, tested through the local OpenAI-compatible API at localhost:1234/v1/chat/completions):

1. Download this repo, or symlink/copy the folder into ~/.lmstudio/models/<publisher>/<name>/.
2. LM Studio's model indexer will pick it up automatically (or run "Rescan").
3. Load it in the UI or via lms load <model-name>.

Confirmed a clean, correct response to a basic prompt with no truncation or garbled output.