MiniCPM5-2B-MXFP8

This is openbmb/MiniCPM5-2B quantized to 8-bit MXFP8 (group size 32) for use with MLX on Apple Silicon.

• Size on disk: ~2.6 GB
• Architecture: LlamaForCausalLM (standard, no custom code required)
• Quantized with: mlx_lm.convert -q --q-mode mxfp8

Other quantizations of this model

• MXFP4
• OptiQ 5-bpw

Use with mlx-lm

pip install mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("sahilchachra/MiniCPM5-2B-MXFP8")
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Hello, who are you?"}],
    add_generation_prompt=True, tokenize=False,
)
print(generate(model, tokenizer, prompt=prompt, max_tokens=200))

Verification

Smoke-tested via mlx_lm.load + mlx_lm.generate — produces coherent, on-topic completions matching the base model's expected chat-reasoning style.

Run in LM Studio

Verified working in LM Studio (loads via the MLX engine, tested through the local OpenAI-compatible API at localhost:1234/v1/chat/completions):

1. Download this repo, or symlink/copy the folder into ~/.lmstudio/models/<publisher>/<name>/.
2. LM Studio's model indexer will pick it up automatically (or run "Rescan").
3. Load it in the UI or via lms load <model-name>.

Confirmed a clean, correct response to a basic prompt with no truncation or garbled output.