Instructions to use dalatexcoder/MiniCPM5-1B-Heretic-MLX-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use dalatexcoder/MiniCPM5-1B-Heretic-MLX-4bit with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("dalatexcoder/MiniCPM5-1B-Heretic-MLX-4bit")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use dalatexcoder/MiniCPM5-1B-Heretic-MLX-4bit with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "dalatexcoder/MiniCPM5-1B-Heretic-MLX-4bit"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "dalatexcoder/MiniCPM5-1B-Heretic-MLX-4bit"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

MLX LM

How to use dalatexcoder/MiniCPM5-1B-Heretic-MLX-4bit with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "dalatexcoder/MiniCPM5-1B-Heretic-MLX-4bit"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "dalatexcoder/MiniCPM5-1B-Heretic-MLX-4bit"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "dalatexcoder/MiniCPM5-1B-Heretic-MLX-4bit",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

Hermes Agent

How to use dalatexcoder/MiniCPM5-1B-Heretic-MLX-4bit with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "dalatexcoder/MiniCPM5-1B-Heretic-MLX-4bit"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default dalatexcoder/MiniCPM5-1B-Heretic-MLX-4bit

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use dalatexcoder/MiniCPM5-1B-Heretic-MLX-4bit with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "dalatexcoder/MiniCPM5-1B-Heretic-MLX-4bit"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "dalatexcoder/MiniCPM5-1B-Heretic-MLX-4bit" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Quants of dalatexcoder/MiniCPM5-1B-heretic-som. 2, 3, 4, 5, 6, 8 bit mixed.

[INFO] Quantizing
[INFO] Quantized model with 4.306 bits per weight.
Output model size: 554.8 MB

Done! OptiQ model saved to: ./MiniCPM5-1B-Heretic-MLX-4bit/optiq_mixed

Raw Details:
Top 20 most-sensitive layers at 2-bit:
1. lm_head sens=4.799e-01 params=200,540,160
2. model.layers.1.self_attn.o_proj sens=2.809e-01 params=3,145,728
3. model.layers.4.mlp.down_proj sens=2.541e-01 params=7,077,888
4. model.layers.1.self_attn.v_proj sens=1.466e-01 params=393,216
5. model.layers.23.mlp.down_proj sens=1.338e-01 params=7,077,888
6. model.layers.0.mlp.down_proj sens=1.199e-01 params=7,077,888
7. model.layers.0.self_attn.o_proj sens=1.155e-01 params=3,145,728
8. model.layers.2.self_attn.v_proj sens=9.771e-02 params=393,216
9. model.layers.4.self_attn.v_proj sens=9.069e-02 params=393,216
10. model.layers.8.self_attn.o_proj sens=8.221e-02 params=3,145,728
11. model.layers.11.self_attn.o_proj sens=8.050e-02 params=3,145,728
12. model.layers.4.mlp.up_proj sens=7.259e-02 params=7,077,888
13. model.layers.1.mlp.down_proj sens=7.142e-02 params=7,077,888
14. model.layers.10.self_attn.o_proj sens=6.905e-02 params=3,145,728
15. model.layers.12.self_attn.o_proj sens=6.872e-02 params=3,145,728
16. model.layers.5.mlp.up_proj sens=6.575e-02 params=7,077,888
17. model.layers.1.self_attn.q_proj sens=6.040e-02 params=3,145,728
18. model.layers.23.mlp.up_proj sens=6.036e-02 params=7,077,888
19. model.layers.6.mlp.up_proj sens=5.986e-02 params=7,077,888
20. model.layers.3.mlp.up_proj sens=5.563e-02 params=7,077,888

Top 20 most-sensitive layers at 3-bit:
1. lm_head sens=1.042e-01 params=200,540,160
2. model.layers.4.mlp.down_proj sens=7.892e-02 params=7,077,888
3. model.layers.1.self_attn.v_proj sens=3.553e-02 params=393,216
4. model.layers.23.mlp.down_proj sens=3.540e-02 params=7,077,888
5. model.layers.1.self_attn.o_proj sens=2.688e-02 params=3,145,728
6. model.layers.2.self_attn.v_proj sens=2.617e-02 params=393,216
7. model.layers.0.mlp.down_proj sens=2.576e-02 params=7,077,888
8. model.layers.0.self_attn.o_proj sens=2.521e-02 params=3,145,728
9. model.layers.8.self_attn.o_proj sens=2.308e-02 params=3,145,728
10. model.layers.11.self_attn.o_proj sens=2.019e-02 params=3,145,728
11. model.layers.10.self_attn.o_proj sens=1.672e-02 params=3,145,728
12. model.layers.12.self_attn.o_proj sens=1.593e-02 params=3,145,728
13. model.layers.5.mlp.up_proj sens=1.564e-02 params=7,077,888
14. model.layers.9.self_attn.o_proj sens=1.542e-02 params=3,145,728
15. model.layers.1.mlp.down_proj sens=1.487e-02 params=7,077,888
16. model.layers.7.mlp.up_proj sens=1.464e-02 params=7,077,888
17. model.layers.2.self_attn.o_proj sens=1.432e-02 params=3,145,728
18. model.layers.3.self_attn.v_proj sens=1.425e-02 params=393,216
19. model.layers.6.mlp.up_proj sens=1.402e-02 params=7,077,888
20. model.layers.0.mlp.up_proj sens=1.391e-02 params=7,077,888

Top 20 most-sensitive layers at 4-bit:
1. lm_head sens=2.590e-02 params=200,540,160
2. model.layers.4.mlp.down_proj sens=1.958e-02 params=7,077,888
3. model.layers.1.self_attn.v_proj sens=1.038e-02 params=393,216
4. model.layers.23.mlp.down_proj sens=9.017e-03 params=7,077,888
5. model.layers.0.mlp.down_proj sens=7.697e-03 params=7,077,888
6. model.layers.1.self_attn.o_proj sens=7.600e-03 params=3,145,728
7. model.layers.2.self_attn.v_proj sens=7.303e-03 params=393,216
8. model.layers.0.self_attn.o_proj sens=7.085e-03 params=3,145,728
9. model.layers.8.self_attn.o_proj sens=5.952e-03 params=3,145,728
10. model.layers.11.self_attn.o_proj sens=5.067e-03 params=3,145,728
11. model.layers.5.mlp.up_proj sens=4.940e-03 params=7,077,888
12. model.layers.1.mlp.down_proj sens=4.700e-03 params=7,077,888
13. model.layers.10.self_attn.o_proj sens=4.471e-03 params=3,145,728
14. model.layers.4.self_attn.v_proj sens=4.441e-03 params=393,216
15. model.layers.3.self_attn.v_proj sens=4.279e-03 params=393,216
16. model.layers.12.self_attn.o_proj sens=4.129e-03 params=3,145,728
17. model.layers.3.mlp.up_proj sens=4.090e-03 params=7,077,888
18. model.layers.1.self_attn.q_proj sens=4.058e-03 params=3,145,728
19. model.layers.6.mlp.up_proj sens=4.025e-03 params=7,077,888
20. model.layers.2.self_attn.o_proj sens=3.845e-03 params=3,145,728

Top 20 most-sensitive layers at 5-bit:
1. lm_head sens=6.904e-03 params=200,540,160
2. model.layers.4.mlp.down_proj sens=4.846e-03 params=7,077,888
3. model.layers.1.self_attn.v_proj sens=3.023e-03 params=393,216
4. model.layers.0.self_attn.o_proj sens=2.795e-03 params=3,145,728
5. model.layers.0.mlp.down_proj sens=2.696e-03 params=7,077,888
6. model.layers.1.self_attn.o_proj sens=2.634e-03 params=3,145,728
7. model.layers.2.self_attn.v_proj sens=2.525e-03 params=393,216
8. model.layers.23.mlp.down_proj sens=2.442e-03 params=7,077,888
9. model.layers.1.mlp.down_proj sens=2.005e-03 params=7,077,888
10. model.layers.8.self_attn.o_proj sens=1.998e-03 params=3,145,728
11. model.layers.3.self_attn.v_proj sens=1.904e-03 params=393,216
12. model.layers.2.self_attn.o_proj sens=1.846e-03 params=3,145,728
13. model.layers.0.self_attn.v_proj sens=1.842e-03 params=393,216
14. model.layers.0.mlp.up_proj sens=1.832e-03 params=7,077,888
15. model.layers.11.self_attn.o_proj sens=1.815e-03 params=3,145,728
16. model.layers.4.mlp.up_proj sens=1.793e-03 params=7,077,888
17. model.layers.4.self_attn.v_proj sens=1.793e-03 params=393,216
18. model.layers.3.mlp.up_proj sens=1.780e-03 params=7,077,888
19. model.layers.3.self_attn.o_proj sens=1.770e-03 params=3,145,728
20. model.layers.1.self_attn.q_proj sens=1.769e-03 params=3,145,728

Top 20 most-sensitive layers at 6-bit:
1. model.layers.4.mlp.down_proj sens=2.093e-03 params=7,077,888
2. lm_head sens=1.755e-03 params=200,540,160
3. model.layers.1.self_attn.v_proj sens=1.749e-03 params=393,216
4. model.layers.0.self_attn.o_proj sens=1.682e-03 params=3,145,728
5. model.layers.0.mlp.down_proj sens=1.621e-03 params=7,077,888
6. model.layers.1.self_attn.o_proj sens=1.507e-03 params=3,145,728
7. model.layers.2.self_attn.v_proj sens=1.497e-03 params=393,216
8. model.layers.1.mlp.down_proj sens=1.370e-03 params=7,077,888
9. model.layers.0.self_attn.v_proj sens=1.366e-03 params=393,216
10. model.layers.0.mlp.up_proj sens=1.356e-03 params=7,077,888
11. model.layers.0.self_attn.k_proj sens=1.326e-03 params=393,216
12. model.layers.0.self_attn.q_proj sens=1.293e-03 params=3,145,728
13. model.layers.1.mlp.up_proj sens=1.293e-03 params=7,077,888
14. model.layers.1.self_attn.q_proj sens=1.292e-03 params=3,145,728
15. model.layers.2.self_attn.o_proj sens=1.278e-03 params=3,145,728
16. model.layers.0.mlp.gate_proj sens=1.273e-03 params=7,077,888
17. model.layers.1.mlp.gate_proj sens=1.259e-03 params=7,077,888
18. model.layers.3.self_attn.v_proj sens=1.251e-03 params=393,216
19. model.layers.2.mlp.up_proj sens=1.241e-03 params=7,077,888
20. model.layers.1.self_attn.k_proj sens=1.234e-03 params=393,216

Top 20 most-sensitive layers at 8-bit:
1. model.layers.0.self_attn.v_proj sens=1.262e-03 params=393,216
2. model.layers.0.self_attn.k_proj sens=1.251e-03 params=393,216
3. model.layers.0.self_attn.o_proj sens=1.249e-03 params=3,145,728
4. model.layers.0.mlp.down_proj sens=1.209e-03 params=7,077,888
5. model.layers.1.self_attn.v_proj sens=1.208e-03 params=393,216
6. model.layers.0.self_attn.q_proj sens=1.207e-03 params=3,145,728
7. model.layers.0.mlp.up_proj sens=1.200e-03 params=7,077,888
8. model.layers.0.mlp.gate_proj sens=1.197e-03 params=7,077,888
9. model.layers.1.self_attn.o_proj sens=1.176e-03 params=3,145,728
10. model.layers.1.self_attn.q_proj sens=1.158e-03 params=3,145,728
11. model.layers.1.mlp.up_proj sens=1.125e-03 params=7,077,888
12. model.layers.1.self_attn.k_proj sens=1.123e-03 params=393,216
13. model.layers.1.mlp.down_proj sens=1.122e-03 params=7,077,888
14. model.layers.1.mlp.gate_proj sens=1.118e-03 params=7,077,888
15. model.layers.2.self_attn.v_proj sens=1.109e-03 params=393,216
16. model.layers.4.mlp.down_proj sens=1.096e-03 params=7,077,888
17. model.layers.2.self_attn.o_proj sens=1.085e-03 params=3,145,728
18. model.layers.2.self_attn.q_proj sens=1.075e-03 params=3,145,728
19. model.layers.2.mlp.down_proj sens=1.074e-03 params=7,077,888
20. model.layers.3.self_attn.o_proj sens=1.072e-03 params=3,145,728

Downloads last month: 56

Safetensors

Model size

1B params

Tensor type

U32

BF16

MLX

Hardware compatibility

4-bit

Model tree for dalatexcoder/MiniCPM5-1B-Heretic-MLX-4bit

Base model

openbmb/MiniCPM5-1B

Finetuned

dalatexcoder/MiniCPM5-1B-heretic-som

Quantized

(3)

this model

dalatexcoder
/

MiniCPM5-1B-Heretic-MLX-4bit

Model tree for dalatexcoder/MiniCPM5-1B-Heretic-MLX-4bit

Datasets used to train dalatexcoder/MiniCPM5-1B-Heretic-MLX-4bit