Instructions to use QuantPasture/MiniMax-M2.5-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use QuantPasture/MiniMax-M2.5-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf QuantPasture/MiniMax-M2.5-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf QuantPasture/MiniMax-M2.5-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf QuantPasture/MiniMax-M2.5-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf QuantPasture/MiniMax-M2.5-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf QuantPasture/MiniMax-M2.5-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf QuantPasture/MiniMax-M2.5-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf QuantPasture/MiniMax-M2.5-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf QuantPasture/MiniMax-M2.5-GGUF:Q4_K_M

Use Docker

docker model run hf.co/QuantPasture/MiniMax-M2.5-GGUF:Q4_K_M

LM Studio
Jan
Ollama
How to use QuantPasture/MiniMax-M2.5-GGUF with Ollama:
```
ollama run hf.co/QuantPasture/MiniMax-M2.5-GGUF:Q4_K_M
```

Unsloth Studio

How to use QuantPasture/MiniMax-M2.5-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for QuantPasture/MiniMax-M2.5-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for QuantPasture/MiniMax-M2.5-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for QuantPasture/MiniMax-M2.5-GGUF to start chatting

How to use QuantPasture/MiniMax-M2.5-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf QuantPasture/MiniMax-M2.5-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "QuantPasture/MiniMax-M2.5-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

OpenClaw new

How to use QuantPasture/MiniMax-M2.5-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf QuantPasture/MiniMax-M2.5-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "QuantPasture/MiniMax-M2.5-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use QuantPasture/MiniMax-M2.5-GGUF with Docker Model Runner:
```
docker model run hf.co/QuantPasture/MiniMax-M2.5-GGUF:Q4_K_M
```

Lemonade

How to use QuantPasture/MiniMax-M2.5-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull QuantPasture/MiniMax-M2.5-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.MiniMax-M2.5-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use QuantPasture/MiniMax-M2.5-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf QuantPasture/MiniMax-M2.5-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default QuantPasture/MiniMax-M2.5-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.

by imweijh - opened Feb 21

Discussion

imweijh

Feb 21

•

edited Feb 21

build-cuda/bin/llama-server --version
ggml_cuda_init: found 1 CUDA devices:
  Device 0: NVIDIA GB10, compute capability 12.1, VMM: yes
version: 8120 (07968d53e)
built with GNU 13.3.0 for Linux aarch64

build-cuda/bin/llama-server -m ~/.cache/llama.cpp/AesSedai_MiniMax-M2.5-IQ3_S-00001-of-00003.gguf -c 32000 --no-mmap

slot   load_model: id  3 | task -1 | speculative decoding context not initialized
slot   load_model: id  3 | task -1 | new slot, n_ctx = 32000
srv    load_model: prompt cache is enabled, size limit: 8192 MiB
srv    load_model: use `--cache-ram 0` to disable the prompt cache
srv    load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
init: chat template, example_format: ']~!b[]~b]system
You are a helpful assistant[e~[
]~b]user
Hello[e~[
]~b]ai
Hi there[e~[
]~b]user
How are you?[e~[
]~b]ai
<think>
'
render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
render_message_to_json: Neither string content nor typed content is supported by the template. This is unexpected and may lead to issues.
srv          init: init: chat template, thinking = 1
main: model loaded
main: server is listening on http://127.0.0.1:8080
main: starting the main loop...
srv  update_slots: all slots are idle

First, thank you for your GGUF quantization.
There’s a small warning when running, and using Unsloth’s chat template seems to solve the problem.

AesSedai

Aes Sedai's Quant Pasture org Feb 21

Try using the autoparser branch on llama.cpp from pwilkin: https://github.com/ggml-org/llama.cpp/pull/18675

blankreg

Mar 15

Since autoparser fixes have been merged in llama the template issue should be solved?

AesSedai

Aes Sedai's Quant Pasture org Mar 15

Yeah the autoparser merge should have fixed most of these issues :)

deleted

Apr 15

Still getting this with M2.7 :/ Are your quants doing something unusual that needs a special parser?

AesSedai

Aes Sedai's Quant Pasture org Apr 15

They shouldn't be, I'm using the upstream template from MiniMax directly.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment