Instructions to use baa-ai/MiniMax-M2.7-RAM-100GB-MLX with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use baa-ai/MiniMax-M2.7-RAM-100GB-MLX with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("baa-ai/MiniMax-M2.7-RAM-100GB-MLX")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps
LM Studio

Pi new

How to use baa-ai/MiniMax-M2.7-RAM-100GB-MLX with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "baa-ai/MiniMax-M2.7-RAM-100GB-MLX"

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "baa-ai/MiniMax-M2.7-RAM-100GB-MLX"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use baa-ai/MiniMax-M2.7-RAM-100GB-MLX with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "baa-ai/MiniMax-M2.7-RAM-100GB-MLX"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default baa-ai/MiniMax-M2.7-RAM-100GB-MLX

Run Hermes

hermes

MLX LM

How to use baa-ai/MiniMax-M2.7-RAM-100GB-MLX with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "baa-ai/MiniMax-M2.7-RAM-100GB-MLX"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "baa-ai/MiniMax-M2.7-RAM-100GB-MLX"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "baa-ai/MiniMax-M2.7-RAM-100GB-MLX",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

Which device was benchmarking done?

by spanspek - opened Apr 18

Discussion

spanspek

Apr 18

In the README you mention the inference speed as

36.4 tok/s (wall-gen) / 36.8 tok/s (task-mean)

This would vary by device based on memory bandwidth across the different Apple Silicon devices

Which variant of Apple Silicon or device was used for benchmarking? (M4 Max, M3 Ultra, M5 Pro, etc)

tomkay

baa.ai org Apr 19

That one specifically was on an older M2 Pro, so certainly if you have the lasted M5 Pro you are going to do much better.

spanspek

Apr 20

Thanks, M2 Pro has 200GB/s memory bandwidth

M5 Pro = 307GB/s
M5 Max = 614GB/s

Current king is still M3 Ultra = 819GB/s but M5 Ultra should come out later this year (M4 Ultra does not exist)

tomkay changed discussion status to closed Apr 20

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment