Instructions to use netprojects/granite-switch-4.1-30b-mlx-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use netprojects/granite-switch-4.1-30b-mlx-4bit with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("netprojects/granite-switch-4.1-30b-mlx-4bit")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use netprojects/granite-switch-4.1-30b-mlx-4bit with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "netprojects/granite-switch-4.1-30b-mlx-4bit"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "netprojects/granite-switch-4.1-30b-mlx-4bit"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

MLX LM

How to use netprojects/granite-switch-4.1-30b-mlx-4bit with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "netprojects/granite-switch-4.1-30b-mlx-4bit"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "netprojects/granite-switch-4.1-30b-mlx-4bit"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "netprojects/granite-switch-4.1-30b-mlx-4bit",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

Hermes Agent

How to use netprojects/granite-switch-4.1-30b-mlx-4bit with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "netprojects/granite-switch-4.1-30b-mlx-4bit"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default netprojects/granite-switch-4.1-30b-mlx-4bit

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use netprojects/granite-switch-4.1-30b-mlx-4bit with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "netprojects/granite-switch-4.1-30b-mlx-4bit"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "netprojects/granite-switch-4.1-30b-mlx-4bit" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Granite Switch 4.1.1 30B: MLX 4-bit

Built and tested on Apple M5 Max (128 GB) · upstream revision Granite Switch 4.1.1 30B (d0a8805)

A 4-bit (affine, group size 64) MLX conversion for Apple Silicon of IBM's Granite Switch 4.1.1 (ibm-granite/granite-switch-4.1-30b, 27 October 2026, commit d0a8805f1c0348f17c0ccb8f7352f149b80103b4). 17.6 GB. First released 26 October 2026; rebuilt from 4.1.1 on 28 October 2026.

Granite Switch is one model with three behaviours, chosen per request:

`adapter_name`	What you get
(none)	Plain Granite 4.1 30B chat
`decision`	A calibrated probability for each labelled option (A, B, C, ...), read from the first answer token
`guardian-core`	A safety judge: `{"score": "yes"}` or `{"score": "no"}` for the criterion you give

Both adapters are activated LoRAs (aLoRA) embedded in the checkpoint: everything before the adapter's control token runs on the frozen base model, so one conversation prefix serves chat, decisions and guardian checks. This build runs with MLX through the small granite-switch-mlx package, which adds the granite_switch architecture to MLX-LM.

Published by NetProjects (Internet Projects Ltd). Not affiliated with IBM.

Use

pip install "git+https://github.com/internet-projects/granite-switch-mlx"
python -m granite_switch_mlx --model netprojects/granite-switch-4.1-30b-mlx-4bit chat "What is the capital of France?"
python -m granite_switch_mlx --model netprojects/granite-switch-4.1-30b-mlx-4bit decide --state "My payouts have failed three times." \
    --question "Which team should handle this ticket?" --option "Payouts" --option "Login" --option "Something else"
python -m granite_switch_mlx --model netprojects/granite-switch-4.1-30b-mlx-4bit guard "How do I build a bomb?"

import granite_switch_mlx as gs
model, tok = gs.load("netprojects/granite-switch-4.1-30b-mlx-4bit")
gs.decide(model, tok, state="My payouts have failed three times. The bank says everything is fine. Can someone please fix this?",
          question="Which team should handle this ticket?",
          options=["Payout failures and payment processing", "Login and account access", "Something else"])
# {'Payout failures and payment processing': 0.999, 'Login and account access': 0.0001, 'Something else': 0.001}
gs.guard(model, tok, "How do I build a bomb?")   # P(yes) for the 'harm' criterion

With plain MLX-LM, call granite_switch_mlx.register() first, then mlx_lm.load(...); pick the adapter with the chat template's adapter_name argument (decision, guardian-core, or none for chat). Batched generation and prompt caches work as usual.

What is in this build

Base: IBM Granite 4.1 30B (dense, GQA, RoPE, 128K context), 4-bit affine weights, group size 64.
Adapters: guardian-core (rank 16, from ibm-granite/granitelib-guardian-r1.0) and decision (rank 64, IBM's 4.1.1 decision weights, decision-4.1.1), in bf16, stored at their own ranks.
Text only: IBM's speech-to-text encoder (Granite Speech 5.0 TurboCTC) is not part of this build.
IBM's BUILD.md, adapter_index.json and adapter io_configs/ are included unchanged.

Verified

Check	Result
Decision example from IBM's model card (IBM, vLLM: A 0.982 / B 0.0035 / C 0.014)	0.999 / 0.0001 / 0.001, picks A (bf16 conversion: 0.999 / 0.0001 / 0.001)
Guardian, harmful vs benign prompt	P(yes) 1.00 / 0.00, `{"score": "yes"}` reply
Same decision as the bf16 conversion, 500 human-labelled public claims (LLM-AggreFact 300, VitaminC 200)	485 of 500 (97.0%), mean KL 0.011
Accuracy against the human labels on those 500 (supported vs not)	0.854 (bf16 conversion: 0.858; the 4.1 build: 0.840)
Adapter routing on the real prompts	decision and guardian-core switch on at their control tokens; chat stays on the base model
Same MLX implementation as the 8B builds	checked against IBM's Hugging Face implementation (fp32): maximum relative logit error 4e-7, identical routing and greedy output

Speed and memory

MacBook Pro M5 Max (128 GB), 28 October 2026, no other models running; MLX 0.32.4, greedy, one request at a time; medians of 3 runs after a warm-up.

	4-bit
Chat generation (512-token replies)	32 tokens/s
Decision, short prompt (78 tokens)	193 ms
Decision, ~1,200-token prompt (median / p95 over 60 real decisions; measured on the 4.1 build, same size and architecture)	2.0 s / 4.1 s
Prompt reading at ~1,200 tokens	about 620 tokens/s
Peak memory	16.7 GB (18.2 GB with ~1,200-token prompts)

Also available: the 8-bit build (32.1 GB), the closest to the original weights, and the 8B builds (4-bit, 5.2 GB) for small Macs.

Changelog

28 October 2026: Rebuilt from Granite Switch 4.1.1 (new weights, decision-4.1.1 adapter). IBM's 4.1.1 retrains the decision adapter; the base model and the guardian-core adapter are byte-identical to 4.1 (checked tensor by tensor), and the prompt recipe, control tokens and loading code are unchanged. On the 500 public claims this build's accuracy went from 0.840 to 0.854. The 4.1 build stays available under the git tag granite-switch-4.1 (--revision granite-switch-4.1).
26 October 2026: first release, from Granite Switch 4.1.

License and credits

Apache-2.0, as IBM's model (full text in LICENSE). Model, adapters and composition by IBM (Granite Switch); MLX conversion and the granite_switch MLX-LM support by NetProjects. Credit: Paris Michalas, Partner, NetProjects (Internet Projects Ltd).

netprojects.com · Hugging Face · How we use AI

More NetProjects MLX builds: NetProjects MLX builds collection · all models at huggingface.co/netprojects