Instructions to use GEHIRNS/magi-1.1-4B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use GEHIRNS/magi-1.1-4B with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf GEHIRNS/magi-1.1-4B
# Run inference directly in the terminal:
llama cli -hf GEHIRNS/magi-1.1-4B

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf GEHIRNS/magi-1.1-4B
# Run inference directly in the terminal:
llama cli -hf GEHIRNS/magi-1.1-4B

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf GEHIRNS/magi-1.1-4B
# Run inference directly in the terminal:
./build/bin/llama cli -hf GEHIRNS/magi-1.1-4B

Use Docker

docker model run hf.co/GEHIRNS/magi-1.1-4B

Llama new
LM Studio
Jan

vLLM

How to use GEHIRNS/magi-1.1-4B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "GEHIRNS/magi-1.1-4B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "GEHIRNS/magi-1.1-4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/GEHIRNS/magi-1.1-4B

Ollama
How to use GEHIRNS/magi-1.1-4B with Ollama:
```
ollama run hf.co/GEHIRNS/magi-1.1-4B
```
Unsloth Desktop

How to use GEHIRNS/magi-1.1-4B with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf GEHIRNS/magi-1.1-4B

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "GEHIRNS/magi-1.1-4B"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use GEHIRNS/magi-1.1-4B with Docker Model Runner:
```
docker model run hf.co/GEHIRNS/magi-1.1-4B
```

Lemonade

How to use GEHIRNS/magi-1.1-4B with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull GEHIRNS/magi-1.1-4B

Run and chat with the model

lemonade run user.magi-1.1-4B-{{QUANT_TAG}}

List all available models

lemonade list

Hermes Agent

How to use GEHIRNS/magi-1.1-4B with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf GEHIRNS/magi-1.1-4B

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default GEHIRNS/magi-1.1-4B

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use GEHIRNS/magi-1.1-4B with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf GEHIRNS/magi-1.1-4B

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "GEHIRNS/magi-1.1-4B" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Introduction

magi-1.1-4B is Qwen3.5-4B in 2.61 GiB at a 32K context. On our 2,313-task benchmark it scores 81.7%, against 84.0% for the BF16 model in 9.51 GiB, and it thinks 28% less than magi-1.0-4B.

It is magi-1.0-4B with one LoRA fine-tune merged in before quantization:

The fine-tune teaches the model to finish its thinking. Under the 4,096-token thinking budget, magi-1.0-4B runs out of budget on 35% of tasks. The adapter trains on 360 of the model's own correct, short reasoning traces; 40 of them are for prompts on which magi-1.0-4B loops under greedy decoding.
The magi-1.0-4B mixed-precision recipe, unchanged: the same 13 tensor-class rules, quantizer and importance matrix. The file is the same size as magi-1.0-4B's.

Against magi-1.0-4B on the same 2,313 tasks (paired):

Accuracy: +1.24 points (110 tasks gained, 89 lost, standard error 0.61). That is 2.0 standard errors, at the edge of the 2-SE bar we use to call a gain real.
Thinking: 28% fewer thinking tokens; thinking-budget hits fall from 35% to 18% of tasks.
Time: each benchmark task takes 27% less time.

Capability density

Capability density is ρ = −log₂(1 − score) / GiB: how many times the benchmark error rate is halved per gibibyte of memory. Memory for all three models is measured the same way: llama.cpp with a q4_0 KV cache at a 32K context, from llama-fit-params. The magi-1.0-4B card's chart used Ollama's memory figures, so its numbers are not comparable with these.

Thinking

magi-1.1-4B reaches its answer sooner. It ends thinking on its own more often, so fewer tasks are cut off at the 4,096-token budget and fewer traces end in a repetition loop.

Benchmarks

Every column was run on one RTX 5070 (12 GB) under one protocol:

Tasks: 2,313, run with thinking on and a 4,096-token thinking budget (budget forcing), greedy decoding.
Server: magi-1.0-4B and magi-1.1-4B on llama.cpp b11800 (llama-server) with a q4_0 KV cache, flash attention and the default micro-batch (512). The BF16 column is our earlier run of the same tasks through Ollama. Run both ways, magi-1.0-4B scores 80.64% (Ollama) and 80.49% (llama.cpp), so the runtime moves the score by about 0.15 points.
Code grading: by executing tests (EvalPlus for HumanEval+ and MBPP+). LiveCodeBench uses 150 problems from contests between 2025-01-18 and 2025-04-06.

Memory. Memory is the resident size llama-fit-params projects for a 32K context with one sequence and micro-batch 128: weights, KV cache, recurrent state and compute buffers, GPU plus host, without the CUDA context. These sizes are measured differently from the Ollama figures on the magi-1.0-4B card, so compare them only within this table.

Benchmark	Qwen3.5-4B (BF16) _{9.51 GiB}	magi-1.0-4B _{2.61 GiB}	magi-1.1-4B _{2.61 GiB}
Efficiency
Memory at 32K (GiB)	9.51	2.61	2.61
Benchmark score	83.96%	80.49%	81.72%
Capability density ρ	0.278	0.903	0.939
Decode speed (tok/s)	66	186	189
Mean thinking tokens	2,222	2,233	1,597
Tasks hitting the thinking budget	31.6%	34.9%	18.3%
Traces ending in a loop	5.7%	15.0%	4.4%
Coding
HumanEval+	91.46	86.59	82.93
MBPP+	72.18	70.43	69.42
LiveCodeBench	35.33	24.00	32.67
Math
GSM8K	87.33	86.00	92.67
Science and Knowledge
ARC-Challenge	98.67	97.33	98.00
MMLU	84.50	79.00	81.50
OpenBookQA	95.00	93.50	95.50
SciQ	98.00	97.00	98.00
TruthfulQA	82.50	75.50	76.50
Commonsense
HellaSwag	75.50	73.50	69.50
WinoGrande	87.00	83.00	84.00
Tool Use
Function calling	100.00	100.00	100.00

magi-1.1-4B against BF16: −2.23 points (paired across the two runtimes, standard error 0.63; 77 tasks gained, 134 lost) in 27% of the memory.

Where it gains and loses against magi-1.0-4B: LiveCodeBench +8.7 and GSM8K +6.7 points; HellaSwag −4.0 and HumanEval+ −3.7.

Decode speed: single-stream generation speed (llama-bench, 128 tokens, q4_0 KV), on the same GPU in the same session.

Recipe

magi-1.0-4B's rules, unchanged, on Qwen3.5-4B's 32 layers (24 Gated DeltaNet, 8 full attention):

Tensor group	Type	Share of parameters	bits/weight
FFN gate, up, down (all 32 layers)	IQ3_XXS	53.9%	3.06
Token embedding = tied output head	Q4_K	15.1%	4.50
DeltaNet input projection `attn_qkv` (24)	Q4_K	12.0%	4.50
Full-attention q, k, v, output (8 layers)	Q4_K	7.0%	4.50
Gated-attention output gate (24 DeltaNet layers)	IQ3_XXS	6.0%	3.06
DeltaNet output projection `ssm_out` (24)	Q4_K	6.0%	4.50
DeltaNet gates `ssm_alpha`, `ssm_beta`	Q8_0	<0.1%	8.50
Norms, convolution, A, Δt bias	F32	<0.1%	32.00

The quantizer is llama-quantize --pure from the PrismML-Eng/llama.cpp fork with magi-1.0-4B's override file and base type IQ2_XXS. The importance matrix is magi-1.0-4B's, computed on Qwen3.5-4B (BF16), not recomputed for the fine-tune. The file holds 4,205,751,296 parameters. File SHA-256: cb321e7fd3e11ca6600c23242c8d5738c0d230333513b8ee70e25398795ac1f0.

Training

Adapter: LoRA rank 16, α 32, on all MLP projections and the q/k/v/o projections of the 8 full-attention layers (128 modules, 21.2M parameters).
Optimisation: QLoRA. The BF16 base was loaded in 4-bit NF4 so that training fits a 12 GB GPU. 2 epochs (90 steps), learning rate 1e-4 with warmup and cosine decay, effective batch 8. The adapter is merged into the BF16 weights before quantization.
Data: 360 rows, each a prompt and one of Qwen3.5-4B's own answers with its thinking.
- The answers were sampled from Qwen3.5-4B with thinking on, at Qwen's recommended settings (temperature 0.6, top_p 0.95, top_k 20).
- A sample was kept only if it was correct, finished thinking on its own within 3,000 tokens, and was the shortest such sample for its prompt.
- 40 rows are for prompts on which magi-1.0-4B loops under greedy decoding.
- Benchmark questions were excluded by id, normalized text and hash.
Prompt sources: the licenses are as declared on each dataset's Hugging Face card.

Dataset	Split	Rows	Declared license
cais/mmlu	validation, dev	109	MIT
google-research-datasets/mbpp	train, validation	74	CC BY 4.0
allenai/winogrande	train	42	not declared
openai/gsm8k	train	37	MIT
Rowan/hellaswag	train	30	not declared
allenai/openbookqa	train	24	unknown
allenai/ai2_arc	train	22	CC BY-SA 4.0
allenai/sciq	train	15	CC BY-NC 3.0
livecodebench/code_generation_lite	problems dated before 2025-01-18	7	cc (variant not stated)

The training rows hold the model's own outputs, not the datasets' answer texts. Some sources carry non-commercial or share-alike terms. If those terms matter for your use, check them.

How to run

llama.cpp (the evaluated configuration; uses the Qwen3.5 chat template embedded in the file):

llama-server -m magi-1.1-4B.gguf -c 32768 --jinja --flash-attn on --cache-type-k q4_0 --cache-type-v q4_0

Ollama:

ollama create magi-1.1-4B -f Modelfile
ollama run magi-1.1-4B

The benchmark used greedy decoding. For interactive use with thinking enabled, Qwen recommends temperature 0.6, top_p 0.95 and top_k 20.

Limitations

The accuracy gain over magi-1.0-4B is small and at the edge of significance: +1.24 points, 2.0 SE. The large, clear change is in thinking length.
It is not better everywhere: HellaSwag (−4.0) and HumanEval+ (−3.7) are lower than magi-1.0-4B.
Still loops sometimes: 4% of greedy benchmark traces end in a repetition loop (magi-1.0-4B: 15%).
The name is ours. "4B" follows Qwen's naming; the file holds 4,205,751,296 parameters.
Narrow evaluation: one benchmark, one GPU, each configuration run once; the magi columns on one runtime (llama.cpp b11800), the BF16 column on Ollama. Text only: the base model's vision encoder is not included.

GEHIRNS/magi-1.0-4B: the model and recipe this release builds on.
GEHIRNS/magi-1.1-2B: the same approach on Qwen3.5-2B.
GEHIRNS/magi-1.0-2B: the recipe alone on Qwen3.5-2B.

Citation

The recipe is described in:

@misc{sun2026wherethebitsgo,
  title     = {Where the Bits Go: Sensitivity-Allocated Mixed Precision for a 4B Reasoning Model on a 12 GB Consumer GPU},
  author    = {Sun, Eason},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.23630199},
  url       = {https://doi.org/10.5281/zenodo.23630199}
}

License

The weights are Apache 2.0, inherited from Qwen/Qwen3.5-4B; the license text is included in this repository. See Training for the terms of the datasets the fine-tuning prompts came from.