Instructions to use deucebucket/North-Mini-Code-Cerebellum-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use deucebucket/North-Mini-Code-Cerebellum-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="deucebucket/North-Mini-Code-Cerebellum-GGUF",
	filename="North-Mini-Code-Cerebellum-v1.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use deucebucket/North-Mini-Code-Cerebellum-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf deucebucket/North-Mini-Code-Cerebellum-GGUF
# Run inference directly in the terminal:
llama cli -hf deucebucket/North-Mini-Code-Cerebellum-GGUF

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf deucebucket/North-Mini-Code-Cerebellum-GGUF
# Run inference directly in the terminal:
llama cli -hf deucebucket/North-Mini-Code-Cerebellum-GGUF

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf deucebucket/North-Mini-Code-Cerebellum-GGUF
# Run inference directly in the terminal:
./llama-cli -hf deucebucket/North-Mini-Code-Cerebellum-GGUF

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf deucebucket/North-Mini-Code-Cerebellum-GGUF
# Run inference directly in the terminal:
./build/bin/llama-cli -hf deucebucket/North-Mini-Code-Cerebellum-GGUF

Use Docker

docker model run hf.co/deucebucket/North-Mini-Code-Cerebellum-GGUF

LM Studio
Jan

vLLM

How to use deucebucket/North-Mini-Code-Cerebellum-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "deucebucket/North-Mini-Code-Cerebellum-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "deucebucket/North-Mini-Code-Cerebellum-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/deucebucket/North-Mini-Code-Cerebellum-GGUF

Ollama
How to use deucebucket/North-Mini-Code-Cerebellum-GGUF with Ollama:
```
ollama run hf.co/deucebucket/North-Mini-Code-Cerebellum-GGUF
```

Unsloth Studio

How to use deucebucket/North-Mini-Code-Cerebellum-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for deucebucket/North-Mini-Code-Cerebellum-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for deucebucket/North-Mini-Code-Cerebellum-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for deucebucket/North-Mini-Code-Cerebellum-GGUF to start chatting

How to use deucebucket/North-Mini-Code-Cerebellum-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf deucebucket/North-Mini-Code-Cerebellum-GGUF

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "deucebucket/North-Mini-Code-Cerebellum-GGUF"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use deucebucket/North-Mini-Code-Cerebellum-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf deucebucket/North-Mini-Code-Cerebellum-GGUF

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default deucebucket/North-Mini-Code-Cerebellum-GGUF

Run Hermes

hermes

Atomic Chat new
Docker Model Runner
How to use deucebucket/North-Mini-Code-Cerebellum-GGUF with Docker Model Runner:
```
docker model run hf.co/deucebucket/North-Mini-Code-Cerebellum-GGUF
```

Lemonade

How to use deucebucket/North-Mini-Code-Cerebellum-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull deucebucket/North-Mini-Code-Cerebellum-GGUF

Run and chat with the model

lemonade run user.North-Mini-Code-Cerebellum-GGUF-{{QUANT_TAG}}

List all available models

lemonade list

Cerebellum

North-Mini-Code 1.0 — Cerebellum GGUF

Coding-ablation-guided mixed-precision quantization of CohereLabs/North-Mini-Code-1.0 (cohere2moe, 128-expert MoE).

Variant	File	Size
Cerebellum v1	`North-Mini-Code-Cerebellum-v1.gguf`	13.57 GB

Cerebellum measures, per tensor group, what actually breaks the model's coding ability when crushed to Q2_K — using real HumanEval pass@1 deltas, not perplexity (perplexity is blind to code generation). It then writes one GGUF that protects the groups that matter and crushes the rest. On North, the per-group HumanEval ablation found exactly one coding-critical group — the routed down-projection experts — so v1 keeps those at Q4_K_M and crushes everything else to Q2_K.

Benchmarks

Measured directly on this GGUF with llama.cpp (current master). --parallel 1, temperature 0.

Benchmark	North-Mini-Code-Cerebellum-v1 (13.57 GB)
HumanEval base (thinking on / off)	86.6% / 84.8%
HumanEval+ (thinking on / off)	82.9% / 79.3%
ARC-Challenge (1172)	92.2%
MMLU-Redux (250)	78.0%
HellaSwag (250)	55.2%

Important: measuring HumanEval correctly

North-Mini-Code is a reasoning-native model — it emits its full chain-of-thought in the response content (no <think> delimiter for llama.cpp to split on). A naive EvalPlus/HumanEval harness extracts the reasoning prose instead of the final code and massively under-scores the model (~51% on a stock extractor). The numbers above use a reasoning-aware extractor that recovers the model's actual final code block before execution; on inspection, the remaining failures are genuine logic errors plus a small number of reasoning-overflow cases, not extraction noise. If you bench this model, strip the reasoning to the final code block, or disable thinking (below).

Per the project's standing rule, comparisons are to the base model and same-size baselines only.

v1 Allocation

Built from the complete per-group HumanEval coding ablation (baseline = uniform Q4_K_M, 8 groups, real pass@1 deltas):

Group	Precision	Why
`ffn_down_exps` (routed)	Q4_K_M	The one coding-critical group — Q2_K here dropped HumanEval ~37 pts. Protected.
`ffn_(gate\|up)_exps` (routed)	Q2_K	Crushing it improved coding; ~4.5 GB of the savings
`attn_q` / `attn_k` / `attn_v` / `attn_output`	Q2_K	Free or beneficial at Q2_K per the ablation
`token_embd` (tied)	Q2_K	Free / beneficial
`blk.0` dense ffn	Q2_K	Free

Norms protected (default). Base type Q4_K_M; only the groups above are overridden.

Requirements

This is a cohere2moe model — it needs a llama.cpp build with cohere2-MoE support (merged to master, PR #24260 / commit 4988f6e). The GGUF uses the tiny_aya pre-tokenizer; older/PR-head builds that expect cohere2moe will fail to load. Build from current ggml-org/llama.cpp master.

Usage

# default (reasoning on — the model performs best with thinking enabled)
llama-server --model North-Mini-Code-Cerebellum-v1.gguf -ngl 99 -c 8192 --jinja

# no-thinking (faster, clean final output)
llama-server --model North-Mini-Code-Cerebellum-v1.gguf -ngl 99 -c 8192 --jinja \
  --reasoning off --reasoning-budget 0
# (also pass chat_template_kwargs {"enable_thinking": false} in the request)

Fits a 16 GB card. The model reasons in the content channel; for agent loops, pass the reasoning content forward between turns (per the base model's guidance).

Evidence

Per-benchmark detail and the coding-ablation results are in benchmark_results/.

Downloads last month: 486

GGUF

Model size

30B params

Architecture

cohere2moe

Hardware compatibility

We're not able to determine the quantization variants.

View all variants

Model tree for deucebucket/North-Mini-Code-Cerebellum-GGUF

Base model

CohereLabs/North-Mini-Code-1.0

Quantized

(33)

this model

Evaluation results

pass@1 on HumanEval (pass@1)
test set Local run (RTX 3090, llama.cpp, reasoning-aware extraction; thinking-off)

0.848
pass@1 on HumanEval+ (pass@1)
test set Local run (RTX 3090, llama.cpp, reasoning-aware extraction; thinking-off)

0.793
normalized accuracy on AI2 Reasoning Challenge
test set Local benchmark run (RTX 3090, llama.cpp)

0.922
accuracy on HellaSwag
validation set Local benchmark run (RTX 3090, llama.cpp)

0.552
accuracy on MMLU-Redux
test set Local benchmark run (RTX 3090, llama.cpp)

0.780