Instructions to use Zephyrs33/aurelius-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Zephyrs33/aurelius-v2 with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="Zephyrs33/aurelius-v2",
	filename="aurelius-v2-Q4_K_M.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use Zephyrs33/aurelius-v2 with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Zephyrs33/aurelius-v2:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Zephyrs33/aurelius-v2:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Zephyrs33/aurelius-v2:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Zephyrs33/aurelius-v2:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Zephyrs33/aurelius-v2:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf Zephyrs33/aurelius-v2:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Zephyrs33/aurelius-v2:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Zephyrs33/aurelius-v2:Q4_K_M

Use Docker

docker model run hf.co/Zephyrs33/aurelius-v2:Q4_K_M

LM Studio
Jan
Ollama
How to use Zephyrs33/aurelius-v2 with Ollama:
```
ollama run hf.co/Zephyrs33/aurelius-v2:Q4_K_M
```

Unsloth Studio

How to use Zephyrs33/aurelius-v2 with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Zephyrs33/aurelius-v2 to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Zephyrs33/aurelius-v2 to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Zephyrs33/aurelius-v2 to start chatting

How to use Zephyrs33/aurelius-v2 with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Zephyrs33/aurelius-v2:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Zephyrs33/aurelius-v2:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use Zephyrs33/aurelius-v2 with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Zephyrs33/aurelius-v2:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Zephyrs33/aurelius-v2:Q4_K_M

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use Zephyrs33/aurelius-v2 with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Zephyrs33/aurelius-v2:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Zephyrs33/aurelius-v2:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use Zephyrs33/aurelius-v2 with Docker Model Runner:
```
docker model run hf.co/Zephyrs33/aurelius-v2:Q4_K_M
```

Lemonade

How to use Zephyrs33/aurelius-v2 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Zephyrs33/aurelius-v2:Q4_K_M

Run and chat with the model

lemonade run user.aurelius-v2-Q4_K_M

List all available models

lemonade list

Aurelius v2 — verifier-native serving system (model card)

Base: Qwen/Qwen3-Coder-30B-A3B-Instruct (Apache-2.0, MoE 30.5B total / 3.3B active). Clean-teacher provenance ✓. What v2 is: the strong open base + a thin, MEASURED serving layer — one lever per axis, each the one that actually worked.

Results (measured on this base)

axis	single-pass	+ serving lever	lift	lever
MATH-500 L5 (n=134)	greedy 65.7%	maj@8 73.9%	+8.2pp	self-consistency (verifier-free)
MATH-500 (n=100)	85.0%	—	—	(near-saturated)
GSM8K (n=100)	98.0%	—	—	(saturated)
HumanEval	~96.7%	best-of-N+repair	(small; saturated)	exec-verifier
MBPP	~70.5%	best-of-N+repair	~+10pp*	exec-verifier
*code best-of-N+repair lift measured on the v1 14B (+10.5pp MBPP / +8.5pp HumanEval); same mechanism, re-runnable here.

Capability lever = CAPACITY (the real win)

30B-A3B vs 14B, zero training: MATH-500 +33pp, GSM8K +20pp, MATH-L5 greedy 65.7 vs ~47. The base swap is the capability gain; the serving layer cashes the cheap selection/repair headroom on top.

What worked / what didn't (honest)

✅ Capacity (bigger base) — the capability lever.
✅ maj@N self-consistency (math) — free, +8.2pp on hard math. SHIPPED.
✅ best-of-N + repair (code, exec-verifier) — +10.5pp MBPP on v1; carried to v2.
✗ LLM-as-verifier rerank (math) — NULL: verifier@8 73.1% ≈ maj@8 73.9% (1 worse). Zero-shot self-verification does not beat majority voting; the remaining ~7.4pp selection gap (maj→oracle 81.3%) needs a TRAINED verifier/RLVR.
✗ SFT-for-reasoning — confirmed neutral (v1: MATH 46.5%=base 46.5%). Reasoning is inherited, not SFT-able.
✗ RLVR on frozen 8B / flywheel-into-greedy — prior nulls; not revisited.

Usage

solve("math", "<problem>")                      # -> {answer, confidence, votes}
solve("code", "<task asking for one ```python block>", tests="<assert lines>")   # -> {code, how, passed}

maj@N: no verifier needed (ships anywhere). Code best-of-N: needs caller-supplied tests as the verifier. Compute ~Nx greedy. Math max_new=4096 (think=1 needs room; check no-box). Apache → public-release-eligible (keep private until gated). Future work (gated on compute): trained verifier / RLVR-for-selection for the residual ~7.4pp.

Limitations

Code lever is only as strong as the caller's tests. Tests stricter than the true spec (e.g. demanding a list when a correct algorithm returns a tuple) report failure even on correct code — spec the output type in the prompt and/or normalize in the tests. solve_code surfaces feedback on failure to debug this.
maj@N self-consistency helps only where the model samples the right answer sometimes (no help on hard-capability problems it never gets — ~15% of MATH-L5 never-recovered).

Files

aurelius_serve.py — self-contained serving (maj@N math + best-of-N+repair code)
*.gguf — quantized weights for local serving (if present)

from aurelius_serve import AureliusServe
s = AureliusServe()
print(s.solve('math', 'What is 12*13?'))

Downloads last month: 10

GGUF

Model size

31B params

Architecture

qwen3moe

Hardware compatibility

4-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Zephyrs33/aurelius-v2

Base model

Qwen/Qwen3-Coder-30B-A3B-Instruct

Quantized

(153)

this model