Instructions to use AtomicChat/Nanbeige4.3-33B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use AtomicChat/Nanbeige4.3-33B-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M

Use Docker

docker model run hf.co/AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M

Llama new
LM Studio
Jan

vLLM

How to use AtomicChat/Nanbeige4.3-33B-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "AtomicChat/Nanbeige4.3-33B-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AtomicChat/Nanbeige4.3-33B-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M

Ollama
How to use AtomicChat/Nanbeige4.3-33B-GGUF with Ollama:
```
ollama run hf.co/AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M
```
Unsloth Desktop

How to use AtomicChat/Nanbeige4.3-33B-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use AtomicChat/Nanbeige4.3-33B-GGUF with Docker Model Runner:
```
docker model run hf.co/AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M
```

Lemonade

How to use AtomicChat/Nanbeige4.3-33B-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Nanbeige4.3-33B-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use AtomicChat/Nanbeige4.3-33B-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use AtomicChat/Nanbeige4.3-33B-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Nanbeige4.3-33B · GGUF

GGUF quants of Nanbeige's Nanbeige4.3-33B, built from Nanbeige's original BF16 weights on release day.

Nanbeige4.3-33B is Nanbeige's 33B dense agent and reasoning model (Llama architecture, 512K context, thinking on by default), Apache-2.0.
You can now run Nanbeige4.3-33B in Atomic Chat, with thinking on by default.
Atomic Dynamic (AD) files with our own importance matrix, from 13.7 GB to 30.1 GB on disk, plus plain Q8_0 and Q5_K_M; every one measured against the BF16 original on one stand.

Built on release day, 25 October. Q8_0 and Q5_K_M are llama.cpp's own presets, made without an importance matrix; its Q6_K and Q4_K_M were up for the first hours and are removed now that AD- files beat them at the same size (their numbers stay below as the baseline). The AD- files use our own importance matrix, collected on the BF16 weights (--parse-special) over 8,000 chunks of 512 tokens of our calibration corpus. The corpus build was rendered for Qwen3.8-27B, whose chat format (ChatML) Nanbeige4.3 shares; it is imatrix/imatrix.gguf in the metrics repo.

Pick a file

KL divergence is how far the quantized model's predictions drift from the original BF16 weights. Lower is better, and zero means identical. top-1 is how often it picks the same next token the original would have picked. Every file was measured on the same stand: our neutral multilingual text, 24 chunks of 4,096 tokens, llama.cpp 36e36e6. Raw logs and the importance matrix are in the metrics repo.

File	Size on disk	Built	KL divergence	top-1
Nanbeige4.3-33B-Q8_0.gguf	35.2 GB	plain 8-bit	0.0032	98.0%
Nanbeige4.3-33B-AD-Q6_K.gguf	30.1 GB	Atomic Dynamic, our imatrix	0.0052	97.5%
Nanbeige4.3-33B-AD-Q6_K-Q5_K.gguf	27.9 GB	Atomic Dynamic, our imatrix	0.0102	96.1%
Nanbeige4.3-33B-AD-Q5_K_M.gguf	24.7 GB	Atomic Dynamic, our imatrix	0.0213	94.7%
Nanbeige4.3-33B-Q5_K_M.gguf	23.5 GB	llama.cpp preset, no imatrix	0.0304	93.4%
Nanbeige4.3-33B-AD-Q5_K_M-Q4_K_M.gguf	22.6 GB	Atomic Dynamic, our imatrix	0.0332	93.1%
Nanbeige4.3-33B-AD-Q4_K_M.gguf	21.2 GB	Atomic Dynamic, our imatrix	0.0380	92.6%
Nanbeige4.3-33B-AD-IQ4_XS.gguf	20.3 GB	Atomic Dynamic, our imatrix	0.0462	92.0%
Nanbeige4.3-33B-AD-IQ3_S.gguf	17.1 GB	Atomic Dynamic, our imatrix	0.1019	87.5%
Nanbeige4.3-33B-AD-IQ3_XXS.gguf	14.9 GB	Atomic Dynamic, our imatrix	0.1593	84.6%
Nanbeige4.3-33B-AD-IQ2_S.gguf	13.7 GB	Atomic Dynamic, our imatrix	0.2448	80.5%

Removed, measured on the same stand: llama.cpp's plain Q6_K (27.2 GB, KL divergence 0.0165, top-1 95.5%) and Q4_K_M (20.1 GB, 0.0693, 89.9%).

This model is unusually sensitive to quantization. llama.cpp's plain Q6_K picks the original's next token only 95.5% of the time (most models reach 98% at 6 bits) and plain Q4_K_M 89.9%. At about the same size on disk, the AD- files keep much more: AD-IQ4_XS (20.3 GB) has a third less KL divergence than Q4_K_M (20.1 GB), and AD-Q6_K-Q5_K (27.9 GB) 39% less than Q6_K (27.2 GB). AD-Q6_K (30.1 GB) is within reach of Q8_0 (97.5% against 98.0%). Below 4 bits the model loses a lot: AD-IQ3_XXS and AD-IQ2_S change the next token often (top-1 below 85%); use them only if nothing larger fits.

AD- marks an Atomic Dynamic layout: the FFN of the first and last layers one step up, attention output and the head at q6_k or higher, the small key and value projections at q8_0. The rest of the name is the type of the FFN up projections. Compare files by size on disk, not by name.

Run

llama.cpp:

llama-server -hf AtomicChat/Nanbeige4.3-33B-GGUF:AD-Q4_K_M --jinja -c 32768

Thinking is on by default. To turn it off: --chat-template-kwargs '{"enable_thinking": false}'.

Nanbeige's recommended sampling: reasoning and chat --temp 0.6 --top-p 0.95; agentic and tool use --temp 1.0 --top-p 0.95.

Or download a file and open it in Atomic Chat.

License and credits

The weights are Nanbeige's, released under Apache-2.0 (LICENSE in this repo). Quantized by AtomicChat.

Downloads last month: 8,885

GGUF

Model size

33B params

Architecture

llama

Hardware compatibility

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

Model tree for AtomicChat/Nanbeige4.3-33B-GGUF

Base model

Nanbeige/Nanbeige4.3-33B-Base

Finetuned

Nanbeige/Nanbeige4.3-33B

Quantized

(70)

this model

Collection including AtomicChat/Nanbeige4.3-33B-GGUF

Nanbeige4.3

Collection

GGUF quants of Nanbeige4.3-33B by AtomicChat • 2 items • Updated 1 day ago