Instructions to use SmallAICreator/AuroraGPT-ToolFix with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use SmallAICreator/AuroraGPT-ToolFix with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="SmallAICreator/AuroraGPT-ToolFix")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("SmallAICreator/AuroraGPT-ToolFix")
model = AutoModelForCausalLM.from_pretrained("SmallAICreator/AuroraGPT-ToolFix", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use SmallAICreator/AuroraGPT-ToolFix with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf SmallAICreator/AuroraGPT-ToolFix:Q8_0
# Run inference directly in the terminal:
llama cli -hf SmallAICreator/AuroraGPT-ToolFix:Q8_0

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf SmallAICreator/AuroraGPT-ToolFix:Q8_0
# Run inference directly in the terminal:
llama cli -hf SmallAICreator/AuroraGPT-ToolFix:Q8_0

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf SmallAICreator/AuroraGPT-ToolFix:Q8_0
# Run inference directly in the terminal:
./llama-cli -hf SmallAICreator/AuroraGPT-ToolFix:Q8_0

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf SmallAICreator/AuroraGPT-ToolFix:Q8_0
# Run inference directly in the terminal:
./build/bin/llama-cli -hf SmallAICreator/AuroraGPT-ToolFix:Q8_0

Use Docker

docker model run hf.co/SmallAICreator/AuroraGPT-ToolFix:Q8_0

LM Studio
Jan

vLLM

How to use SmallAICreator/AuroraGPT-ToolFix with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "SmallAICreator/AuroraGPT-ToolFix"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SmallAICreator/AuroraGPT-ToolFix",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/SmallAICreator/AuroraGPT-ToolFix:Q8_0

SGLang

How to use SmallAICreator/AuroraGPT-ToolFix with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "SmallAICreator/AuroraGPT-ToolFix" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SmallAICreator/AuroraGPT-ToolFix",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "SmallAICreator/AuroraGPT-ToolFix" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SmallAICreator/AuroraGPT-ToolFix",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use SmallAICreator/AuroraGPT-ToolFix with Ollama:
```
ollama run hf.co/SmallAICreator/AuroraGPT-ToolFix:Q8_0
```

Unsloth Studio

How to use SmallAICreator/AuroraGPT-ToolFix with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for SmallAICreator/AuroraGPT-ToolFix to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for SmallAICreator/AuroraGPT-ToolFix to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for SmallAICreator/AuroraGPT-ToolFix to start chatting

How to use SmallAICreator/AuroraGPT-ToolFix with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf SmallAICreator/AuroraGPT-ToolFix:Q8_0

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "SmallAICreator/AuroraGPT-ToolFix:Q8_0"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use SmallAICreator/AuroraGPT-ToolFix with Docker Model Runner:
```
docker model run hf.co/SmallAICreator/AuroraGPT-ToolFix:Q8_0
```

Lemonade

How to use SmallAICreator/AuroraGPT-ToolFix with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull SmallAICreator/AuroraGPT-ToolFix:Q8_0

Run and chat with the model

lemonade run user.AuroraGPT-ToolFix-Q8_0

List all available models

lemonade list

Hermes Agent

How to use SmallAICreator/AuroraGPT-ToolFix with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf SmallAICreator/AuroraGPT-ToolFix:Q8_0

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default SmallAICreator/AuroraGPT-ToolFix:Q8_0

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use SmallAICreator/AuroraGPT-ToolFix with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf SmallAICreator/AuroraGPT-ToolFix:Q8_0

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "SmallAICreator/AuroraGPT-ToolFix:Q8_0" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

AuroraGPT-ToolFix (700M)

A 707M-parameter chat model by UltraLabs. This is AuroraGPT-Math with a tool-robustness LoRA applied — it keeps the math gains while restoring reliable tool-calling across any system-prompt wording.

This model supersedes AuroraGPT-Math for tool use and math. For code generation use AuroraGPT-Qwen-Distill instead — see the HumanEval numbers below.

The problem it fixes

AuroraGPT-Math added ~250k math examples, which diluted the tool-calling signal ~5x in the training mix. The result: tool calls still fired on the exact system-prompt wording used in training, but became unreliable when an app phrased the tool spec differently (e.g. a <tools> JSON block instead of the trained natural-language list). In practice this meant tool-calling broke inside a real on-device chat app.

The fix: prompt-variation augmentation

Every one of the 2,751 tool examples was replayed under 12 different system-prompt wordings — the original phrasing, a <tools> JSON spec block, terse, verbose, bulleted, numbered, XML-ish, generic-assistant, JSON-only, and with no system prompt at all — producing ~33k tool examples. This teaches the model that tool-calling binds to intent, not to one memorized string. Mixed with chat/identity data so nothing else drifts. LoRA r16, 1 epoch, merged.

Measured results

Valid tool calls by system-prompt wording (3 prompts each, locally benchmarked):

system prompt style	AuroraGPT-Math	AuroraGPT-ToolFix
exact trained wording	3/3	3/3
`<tools>` JSON block	2/3	3/3
terse	3/3	3/3
generic assistant	3/3	3/3
numbered list	3/3	3/3
total	14/15	15/15

Verified working in a real on-device GGUF chat app, where the previous model failed.

Inherited from AuroraGPT-Math: closed-book arithmetic roughly doubled vs the original flagship (5/16 → 10/16 on a 16-question set) via ~250k procedurally generated, correct-by-construction chain-of-thought math examples.

vs LiquidAI LFM2-700M

Same size class, both Q8_0, each model in its own native chat format.

Tool-calling (5 prompts):

	AuroraGPT-ToolFix	LFM2-700M
valid tool calls	5/5	0/5

LFM2 attempts tool use but emits unparseable pseudo-code (e.g. <tool_call>fetch_url(url="...")</tool_call>) rather than JSON, so no app harness can execute it.

General chat quality (20-question objective set), with AuroraGPT's tools enabled and calculator calls actually executed — i.e. how it's actually deployed:

section	AuroraGPT-ToolFix	LFM2-700M
facts	6/6	6/6
math	4/5	5/5
instruction-following	3/5	5/5
safety (should refuse)	1/2	1/2
over-refusal (should answer)	2/2	2/2
TOTAL	16/20	19/20

On math, AuroraGPT mostly delegates to the calculator tool rather than computing inline — 4 of 5 math questions were answered by emitting a calculator call and reading back the result. That is the intended design ("own the behavior, rent the facts"), and it is why tools-enabled scoring is the honest measure: with tools disabled the same model scores 12/20, because correct tool calls go unexecuted.

Honest read: LFM2-700M is still better at general chat quality (19/20 vs 16/20). AuroraGPT's advantage is tool-calling, which LFM2 cannot do at all.

HumanEval (code generation)

Measured on all 164 problems, greedy decoding, completions executed against the real unit tests.

model	raw completion	chat-formatted
AuroraGPT-Qwen-Distill	0/164 (0.00%)	15/164 (9.15%)
AuroraGPT-Math	–	12/164 (7.32%)
AuroraGPT-ToolFix	0/164 (0.00%)	6/164 (3.66%)
AuroraGPT-700M (pre-chat-SFT, earlier run)	9/164 (5.49%)	–
Qwen2.5-0.5B-Instruct (reference)	46/164 (28.05%)	–

Read the chat column, not the raw one. Under the standard HumanEval protocol (bare function stub, no chat template) these models emit literally nothing and score a clean 0/164 — the chat SFT taught them that text outside <|user|>...<|assistant|> terminates immediately. That 0 measures prompt-format incompatibility, not coding ability. Wrapping the same problems in the chat template and extracting the code block recovers a real score. Anyone benchmarking a chat-tuned small model should check for this failure mode before reporting a zero.

Code ability declined across the fine-tuning stages, monotonically: Qwen-Distill 15 → Math 12 → ToolFix 6. The endpoints differ significantly (two-proportion z = 2.03, p = 0.042), but neither individual step does (p = 0.55 and p = 0.15 respectively), so the data does not identify a single culprit — it is a gradual alignment tax across both the math SFT and the tool LoRA rather than one bad stage. None of the fine-tuning data contained code.

If code generation matters to you, use AuroraGPT-Qwen-Distill instead — it is ~2.5x better at HumanEval. ToolFix wins on tool robustness and math; it is not strictly superior.

Honest limitations

Little/no safety refusal training. On a 2-prompt safety probe it refused only 1/2 — it will comply with some requests it should decline. This is a real gap, not a benchmark artifact. Do not deploy user-facing without a separate safety layer.
General chat quality trails LFM2-700M (16/20 vs 19/20 even with tools enabled). AuroraGPT's edge is tool-calling, not across-the-board quality.
Instruction-following is inconsistent (3/5) — it can miss exact-format constraints like "reply with only yes or no" or "say hello in all caps".
Inline arithmetic is still fragile — it usually routes math to the calculator tool (good), but when it answers inline it can be wrong (e.g. "15% of 80" → 20, correct is 12). Keep the calculator tool available when exactness matters.
False-premise correction is weak — it can confidently agree with a popular myth.
700M capacity limits apply: closed-book knowledge is thin by design — pair with web_search/fetch_url.

Chat format (NOT ChatML)

<|system|>{system}<|end|><|user|>{user}<|end|><|assistant|>{reply}<|end|>

Tool call (model emits): <tool_call>\n{"name": "...", "arguments": {...}}\n</tool_call> Tool result (feed back as a user turn): <|user|><tool_response>\n{result}\n</tool_response><|end|>

Tool-calling now works with a wide range of system-prompt phrasings, so most app-provided tool specs should trigger it.

Usage (transformers)

from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("SmallAICreator/AuroraGPT-ToolFix")
model = AutoModelForCausalLM.from_pretrained("SmallAICreator/AuroraGPT-ToolFix")
msgs = [{"role": "user", "content": "What's the capital of Burkina Faso? Look it up."}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt")
print(tok.decode(model.generate(ids, max_new_tokens=100)[0][ids.shape[1]:], skip_special_tokens=True))

On-device (llama.cpp / GGUF)

AuroraGPT-ToolFix.Q8_0.gguf (753MB) is included with a tool-capable chat template embedded.

Made by UltraLabs. EOS token is <|end|>.

Downloads last month: 685

Safetensors

Model size

0.7B params

Tensor type

F16

Model tree for SmallAICreator/AuroraGPT-ToolFix

Quantizations

1 model