Instructions to use AnwarPy/qwen3.5-9b-anwar-hermes-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AnwarPy/qwen3.5-9b-anwar-hermes-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="AnwarPy/qwen3.5-9b-anwar-hermes-GGUF",
	filename="qwen3.5-9b-anwar-hermes-Q4_K_M.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use AnwarPy/qwen3.5-9b-anwar-hermes-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf AnwarPy/qwen3.5-9b-anwar-hermes-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf AnwarPy/qwen3.5-9b-anwar-hermes-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf AnwarPy/qwen3.5-9b-anwar-hermes-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf AnwarPy/qwen3.5-9b-anwar-hermes-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf AnwarPy/qwen3.5-9b-anwar-hermes-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf AnwarPy/qwen3.5-9b-anwar-hermes-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf AnwarPy/qwen3.5-9b-anwar-hermes-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf AnwarPy/qwen3.5-9b-anwar-hermes-GGUF:Q4_K_M

Use Docker

docker model run hf.co/AnwarPy/qwen3.5-9b-anwar-hermes-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use AnwarPy/qwen3.5-9b-anwar-hermes-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "AnwarPy/qwen3.5-9b-anwar-hermes-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AnwarPy/qwen3.5-9b-anwar-hermes-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/AnwarPy/qwen3.5-9b-anwar-hermes-GGUF:Q4_K_M

Ollama
How to use AnwarPy/qwen3.5-9b-anwar-hermes-GGUF with Ollama:
```
ollama run hf.co/AnwarPy/qwen3.5-9b-anwar-hermes-GGUF:Q4_K_M
```

Unsloth Studio

How to use AnwarPy/qwen3.5-9b-anwar-hermes-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for AnwarPy/qwen3.5-9b-anwar-hermes-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for AnwarPy/qwen3.5-9b-anwar-hermes-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for AnwarPy/qwen3.5-9b-anwar-hermes-GGUF to start chatting

How to use AnwarPy/qwen3.5-9b-anwar-hermes-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf AnwarPy/qwen3.5-9b-anwar-hermes-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "AnwarPy/qwen3.5-9b-anwar-hermes-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use AnwarPy/qwen3.5-9b-anwar-hermes-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf AnwarPy/qwen3.5-9b-anwar-hermes-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default AnwarPy/qwen3.5-9b-anwar-hermes-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use AnwarPy/qwen3.5-9b-anwar-hermes-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf AnwarPy/qwen3.5-9b-anwar-hermes-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "AnwarPy/qwen3.5-9b-anwar-hermes-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use AnwarPy/qwen3.5-9b-anwar-hermes-GGUF with Docker Model Runner:
```
docker model run hf.co/AnwarPy/qwen3.5-9b-anwar-hermes-GGUF:Q4_K_M
```

Lemonade

How to use AnwarPy/qwen3.5-9b-anwar-hermes-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull AnwarPy/qwen3.5-9b-anwar-hermes-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.qwen3.5-9b-anwar-hermes-GGUF-Q4_K_M

List all available models

lemonade list

Qwen3.5-9B-Anwar-Hermes model overview by AnwarPy

Qwen3.5-9B-Anwar-Hermes — GGUF Q4_K_M

A compact GGUF release derived from Qwen/Qwen3.5-9B, fine-tuned for tool selection, structured output, function calling, and multi-step agent workflows in English and Arabic.

Built for Hermes Agent

Qwen3.5-9B-Anwar-Hermes is the result of a month-long training, dataset-curation, refinement, and evaluation cycle focused on practical compatibility with Hermes Agent and its tool ecosystem.

The model was shaped around function calling, structured tool schemas, terminal workflows, multi-step chains, and agent-style instructions. The goal is to provide a local model that works naturally with Hermes workflows—not a generic chat model with tool use added as an afterthought. Compatibility was validated against the documented Hermes-oriented behavior and terminal gates; it is not a claim of perfect parity across every tool, runtime, or future Hermes release.

At a glance


Format	GGUF · Q4_K_M · 5.24 GiB
Base model	Qwen3.5-9B
Development cycle	One month of training, curation, refinement, and evaluation
Target integration	Hermes Agent tools and multi-step workflows
Validated context	8,192 tokens
Primary strengths	Tool use, JSON, function calling, agent workflows
Languages	English and Arabic
Validated runtimes	llama.cpp, LM Studio, Windows llama.cpp

Performance snapshot

Evaluation	Result
tool-eval-bench v2.1.0	91/100 · 126/138 points
Tool Selection	100%
Multi-Step Chains	100%
Structured Output	100%
Context & State	90%
Frozen behavior suite	12/16 · threshold 11/16
Terminal contract suite	7/7

The benchmark covered 69 sequential scenarios with 52 tool definitions at temperature 0.0 and seed 42. These are task-specific results, not a claim of broad production readiness. The complete reproducibility summary is available in evals/tool-eval-bench-summary.md.

Quick start

1. Download

hf download AnwarPy/qwen3.5-9b-anwar-hermes-GGUF \
  qwen3.5-9b-anwar-hermes-Q4_K_M.gguf \
  --local-dir .

2. Start llama.cpp

llama-server \
  -m qwen3.5-9b-anwar-hermes-Q4_K_M.gguf \
  --alias qwen3.5-9b-anwar-hermes \
  -ngl 99 -c 8192 -np 1 \
  --host 127.0.0.1 --port 11442 \
  --jinja --reasoning off --reasoning-budget 0

3. Verify the API

curl http://127.0.0.1:11442/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "qwen3.5-9b-anwar-hermes",
    "messages": [{"role": "user", "content": "Reply with exactly: MODEL_OK"}],
    "temperature": 0
  }'

Expected assistant content:

MODEL_OK

The exact command above was validated against the published GGUF with llama.cpp commit e8bcb772d749a581d717b79ac8b9cd57301d9651. The same artifact passed load and inference canaries with official Windows llama.cpp b10078 (f04801018) and LM Studio engine 2.25.2. Runtime flags may vary in other versions.

Recommended use

Function calling and tool selection.
Strict or structured JSON generation.
Multi-step local agent workflows.
English and Arabic assistant tasks.
Local OpenAI-compatible inference through llama.cpp-based runtimes.

This release is text/tool-use oriented. Vision capability inherited from the base model was not qualified for this GGUF release and should not be assumed.

Files and verification

File	Purpose
`qwen3.5-9b-anwar-hermes-Q4_K_M.gguf`	Q4_K_M release · 5,629,108,704 bytes
`SHA256SUMS`	Release checksums
`PROVENANCE.json`	Machine-readable model, dataset, and compatibility lineage
`data/terminal-curriculum-build-v2.json`	Terminal curriculum build report
`evals/tool-eval-bench-summary.md`	Reproducible benchmark summary

Release GGUF SHA-256:

cf0cc3541a75739861c4208b47a7f163c8273f0e56b4132b9bebf2a357a2c7c3

License

Apache License 2.0. See LICENSE. Users are also responsible for reviewing the base-model and dataset terms.

Acknowledgements

Built from Qwen3.5 by the Qwen team and trained with pinned data from Nous Research, the Hermes Agent reasoning-trace dataset, and the audited derived terminal curriculum sourced from Roman1111111/gpt5.5-terminal. Quantization and inference use the GGUF and llama.cpp ecosystem.

Downloads last month: 274

GGUF

Model size

9B params

Architecture

qwen35

Hardware compatibility

4-bit

Model tree for AnwarPy/qwen3.5-9b-anwar-hermes-GGUF

Base model

Qwen/Qwen3.5-9B-Base

Finetuned

Qwen/Qwen3.5-9B

Finetuned

(528)

this model

AnwarPy
/

qwen3.5-9b-anwar-hermes-GGUF

Qwen3.5-9B-Anwar-Hermes — GGUF Q4_K_M

Built for Hermes Agent

At a glance

Performance snapshot

Quick start

1. Download

2. Start llama.cpp

3. Verify the API

Recommended use

Files and verification

License

Acknowledgements

Model tree for AnwarPy/qwen3.5-9b-anwar-hermes-GGUF

Datasets used to train AnwarPy/qwen3.5-9b-anwar-hermes-GGUF