Instructions to use JamieBradfield/qwen3.8-9b-hermes-function-calling-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use JamieBradfield/qwen3.8-9b-hermes-function-calling-v2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="JamieBradfield/qwen3.8-9b-hermes-function-calling-v2")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("JamieBradfield/qwen3.8-9b-hermes-function-calling-v2")
model = AutoModelForMultimodalLM.from_pretrained("JamieBradfield/qwen3.8-9b-hermes-function-calling-v2", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use JamieBradfield/qwen3.8-9b-hermes-function-calling-v2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "JamieBradfield/qwen3.8-9b-hermes-function-calling-v2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "JamieBradfield/qwen3.8-9b-hermes-function-calling-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/JamieBradfield/qwen3.8-9b-hermes-function-calling-v2

SGLang

How to use JamieBradfield/qwen3.8-9b-hermes-function-calling-v2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "JamieBradfield/qwen3.8-9b-hermes-function-calling-v2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "JamieBradfield/qwen3.8-9b-hermes-function-calling-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "JamieBradfield/qwen3.8-9b-hermes-function-calling-v2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "JamieBradfield/qwen3.8-9b-hermes-function-calling-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use JamieBradfield/qwen3.8-9b-hermes-function-calling-v2 with Docker Model Runner:
```
docker model run hf.co/JamieBradfield/qwen3.8-9b-hermes-function-calling-v2
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Qwen3.8-9B Hermes Function Calling v2

A QLoRA fine-tune of Empero/Qwen3.8-9B (Qwen3.5 hybrid-architecture distillation, 9B dense, 262,144-token context).

Base model: Empero/Qwen3.8-9B (Apache-2.0)
Training data: 979 tool-call conversations in ShareGPT format. 933 are real Hermes agent trajectories (multi-turn sessions with tool execution results, avg 12 turns, max 35). 46 are gap-fill rows that bridge sparse tool sequences.
Method: QLoRA 4-bit, rank 16 / alpha 16, dropout 0, targets q/k/v/o/gate/up/down_proj, lr 2e-4, batch 1 x grad-accum 8 (effective 8), MAX_SEQ 4096, warmup 0.1, 2 epochs = 246 planned steps
Trainable params: 0.32% (embeddings frozen)
Vocabulary: 248077 → 248079 (2 added tool tokens: <|tool_call|>, <|tool_response|>)
Checkpoint note: training stopped at step 213/246 (86%). The published weights are the checkpoint-200 merge (loss 0.131). The remaining ~33 steps offered marginal gain for roughly 8 more hours of compute.
Weights: BF16 full merge (12 shards, 18.4 GB) plus a ROCmFPX quantized GGUF (see companion repo)

Intended use

This model is intended for experimenters working on tool-call behavior in 9B-class models. It is a research artifact, not a product.

What this repo contains

The BF16 merged weights (this repo)
All scripts that produced the model and dataset (scripts/)

Evaluation status

Independent evaluation of this checkpoint is not complete. The training data, scripts, and harness in scripts/ are published so that anyone — including the author — can evaluate it. Do not treat this model as benchmarked or validated for any specific behavior.

Training data note

The training data itself is not published. The rows derive from the author's own agent sessions, and some rows contain private strings (hostnames, session identifiers). The scripts that build the dataset are published; the data is not. The weights do not contain those strings.

Reproducibility

All scripts that produced this model are in scripts/:

file	purpose
`build_v2.py` / `gapfill_v2.py`	dataset build from mined trajectories (the 979 rows)
`qwen38_train_v2.py`	QLoRA training via Unsloth (246 planned steps)
`merge_qwen38_v2_pure.py`	shard-streamed BF16 merge (pure torch; vision tower dropped; embedding/lm_head resized with EOS-anchor rows)
`subagent-qwen38-workloads.py` + `subagent-qwen38.sh`	tool-call evaluation harness (workloads A–E)

The scripts carry the author's machine paths (F:/models/..., C:/AI/...) and run on this rig's ROCm stack (Unsloth 2026.8.18, torch 2.11 ROCm, RX 7700 XT); adjust paths for your environment.

Convert/quantize: llama-rocmfpx fork, convert_hf_to_gguf.py --outtype bf16 → llama-quantize Q4_0_ROCMFP4_FAST (see GGUF repo).

Quantized version

Q4_0_ROCMFP4_FAST GGUF (ROCmFPX format, for AMD RDNA3 kernels) is published in the companion repo. ROCmFPX quants target AMD ROCm inference; for portable use, convert from the BF16 merge here.

Acknowledgements

Base model and license inherited from Empero/Qwen3.8-9B (Apache-2.0). Training data derived from the author's own Hermes agent sessions.

Downloads last month: -

Safetensors

Model size

9B params

Tensor type

BF16

Model tree for JamieBradfield/qwen3.8-9b-hermes-function-calling-v2

Quantizations

1 model