loom-fun/unsloth-Qwen3.5-9B-loom

Qwen3.5-9B (Q4_K_M) for AMD Loom

A Loom package (.loom programs and a model.json card) for the unmodified GGUF checkpoint unsloth/Qwen3.5-9B-GGUF (Qwen3.5-9B-Q4_K_M.gguf), generated on an AMD Ryzen AI Max+ 395 "Strix Halo" with the loom-skills workflow.

Quick start

pip install "hrx-loom[hf]"
loom-fetch https://huggingface.co/loom-fun/unsloth-Qwen3.5-9B-loom/resolve/main/model.json --materialize=copy --output=Qwen3.5-9B
loom-serve --model=Qwen3.5-9B/model.json --device=amdgpu --memory_bytes=30000000000

The server listens on 127.0.0.1:8080 and answers OpenAI-style /v1/chat/completions requests.

Performance vs llama.cpp

metric	llama.cpp	Loom	margin	roofline	Loom % of roofline	DevLab llama.cpp	DevLab Loom	DevLab llama.cpp Vulkan	local ÷ DevLab (llama.cpp, Loom)
decode tok/s, short prompt	39.9	51.4	+29 %	104.4	49 %	43.5	52.0 ¹	45.3	0.92, 0.99
decode tok/s, 2k context	39.1	43.9	+12 %	102.5	43 %	43.2	41.0 ¹	43.5	0.91, 1.07
decode tok/s, 32k context	32.5	29.5	-9 %	80.2	37 %	31.8	23.9 ¹	29.9	1.02, 1.23
prefill tok/s, 1k document	943.0	1472.9	+56 %	4243.3	35 %	988.0	1341.9 ¹	856.0	0.95, 1.10
prefill tok/s, 2k document	941.2	1505.1	+60 %	4223.4	36 %	988.2	1322.8 ¹	877.2	0.95, 1.14
prefill tok/s, 8k document	950.3	1549.7	+63 %	4107.8	38 %	779.0	1380.2 ¹	889.2	1.22, 1.12
prefill tok/s, 16k document	857.0	1490.5	+74 %	3963.1	38 %	737.3	1321.0 ¹	843.1	1.16, 1.13
prefill tok/s, 32k document	735.2	1373.5	+87 %	3702.4	37 %	664.1	1180.0 ¹	754.2	1.11, 1.16
4 streams, aggregate decode tok/s	101.9	99.5	-2 %	402.9	25 %	—	99.6 ¹	109.9	—, 1.00
speculative decoding, drafted tokens accepted	—	45 %	—	n/a	n/a	n/a	n/a	n/a	n/a

• llama.cpp: build llamacpp-rocm/b1369, server arguments -ngl 99 -cram 0 --mmproj ~/.cache/huggingface/hub/models--unsloth--Qwen3.5-9B-GGUF/snapshots/3885219b6810b007914f3a7950a8d1b469d598a5/mmproj-F16.gguf --spec-type draft-dflash --spec-draft-model ~/.cache/huggingface/hub/models--AtomicChat--Qwen3.5-9B-DFlash-GGUF/snapshots/de44b8556576637d1049951f8fdbd98bcb2462f3/Qwen3.5-9B-DFlash.Q8_0.gguf --spec-draft-n-max 5 -ngld 99 -c 35024 -kvu -np 4.
• Loom: speculative decoding on (DFlash, depth 7).
• Same GGUF on both engines, tokens per second. Loom: client-side timing; llama.cpp: server-side timings. Runs per value: decode and prefill 2 (mean), 4 streams 3 (median).
• Measured 2026-11-06; device AMD RYZEN AI MAX+ 395 w/ Radeon 8060S, AMD Radeon 8060S Graphics, Ubuntu 26.04; hrx-loom 0.1.0 (hrx-system d01d1c1); source: local run.
• ¹ Measured 2026-11-07; device not recorded; hrx-loom unknown; source: DevLab Dispatch run 42395970032.
• DevLab: the same measurement on an AMD DevLab Dispatch Strix Halo runner; local ÷ DevLab is the local value over the DevLab value of each engine, recorded for comparison, not a verdict.
• DevLab llama.cpp Vulkan: build not recorded, server arguments -ngl 99 -cram 0 --spec-type draft-dflash --spec-draft-model card:parameter/Qwen3.5-9B-DFlash.Q8_0.gguf --spec-draft-n-max 5 -ngld 99 -c 65536 -kvu -np 4; same GGUF, prompts and settings as llama.cpp; measured 2026-11-09 in its own DevLab job (a different runner than the other DevLab columns: compare the engines through ratios, not across jobs); source: DevLab Dispatch run 42580835796. Differs from llama.cpp: decode_2k measured with unified KV, draft, 4 slots: -ngl 99 -cram 0 --spec-type draft-dflash --spec-draft-model card:parameter/Qwen3.5-9B-DFlash.Q8_0.gguf --spec-draft-n-max 5 -ngld 99 -c 65536 -kvu -np 4 (the original job, run 42574818485). Differs from llama.cpp: decode measured with 1 slot, draft: -ngl 99 -cram 0 --spec-type draft-dflash --spec-draft-model card:parameter/Qwen3.5-9B-DFlash.Q8_0.gguf --spec-draft-n-max 5 -ngld 99 -c 36864 -np 1. Differs from llama.cpp: decode_32k, prefill_1k, prefill_2k, prefill_8k, prefill_16k, prefill_32k measured with 1 slot, no draft: -ngl 99 -cram 0 -c 36864 -np 1. Differs from llama.cpp: streams4 measured with 4 per-slot streams, no draft: -ngl 99 -cram 0 -c 147456 -np 4. Differs from llama.cpp: each metric is the highest of the Vulkan server configurations measured (b12211, separate DevLab jobs); the ROCm baseline's own configuration is the unified pool with drafting. b12211's Vulkan backend has no GPU kernel for the fused flash attention over K/V rows that a unified pool uses (the server logs the CPU fallback). Differs from llama.cpp: vision projector (--mmproj) not loaded: the workloads are text-only.
• Drafted tokens accepted: the share of the drafter's proposed tokens the model accepted during the spec gate's requests.
• Roofline: the hardware ceiling for this GGUF on AMD Ryzen AI Max+ 395 "Strix Halo" (gfx1151) at 216 GB/s and 59 TFLOP/s, from loom-roofline roofline.py (loom-skills 4da7fb0); decode and 4 streams with speculative decoding: verify passes × (1 + acceptance × depth 7) tokens per pass, the drafter's cost not counted; acceptance measured in the row's own session: decode 21 %, decode at 2k 21 %, decode at 32k 18 %, 4 streams 20 %.
• — not measured; n/a not applicable.

Features

feature	status	detail
API	✅	OpenAI-style /v1/chat/completions
tool calling	✅	OpenAI-style tools and tool_calls
reasoning / thinking	✅	off by default; chat_template_kwargs: {"enable_thinking": true} turns it on
reasoning effort	—	not applicable
speculative decoding	✅	DFlash, depth 7, on by default; drafter AtomicChat/Qwen3.5-9B-DFlash-GGUF
image input	✅	OpenAI-style image_url parts; projector mmproj-F16.gguf
audio input / output	—	not applicable
sampling	✅	temperature, top_p, top_k, seed
context size	✅	262,144 tokens per request
concurrency	✅	4 concurrent requests
max output tokens	✅	4,096 (preset max_tokens)

Correctness

gate	description	result
kernel_tests	Unit tests of the generated kernels against reference implementations	✅ pass (52/52) ¹
golden	llama.cpp reference recorded on the same GGUF: greedy tokens and top-5 log-probs, short prompts and 1k-32k documents	✅ pass ²
compare	Greedy output vs the llama.cpp reference, token by token; every divergence is a near-tie	✅ pass ²
vision	Image input vs llama.cpp with the same projector: identical prompt tokens, replies equal up to near-ties	✅ pass ²
consistency	Concurrent requests give the same output as the same requests run alone	✅ pass ²
spec	Speculative decoding on gives exactly the output of speculative decoding off	✅ pass ²
agent	Multi-turn tool-calling loop: exact prompt rendering, history reuse, valid tool calls, output vs llama.cpp	✅ pass ²
agent_think_off	The tool-calling loop with thinking off	✅ pass ²
sampling	temperature, top_k and seed behave as specified; first-token distribution matches llama.cpp	✅ pass ²
needle	Needle-in-a-haystack prompt at up to half the context, compared with llama.cpp; over-capacity prompt refused	✅ pass ³
needle_full	Needle-in-a-haystack prompt at the full context is served	✅ pass ²
decision	Decision answers (choice, score, noul probabilities) vs llama.cpp on the same prompts	— not applicable

• ¹ local run, 2026-11-06; device AMD RYZEN AI MAX+ 395 w/ Radeon 8060S, AMD Radeon 8060S Graphics, Ubuntu 26.04; hrx-loom 0.1.0 (hrx-system d01d1c1).
• ² DevLab Dispatch run 42331885837, LEVEL=full, 2026-11-07; device AMD RYZEN AI MAX+ 395 w/ Radeon 8060S, AMD Radeon 8060S Graphics, Ubuntu 24.04; hrx-loom 0.1.0 (hrx-system d01d1c1).
• ³ DevLab Dispatch run 42328045047, LEVEL=quick, 2026-11-07; device AMD RYZEN AI MAX+ 395 w/ Radeon 8060S, AMD Radeon 8060S Graphics, Ubuntu 24.04; hrx-loom 0.1.0 (hrx-system d01d1c1).
• Gates are the loom-benchmark correctness gates (gates.sh); a gate without a recorded run shows no evidence.

Files

• model.json / model.json.lock: the card and the pinned source files (sha256 checked by loom-fetch).
• program/*.loom, weights.loom: the generated Loom programs.
• tokenizer.json: the tokenizer.
• llm_config.json: the configuration the programs were generated from.
• info.json: the measurements, gate evidence and versions this card shows; everything else comes from the files above.
• REPORT.md: how the values above were measured: setup, commands and the value of every run.

Versions: card schema loom-card/2; loom-skills 62c8af3; hrx-loom 0.1.0 (hrx-system d01d1c1); card generator 1ed9a3e.

Source files, fetched by loom-fetch from their own repositories:

• checkpoint: unsloth/Qwen3.5-9B-GGUF @ 3885219, Qwen3.5-9B-Q4_K_M.gguf
• drafter: AtomicChat/Qwen3.5-9B-DFlash-GGUF @ de44b85, Qwen3.5-9B-DFlash.Q8_0.gguf
• projector: unsloth/Qwen3.5-9B-GGUF @ 3885219, mmproj-F16.gguf