loom-fun/InternScience-Sage-A1.5-loom

Sage-A1.5-Q4_K_M (Q4_K_M) for AMD Loom

A Loom package (.loom programs and a model.json card) for the unmodified GGUF checkpoint InternScience/Sage-A1.5-Q4_K_M-GGUF (Sage-A1.5-Q4_K_M.gguf), generated on an AMD Ryzen AI Max+ 395 "Strix Halo" with the loom-skills workflow.

Quick start

pip install "hrx-loom[hf]"
loom-fetch https://huggingface.co/loom-fun/InternScience-Sage-A1.5-loom/resolve/main/model.json --materialize=copy --output=Sage-A1.5
loom-serve --model=Sage-A1.5/model.json --device=amdgpu

The server listens on 127.0.0.1:8080 and answers OpenAI-style /v1/chat/completions requests.

Performance vs llama.cpp

metric	llama.cpp	Loom	margin	roofline	Loom % of roofline
decode tok/s, short prompt	76.7	72.7	-5 %	141.9	51 %
decode tok/s, 2k context	69.6	65.9	-5 %	132.7	50 %
decode tok/s, 32k context	77.0	63.0	-18 %	150.9	42 %
prefill tok/s, 1k document	1132.2	1714.7	+51 %	11600.2	15 %
prefill tok/s, 2k document	1155.5	1706.3	+48 %	11416.3	15 %
prefill tok/s, 8k document	1139.5	1675.2	+47 %	10424.8	16 %
prefill tok/s, 16k document	958.4	1582.1	+65 %	9343.0	17 %
prefill tok/s, 32k document	770.6	1423.6	+85 %	7737.1	18 %
4 streams, aggregate decode tok/s	140.7	88.9	-37 %	245.9	36 %
speculative decoding, drafted tokens accepted	—	56 %	—	n/a	n/a

• llama.cpp: build llamacpp-rocm/b1369, server arguments -ngl 99 -cram 0 --mmproj ~/.cache/huggingface/hub/models--InternScience--Sage-A1.5-Q4_K_M-GGUF/snapshots/10b179b52f2c841030db327627e707d076b4c8cf/Sage-A1.5-mmproj.gguf --spec-type draft-mtp --spec-draft-n-max 2 -c 35024 -kvu -np 4.
• Loom: speculative decoding on (MTP, depth 3).
• Same GGUF on both engines, tokens per second. Loom: client-side timing; llama.cpp: server-side timings. Runs per value: decode and prefill 1–2 (mean), 4 streams 1 (median).
• Measured 2026-11-07; device AMD RYZEN AI MAX+ 395 w/ Radeon 8060S, AMD Radeon 8060S Graphics, Ubuntu 24.04; hrx-loom 0.1.0 (hrx-system d01d1c1); source: DevLab Dispatch run 42376130174.
• Drafted tokens accepted: the share of the drafter's proposed tokens the model accepted during the spec gate's requests.
• Roofline: the hardware ceiling for this GGUF on AMD Ryzen AI Max+ 395 "Strix Halo" (gfx1151) at 216 GB/s and 59 TFLOP/s, from loom-roofline roofline.py (loom-skills 7db8179); decode and 4 streams with speculative decoding: verify passes × (1 + acceptance × depth 3) tokens per pass, the drafter's cost not counted; acceptance measured in the row's own session: decode 49 %, decode at 2k 45 %, decode at 32k 70 %; for 4 streams, the spec gate's acceptance 56 % (× 2.68 tokens per pass), as the row's session recorded none: the gate's text can accept more or less than the row's.
• — not measured; n/a not applicable.

Features

feature	status	detail
API	✅	OpenAI-style /v1/chat/completions
tool calling	✅	OpenAI-style tools and tool_calls
reasoning / thinking	✅	on by default; chat_template_kwargs: {"enable_thinking": false} turns it off
reasoning effort	❌	not supported
speculative decoding	✅	MTP, depth 3, on by default
image input	✅	OpenAI-style image_url parts; projector Sage-A1.5-mmproj.gguf
audio input / output	❌	not supported
sampling	✅	temperature, top_p, top_k, seed
context size	✅	262,144 tokens per request
concurrency	✅	4 concurrent requests
max output tokens	✅	4,096 (preset max_tokens)

Correctness

gate	description	result
kernel_tests	Unit tests of the generated kernels against reference implementations	✅ pass (45/45) ¹
golden	llama.cpp reference recorded on the same GGUF: greedy tokens and top-5 log-probs, short prompts and 1k-32k documents	✅ pass ²
compare	Greedy output vs the llama.cpp reference, token by token; every divergence is a near-tie	✅ pass ²
vision	Image input vs llama.cpp with the same projector: identical prompt tokens, replies equal up to near-ties	✅ pass ²
consistency	Concurrent requests give the same output as the same requests run alone	✅ pass ²
spec	Speculative decoding on gives exactly the output of speculative decoding off	✅ pass ²
agent	Multi-turn tool-calling loop: exact prompt rendering, history reuse, valid tool calls, output vs llama.cpp	✅ pass ²
agent_think_off	The tool-calling loop with thinking off	✅ pass ²
sampling	temperature, top_k and seed behave as specified; first-token distribution matches llama.cpp	✅ pass ²
needle	Needle-in-a-haystack prompt at up to half the context, compared with llama.cpp; over-capacity prompt refused	✅ pass ²
needle_full	Needle-in-a-haystack prompt at the full context is served	❌ no evidence
decision	Decision answers (choice, score, noul probabilities) vs llama.cpp on the same prompts	— not applicable

• ¹ DevLab Dispatch run 42376130174, 2026-11-07; device AMD RYZEN AI MAX+ 395 w/ Radeon 8060S, AMD Radeon 8060S Graphics, Ubuntu 24.04; hrx-loom 0.1.0 (hrx-system d01d1c1).
• ² DevLab Dispatch run 42376130174, LEVEL=full, 2026-11-07; device AMD RYZEN AI MAX+ 395 w/ Radeon 8060S, AMD Radeon 8060S Graphics, Ubuntu 24.04; hrx-loom 0.1.0 (hrx-system d01d1c1).
• Gates are the loom-benchmark correctness gates (gates.sh); a gate without a recorded run shows no evidence.

Files

• model.json / model.json.lock: the card and the pinned source files (sha256 checked by loom-fetch).
• program/*.loom, weights.loom: the generated Loom programs.
• tokenizer.json: the tokenizer.
• llm_config.json: the configuration the programs were generated from.
• info.json: the measurements, gate evidence and versions this card shows; everything else comes from the files above.
• REPORT.md: how the values above were measured: setup, commands and the value of every run.

Versions: card schema loom-card/2; loom-skills 620faf8; hrx-loom 0.1.0 (hrx-system d01d1c1); card generator 7db8179.

Source files, fetched by loom-fetch from their own repositories:

• checkpoint: InternScience/Sage-A1.5-Q4_K_M-GGUF @ 10b179b, Sage-A1.5-Q4_K_M.gguf
• projector: InternScience/Sage-A1.5-Q4_K_M-GGUF @ 10b179b, Sage-A1.5-mmproj.gguf