kingjones777/Ornith-1.5-9B-ROCmFP4-STRIX-LEAN-GGUF

Ornith-1.5-9B — LEAN — ROCmFP4 / ROCmFPX GGUF

The size-optimised 4-bit of ornith-ai/Ornith-1.5-9B — ftype 106 Q4_0_ROCMFP4_STRIX_LEAN, with Q5_K token embeddings and the output head explicitly protected at Q6_K.

Built for AMD Strix Halo (gfx1151) — Ryzen AI MAX+ 395, 128 GB unified memory, ROCm 7.13.0 — using the ROCmFPX llama.cpp fork, which adds AMD-native FP4/FP8 tensor types that mainline llama.cpp does not have.

⚠️ These files require a ROCmFPX-capable llama.cpp build. They will not load in stock llama.cpp / Ollama / LM Studio — the Q4_0_ROCMFP4_* and Q*_0_ROCMFPX* tensor types are not in mainline.

Variants in this repo

file	ftype	size	BPW	token_embd	output.weight	decode
Ornith-1.5-9B-Q4_0_ROCMFP4_STRIX_LEAN.gguf	106	4.89 GiB	4.69	Q5_K	Q6_K	37.12 t/s

Which to pick: LEAN is the smallest file (4.89 GiB, 4.69 BPW) at 37.12 t/s — within 1.5% of the fastest. It gets there with Q5_K token embeddings while keeping the output head at Q6_K, and zero *shexp tensors at 4-bit.

Head protection — verified in the file, not assumed

tie_word_embeddings is false on this model, so output.weight is a real standalone tensor and --output-tensor-type genuinely bites. Every artifact was re-opened after quantization and its header read back with an independent parser:

ftype	token_embd.weight	output.weight
106 STRIX_LEAN	Q5_K	Q6_K
104 FAST_COHERENT	Q6_K	Q6_K
102 COHERENT	Q6_K	Q6_K
114 / 111 / 115	Q8_0	Q8_0

⚠️ Note the LEAN row: it keeps the Q5_K token embeddings that make it lean while still protecting the output head at Q6_K. ftype 106 does not do this on its own — left to its defaults it emits a 4-bit output.weight. Head protection has to be requested and then verified in the file; an ftype name is not a guarantee.

Measured — not estimated

Hardware: AMD Ryzen AI MAX+ 395 (Strix Halo, gfx1151), 128 GB unified, ROCm 7.13.0. Idle box, 2 warm-ups discarded, median of 5, 300 tokens, identical prompt across every sample.

quant	run 1	run 2	run 3	run 4	run 5	median
106	36.37	37.10	37.12	37.42	37.39	37.12

Speculative decoding

⛔ Not available for this model. config.json declares mtp_num_hidden_layers: 1, but the published safetensors contain no MTP weights and the converted GGUF has no nextn_predict_layers key — we checked both. No draft head ships with this model, and no GGUF drafter is published for it. The decode figures below are therefore the deployed numbers, with no speculative multiplier left on the table.

Source — byte-verified, not re-converted

Quantized from ornith-ai/Ornith-1.5-9B-GGUF → Ornith-1.5-9B-BF16.gguf, 17,920,696,768 bytes. The vision projector mmproj-Ornith-1.5-9B-BF16.gguf (921,704,416 bytes) is included. Both byte-verified against the Hub before quantization. No re-conversion from safetensors.

Architecture: Qwen3_5ForConditionalGeneration / qwen35, 427 tensors, 32 layers, hidden 4096, vocab 248,320 — a hybrid stack (24 gated-deltanet + 8 full-attention layers), not a pure dense transformer and not an MoE.

Verification

Every artifact: loaded at -ngl 999 -c 4096 -fit off -fa on, 3/3 correctness (17×23 → 391, capital of Japan → Tokyo, days in 2024 → 366) asserted on both content and reasoning_content, and 4/4 vision on a four-quadrant colour image via the mmproj.

File sizes were checked against --dry-run projections; the header delta is constant across artifacts, which is the signature of complete, untruncated files.