kingjones777/Ornith-1.5-9B-Abliterated-ROCmFP4-GGUF

Ornith-1.5-9B-Abliterated — ROCmFP4 for AMD Strix Halo (gfx1151)

The model here is not our work. The abliteration, conversion and original validation are PocketAI Model Lab's, published as PocketAiHub/Ornith-1.5-9B-Abliterated-GGUF; the base model is ornith-ai's. This repository adds only the ROCmFP4/ROCmFPX quantisation ladder, the dual-backend build, and the measurements. Go star their repo.

Six ROCmFP4 / ROCmFPX quantisations of Ornith-1.5-9B-Abliterated, built and measured on a Ryzen AI Max+ 395 (Radeon 8060S). Every file runs on BOTH the HIP (ROCm) and Vulkan backends from a single binary — the backend is a runtime -dev flag, not a rebuild. The F16 vision projector works with every tier.

Lead result — this model is the opposite of its 35B sibling. On the 35B (MoE) the backend was worth ~10× the tier. Here, on the 9B (dense), the backends are within ~1 % on the 4-bit tiers and the interesting spread is between tiers instead. Do not carry a backend ranking across architectures.

Which file should I use?

File	ftype	Size	PPL (wikitext-2, 100 chunks)	Use it for
Q4_0_ROCMFP4_STRIX_LEAN	106	4.89 GiB	9.4313	Best all-round. Fastest tier measured, and better perplexity than full Q8.
Q4_0_ROCMFP4_FAST	103	4.72 GiB	9.4571	Smallest file; within noise of STRIX_LEAN on speed.
Q4_0_ROCMFP4_COHERENT	102	5.19 GiB	9.4576	q6_K embeddings + head. ⚠️ See the Vulkan note below.
Q8_0_ROCMFPX_AGENT	115	8.77 GiB	9.4163	Best perplexity. Pick when quality outranks speed.
Q8_0_ROCMFPX	111	8.64 GiB	9.4559	Plain Q8 reference.
Q6_0_ROCMFPX_AGENT	114	7.88 GiB	9.5850	⛔ Not recommended — see below.
mmproj-*-F16.gguf	—	0.86 GiB	—	Vision projector. Required for images; works with every tier above.

⛔ Q6 is strictly dominated on this model — don't use it

Q6_0_ROCMFPX_AGENT is worse on every axis than the 4-bit tiers: slower (24.4 vs ~35.5 tok/s), larger (7.88 vs ~4.9 GiB), and worse perplexity (9.5850 — the worst of all six). It is also the one tier where Vulkan loses badly to ROCm (21.1 vs 24.4, −13.6 %). There is no workload on this model where Q6 is the right answer. It is published only for completeness of the ladder.

This is not a general claim about Q6 — it is what this model measured.

---

Quick start — the backend is one flag

# ROCm — the conservative default (zero truncated generations in 72)
llama-server -m Ornith-1.5-9B-Abliterated-Q4_0_ROCMFP4_STRIX_LEAN.gguf \
  --mmproj mmproj-Ornith-1.5-9B-Abliterated-F16.gguf \
  -dev ROCm0 -fa on -ngl 999 -fit off -np 1 \
  -ctk q8_0 -ctv q8_0 -c 32768 -b 8192 -ub 2048 -t 4 --poll 100 --jinja \
  --repeat-penalty 1.05 --repeat-last-n 256 \
  --host 0.0.0.0 --port 8080

# Vulkan — SAME FILE, only -dev changes. Within ~1 % on this model.
#   -dev Vulkan0

⚠️ This model reasons before answering — give it enough max_tokens

It emits a reasoning block first, then the answer. If your token budget runs out during reasoning you get an empty content field — a valid HTTP 200 with nothing in it, which reads like a broken model.

Measured on the six tiers with a single "write a function" prompt (reasoning chars → total completion tokens):

Tier	reasoning	content	completion tokens
STRIX_LEAN	1221	917	708
COHERENT	979	477	378
Q8_0_ROCMFPX	356	1031	410
Q8_AGENT	528	616	338
Q6_AGENT	816	364	326
FAST	358	685	301

Every tier needed more than 300 tokens for that one short request, and STRIX_LEAN needed 708. A max_tokens: 300 cap returns empty content from this model — we hit exactly that in our own first test run. Budget ≥ 1024, and read reasoning_content separately if your client exposes it.

--repeat-penalty 1.05 --repeat-last-n 256 is recommended, not decorative: with all repetition control off, this family can fall into a repeating-candidate loop on long agentic turns.

⛔ You need a ROCmFPX build with Vulkan enabled — stock llama.cpp will NOT load these files

ROCmFP4/ROCmFPX use ggml tensor types 100–119. Upstream llama.cpp's type table stops at 43, so a stock build rejects these files outright, on any backend. Build one with both backends — this is the exact configuration behind every number on this card:

cmake -S . -B build-hipvk -DCMAKE_BUILD_TYPE=Release \
  -DGGML_HIP=ON -DGGML_VULKAN=ON -DGGML_NATIVE=ON \
  -DVulkan_GLSLC_EXECUTABLE=/usr/bin/glslc \
  -DAMDGPU_TARGETS=gfx1151
cmake --build build-hipvk -j

# verify BOTH backends are present before trusting any of this:
build-hipvk/bin/llama-server --list-devices   # must list ROCm0 AND Vulkan0

Traps: glslc is not always auto-detected (pass -DVulkan_GLSLC_EXECUTABLE); a stale CMakeCache poisons a generator switch (rm -rf the build dir).

---

Measured performance — all six tiers, both backends

Decode tok/s, median of 2 scored reps (warm-up discarded), n_predict = 400, -c 32768, single seat. Medians are computed over full-length generations only; any truncated generation is excluded and noted.

Tier	Backend	code	prose	reason	json	prefill (code)
COHERENT	ROCm0	35.22	34.78	34.69	35.17	407
COHERENT	Vulkan0	34.57 †	34.58 †	34.62 †	34.58 †	367
FAST	ROCm0	35.52	35.53	35.44	35.98	443
FAST	Vulkan0	35.80	35.67	35.77	35.58	385
STRIX_LEAN	ROCm0	35.40	36.25	36.17	36.62	435
STRIX_LEAN	Vulkan0	35.23	35.73	35.84	35.78	381
Q6_AGENT	ROCm0	24.37	24.39	24.43	24.59	255
Q6_AGENT	Vulkan0	21.05	21.17	21.17	21.23	248
Q8_AGENT	ROCm0	22.44	22.49	22.44	22.59	377
Q8_AGENT	Vulkan0	22.73	22.68	22.74	22.75	231
Q8	ROCm0	22.29	22.29	22.26	22.37	353
Q8	Vulkan0	22.61 ‡	22.62	22.62	22.65	272

‡ Q8/Vulkan0 code had 1 truncated generation of 2 scored, so that single figure is a median over one surviving generation — treat it as indicative, not measured. Every other cell in this table is a median over 2 scored generations with zero truncations.

† COHERENT/Vulkan0 is reported from a deeper 48-generation re-run rather than the 2-rep protocol, because 3 of its 4 workloads truncated under Vulkan and the 2-rep medians were unusable — one workload had no valid sample at all. Medians are over full-length generations only (10, 12, 8 and 10 of 12 respectively). See the Vulkan section below.

12 of 12 arms loaded. Zero load failures on either backend.

The three findings that matter

1. Backend barely matters here — and that is the opposite of the 35B sibling. On the 4-bit tiers the two backends are within ~1 %. On the 35B (MoE) Vulkan was worth +14 %. Same family, same recipe, same box: the backend ranking did not transfer across architectures. Measure it on your model.
2. A 4-bit tier beats full Q8 on perplexity. STRIX_LEAN (4.89 GiB, 9.4313) is better than Q8_0_ROCMFPX (8.64 GiB, 9.4559) while running ~1.6× faster. The 4-bit ROCmFP4 tiers are not a quality compromise on this model.
3. Q6 loses on every axis — slower and larger than 4-bit, worse PPL than everything. See above.

---

Perplexity — measured

wikitext-2-raw, 100 chunks, -c 512, ROCm0, -ngl 999 -fa on. Lower is better.

Tier	PPL	±
Q8_0_ROCMFPX_AGENT	9.4163	0.16280
Q4_0_ROCMFP4_STRIX_LEAN	9.4313	0.16241
Q8_0_ROCMFPX	9.4559	0.16386
Q4_0_ROCMFP4_FAST	9.4571	0.16270
Q4_0_ROCMFP4_COHERENT	9.4576	0.16318
Q6_0_ROCMFPX_AGENT	9.5850	0.16705

The spread across the top five is 0.04 PPL — smaller than the ± on any single measurement. Treat those five as quality-equivalent and choose on size and speed.

---

⚠️ Vulkan silently truncated generation on this model — use ROCm

This is the most important thing on this page. Under Vulkan this model intermittently emits 1 token with empty content and stops. Under ROCm it never did once, in 120 generations.

The deep check — 48 generations per cell

Model	Backend	Truncated
This model (9B, dense)	Vulkan0	8 / 48 — 17 %
This model (9B, dense)	ROCm0	0 / 48
Sibling Ornith-1.5-35B-A3B-Abliterated (MoE)	Vulkan0	0 / 48
Sibling Ornith-1.5-35B-A3B-Abliterated (MoE)	ROCm0	0 / 48

Same recipe, same box, same binary, same driver — and only this model on Vulkan fails. The failure lives at the (this model × Vulkan) intersection, not in the format or the backend alone.

Per workload, on COHERENT/Vulkan0 (12 generations each):

Workload	Truncated	Median over full-length gens
prose	0 / 12	34.58
code	2 / 12	34.57
json	2 / 12	34.58
reason	4 / 12	34.62

Throughput on the surviving generations is normal — this does not look like a slow or overloaded GPU. It either produces the full answer at full speed, or produces nothing.

Across the full six-tier ladder — 12 generations per arm

Tier	ROCm0	Vulkan0
Q4_0_ROCMFP4_COHERENT	0 / 12	3 / 12
Q8_0_ROCMFPX	0 / 12	1 / 12
FAST · STRIX_LEAN · Q6_AGENT · Q8_AGENT	0 / 12 each	0 / 12 each
Total	0 / 72	4 / 72

Tier-variable, not tier-exclusive. It is worst on COHERENT, but it also hit Q8_0_ROCMFPX — a tier passing once does not make it immune.

It fails silently — plan for that

No error. No crash. Clean exit code. /health still returns 200. The response is simply empty (one case emitted <response> and stopped mid-tag). A benchmark that reports only median throughput cannot see this: it surfaces as an absurd tg = 1000000 tok/s (a 1-token / 0 ms decode) or as a plausible "fast" outlier inside a range. Every median on this card is therefore computed over full-length generations only, with truncations excluded and counted separately.

Recommendation for this model: run ROCm. It was clean in all 120 generations we put through it, and on the 4-bit tiers it is within ~1 % of Vulkan anyway — you give up nothing. If you do run Vulkan, count how many completions reach your n_predict rather than trusting throughput. The same GGUF serves correctly on -dev ROCm0; switching is one flag, no re-download and no rebuild.

---

Source integrity — verified before we built anything

Built from PocketAiHub/Ornith-1.5-9B-Abliterated-GGUF (BF16 GGUF), itself derived from ornith-ai/Ornith-1.5-9B pinned at revision c927ad73b7eb20f00aafcaa0a11a9d58ed5487bc.

The abliteration manifest was checked against the base before quantising — a de-fused or partially-applied checkpoint produces a model that quantises fine and then fails behaviourally, which is expensive to discover late.

Field	Value
Method	refusal-direction orthogonal weight projection
Direction source	residual_post at the assistant-generation boundary, layer 23
Direction sha256	97a2519200…3852aadffb
Destination layers	12 – 31 (20 layers)
Target kinds	full_attention_out (5), linear_attention_out (15), mlp_down (20)
Modified tensors	40
Scale	1.0, norm_preserve: true
Architecture	qwen3_5, hidden 4096, 32 layers, full-attention interval 4
BF16 source sha256	099f1d1c2942e0f9b071681175154ef86565df459b60d593953778b054962b64
mmproj sha256	5b0fde273fd6b54bc19e700fa5cb6501d58d2b61acafc4a73c7d6fdad87d96d1

---

Reproduction block

A number without its binary is a rumour. Everything on this card came from:

Host	Ryzen AI Max+ 395 (Strix Halo), Radeon 8060S, gfx1151, 128 GB unified
Build	ROCmFPX fork @ e7712358806055c70a9753b070202b0cc7c637e3
	GGML_HIP=ON GGML_VULKAN=ON GGML_NATIVE=ON, Release, AMDGPU_TARGETS=gfx1151
llama-server sha256	e860b763d5e8f496ddb4b01236d8a8f37ae09dda456b5d076ab3abe2d9deb9fc
Serve flags	-fa on -ngl 999 -fit off -np 1 -ctk q8_0 -ctv q8_0 -c 32768 -b 8192 -ub 2048 -t 4 --poll 100 --jinja --repeat-penalty 1.05 --repeat-last-n 256
ROCm arms only	ROCBLAS_USE_HIPBLASLT=1
Perf protocol	4 workloads × 3 reps, warm-up discarded, median of remaining full-length gens
Truncation protocol	predicted_n recorded per generation; any gen < 50 tokens excluded from the median and counted separately
PPL	llama-perplexity -f wiki.test.raw --chunks 100 -c 512 -t 8 -dev ROCm0

---

Verification — every tier, measured

Tier	loads ROCm0	loads Vulkan0	mmproj	coherent output	tools	trunc ROCm	trunc Vulkan
Q4_0_ROCMFP4_COHERENT	✅	✅	✅ LOADED	✅ PASS (477 chars)	3/3	0 / 12	3 / 12
Q4_0_ROCMFP4_FAST	✅	✅	✅ LOADED	✅ PASS (685 chars)	3/3	0 / 12	0 / 12
Q4_0_ROCMFP4_STRIX_LEAN	✅	✅	✅ LOADED	✅ PASS (917 chars)	3/3	0 / 12	0 / 12
Q6_0_ROCMFPX_AGENT	✅	✅	✅ LOADED	✅ PASS (364 chars)	3/3	0 / 12	0 / 12
Q8_0_ROCMFPX_AGENT	✅	✅	✅ LOADED	✅ PASS (616 chars)	3/3	0 / 12	0 / 12
Q8_0_ROCMFPX	✅	✅	✅ LOADED	✅ PASS (1031 chars)	3/3	0 / 12	1 / 12

12 of 12 backend load arms passed. 0 load failures. Tool calling is 18/18 correct structured calls across all six tiers. Every tier returns real, runnable code when given an adequate token budget.

The coherence probe was run at max_tokens: 1024 with reasoning_content captured separately. An earlier run of this same probe at max_tokens: 300 reported STRIX_LEAN as producing empty output — that was a probe defect, not a model defect: the server had generated a full 300 tokens, all of them reasoning. See the max_tokens note above.

Vision: the F16 projector loads against every tier. Image-grounded accuracy is not scored here — see Not yet measured.

---

Not yet measured

• Image-grounded vision accuracy (the projector loads and passes a smoke test; output is not scored against ground truth).
• Context beyond 32768.
• Speculative decoding / MTP — the source ships no MTP head.
• Full-corpus perplexity (the figures above are a 100-chunk run).
• Whether the Vulkan truncation reproduces on other gfx1151 hosts or driver versions. We saw it on one box.

---

License and attribution

Released under MIT, matching the source derivative.

• Base model: ornith-ai/Ornith-1.5-9B by the Ornith team, pinned at c927ad73b7eb20f00aafcaa0a11a9d58ed5487bc.
• Abliterated derivative and original validation: PocketAI Model Lab, published as PocketAiHub/Ornith-1.5-9B-Abliterated-GGUF.
• This repository contributes only the ROCmFP4/ROCmFPX quantisation ladder, the dual-backend build, and the measurements above.

Responsible use

This is an experimental derivative in which learned refusal behaviour has been reduced. It is published for research and legitimate local use. The edit reduces refusal broadly rather than judging whether a request is legitimate — deployers should evaluate it in their own context and apply their own safeguards. Abliteration is not truthfulness training, not a capability improvement, and not a guarantee of compliance.

Acknowledgements

The Ornith team for the base model; PocketAI Model Lab for the abliteration and its machine-readable provenance, without which the source-integrity check above would not have been possible; and the ROCmFPX project for the FP4 tensor types that make these tiers exist at all.