JoJoLabs/Qwen3.6-27B-MTP-ROCmFPX-GGUF

Qwen3.6-27B-MTP — ROCmFPX quants

⚠ This collection REQUIRES the ROCmFPX fork of llama.cpp

These files use custom ggml type IDs 100–112, which upstream does not know about. Do not download them expecting them to run on mainline llama.cpp, ollama, LM Studio, KoboldCpp, text-generation-webui or anything else. They fail at model-load with an unknown/invalid tensor type — this is not a bug you can work around with flags.

Built and measured against source commit 969a8581bf2e65b9c763397ceef089ca94566ba3.

ROCmFPX is a set of block quantisation formats built for AMD RDNA3.5 iGPUs (gfx1151, "Strix Halo"), where the memory bus is the binding constraint at 4+ bits and the dequant kernel is the binding constraint below 4. They pair a per-32 affine layout with a mini-float (UE4M4) block scale, and an encoder that searches 225–325 (scale, zero) hypotheses per block against imatrix weights.

Where the value actually is: at 2–3 bits, where stock formats leave both quality and bandwidth on the table. At 5 bits and above both families are near-lossless and bandwidth-bound, and the differences stop being measurable — see the honest "unresolved" rows below rather than a marketing table.

Quant collection

Sizes are whole-file (metadata included). KLD is against a pedestal-free BF16 reference; lower is better. "vs stock" is a paired per-chunk delta against the stock peer at the same nominal width, with a 95% bootstrap CI.

These exact files were built and measured on MI300X (gfx942), so the KLD column below describes the artifacts you are downloading, not a same-recipe rebuild. Decode speed is from gfx1151, the target hardware, and is marked as such.

File	ggml type	id	Size	KLD (gfx942)	vs peer, paired 95% CI	t	Decode (gfx1151)
...-q2_1_rocmfpx.gguf	q2_1_rocmfpx	109	8.96 GiB	0.207800	−16.3% [−17.9, −14.8] vs IQ2_S	−20.38	+37.7%
...-q3_1_rocmfpx.gguf	q3_1_rocmfpx	108	10.95 GiB	0.067230	−36.6% [−39.7, −33.8] vs Q3_K	−24.01	+10.8%
...-q4_0_rocmfp4.gguf	q4_0_rocmfp4	100	14.33 GiB	0.025140	−18.4% [−24.2, −13.0] vs Q4_K	−6.45	+5.3%
...-q5_0_rocmfpx.gguf	q5_0_rocmfpx	105	17.51 GiB	0.015680	−1.3% [−9.0, +6.5] unresolved	−0.32	+2.1%
...-q6_0_rocmfpx.gguf	q6_0_rocmfpx	102	20.69 GiB	0.005220	−5.1% [−32.7, +19.1] unresolved	−0.39	+3.0%
...-q8_0_rocmfpx.gguf	q8_0_rocmfpx	110	26.26 GiB	0.002930	−17.5% [−48.8, +9.3] unresolved	−1.18	+3.3%
...-UD-Q4_K_XL-mirror.gguf	mixed	—	16.60 GiB	0.017540	−15.7% [−22.5, −9.2] vs Unsloth UD-Q4_K_XL	−4.64	+21.2%
...-UD-Q5_K_XL-mirror.gguf	mixed	—	18.85 GiB	0.010800	−5.0% [−14.3, +4.3] unresolved	−1.06	—
...-UD-Q6_K_XL-mirror.gguf	mixed	—	23.89 GiB	0.004190	+2.9% [−16.6, +22.4] unresolved	+0.29	—
...-UD-Q8_K_XL-mirror.gguf	mixed	—	32.73 GiB	0.002600	−0.8% [−30.9, +31.8] unresolved	−0.05	—

The four UD rows compare against the actual downloaded Unsloth artifact, not a rebuild.

"Unresolved" means unresolved, not "tied." At 5/6/8 bits and UD-Q5/Q6/Q8 the intervals span zero by tens of points. They rule out a large deficit and nothing finer. In particular do not quote the 8-bit −17.5% — its CI runs from −48.8% to +9.3%, which is noise wearing a large point estimate. Reporting these as wins would be dishonest; calling them ties would be equally wrong.

Four rungs are iso-byte to the byte with their stock peer, so those deltas carry no size confound. At 6 and 8 bits ours is smaller as well, making the comparison conservative against us.

Behavioural check — MMLU 5-shot, 2000 tasks

KLD measures distributional distance; this measures whether it changes answers. Holm- Bonferroni corrected across the 10 designed pairs.

pair	accuracy	Δ vs peer	McNemar (Holm)	mean Δ logP(correct)
BF16 reference	86.20%	—	—	−0.531
q2_1 vs IQ2_S	82.15% vs 77.75%	+4.40 pp	p<0.0001 REJECT	+1.144 (t=+20.69)
q3_1 vs Q3_K	85.55% vs 83.90%	+1.65 pp	p=0.0615 ns	+0.165 (t=+6.86, sign p=0.003)
q4_0 vs Q4_K	86.05% vs 85.75%	+0.30 pp	p=1.0 ns	−0.009 ns

MMLU saturates above 3 bits — every arm from 4-bit up sits within noise of BF16's 86.2%, so the instrument has no resolution there. It discriminates exactly where KLD says the value is.

At 3 bits the observed +1.65 pp sits on the minimum detectable effect (1.66 pp), so accuracy is underpowered; the log-probability metric, which uses more information per item, resolves it. Stock IQ2_S's mean logP(correct) of −1.853 against ours at −0.709 shows 2-bit stock is not merely wrong more often, it is badly miscalibrated.

Four of six uniform rungs are iso-byte to the byte with their stock peer, so those quality deltas carry no size confound. At 6 and 8 bits ours is smaller as well, which makes the comparison conservative against us.

Which file should I choose?

• 8–11 GiB of VRAM → q2_1 or q3_1. This is where ROCmFPX earns its existence: 3-bit is 36.6% closer to BF16 than Q3_K at identical bytes, and decodes 10.8% faster.
• 14–18 GiB → q4_0_rocmfp4, a decisive 18.4% quality win at iso-byte.
• 20 GiB+ → q6_0 or q8_0 if you want the small decode and size edge, but be honest with yourself that the quality difference from stock is below measurement here.
• Mirroring an Unsloth UD mix → the UD-*-mirror files reproduce Unsloth's per-tensor allocation using ROCmFPX types.

Quick start

Build the fork, then:

# decode-oriented, single GPU
./build/bin/llama-server \
    --model Qwen3.6-27B-MTP-ROCmFPX-q3_1_rocmfpx.gguf \
    --ctx-size 16384 -ngl 99 -fa on

# geometry trap: for --kl-divergence and other batch-sensitive paths on the
# hybrid gated_delta_net architecture, -b and -ub MUST be >= -c, or the
# recurrent state is silently wrong and perplexity comes back in the thousands.
./build/bin/llama-perplexity -m <model> -f wiki.test.raw \
    -c 512 -b 512 -ub 512 -ngl 99 -fa 1

How quality was measured

Getting this wrong is easy and we did, twice, before the numbers above stabilised:

• Pedestal-free BF16 reference, not a Q8_0 stand-in. A Q8_0 reference adds its own error floor and compresses every delta toward zero.
• 300 chunks, wikitext-2 test, -c 512 -b 512 -ub 512 -ngl 99 -fa 1. Note --chunks is ignored in --kl-divergence mode; the count comes from the base file.
• Paired per-chunk bootstrap, keyed on chunk index. llama-perplexity intermittently omits the chunk-1 row, and pairing by position instead of index silently compares off-by-one chunks — that alone moved the 4-bit result from t=−1.78 to t=−5.05.
• Identical imatrix on both sides. An earlier version of this matrix compared blind ROCmFPX arms against imatrix-guided stock peers and reported a 20% loss that did not exist. Every arm here is imatrix-guided from the same file.

Cross-architecture reproduction

The 2-bit result was re-derived from scratch on MI300X (gfx942, CDNA3, wave64) — different microarchitecture, different wave size, different toolchain — to check it was not an artifact of RDNA3.5-tuned tile configs:

	gfx1151 (RDNA3.5, wave32)	gfx942 (CDNA3, wave64)
paired delta	−15.8%	−16.3%
t	−20.02	−20.38

See tripwire-2bit.txt and TRIPWIRE-A.txt.

Provenance

Every file was quantized from one merged BF16 source (unsloth/Qwen3.6-27B-MTP-GGUF @ 5cb35eb3, BF16/*) with one imatrix, on a single MI300X. Mismatched imatrix provenance invalidated an entire earlier matrix in this project, so it is pinned deliberately rather than by convention.

q3_1 and q6_0 include three encoder fixes: the FP3 imatrix path (it previously discarded quant_weights outright, quantizing blind), the FP6 clip_delta constant, and the MSE search weight. The first is why 3-bit moved from a tie to the largest win here.

Full environment, including the exact source commit and both ROCm toolchain paths, is in environment.txt. Measurements shipped alongside (KLD-RESULTS.md, MMLU-RESULTS.md) were taken on gfx942 and should not be merged into a gfx1151 table.

Known limitations

• Prefill regresses 4–11% versus stock at 4 bits and above. Not yet investigated.
• wikitext-2 is not the target workload. These formats are aimed at agent/tool-call use; KLD on wikitext is a sensitive tripwire but a weak ranking instrument near baseline.
• q6_1_rocmfpx (id 112, UE3M5 scale) is deliberately absent. At bias shift 5 it was broken — 801 of 1.71e9 half-blocks clipped, carrying 99.89% of the weighted error, KLD 0.019998 (+240% vs Q6_K). Shift 4 repairs it to 0.006427, still behind q6_0. It exists in the fork so the measurement reproduces; it is not worth downloading.
• Retired type ids 101, 103, 104, 106, 107 must never be reused.