magiccodingman/Qwen3.8-27B-MXFP4-MagicQuant-GGUF

MagicQuant MXFP4 Hybrids — AMD Qwen3.8-27B Quark AWQ

Native MXFP4 GGUF repack of AMD's Quark-AWQ Qwen3.8-27B checkpoint, plus benchmark-validated MagicQuant and Unsloth downward hybrid quantizations with complete tensor maps, KLD results, imatrix provenance, MTP support, and vision projectors.

This release is unusual: its model weights come from AMD's officially post-trained/quantized amd/Qwen3.8-27B-Quark-AWQ-MXFP4, while most of its recipe identities come from magiccodingman/Qwen3.8-27B-MagicQuant-GGUF.

It is therefore an adapted downward clone, not a pure repository clone and not a new full MagicQuant evolution/search run. Each selected tensor configuration was applied directly to the native MXFP4 GGUF under a strict rule: never increase a tensor's storage precision just to match the source recipe. If AMD's source tensor was already smaller, it stayed smaller and byte-exact.

MagicQuant Info & Wiki

MagicQuant's methodology and protocols are documented at the MagicQuant Wiki.

The exact adapted tensor maps, source recipes, imatrix identities, benchmark results, verification counts, and hashes for this release are published under magicquant-manifest/.

Support MagicQuant

I’m a solo developer working full time for myself to achieve my dream. I build open source code on the side. If you like any of my work, buying me a coffee is always appreciated. Otherwise, I hope you enjoy, maybe give me a star or something—or just send good vibes.

Click here to see ways to support — BTC, PayPal, GitHub Sponsors.

Adapted clone notice

The recipe source is magiccodingman/Qwen3.8-27B-MagicQuant-GGUF. The actual parent checkpoint is amd/Qwen3.8-27B-Quark-AWQ-MXFP4.

This repository did not rerun the full MagicQuant search space. It tested eleven configurations from the established Qwen3.8-27B recipe ladder, adapted each one to AMD's already-quantized source using a strict downward-only policy, rebuilt it directly from the native MXFP4 GGUF, and benchmarked it against native-MXFP4 reference logits. Ten hybrids passed the release cut; one completed result is documented but deliberately not distributed.

Because the parent weights and the adaptation policy differ, the KLD values here are fresh measurements for this repository. They are not copied from the source MagicQuant release and should not be compared as if the two repositories used the same reference model.

---

Quantized models

Recommended starting points: MQ-IQ4_XS_1, UD-Q4_K_S, UD-IQ4_XS, and UD-Q3_K_XL are the standout quality-to-size models in this release. Across the tested ladder, these four preserve the native MXFP4 logits exceptionally well while still delivering meaningful size reductions; they are the strongest first choices before moving into the more aggressive low-bit experiments.

Name	Recipe source	Imatrix	KLD vs native MXFP4	Size (GB)	Saved	Download
Native MXFP4	AMD / lossless GGUF repack	None	0.000000	18.89	0.00%	Link
MQ-IQ4_XS_1	MagicQuant	Generic	0.000940	14.98	20.70%	Link
UD-Q4_K_S	Unsloth	Unsloth	0.003450	14.55	23.00%	Link
UD-IQ4_XS	Unsloth	Unsloth	0.009018	13.89	26.49%	Link
UD-Q3_K_XL	Unsloth	Unsloth	0.022344	13.03	31.05%	Link
UD-IQ3_S	Unsloth	Unsloth	0.042218	11.99	36.55%	Link
MQ-IQ2_M_1	MagicQuant	Generic	0.058879	11.91	36.94%	Link
UD-IQ3_XXS	Unsloth	Unsloth	0.072327	10.90	42.29%	Link
MQ-IQ2_M_2	MagicQuant	Generic	0.102424	10.69	43.41%	Link
UD-Q2_K_XL	Unsloth	Unsloth	0.111015	9.81	48.08%	Link
UD-IQ2_XXS	Unsloth	Unsloth	1.172122	9.01	52.29%	Removed — failed quality floor
MQ-IQ2_XXS_1	MagicQuant	Unsloth	0.321797	8.22	56.47%	Link

The crossed-out UD-IQ2_XXS row is retained as a transparent failed experiment, but its GGUF has been removed and will not be uploaded. Its KLD was 1.172122 and its PPL was 17.87418 versus native 5.801511; meanwhile, MQ-IQ2_XXS_1 is both smaller (8.22 GB versus 9.01 GB) and dramatically closer to the native logits (KLD 0.321797). There is no practical reason to distribute the dominated 9.01 GB file. Its benchmark, tensor map, hash, and removal reason remain in the manifest for auditability.

The names describe the cloned recipe/configuration targets. They do not mean every tensor in the file uses that nominal type. The exact effective type of all 866 tensors is recorded in magicquant.clone-configs.json, including every place where native MXFP4 was retained instead of being increased.

Native reference and vision projectors

Artifact	Precision	Size (GB)	Purpose	Download
Native language GGUF	Native MXFP4 + source BF16/F32	18.89	Immutable benchmark/requantization parent	Link
Vision projector	BF16/F32	0.93	Archival/highest-fidelity projector	Link
Vision projector	Q8_0 + safe F16/F32 fallbacks	0.63	Smaller recommended distribution projector	Link

The Q8_0 projector was converted directly from the original source tensors—not requantized from the BF16 GGUF. It contains 83 Q8_0, 27 F16, and 224 F32 tensors. Unsupported shapes safely remained F16/F32. A CPU-only multimodal load encoded an image successfully and exited cleanly with projector offload disabled.

The language KLD benchmark does not evaluate vision quality, so no vision-quality score is claimed. Keep the BF16 projector when maximum vision fidelity matters; use Q8_0 when saving roughly 302 MB is preferable.

---

What “native MXFP4” means here

The immutable parent GGUF is a lossless repack of AMD's Quark/AWQ checkpoint:

• 496 native MXFP4 tensors and 12,936,232,960 MXFP4 payload bytes;
• 10 source BF16 tensors;
• 360 source F32 tensors;
• 866 tensors total;
• no BF16 expansion or intermediate for the MXFP4 weights.

The converter packed the original E2M1 four-bit codes and E8M0 block scales into llama.cpp's block_mxfp4 layout, applying only the required Qwen linear-attention permutations by moving whole codes/scales. Independent reconstruction verified every native MXFP4 payload byte after those lossless layout transforms.

AMD's upstream release also describes dynamically quantized MXFP4 activations in its Quark runtime. GGUF preserves the trained/static checkpoint weights; runtime activation behavior is controlled by llama.cpp and is not a claim of reproducing AMD's original activation kernel path.

Strict downward-only adaptation

Each source tensor type request was compared with the actual AMD-derived source tensor:

1. If the cloned recipe requested smaller storage, the tensor was requantized downward.
2. If it requested equal or larger storage, the source tensor was preserved byte-exact.
3. No candidate passed through a BF16 language-model intermediate.
4. Every unchanged tensor payload was byte-compared during verification before promotion.
5. Final files were hash-verified after serialized promotion into the release directory.

This is why these outputs can be much smaller without discarding AMD's native MXFP4 values wherever the cloned recipe would otherwise have increased precision.

MTP tensors retained

The native GGUF and all derivatives retain Qwen's MTP/next-token-prediction metadata and block 64 tensors (qwen35.nextn_predict_layers = 1). MQ-IQ2_XXS_1 preserves the full MTP block at its source BF16/F32 types; other recipes retain the block structurally while quantizing some of its larger tensors. Exact per-artifact MTP types are recorded in magicquant.hybrid-map.json.

Current llama-perplexity evaluation reports the MTP block as unused, so the published KLD values validate the standard autoregressive path only—not MTP prediction quality.

Benchmark protocol and reproducibility

All candidates used the same native MXFP4 reference logits and the same normalized MagicQuant corpus:

llama-perplexity \
  -m <candidate.gguf> \
  -ngl 0 -t 4 -c 2048 \
  --file <ppl_corpus_general.txt> \
  --kl-divergence-base <native_mxfp4_logits.bin> \
  --kl-divergence

• Dataset: Salesforce/wikitext
• Token target: 32,768
• Context: 2,048
• Benchmark threads: 4
• GPU layers: 0
• GPU visibility: disabled
• Corpus SHA-256: 5d38d98dce15f54e9a1a926187b6058e65cd8b3dd9b5cc2729b0a5fd249228b0
• Native-logits SHA-256: 604db2df7253cba18a6c0a569ed819ee3065d4ac20cd94ae72f4329e7bba8516

Full KLD distributions, PPL values, probability deltas, and benchmark scope are in magicquant.clone-benchmarks.json.

Generic vs Unsloth imatrix use

This release reuses two documented Qwen3.8-27B importance matrices rather than generating a new AMD-specific matrix:

Imatrix	SHA-256	Used by
Generic MagicQuant	123a92c3...899eaa	MQ-IQ4_XS_1, MQ-IQ2_M_1, MQ-IQ2_M_2
Unsloth	0ee5b10b...99f1c1	all tested UD-* configurations (including the removed failure) and MQ-IQ2_XXS_1

The generic matrix came from magiccodingman/Qwen3.8-27B-MagicQuant-GGUF. The Unsloth matrix came from unsloth/Qwen3.8-27B-GGUF.

An imatrix only influences a tensor when llama.cpp actually requantizes a covered tensor. For MQ-IQ4_XS_1, all 496 covered native MXFP4 tensors stayed byte-exact and the ten changed source-BF16 tensors had no matrix entries, so the supplied generic imatrix did not numerically affect that artifact. The remaining downward candidates requantized covered tensors and used the matrix identified in the table.

This fixed-recipe experiment does not claim that a newly generated AMD-specific imatrix could never improve a result; it records exactly what was reused and measured.

Provider and lineage credits

• Qwen — base model.
• AMD — Quark AWQ MXFP4 post-training/quantized checkpoint and BF16 vision source.
• MagicQuant Qwen3.8 release — selected tensor recipes and generic imatrix.
• Unsloth — learned Dynamic tensor configurations and Unsloth imatrix.
• llama.cpp — GGUF, MXFP4, quantization, multimodal, and benchmark tooling.

Warning — what this release proves

These measurements show how the selected configurations behave when adapted to this specific AMD-derived native MXFP4 parent, under the same corpus, reference logits, and CPU benchmark flow.

They do not prove that the original external artifacts are universally better or worse, and they do not imply that this repository completed the full MagicQuant evolutionary search. The source checkpoint, tensor adaptation policy, imatrix, runtime, and benchmark reference all matter.

---

Release metadata

• Manifest guide — file semantics and publication boundaries.
• Final artifact metrics — published and removed status, download names, KLD/PPL, sizes, savings, hashes, and verification counts.
• Hybrid tensor map — effective tensor-group distributions, MTP treatment, and downward-adaptation summaries.
• Clone tensor configs — exact 866-tensor effective maps for every GGUF.
• Clone benchmark summary — fresh native-MXFP4 benchmark results and scope.
• Imatrix provenance — identities, hashes, sources, and per-artifact use.
• Projector manifest — BF16/Q8_0 projector construction, types, hashes, and validation.
• Lineage and conversion provenance — base, AMD source, recipe source, conversion commit, and downward policy.
• Artifact hashes — SHA-256 and exact byte sizes.
• Release boundary — explicit public include/private exclude patterns for upload safety.
• Per-experiment records — commands, configs, effective type files, all tensor decisions, result distributions, and byte-exact verification summaries.
• Source recipe snapshots — the selected source-repository recipes preserved for durable provenance.

Upstream AMD benchmark context

AMD reports the following GSM8K 5-shot results for the original amd/Qwen3.8-27B-Quark-AWQ-MXFP4 checkpoint:

Mode	AMD MXFP4 AWQ — flexible / strict	BF16 base — flexible / strict	Recovery
Thinking	94.996% / 95.30%	93.33% / 93.33%	101.8%
Non-thinking	89.92% / 89.76%	90.67% / 89.76%	99.2%

Recovery is AMD's flexible-extract score for the MXFP4 AWQ checkpoint divided by the BF16 base score. AMD produced the parent with Quark using AWQ, 128 pileval calibration samples at sequence length 512, OCP MXFP4 weights with group size 32 and E8M0 shared scales, and dynamic MXFP4 activations in its original runtime. The vision tower remained BF16.

These are upstream results quoted for parent-model context, not fresh GSM8K evaluations of this repository's GGUF files. This release's directly measured comparisons are the native-logit KLD/PPL results above. GGUF preserves the trained checkpoint weights, but llama.cpp does not claim to reproduce AMD's original dynamic-MXFP4 activation runtime.

---