Qwen3.8-27B MQ V2 ladder for hipfire

Text-only Qwen3.8-27B artifacts for hipfire, the Rust-native LLM inference engine for AMD GPUs.

This repository contains the admitted MQ3V2 through MQ6V2 product ladder and matching DFlash drafters. MQ4V2 supersedes MQ4V1 as hipfire's canonical mq4 product. Artifacts remain self-describing by HFQM quant-type ID; old MQ4V1 files are still readable but are not the current product.

Product filenames

Each bit-width has exactly three tiers:

• mqN-xt: embeddings protected; lm_head and recurrent outputs remain at the base codec.
• mqN: embeddings and lm_head protected.
• mqN-pro: recurrent/SSM outputs also lifted. MQ3 Pro uses ssm_out:mq6v2 to stay inside the 3.x model-bpw class.

Product	File	Model bpw	Size (GB)	WT2 KLD
mq3-xt	qwen3.8-27b.mq3-xt	3.503	11.778	0.248348
mq3	qwen3.8-27b.mq3	3.753	12.619	0.153658
mq3-pro	qwen3.8-27b.mq3-pro	3.922	13.184	0.130314
mq4-xt	qwen3.8-27b.mq4-xt	4.456	14.980	0.057449
mq4	qwen3.8-27b.mq4	4.659	15.663	0.039033
mq4-pro	qwen3.8-27b.mq4-pro	4.897	16.464	0.032495
mq5-xt	qwen3.8-27b.mq5-xt	5.408	18.183	0.015028
mq5	qwen3.8-27b.mq5	5.564	18.706	0.010255
mq5-pro	qwen3.8-27b.mq5-pro	5.746	19.319	0.009006
mq6-xt	qwen3.8-27b.mq6-xt	6.361	21.386	0.004389
mq6	qwen3.8-27b.mq6	6.469	21.750	0.002771
mq6-pro	qwen3.8-27b.mq6-pro	6.596	22.174	0.002208

MQ2V2 was fully measured but is deliberately not published as a product: all three tiers produced catastrophic WT2 KLD around 12–13 and degraded/repetitive text.

DFlash drafters

Draft	File	Size (GB)	Role
MQ3V2	qwen38-27b-dflash-mq3.hfq	0.985	Optional same-bit controller
MQ4V2	qwen38-27b-dflash-mq4.hfq	1.210	Recommended controller
MQ5V2	qwen38-27b-dflash-mq5.hfq	1.434	Optional same-bit controller
MQ6V2	qwen38-27b-dflash-mq6.hfq	1.659	Optional same-bit controller

Sizes are decimal GB, matching Hugging Face.

The MQ4V2 control drafter won the matched decode screen for every admitted MQ3/MQ5/MQ6 target tier and is therefore the recommended default controller. The same-bit drafts remain available for explicit size/codec experiments.

DFlash is opt-in:

hipfire pull qwen3.8:27b
hipfire pull qwen3.8:27b-draft-mq4
hipfire run --spec dflash \
  --model-draft ~/.hipfire/models/qwen38-27b-dflash-mq4.hfq \
  qwen3.8:27b "Design a streaming Rust file deduplicator."

Usage

# Canonical MQ4V2 base tier
hipfire pull qwen3.8:27b
hipfire run qwen3.8:27b "Design a streaming Rust file deduplicator."

# Explicit ladder cells
hipfire pull qwen3.8:27b-mq3-xt
hipfire pull qwen3.8:27b-mq4-pro
hipfire pull qwen3.8:27b-mq6

Runtime defaults

Every non-draft Qwen3.8 ladder tag receives the same registry tag policy:

Setting	Value
KV cache	Q8
KV backend	VMM
Native context	262,144 tokens
Maximum output	81,920 tokens
Temperature	1.0
Top-p	0.95
Top-k	20
Min-p	0.0
Presence penalty	0.0
Repeat penalty	1.0
Reasoning effort	xhigh
Thinking budget	uncapped

Measurement status

The WT2 KLD values above use 24 scored chunks against the pinned WT2 teacher, Q8 KV, and prefill scoring. They measure quantization quality rather than runtime speed.

The earlier throughput rows have been withdrawn because they predated the restored gfx1100 MQ4V2 multi-wave dispatch. Fresh hipfire bench --matrix results for gfx1100, gfx1151, and gfx1201 are being collected against corrected master bca38df0a; no pre-restoration throughput numbers are shown here.

Wire format

MQ V2 uses an 8-byte dual-FP16 affine header per 256-weight group: one scale/zero pair for weights 0–127 and one for 128–255. Wire IDs are MQ4V2 qt44, MQ6V2 qt47, MQ5V2 qt48, and MQ3V2 qt49.

License and attribution

The upstream Qwen3.8 weights are Apache-2.0 licensed. These quantized artifacts were produced for hipfire using its MagnumQuant FWHT-rotated quantization family. See the upstream model card and hipfire provenance.