StillDeadcode/qwen3.8-27b-iqr-v4

Qwen3.8-27B IQR-V4: radiance containers, every size in one repository

Qwen3.8-27B quantised with IQR-V4, radiance's trellis format. Every trunk linear is rotated by a 128-point Hadamard and fitted by GPTQ, in act order, against calibration Hessians. Its codes are a tail-biting trellis down each column whose 2^16 states' values are computed from the state, under one f32 scale a row. The projections fed by the attention heads also get a bf16 scale per 128 weights. The widths come from measured KL prices. Below IQ6R-L the lm_head is vocabulary-weighted: four trellis widths, with the tokens that carry the probability on the widest. The GEMMs decode the codes in their tiles; nothing is widened in memory.

Each file is a single .rad container for the radiance inference engine (AMD RDNA4, ROCm). The vision tower and the z-lab/Qwen3.8-27B-DFlash2 block-diffusion drafter are merged in.

Needs the radiance release after 1.16.0: the IQR-V4 kernels (the vocabulary-weighted head, the faster GEMV and 16-token tile) ship there. 1.16.0 and older refuse these files.

Weights	IQR-V4 at 2.5 to 8 bits a weight, a mix of 2- to 7-bit trellises and the int8 grid by layer band and projection; int8 activations under the same rotation
Speculator	DFlash2. Its linears are OCP-MXFP4; its vocabulary head is 2-bit codes, and the engine rescores their top 32 candidates against the model's own head. Drafts are verified, so they change the speed, never the output
Vision	the 27-block vision tower, bf16: images and video in chat requests
Context	262,144 tokens trained; 200K tested

The files

Scores are from the radiance quantisation challenge's private split: 1.05M model-written positions, KL against the bf16 model's distribution. The board is at radiance.stilldeadco.de/leaderboard. bpw counts the language model, as the board does; the drafter and the vision tower come on top.

file	bpw	KL mean	top-1	IQR-V3 at that size	KL
qwen3.8-27b-iq2r-v4.rad	2.470	0.06113	91.23%	0.06807	−10%
qwen3.8-27b-iq2r-l-v4.rad	2.797	0.03602	93.29%	0.04241	−15%
qwen3.8-27b-iq3r-s-v4.rad	3.157	0.02241	94.75%	0.02584	−13%
qwen3.8-27b-iq3r-v4.rad	3.472	0.01495	95.74%	0.01698	−12%
qwen3.8-27b-iq3r-l-v4.rad	3.804	0.00917	96.61%	0.01129	−19%
qwen3.8-27b-iq4r-s-v4.rad	4.142	0.00643	97.12%	0.00762	−16%
qwen3.8-27b-iq4r-v4.rad	4.473	0.00444	97.69%	0.00512	−13%
qwen3.8-27b-iq4r-l-v4.rad	4.793	0.00297	98.07%	0.00342	−13%
qwen3.8-27b-iq5r-s-v4.rad	5.071	0.00223	98.33%	0.00249	−10%
qwen3.8-27b-iq5r-v4.rad	5.433	0.00154	98.60%	0.00182	−15%
qwen3.8-27b-iq6r-s-v4.rad	6.063	0.00100	98.86%	0.00106	−6%
qwen3.8-27b-iq6r-v4.rad	6.439	0.00081	98.96%	0.00088	−8%
qwen3.8-27b-iq6r-l-v4.rad	6.939	0.00068	99.08%	0.00072	−6%
qwen3.8-27b-iq7r-v4.rad	7.420	0.00059	99.15%	0.00060	−2%
qwen3.8-27b-iq8r-v4.rad	8.040	0.00046	99.22%	0.00047	−2%

On one card, IQ4R-S-V4 decodes a single stream 4% faster than the MXFP4 checkpoint, at five times lower KL, and IQ3R-S-V4 is faster at every concurrency. The design, the kernels and the measurements are in the IQR-V4 technical report.

Serve

radiance --model qwen3.8-27b-iq4r-v4.rad --tp 1 --max-model-len 200000 --kv-cache-dtype fp8 \
    --max-num-seqs 8 --host 0.0.0.0 --port 8000

Files up to IQ5R fit one 32 GB card with room for long context; use --tp 2 for the larger ones. The server speaks the OpenAI API (/v1/chat/completions, /v1/completions), with tool calls and structured output, and image_url / video_url parts in chat messages. The drafter's depth is chosen automatically (--num-speculative-tokens N sets it, 0 turns speculation off).

How these files were made

The recipe that converts each file is in recipes/. The radiance repository carries the same mixes without the drafter as data/recipes/q38-27b-*-v4.recipe.

For the entries with the vocabulary-weighted head, first split the head into its width classes with tools/vhead.py and the class file the recipe names. The 4.0, 4.5 and 5.0 b class files are in data/vhead/, here and in the radiance repository.

The embedding's trellis tables are in data/codebooks/. The recipes name them by that relative path, so run rad-convert from the directory that holds data/. CALIB is the directory a radiance --calib-corpus ... --calib-dir run wrote. Serving needs none of this: each container carries its tables.

python3 tools/vhead.py Qwen/Qwen3.8-27B q38-27b-vhead-4.5 data/vhead/q38-27b-vhead-4.5.cls
CALIB=<calibration Hessians> rad-convert q38-27b-vhead-4.5 --draft-model z-lab/Qwen3.8-27B-DFlash2 \
    --recipe recipes/q38-27b-iq4r-v4-df2.recipe \
    --set quantization_config.quant_method=w8a8 --set quantization_config.fmt=fwht128 -o out.rad

Every file is the container the board scored: its SHA-256 is checked against the scored file before upload.