StillDeadcode/qwen3.8-27b-iqr-v3

Qwen3.8-27B IQR-V3 — radiance containers, every size in one repository

Qwen3.8-27B quantised with IQR-V3, radiance's trellis format: every trunk linear rotated by a 128-point Hadamard and fitted by GPTQ against calibration Hessians, its codes a tail-biting trellis down each column whose 2^16 states' values are computed from the state, under one f32 scale a row. The GEMM decodes the codes in its tile; nothing is widened in memory. Each file is a single .rad container for the radiance inference engine (AMD RDNA4, ROCm), with the vision tower and the z-lab/Qwen3.8-27B-DFlash2 block-diffusion drafter merged in, as in our other production 27B containers.

Needs the radiance release after 1.14.0 (1.15.0 or newer): the IQR-V3 kernels and the qwen35_w8a8 plugin that reads them ship there; 1.14.0 and older refuse these files.

Weights	IQR-V3 at 2.5 to 8 bits a weight, a mix of 2- to 7-bit trellises and the int8 grid by layer band; int8 activations under the same rotation
Speculator	DFlash2, its linears OCP-MXFP4, its vocabulary head as 2-bit codes whose top 32 candidates the engine rescores against the model's own head. Drafts are verified, so they change the speed and never the output
Vision	the 27-block vision tower, bf16: images and video in chat requests
Context	262,144 tokens trained; 200K tested

The files

Scores on the radiance quantisation challenge's private split (1.05M model-written positions, KL against the bf16 model's distribution); the board is at radiance.stilldeadco.de/leaderboard. bpw counts the language model the board counts (the drafter and the vision tower are on top).

file	bpw	KL mean	top-1	IQR-V2 at that size	KL
qwen3.8-27b-iq2r-v3.rad	2.465	0.06807	90.54%	0.07303	-7%
qwen3.8-27b-iq2r-l-v3.rad	2.786	0.04241	92.62%	0.04698	-10%
qwen3.8-27b-iq3r-s-v3.rad	3.145	0.02584	94.02%	0.02798	-8%
qwen3.8-27b-iq3r-v3.rad	3.459	0.01698	95.35%	0.01880	-10%
qwen3.8-27b-iq3r-l-v3.rad	3.798	0.01129	96.16%	0.01198	-6%
qwen3.8-27b-iq4r-s-v3.rad	4.131	0.00762	96.90%	0.00832	-8%
qwen3.8-27b-iq4r-v3.rad	4.458	0.00512	97.45%	0.00562	-9%
qwen3.8-27b-iq4r-l-v3.rad	4.783	0.00342	97.86%	0.00377	-9%
qwen3.8-27b-iq5r-s-v3.rad	5.064	0.00249	98.15%	0.00273	-9%
qwen3.8-27b-iq5r-v3.rad	5.422	0.00182	98.38%	0.00204	-11%
qwen3.8-27b-iq6r-s-v3.rad	6.049	0.00106	98.83%	0.00115	-8%
qwen3.8-27b-iq6r-v3.rad	6.372	0.00088	98.93%	0.00094	-6%
qwen3.8-27b-iq6r-l-v3.rad	6.939	0.00072	99.04%	0.00073	-2%
qwen3.8-27b-iq7r-v3.rad	7.420	0.00060	99.12%	0.00061	-2%
qwen3.8-27b-iq8r-v3.rad	8.040	0.00047	99.18%	0.00047	0%

IQ4R-L-V3 is below the official FP8 checkpoint's KL (0.00389) at 55% of its size. The design, the per-weight and GEMM measurements and the comparison with IQR-V1 and V2 are in the IQR-V3 technical report.

Serve

radiance --model qwen3.8-27b-iq4r-v3.rad --tp 2 --max-model-len 200000 --kv-cache-dtype fp8 \
    --max-num-seqs 8 --host 0.0.0.0 --port 8000

The smaller files fit one 32 GB card (--tp 1). The server speaks the OpenAI API (/v1/chat/completions, /v1/completions), with tool calls and structured output, and image_url / video_url parts in chat messages. The drafter's depth is chosen automatically (--num-speculative-tokens N states one, 0 turns speculation off).

How these files were made

Each entry is a mix of per-width bases chosen at the size of its IQR-V2 twin; the recipe that converts it is in recipes/ (the radiance repository carries the same mixes without the drafter as data/recipes/q38-27b-*-v3.recipe). The embedding's trellis tables are in data/codebooks/ here and in the radiance repository; the recipes name them by that relative path, so run rad-convert from the directory that holds data/. CALIB is the directory a radiance --calib-corpus ... --calib-dir run wrote. Serving needs neither: each container carries its tables.

CALIB=<calibration Hessians> rad-convert Qwen/Qwen3.8-27B --draft-model z-lab/Qwen3.8-27B-DFlash2 \
    --recipe recipes/q38-27b-iq4r-v3-df2.recipe \
    --set quantization_config.quant_method=w8a8 --set quantization_config.fmt=fwht128 -o out.rad

Every file's language model scores exactly as the container the board scored: its tier-1 KL rows are checked byte for byte before upload.