edgeai-lab/Qwen3.8-27B-FlipQuant

FlipQuant MixFP4 maps for Qwen3.8-27B

What this is

Format maps for Qwen/Qwen3.8-27B at revision 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0, calibrated with FlipQuant (TM-OPT+TC). A map holds one bit per weight tile of every quantized linear layer (all text-model linears except the LM head): True stores the tile's 4-bit elements as E0M3 (uniform levels 0..7), False as E2M1 (FourOverSix). Everything else is NVFP4: 16-element blocks with an E4M3 scale and a per-tensor FP32 scale. The maps hold no weights: they are applied to the base model's BF16 checkpoint when it is loaded.

file	tile (output rows × input columns)	E0M3 tiles	sha256
flipquant_8x64.pt	8 × 64	169,073 of 47,559,680 (0.36 %)	f9e0dec72ecd4278a7f22f773390b2205293119275285cbdcf773eef7ceb4a5a
flipquant_16x64.pt	16 × 64	105,278 of 23,779,840 (0.44 %)	050f0eead1bb2199bdcdb36b489618ef7177b0e325eefe5e412b067965d43e8d
flipquant_256x64.pt	256 × 64	13,758 of 1,492,480 (0.92 %)	59b38ae511226745a401be7616ed9a4049841fb797918badcddd3911c99fd30f

8x64 is the smallest tile one MMA operand covers on sm_100 (and with weights as the B operand on sm_120); 16x64 and 256x64 are coarser.

The FlipQuant code will be released with the paper.

Calibration

python -m calibration.train_map --model qwen3.8-27b --unit <unit> --fit-windows 256 --epochs 5 --teacher-topk 1000 (FlipQuant commit 120173a4f67c67f5e67f93ea02fa7a9eaff3ef14):

• TM-OPT+TC: one logit per tile, a straight-through hard map, KL to the BF16 model's top-1000 tokens per position plus one tail bucket, per-token FourOverSix activations.
• Adam, learning rate 0.02, initial logit −1 (every tile starts as E2M1), seed 0, micro-batch 2 × accumulation 4 (8 sequences per step), 5 epochs, deterministic.
• Fit set: 256 windows of 512 tokens. They are the FlipQuant paper's 128 math/code windows (OpenWebMath @ fde8ef8, CodeParrot-clean @ 35a59fb), plus 128 more by the extension rule, which skips the paper's fit, development and published C4 evaluation documents. The calibration records every window by document hash, offset and token hash.
• These are not the paper's settings (20 epochs, 128 windows, the full-vocabulary teacher). The maps' meta records them as non_default_settings.

Cost on one NVIDIA RTX PRO 6000 Blackwell (sm_120), per unit:

model	unit	model load	data (+ extension)	teacher	lean packing	setup	per epoch (mean, min–max)	training	trainer total	train_map end-to-end	batch × accum
qwen3.8-27b	8x64	6.7 s	19.2 s	165.9 s	129.7 s	321.6 s	432.8 s (429.7–435.1)	36.1 min	41.4 min	41.6 min	2 × 4
qwen3.8-27b	16x64	6.7 s	19.4 s	163.3 s	130.6 s	319.9 s	430.9 s (428.2–434.2)	35.9 min	41.2 min	41.5 min	2 × 4
qwen3.8-27b	256x64	6.7 s	19.4 s	165.6 s	127.7 s	319.3 s	431.1 s (428.1–432.8)	35.9 min	41.3 min	41.5 min	2 × 4

model	unit	GPU peak allocated	GPU peak reserved	trainer peak RSS (ru_maxrss)	RSS after model load	teacher (top-1000)	wrapper RSS	process-tree peak (10 Hz)	source of the last two
qwen3.8-27b	8x64	91.7 GiB	92.3 GiB	51.9 GiB	1.0 GiB	0.73 GiB	0.9 GiB (VmHWM)	51.9 GiB	live
qwen3.8-27b	16x64	91.3 GiB	92.8 GiB	51.9 GiB	1.0 GiB	0.73 GiB	0.8 GiB (VmHWM)	52.3 GiB	live
qwen3.8-27b	256x64	90.9 GiB	92.3 GiB	51.9 GiB	1.0 GiB	0.73 GiB	0.8 GiB (VmHWM)	52.2 GiB	live

The one-time data preparation (fit and development records, before the first unit):

train_map's first run of a model on a new data root prepares its calibration record and development sets (calibration.tmopt_common.prepare and prepare_development: the builder rule's checks, re-tokenizing the fit windows, hashing the model's weights on the CPU unless an archived record has them, the development draws). Measured on 2026-10-06 in a fresh data root with the same sampler plus each process's VmHWM; the files it wrote are byte-identical to the ones the release runs used. The dataset shards were already in the local Hugging Face cache; a first-ever preparation also downloads them.

model	wall time	process-tree peak (10 Hz)	wrapper	largest process (VmHWM)	records identical to the release run's
qwen3.8-27b	92 s	2.0 GiB	0.5 GiB	1.5 GiB	yes (11 files)

Time and memory: the trainer's own phase timer and peak counters give the phases, the epochs, the GPU peaks, its ru_maxrss, the RSS after the model load and the teacher storage (the release run). The wrapper's RSS, the process-tree peak (wrapper + trainer, sampled at 10 Hz) and the end-to-end time come from an external process sampler: for the runs marked ᴿ (source “re-run”) from a measured re-run of the same calibration that reproduced the same map, otherwise from a sampler attached to the run itself (“live”). The end-to-end time excludes the one-time data preparation, which has its own row.

Perplexity

Native sm_120 kernels, FlipQuant's --paper-convention (per-token FourOverSix activations), WikiText-2 (145 windows of 2048 tokens) and C4 (256 windows of 2048 tokens). ΔNLL is the mean paired per-window difference in nats per token against FourOverSix; * marks |Δ| > 2 SE. The verdict is “better” or “worse” only when |Δ| > 2 SE. The numbers are also in ppl_summary.json.

method	WikiText-2 PPL	C4 PPL	ΔNLL WikiText-2 vs FO6 (± 2 SE)	ΔNLL C4 vs FO6 (± 2 SE)	verdict (WikiText-2 / C4)
BF16	7.0509	9.8935	-0.0377 ± 0.0058 *	-0.0292 ± 0.0026 *	better / better
FourOverSix (NVFP4, stock kernels)	7.3215	10.1869	—	—	reference
FlipQuant 8x64 (this release)	7.1430	10.1358	-0.0247 ± 0.0059 *	-0.0050 ± 0.0009 *	better / better
FlipQuant 16x64 (this release)	7.1705	10.1444	-0.0208 ± 0.0046 *	-0.0042 ± 0.0010 *	better / better
FlipQuant 256x64 (this release)	7.2369	10.1620	-0.0116 ± 0.0037 *	-0.0024 ± 0.0008 *	better / better
paper-setting map 8x64 (reference)	7.1074	10.1235
paper-setting map 16x64 (reference)	7.1232	10.1276
paper-setting map 256x64 (reference)	7.1773	10.1512

The paper-setting rows are maps calibrated at the paper's settings (20 epochs, 128 windows, the full teacher), evaluated under the same convention; they are listed for comparison only.

How to use

The FlipQuant code will be released with the paper. With it, on an sm_120 GPU (RTX PRO 6000 / RTX 50-series class), build the kernels once, then evaluate with the native kernels:

bash kernels/sm120/build_deploy.sh
python -m evaluation.ppl --model qwen3.8-27b --mode native --weight mixfp4 --map flipquant_16x64.pt --paper-convention

Measured here with the 16x64 map: 70 GiB of GPU memory at peak (2048-token windows, batch 1), 6.5 min for both corpora, model load included.

The native command produced the table above. The simulated (fake) path does not fit this model on one 96 GB GPU: it builds both candidate encodings of every weight on the GPU, next to the BF16 model, and ran out of memory while doing so (smoke test, 2026-10-06). Use the native kernels, or a GPU with more memory.

License

These maps are released under the same license as the base model, the Apache License 2.0.

Files

• flipquant_<unit>.pt: the maps (flipquant-map/1: {format, unit, tiles, meta}; flipquant.maps.load reads them).
• ppl_summary.json: the perplexities, the paired differences against FourOverSix and the evaluation settings.

Provenance of the map files. They are the calibration outputs with only their meta changed: the run directory and calibration-record paths are relative instead of local absolute paths, the data-extension code reference names the file without its repository owner, and the internal branch and commit of the reference implementation are removed (the FlipQuant commit is kept). The tiles are bit-identical. sha256 of the calibration output → this repository's file:

• flipquant_8x64.pt: 17c8e5c79c9c3900d448765ba7324c3b8149cc5883f208e0a18e15ae4ef88ad0 → f9e0dec72ecd4278a7f22f773390b2205293119275285cbdcf773eef7ceb4a5a
• flipquant_16x64.pt: 9b051a0c76796d12236bbdd3c80bb0815db7c00f5f370110c5e7dc5d04cc710e → 050f0eead1bb2199bdcdb36b489618ef7177b0e325eefe5e412b067965d43e8d
• flipquant_256x64.pt: 914e843b1ce4aa494ff0a093188a4435f45dac81f9cb56fe6147c13901188e5a → 59b38ae511226745a401be7616ed9a4049841fb797918badcddd3911c99fd30f