bouroo/Qwopus3.8-27B-Coder-Flash-MXFP4-CT

Qwopus3.8-27B-Coder-Flash MXFP4 (compressed-tensors)

MXFP4 (OCP, group 32) compressed-tensors checkpoint of Jackrong/Qwopus3.8-27B-Coder-Flash, the format the vllm-sly-radiance image serves.

The upstream checkpoint is BF16 (56 GB). This repository is the same model quantized to MXFP4, 19.6 GB on disk.

Format

quantization_config declares:

Field	Value
quant_method	compressed-tensors
format	mxfp4-pack-quantized
quantization_status	compressed
config_groups.MXFP4A16.weights	num_bits 4, type float, strategy group, group_size 32, symmetric true, dynamic false, scale_dtype torch.uint8
config_groups.MXFP4A16.targets	["Linear"]

Each quantized Linear is stored as two tensors:

• .weight_packed — uint8, shape [out, in // 2]. Two E2M1 codes per byte, low nibble first.
• .weight_scale — uint8, shape [out, in // 32]. E8M0 exponents (127 + floor(log2(scale))), one per group of 32 weights.

Kept in BF16: lm_head, the whole vision tower, the MTP head (mtp.*), the embedding table, the GDN A_log / dt_bias vectors and the GDN convolution. They are listed in quantization_config.ignore, which the loader matches by exact name and by re: regex.

This is the same layout and the same config fields as slybase/Swift-1.5-Qwen3.8-27B-heretic-MXFP4-GPTQ, the checkpoint the Radiance image benchmarks, except that this one is RTN (calibration-free) and the vision tower, embedding and GDN tensors are kept in BF16 rather than quantized.

Serve it with vllm-sly-radiance

docker pull ghcr.io/slybase/vllm-sly-radiance:1.8.1-rocm10.1

The image's serve-mxfp4.sh launcher points at a checkpoint snapshot. To use this one, point the same launcher at it:

vllm serve /models/qwopus-ct-mxfp4 \
  --served-model-name qwopus-coder-flash \
  --host 0.0.0.0 --port 8080 \
  --kv-cache-dtype fp8 \
  --gpu-memory-utilization 0.93 \
  --max-model-len 262144 \
  --max-num-seqs 8 \
  --max-num-batched-tokens 4096 \
  --attention-backend R4D \
  --speculative-config '{"method":"mtp","num_speculative_tokens":4,"disable_padded_drafter_batch":true}' \
  --no-async-scheduling \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder --reasoning-parser qwen3 \
  --override-generation-config '{"temperature":0.7,"top_p":0.95,"top_k":20}'

Image environment variables that select the native MXFP4 path:

RADIANCE_MXFP4=1
RADIANCE_MXFP4_W4A8=1
RADIANCE_MXFP4_W4A8_MIN_M=0
VLLM_ROCM_USE_AITER=1

Because the checkpoint declares re:^mtp.* in its ignore list, the MTP head stays in BF16 and needs no post-processing; the DFlash2 drafter (tcclaviger/Qwen3.8-27B-DFlash2-FP8 for the MXFP4 launcher, syvai/Qwen3.8-27B-DFlash2-W4A16 for the general one) is the speculative path.

The MTP tensors live in model-00018-of-00018.safetensors of the main checkpoint, not in a separate model_mtp.safetensors as in the reference checkpoint. Point the loader at the whole directory and it will find them.

How it was made

pip install llmcompressor

python - <<'PY'
from llmcompressor import model_free_ptq

model_free_ptq(
    model_stub="./src",                  # the BF16 safetensors checkpoint
    save_directory="./out-ct-mxfp4",
    scheme="MXFP4A16",
    ignore=[
        "lm_head",
        "re:.*visual.*",
        "re:^mtp.*",
        "re:.*embed_tokens",
        "re:.*A_log",
        "re:.*dt_bias",
        "re:.*conv1d",
    ],
    device="cpu",
    max_workers=1,
)
PY

model_free_ptq walks the safetensors shards one at a time, so the 56 GB source is never materialised in memory. The run took 2m50s on an M4 Pro.

The extra ignore entries (embed_tokens, A_log, dt_bias, conv1d) are needed because the model-free matcher is name-based and offers every 2-D tensor to the scheme, while a module-class matcher would skip them.

Verification

Dequantizing a sample of 12 quantized tensors with compressed-tensors' own MXFP4 decompressor and comparing against the BF16 source gives a relative mean absolute error of 0.109 to 0.126, which is the E2M1 code resolution at group size 32. The non-quantized tensors are byte-identical to the source.

License and limitations

Apache-2.0, inherited from the upstream checkpoint. The upstream model card's stated limitations apply: the reported evaluations are small local runs, and generated code and tool actions can be incorrect. See the upstream model card for the full description and the intended use.