Qwopus3.8-27B-Flash-V2-FP8

FP8 quantization of Jackrong/Qwopus3.8-27B-Flash-V2, produced with llm-compressor (FP8_BLOCK), for vLLM.

The upstream repo publishes BF16 weights (55.6 GB) plus GGUF builds. This repo is the same model at 29.1 GiB, with the vision tower and the MTP drafter intact.

Base revision	13f92e09a46fa364f8de1edb85684d57bda01126
Size	29.09 GiB (31.2 GB), 8 shards
Recommended	chat_template_qwen_xml.jinja + --tool-call-parser qwen3_coder (see below)

Scheme

Same scheme and the same module coverage as the official Qwen/Qwen3.8-27B-FP8 release, so it runs on the same well-trodden vLLM kernel path.

Weights	FP8 e4m3, static, 128×128 blocks
Activations	FP8 e4m3, dynamic, per-token groups of 128
Format	compressed-tensors (float-quantized)
Calibration	none (data-free)

400 Linear modules are quantized:

• q_proj/k_proj/v_proj/o_proj of the 16 full-attention layers
• in_proj_qkv/in_proj_z/out_proj of the 48 gated-delta-net (linear-attention) layers
• gate_proj/up_proj/down_proj of all 64 MLPs

Kept in BF16: in_proj_a/in_proj_b, conv1d, all norms, embed_tokens, lm_head, the whole vision tower, and the MTP head (mtp.*, so MTP speculative decoding works).

Every non-quantized tensor is byte-identical to upstream, and so are the tokenizer, processor, generation config and chat template. This was checked tensor by tensor, and it needed a fix: llm-compressor 0.13 converts Qwen3.5's offset-style RMSNorm weights (1 + w) for its pipeline and converts them back in BF16, which rounds every input_layernorm/post_attention_layernorm/q_norm/k_norm weight to a ~0.004 grid (up to ~7% relative error on individual values). The shipped checkpoint takes those tensors straight from the source instead.

Dequantized-vs-source relative error of the quantized weights is ~2.7%, as expected for FP8 e4m3.

Serving with vLLM

vllm serve batsclamp/Qwopus3.8-27B-Flash-V2-FP8 \
  --max-model-len 262144 \
  --speculative-config '{"method": "mtp", "num_speculative_tokens": 3}' \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --chat-template <snapshot>/chat_template_qwen_xml.jinja \
  --tool-call-parser qwen3_coder \
  --override-generation-config '{"temperature": 1.0, "top_p": 0.95, "top_k": 20}'

Which chat template

The repo carries two templates that differ only in the tool-call format:

File	Tool-call format	vLLM parser
chat_template.jinja (upstream, unchanged)	JSON inside <tool_call>	hermes
chat_template_qwen_xml.jinja (= stock Qwen3.8-27B template)	XML <function=…><parameter=…>	qwen3_coder

The parser must match the template. chat_template.jinja is upstream's file byte for byte and is what vLLM picks up by default; we recommend overriding it with the XML one. A/B on the same weights, same server, reasoning_effort: xhigh, temperature 1.0:

	upstream JSON + hermes	Qwen XML + qwen3_coder
tool-eval-bench v2.0.6, 69 scenarios	84 (18 not fully passed)	91 (10 not fully passed)
Python file written through a tool call compiles, temp 1.0 (n=10)	7/10	9/10
same, temp 0.3 (upstream's coding recommendation) (n=10)	3/10	10/10

With the JSON format the model has to emit code as an escaped JSON string, and it often collapses Python indentation there (if x:\n raise …). In raw XML parameters it does not. Single runs at n=10 are noisy, but both measurements point the same way.

The chat template (upstream's and Qwen's alike) accepts only reasoning_effort values xhigh, medium and low, and raises on anything else (high, max, minimal), which an OpenAI-compatible server turns into HTTP 400. If your client sends other values, map them in a small Jinja prelude before the template.

For coding and agentic work the upstream card recommends reasoning_effort: xhigh with temperature: 0.3 supplied by the caller.

Measured on one DGX Spark (GB10)

vLLM 0.32.0rc1 (nightly of 2026-10-16), --max-model-len 262144, MTP k=3, --gpu-memory-utilization 0.55, kv-cache-dtype auto:

Weights resident	29.1 GiB
KV cache	29.2 GiB, ~424k tokens (1.62× concurrency at 262k)
Decode, single stream, no thinking	14.8 tok/s
TTFT, short prompt	185 ms
MTP	1.58 accepted tokens per draft
Kernel	CutlassFp8BlockScaledMMKernel

The reasoning effort levels behave as upstream: with xhigh a simple factual question used ~290 characters of reasoning.

Caveats

• Quantization was verified structurally (tensor-by-tensor against the source) and by generation and tool-calling tests on vLLM; no perplexity or benchmark-suite numbers are claimed beyond what is listed above.
• All model behaviour, strengths and limitations are upstream's; see the original model card.