maci0/Qwopus3.6-27B-Coder-NVFP4

◆ Rogue Quants · NVFP4

🪐 Qwopus3.6-27B-Coder · NVFP4

💻 Coder

27B coder vision-language · agentic + tool-calling · thinking · GPTQ NVFP4 W4A4

⚙️ NVFP4 · W4A4 💾 ~18 GB 📉 PPL 6.63 📐 256K context 🚀 vLLM · Blackwell 💻 Coder 🛠️ Tool-calling

Size on disk 18 GB vs 55.6 GB bf16 (~33%)

wikitext-2 PPL 6.63 near-lossless vs bf16

Context 256K 262144 tokens

Scheme NVFP4 W4A4 · GPTQ + MSE

TL;DR: Qwopus3.6-27B-Coder, quantized to NVFP4 (W4A4) for vLLM on NVIDIA Blackwell. 18 GB, wikitext-2 PPL 6.63, 256K agentic coder.

Qwopus3.6-27B-Coder NVFP4

NVFP4 (W4A4) quantization of Jackrong/Qwopus3.6-27B-Coder, packed in the compressed-tensors nvfp4-pack-quantized format with llm-compressor. Weights are quantized with GPTQ (error-compensated rounding) and an MSE observer, on a domain-matched calibration blend that includes code.

Near-lossless. Fused layers (q/k/v, gate/up) share one NVFP4 global scale, so vLLM loads it cleanly with no per-layer-scale warning or fallback. wikitext-2 perplexity for this build: 6.63.

• About 18 GB on disk versus about 55.6 GB for the bf16 source (about 33%).
• Built for vLLM on NVIDIA Blackwell, where both the 4-bit weight and 4-bit activation paths are accelerated. On pre-Blackwell GPUs vLLM runs it weight-only.
• Loading and generation verified in vLLM v0.23.0 on an NVIDIA GB10 (Blackwell, sm_121).

Fidelity

Near-lossless versus the bf16 source: wikitext-2 perplexity for this build is 6.63.

Metric	Value
wikitext-2 PPL	6.63
Weights	NVFP4 W4A4, group 16
Size	18 GB vs 55.6 GB bf16 (~33%)

NVFP4 uses GPTQ error compensation, an MSE observer, and shared fused-layer scales, so the drop from bf16 is minimal.

Quickstart

NVFP4 is auto-detected from config.json (compressed-tensors); no quantization flag needed. --reasoning-parser qwen3 splits the <think> block into reasoning_content; --tool-call-parser qwen3_coder enables tool/function calling for agentic coding.

vllm serve maci0/Qwopus3.6-27B-Coder-NVFP4 \
  --served-model-name qwopus-27b-coder-nvfp4 \
  --max-model-len 131072 \
  --gpu-memory-utilization 0.90 \
  --kv-cache-dtype fp8 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder

• Supports up to 262144 tokens; keep at least 128K to preserve thinking quality. --max-model-len 131072 is a safe default; raise it if memory allows.
• Add --language-model-only to skip the vision tower and free KV cache for text use.
• The parser flags are not auto-detected; pass them explicitly.

Python (OpenAI client)

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="x")
r = client.chat.completions.create(
    model="qwopus-27b-coder-nvfp4",
    messages=[{"role": "user", "content": "Write a Python function that merges two sorted lists."}],
)
print(r.choices[0].message.content)

curl

curl http://localhost:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
  "model": "qwopus-27b-coder-nvfp4",
  "messages": [{"role": "user", "content": "Write a Python function that merges two sorted lists."}]
}'

About the base model

A 27B Qwen3.5-family vision-language model specialized for code (Qwopus 3.6 Coder), with thinking-mode reasoning and a 256K context window.

• 64 decoder layers: hybrid gated delta-net linear attention plus full attention, dense MLP, plus a vision tower for image and video input.
• 256K context (max_position_embeddings 262144).
• Thinking mode by default, with an instruct toggle.

Quantization

Scheme	NVFP4, W4A4
Weight rounding	GPTQ (Hessian-based error compensation), MSE observer
Weights	FP4 (E2M1), group_size=16, tensor_group, FP8 (E4M3) group scales, shared across fused layers
Activations	FP4, dynamic per-group, FP8 (E4M3) scales
Quantized	all language-model Linear layers
Kept in bf16	vision tower (model.visual.*), lm_head, MTP head
Untouched	gated delta-net Conv1d and SSM params (A_log, dt_bias), never Linear

GPTQ is a quantization-time cost only; inference speed and format are identical to plain round-to-nearest NVFP4, but it chooses better 4-bit values.

Calibration: 512 domain-matched samples (long reasoning + general chat + code), max_seq_len=2048, text-only path through the VL model.

Recommended sampling

Thinking mode is the default.

• Thinking, precise coding: temperature=0.6, top_p=0.95, top_k=20
• Thinking, general: temperature=1.0, top_p=0.95, top_k=20
• Instruct / non-thinking: temperature=0.7, top_p=0.80, top_k=20
• To run non-thinking, set {%- set enable_thinking = false %} in the chat template, or pass extra_body={"chat_template_kwargs": {"enable_thinking": false}}.

Reproduction

Toolchain: llmcompressor==0.12.0, compressed-tensors==0.17.1, transformers==5.12.1, torch==2.11.0+cu130, on an NVIDIA GB10 (Blackwell, sm_121). llm-compressor 0.12 shares the NVFP4 global scale across fused layers automatically (q/k/v, gate/up).

Related

• Base model: Jackrong/Qwopus3.6-27B-Coder
• Space: Rogue Quants
• Collection: NVFP4 Quants
• Sibling NVFP4 quants:
  • Qwen3.6-40B Deckard
  • Huihui-Qwythos-9B Claude-Mythos
  • Ornith-1.0-9B abliterated

Notes

• Needs NVIDIA Blackwell (sm_121, e.g. GB10) for accelerated W4A4; pre-Blackwell GPUs run it weight-only.
• --reasoning-parser and --tool-call-parser are not auto-detected; pass them explicitly.
• Thinking mode is on by default; toggle it via the chat template or chat_template_kwargs.

License

Apache-2.0, following the base model. Intended use and all responsibility for use follow the base model.

Credits

• Base model: Jackrong
• Quantization tooling: llm-compressor / compressed-tensors

Part of 🎲 Rogue Quants, a set of NVFP4 (W4A4) quants for vLLM on Blackwell. See the full NVFP4 Quants collection.
Built on NVIDIA GB10 (Blackwell, sm_121) with llm-compressor · GPTQ + MSE · shared fused-layer scales.