maci0/Qwen3.6-40B-Claude-4.6-Opus-Deckard-Heretic-Uncensored-Thinking-NVFP4

RQ-40B-DKDUncensored

Qwen3.6-40B Deckard · NVFP4

40B dense VL · Claude 4.6 Opus reasoning distill · thinking · uncensored.

Params	40B
Active	40B (dense)
Size	25.6 GB
Perplexity	6.89
Refusals	n/a
Context	256K
MTP head	n/a

TL;DR: Qwen3.6-40B Deckard, quantized to NVFP4 (W4A4) for vLLM on NVIDIA Blackwell. 25.6 GB, wikitext-2 PPL 6.89, flagship reasoner, uncensored.

Qwen3.6-40B-Claude-4.6-Opus-Deckard-Heretic-Uncensored-Thinking NVFP4

NVFP4 (W4A4) quantization of DavidAU/Qwen3.6-40B-Claude-4.6-Opus-Deckard-Heretic-Uncensored-Thinking, packed in the compressed-tensors nvfp4-pack-quantized format with llm-compressor. Weights are quantized with GPTQ (error-compensated rounding) and an MSE observer, using a domain-matched calibration blend.

Near-lossless versus bf16. SWE-bench Lite resolves land within one instance of the bf16 source at every size, the 27B lm-eval gap averages under 1.5 points, and wikitext-2 perplexity for this build is 6.89. See Benchmarks.

• About 25.6 GB on disk versus about 73.7 GB for the bf16 source (about 35%).
• Built for vLLM on NVIDIA Blackwell, where both the 4-bit weight and 4-bit activation paths are accelerated. On pre-Blackwell GPUs vLLM runs it weight-only.
• Loading and generation verified in vLLM on an NVIDIA GB10 (Blackwell, sm_121).

Uncensored model. This is a quantization of an uncensored / abliterated derivative. It follows instructions without content guardrails, including NSFW. Behaviour and alignment are inherited entirely from the base model.

Benchmarks

Near-lossless versus the bf16 source:

• SWE-bench Lite (agentic, mini-swe-agent, instances 0:20): resolves land within one instance of bf16 at every size (NVFP4 15/13, bf16 16/14 of 20).
• lm-eval (27B pair, the clean apples-to-apples): average accuracy gap under 1.5 points.
• wikitext-2 perplexity (this 40B build, vLLM prompt-logprobs): 6.89.

Full head-to-head tables and method in BENCHMARKS.md.

Fidelity

Near-lossless versus the bf16 source, 25.6 GB vs 73.7 GB bf16 (~35%), at wikitext-2 perplexity 6.89. See Benchmarks for the full head-to-head. GPTQ error compensation and an MSE observer keep the drop from bf16 minimal; the header lists the full characteristics and Quantization covers the recipe.

Quickstart

Offline (vLLM)

NVFP4 activation acceleration needs a Blackwell-class GPU. The if __name__ == "__main__" guard is required for offline LLM(...) because the vLLM v1 engine spawns workers.

from vllm import LLM, SamplingParams

def main():
    llm = LLM(
        model="maci0/Qwen3.6-40B-Claude-4.6-Opus-Deckard-Heretic-Uncensored-Thinking-NVFP4",
        max_model_len=16384,
    )
    msgs = [{"role": "user", "content": "Write the opening paragraph of a noir short story."}]
    sp = SamplingParams(temperature=1.0, top_p=0.95, top_k=20, max_tokens=2048)
    out = llm.chat(msgs, sp)
    print(out[0].outputs[0].text)

if __name__ == "__main__":
    main()

Server (OpenAI-compatible)

Recommended baseline for a single Blackwell GPU. The NVFP4 quantization is auto-detected from config.json (compressed-tensors), so no quantization flag is needed. --reasoning-parser qwen3 splits the <think> block into a separate reasoning_content field.

vllm serve maci0/Qwen3.6-40B-Claude-4.6-Opus-Deckard-Heretic-Uncensored-Thinking-NVFP4 \
  --served-model-name qwen3.6-40b-nvfp4 \
  --tensor-parallel-size 1 \
  --max-model-len 131072 \
  --gpu-memory-utilization 0.90 \
  --kv-cache-dtype fp8 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder

These parser flags are not auto-detected; you must pass them explicitly. Drop the last line if you do not need tool calling; --enable-auto-tool-choice requires --tool-call-parser.

Verified flag values for this model on vLLM:

Goal	Add
Tool / function calling	--enable-auto-tool-choice --tool-call-parser qwen3_coder
Text-only (skip vision tower, free KV cache)	--language-model-only
Bound multimodal inputs	--limit-mm-per-prompt '{"image":4,"video":1}'
Hour-scale video	--media-io-kwargs '{"video":{"num_frames":-1}}' (and raise longest_edge in video_preprocessor_config.json)

Context notes:

• The model supports up to 262144 tokens. Upstream guidance is to keep at least 128K to preserve thinking quality, so --max-model-len 131072 is the recommended default. Go to 262144 if memory allows, or lower it if you hit OOM.
• On unified-memory parts (e.g. GB10), --gpu-memory-utilization carves from RAM shared with the rest of the system. Use about 0.90 when this is the only model, and leave more headroom (about 0.80) when co-hosting other processes.

Python (OpenAI client)

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="x")
r = client.chat.completions.create(
    model="qwen3.6-40b-nvfp4",
    messages=[{"role": "user", "content": "Write the opening paragraph of a noir short story."}],
)
print(r.choices[0].message.content)

curl

curl http://localhost:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
  "model": "qwen3.6-40b-nvfp4",
  "messages": [{"role": "user", "content": "Write the opening paragraph of a noir short story."}]
}'

KV-cache quantization

The NVFP4 here quantizes weights and activations, not the KV cache (the checkpoint ships kv_cache_scheme: null). KV-cache quantization is a separate runtime vLLM option. Only the 24 full-attention layers hold a standard KV cache; the gated delta-net linear-attention layers use recurrent state and are unaffected.

• FP8 KV cache (recommended, safe): --kv-cache-dtype fp8 (or fp8_e4m3). About 2x KV savings with small quality cost. Used in the baseline command above.
• TurboQuant (vLLM, experimental here): lower-bit KV quant via Hadamard rotation plus per-coordinate Lloyd-Max scalar quantization. Values: turboquant_k8v4 (FP8/4-bit, 2.6x, +1.17% PPL), turboquant_4bit_nc (3.8x), turboquant_k3v4_nc (~3.5x), turboquant_3bit_nc (4.9x). turboquant_k8v4 is the quality sweet spot. Caveat: TurboQuant uses a dedicated attention backend whose interaction with this model's linear-attention layers was not verified. Treat as experimental; prefer fp8 for a known-good KV quant.

Performance / backend notes (verified on vLLM)

• FlashInfer is bundled and autotune is on by default. It is used automatically for the full-attention and NVFP4 GEMM paths on Blackwell; there is nothing to enable. Optional: VLLM_USE_FLASHINFER_SAMPLER=1 for faster sampling.
• NVFP4 GEMM auto-selects cutlass FP4 on Blackwell. Do not set VLLM_NVFP4_GEMM_BACKEND (deprecated in 0.23.0). Leave VLLM_USE_NVFP4_CT_EMULATIONS=0 (the default; emulation is for pre-Blackwell).
• Attention backend: leave on auto. This is a hybrid model, so vLLM assigns the per-layer backends (GDNAttentionBackend / LinearAttentionBackend) automatically. Forcing a single global attention backend breaks the linear-attention layers.
• No sparse-attention knob applies. The efficiency comes from the hybrid 3:1 linear:full attention layout, handled automatically.

About the base model

A 40B dense (not MoE) vision-language model expanded from Qwen3.6-27B, made uncensored via Heretic, trained on the internal Deckard/PKD datasets (character, depth, point of view) and on a Claude 4.6 Opus high-reasoning distillation set to sharpen and stabilize reasoning.

• 96 decoder layers: hybrid gated delta-net linear attention (72) plus full attention (24), dense MLP, plus a vision tower for image and video input.
• 256K context (max_position_embeddings 262144).
• Thinking mode by default (variable-length reasoning), with an instruct toggle.

Quantization

Scheme	NVFP4, W4A4
Weight rounding	GPTQ (Hessian-based error compensation), MSE observer
Weights	FP4 (E2M1), group_size=16, tensor_group, symmetric, FP8 (E4M3) group scales
Activations	FP4, dynamic per-group (dynamic: local), FP8 (E4M3) scales
Targets	all language-model Linear layers, 744 modules (360 linear-attn projections + 288 MLP + 96 full-attn)
Kept in bf16	vision tower (model.visual.*), lm_head
Untouched	gated delta-net Conv1d and SSM params (A_log, dt_bias), not Linear, never targeted

GPTQ is a quantization-time cost only. The output is the same nvfp4-pack-quantized format with identical inference speed; GPTQ just chooses better 4-bit values than plain round-to-nearest.

Calibration

512 samples, domain-matched to the model's actual traffic, max_seq_len=2048, text-only path through the VL model:

source	samples	domain
TeichAI/claude-4.5-opus-high-reasoning-250x	250	long reasoning (the base model's own training data)
HuggingFaceH4/ultrachat_200k	150	general chat
m-a-p/Code-Feedback	112	code

Quality

GPTQ with the domain-matched calibration measurably beats plain round-to-nearest. The fused layers (q/k/v, gate/up) share one NVFP4 global scale, so vLLM does not warn or fall back. Measured wikitext-2 perplexity for this build is 6.89 (see Benchmarks); the gain is expected to be larger on the model's own domains (reasoning, creative, code), which wikitext does not cover.

Recommended sampling

Thinking mode is the default.

• Thinking, general: temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, repetition_penalty=1.0
• Thinking, precise coding: temperature=0.6, top_p=0.95, top_k=20
• Instruct / non-thinking: temperature=0.7, top_p=0.80, top_k=20, presence_penalty≈1.5
• If the model loops on thin prompts, add a one-line system prompt (e.g. Be vivid and precise.) and/or set repetition_penalty 1.05 to 1.1.

To run instruct (non-thinking), set {%- set enable_thinking = false %} in the Jinja chat template, or pass extra_body={"chat_template_kwargs": {"enable_thinking": false}} on OpenAI-compatible endpoints.

Reproduction

See scripts/quantize_nvfp4.py for the full recipe and QUANTIZATION.md for the end-to-end methodology.

Toolchain: llmcompressor==0.12.0, compressed-tensors==0.17.1, transformers==5.12.1, torch==2.11.0+cu130, on an NVIDIA GB10 (Blackwell, sm_121).

Related

• Base model: DavidAU/Qwen3.6-40B-Claude-4.6-Opus-Deckard-Heretic-Uncensored-Thinking
• Space: Rogue Quants
• Collection: NVFP4 Quants
• Sibling NVFP4 quants:
  • Ornith-1.0-35B MoE abliterated
  • Qwopus3.6-27B-Coder
  • Qwopus3.6-27B-Coder abliterated
  • Qwopus3.6-27B-v2 abliterated
  • Huihui-Qwythos-9B Claude-Mythos
  • Ornith-1.0-9B abliterated

Notes

• Needs NVIDIA Blackwell (sm_121, e.g. GB10) for accelerated W4A4; pre-Blackwell GPUs run it weight-only.
• --reasoning-parser and --tool-call-parser are not auto-detected; pass them explicitly.
• Thinking mode is on by default; toggle it via the chat template or chat_template_kwargs.

License

Apache-2.0, following the base model. Intended use and all responsibility for use follow the base model.

Credits

• Base model: DavidAU
• Quantization tooling: llm-compressor / compressed-tensors

Part of Rogue Quants · NVFP4 component datasheets · collection. Fabricated on GB10 (Blackwell) with llm-compressor. Refusals shown per 100 harmful prompts; "n/a" = not separately measured (base-inherited).