maci0/ThinkingCap-Qwen3.6-27B-abliterated-NVFP4

◆ Rogue Quants · NVFP4

🎩 ThinkingCap-Qwen3.6-27B abliterated · NVFP4

🔓 Abliterated

27B thinking vision-language · abliterated (2-round Heretic) · MTP head kept bf16 for speculative decoding · GPTQ NVFP4 W4A4

⚙️ NVFP4 · W4A4 💾 ~18 GB 📉 PPL 7.25 📐 256K context 🔮 MTP head bf16 🚀 vLLM · Blackwell 🚫 Refusals 8/100

Refusals 99 → 8 2 rounds (of 100 prompts)

KL divergence 0.0408 <0.5 = capability kept

MTP head bf16 speculative decoding intact

Size on disk 18 GB vs ~55 GB bf16 (~33%)

TL;DR: ThinkingCap-Qwen3.6-27B abliterated, quantized to NVFP4 (W4A4) for vLLM on NVIDIA Blackwell. 18 GB, wikitext-2 PPL 7.25, 256K thinking VL, refusals removed (2-round Heretic). The MTP (multi-token prediction) head is kept at bf16 so speculative decoding runs at full precision.

ThinkingCap-Qwen3.6-27B abliterated NVFP4

bottlecapai/ThinkingCap-Qwen3.6-27B, abliterated (refusal direction removed) with Heretic in two iterative rounds, then quantized to NVFP4 (W4A4) in the compressed-tensors nvfp4-pack-quantized format with llm-compressor (GPTQ + MSE, shared fused-layer scales) — with the MTP head left at bf16.

Near-lossless, decensored, and speculative-decoding-ready. Two Heretic rounds cut refusals from 99/100 to 8/100 while keeping a KL divergence of 0.0408 to the original model (well under the 0.5 line that signals capability damage). NVFP4 compresses to ~18 GB at wikitext-2 perplexity 7.25. The multi-token-prediction head that drives self-speculative decoding is excluded from quantization and kept at bf16, so the speedup it provides is not degraded by 4-bit draft predictions.

• Built for vLLM on NVIDIA Blackwell (4-bit weight + 4-bit activation). Pre-Blackwell GPUs run it weight-only.
• Loading and generation verified in vLLM v0.23.0 on an NVIDIA GB10 (Blackwell, sm_121).

Uncensored / abliterated model. It follows instructions without refusal guardrails. The abliteration only removes refusals; all other behaviour comes from the base model. You are responsible for how you use it.

Why the MTP head is kept at bf16

ThinkingCap ships a multi-token prediction (MTP) head (mtp_num_hidden_layers: 1) — a small extra module that predicts the next token(s) so the model can self-speculate: it drafts ahead and verifies in one pass, cutting decode latency without a separate draft model.

That head is on the critical path for acceptance rate. If it were quantized to 4-bit, its draft predictions would drift from what the full-precision main model would have produced, the verifier would reject more drafts, and the speculative speedup would shrink — you would pay the MTP compute and get less of the latency back. So the quant recipe ignores re:.*mtp.* and leaves the head at bf16 (it is tiny relative to the 27B backbone, so the size cost is negligible). The 4-bit main model does the heavy lifting; the full-precision MTP head keeps acceptance high.

Fidelity

Near-lossless versus the bf16 source: wikitext-2 perplexity for this build is 7.25.

Metric	Value
wikitext-2 PPL	7.25
Weights	NVFP4 W4A4, group 16
Size	18 GB vs 55 GB bf16 (33%)
KL divergence	0.0408 (capability preservation, lower is better)
MTP head	kept bf16 (speculative decoding unaffected)

NVFP4 uses GPTQ error compensation, an MSE observer, and shared fused-layer scales, so the drop from bf16 is minimal.

Quickstart

NVFP4 is auto-detected from config.json (compressed-tensors); no quantization flag needed. --reasoning-parser qwen3 splits the <think> block into reasoning_content.

vllm serve maci0/ThinkingCap-Qwen3.6-27B-abliterated-NVFP4 \
  --served-model-name thinkingcap-27b-abliterated-nvfp4 \
  --max-model-len 131072 \
  --gpu-memory-utilization 0.90 \
  --kv-cache-dtype fp8 \
  --reasoning-parser qwen3

• Supports up to 262144 tokens; keep at least 128K to preserve thinking quality. --max-model-len 131072 is a safe default; raise it if memory allows.
• Add --language-model-only to skip the vision tower and free KV cache for text use.
• The parser flag is not auto-detected; pass it explicitly.

Speculative decoding (MTP)

The bf16 MTP head enables vLLM's built-in self-speculative decoding. Enable it with a speculative config that uses the model's own next-token-prediction module (no separate draft model):

vllm serve maci0/ThinkingCap-Qwen3.6-27B-abliterated-NVFP4 \
  --served-model-name thinkingcap-27b-abliterated-nvfp4 \
  --max-model-len 131072 --kv-cache-dtype fp8 --reasoning-parser qwen3 \
  --speculative-config '{"method": "mtp", "num_speculative_tokens": 1}'

num_speculative_tokens matches the head's depth (mtp_num_hidden_layers: 1). Measure acceptance rate on your traffic; because the head is bf16, acceptance is not penalized by quantization.

Python (OpenAI client)

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="x")
r = client.chat.completions.create(
    model="thinkingcap-27b-abliterated-nvfp4",
    messages=[{"role": "user", "content": "Explain why a heap is better than a sorted array for a priority queue."}],
)
print(r.choices[0].message.content)

About the base model

A 27B Qwen3.5-family vision-language model (ThinkingCap-Qwen3.6-27B) with thinking-mode reasoning, a 256K context window, and an MTP head for speculative decoding.

• 64 decoder layers: hybrid gated delta-net linear attention plus full attention, dense MLP, plus a vision tower for image and video input.
• MTP head (mtp_num_hidden_layers: 1) for multi-token / speculative decoding.
• 256K context (max_position_embeddings 262144).
• Thinking mode by default, with an instruct toggle.

Abliteration

Heretic removes the refusal direction with a TPE-optimized search over per-component ablation strength, jointly minimizing refusal rate and KL divergence from the original model, then merges the best trial. This model was abliterated in two iterative rounds: round 1 removed the dominant refusal direction, then Heretic was re-run on the round-1 model to remove the residual refusal direction that surfaced once the first was gone. Because ThinkingCap is a thinking model, evaluation ran in non-thinking mode so each judged response is a real answer rather than an unfinished <think> block; the shipped model restores the original thinking chat template.

Round	Refusals	Note
Baseline	99/100	original model
Round 1	32/100	dominant refusal direction removed
Round 2	8/100	residual direction removed (final)

• Datasets: mlabonne/harmless_alpaca (good) vs mlabonne/harmful_behaviors (bad).
• Final KL divergence 0.0408 (capability preserved, well under the 0.5 damage line).

Quantization

Scheme	NVFP4, W4A4
Weight rounding	GPTQ (Hessian-based error compensation), MSE observer
Weights	FP4 (E2M1), group_size=16, tensor_group, FP8 (E4M3) group scales, shared across fused layers
Activations	FP4, dynamic per-group, FP8 (E4M3) scales
Quantized	all language-model Linear layers
Kept in bf16	MTP head (model.mtp.*), vision tower (model.visual.*), lm_head
Untouched	gated delta-net Conv1d and SSM params (A_log, dt_bias), never Linear

The recipe ignores re:.*mtp.*, re:.*visual.*, and lm_head. Keeping the MTP head at full precision is the point: it preserves speculative-decoding acceptance (see above). GPTQ is a quantization-time cost only; inference speed and format are identical to plain round-to-nearest NVFP4, but it chooses better 4-bit values.

Calibration: 512 domain-matched samples (long reasoning + general chat + code), max_seq_len=2048, text-only path through the VL model.

Recommended sampling

Thinking mode is the default.

• Thinking, precise: temperature=0.6, top_p=0.95, top_k=20
• Thinking, general: temperature=1.0, top_p=0.95, top_k=20
• Instruct / non-thinking: temperature=0.7, top_p=0.80, top_k=20
• To run non-thinking, set {%- set enable_thinking = false %} in the chat template, or pass extra_body={"chat_template_kwargs": {"enable_thinking": false}}.

Related

• Base model: bottlecapai/ThinkingCap-Qwen3.6-27B
• Space: Rogue Quants
• Collection: NVFP4 Quants
• Sibling NVFP4 quants:
  • Qwopus3.6-27B-Coder abliterated
  • Qwopus3.6-27B-v2 abliterated
  • Ornith-1.0-35B MoE abliterated
  • Qwen3.6-40B Deckard
  • Huihui-Qwythos-9B Claude-Mythos
  • Ornith-1.0-9B abliterated

Notes

• Needs NVIDIA Blackwell (sm_121, e.g. GB10) for accelerated W4A4; pre-Blackwell GPUs run it weight-only.
• The MTP head is bf16; enable speculative decoding via --speculative-config to use it.
• --reasoning-parser is not auto-detected; pass it explicitly.
• Thinking mode is on by default; toggle it via the chat template or chat_template_kwargs.
• No refusal guardrails; you are responsible for how you use it.

License

Apache-2.0, following the base model. Intended use and all responsibility for use follow the base model.

Credits

• Base model: bottlecapai
• Abliteration: Heretic by Philipp Emanuel Weidmann
• Quantization tooling: llm-compressor / compressed-tensors

Part of 🎲 Rogue Quants, a set of NVFP4 (W4A4) quants for vLLM on Blackwell. See the full NVFP4 Quants collection.
Built on NVIDIA GB10 (Blackwell, sm_121) with llm-compressor · GPTQ + MSE · shared fused-layer scales · MTP head kept bf16.