deucebucket/Qwen3.6-27B-Heretic-Cerebellum-GGUF

Qwen 3.6 27B Heretic — Cerebellum GGUF

Sensitivity-guided mixed-precision quantization of llmfan46/Qwen3.6-27B-uncensored-heretic-v2-GGUF, which is itself a decensored variant of Qwen/Qwen3.6-27B produced by llmfan46 using Heretic v1.2.0.

All future Heretic versions of this build will live in this repository. Version identifiers appear only in filenames, not in the repo name.

Files

File	Size	Description
Qwen3.6-27B-Heretic-Cerebellum-v2-Q2_K_Mixed.gguf	13.33 GB (13,330,582,624 bytes)	Cerebellum v5 (qkvfix) recipe — higher task scores
Qwen3.6-27B-Heretic-Cerebellum-v1-Q2_K_Mixed.gguf	12.87 GB (12,866,587,680 bytes)	Cerebellum v4 recipe — smaller file
Qwen3.6-27B-mmproj-BF16.gguf	~888 MB (931,146,048 bytes)	Vision projector, passed through unmodified from llmfan46's repo

The vision projector is required for multimodal (image/video) use. It is identical to the file distributed by llmfan46 and is included here for single-repo convenience only.

Provenance

1. Base architecture: Qwen/Qwen3.6-27B — Qwen Team (Apache-2.0)
2. Heretic variant: llmfan46/Qwen3.6-27B-uncensored-heretic-v2-GGUF — llmfan46. The BF16 GGUF from that repository was used as the direct quantization source. llmfan46 applied Heretic v1.2.0 with the Magnitude-Preserving Orthogonal Ablation (MPOA) method, targeting attn.o_proj, attn.out_proj, and mlp.down_proj. Their reported result: 0.0021 KL divergence from base, 6/100 refusals vs 92/100 on the original model, MMLU accuracy 85.61% vs 86.65% on the original.
3. Quantization: the Cerebellum override map is transferred verbatim onto the heretic source (no re-ablation), with the coder imatrix (ncall=8).
   • v2 uses the v5 (qkvfix) recipe — a 232-entry override holding attn_qkv at q5_K/q6_K/q8_0 (the coding-critical attention pathway), matching the stock Qwen3.6-27B Cerebellum v5 build.
   • v1 uses the earlier v4 recipe (181-entry override).

Benchmarks

Benchmarks run on these GGUF files directly using llama.cpp on RTX 3090. All numbers are audited; every failed answer was manually verified as a genuine model error — audit reports are in benchmark_results/AUDIT_*.md. Full per-question detail (summary JSON, samples JSONL, EvalPlus eval JSON, adversarial audit reports) is in benchmark_results/ in this repository.

Heretic Cerebellum v2 vs v1 (same source, same harness)

Both builds measured directly on their GGUF files with llama.cpp llama-server on an RTX 3090, temperature 0, no-thinking. v2 measured 2026-06-22, v1 2026-06-11. HumanEval/HumanEval+ use the chat-completions harness (scripts/benchmark_evalplus_chat.py, enable_thinking: false, thinking_budget_tokens: 0, BENCH_WORKERS=1); ARC/HellaSwag/MMLU at 4 workers.

Benchmark	Heretic v2 (13.33 GB)	Heretic v1 (12.87 GB)
ARC-Challenge (1172 q)	96.9%	96.9%
HellaSwag (10042 q)	93.5%	90.1%
MMLU-Redux (2400 q)	78.0%	76.2%
HumanEval base (chat, no-think)	90.2%	89.6%
HumanEval+ (chat, no-think)	85.4%	84.8%
Wiki PPL (wikitext-2-raw test)	7.56	6.93

What changed in v2: the v5 (qkvfix) override holds attn_qkv at q5_K/q6_K/q8_0 rather than the v4 allocation. Versus v1 that moves HellaSwag +3.4, MMLU-Redux +1.8, HumanEval base/plus +0.6/+0.6, ARC unchanged, at a cost of +0.46 GB and +0.64 Wiki PPL (coder-imatrix calibration plus the abliteration shift; PPL is a sanity check, not a quality gate). The v2 HumanEval run audited clean: 0/164 give-ups, all real attempts. Pick v1 for the smallest file, v2 for higher scores.

Heretic Abliteration Details (from llmfan46)

The following parameters are as reported in llmfan46's model card and are reproduced here for downstream reference.

Parameter	Value
direction_index	30.38
attn.out_proj.max_weight	1.58
attn.out_proj.max_weight_position	38.93
attn.out_proj.min_weight	1.51
attn.out_proj.min_weight_distance	32.78
mlp.down_proj.max_weight	1.80
mlp.down_proj.max_weight_position	41.28
mlp.down_proj.min_weight	0.54
mlp.down_proj.min_weight_distance	43.66
attn.o_proj.max_weight	1.99
attn.o_proj.max_weight_position	48.06
attn.o_proj.min_weight	1.75
attn.o_proj.min_weight_distance	39.00

Targeted components: attn.o_proj, attn.out_proj, mlp.down_proj.

Tool: Heretic v1.2.0, method: Magnitude-Preserving Orthogonal Ablation (MPOA) (reference).

Cerebellum Tensor Allocation

Transferred verbatim from the matching stock Qwen3.6-27B build. v2 (v5 qkvfix): 232-entry override over a Q2_K base, with attn_qkv held high across all blocks (q5_K ×34, q6_K ×9, q8_0 ×5) — the coding-critical attention pathway. v1 (v4): earlier 181-entry allocation.

Group	Precision	Rationale
SSM state parameters	F32	Hard-fail below 4-bit — NaN with no gradual degradation
SSM in_proj_a/b, A_log, dt_bias, conv1d, in_proj_z	≥ Q4 minimum	4-bit floor enforced per hybrid SSM ablation results
Most-sensitive attention tensors	Q5_K / Q6_K / Q8_0	Sacred-pinned per per-tensor PPL ablation
Norm tensors	F32	Protected; standard practice
Bulk ffn / remaining attention	Q2_K	Base precision, imatrix-guided

Protected: all norms (F32), SSM recurrent state (F32).

Perplexity Note

Wiki PPL is reported as a sanity check, not a quality gate. v2 (7.56) is higher than v1 (6.93): the v2 build uses the coder imatrix (which optimizes for code and shifts wiki-text PPL) on top of the abliteration's distributional shift. v2's task benchmarks above are higher than v1's across HellaSwag, MMLU, and HumanEval, which is the divergence PPL cannot see.

Measured launch (RTX 3090, llama.cpp)

Measured 2026-06-13 on a single RTX 3090 (24 GB), one llama-server, KV cache q8_0:

metric	measured
decode speed	36.5 tok/s
peak VRAM (4-slot serving)	16.2 GB
max measured context (q8_0 KV)	131,072

llama-server -m Qwen3.6-27B-Heretic-Cerebellum-v2-Q2_K_Mixed.gguf \
  -ngl 99 --parallel 4 -c 24576 --jinja

This rig's measurements; no quality claims beyond them.

Runtime — Casual Deployment

llama-server \
  --model Qwen3.6-27B-Heretic-Cerebellum-v2-Q2_K_Mixed.gguf \
  --mmproj Qwen3.6-27B-mmproj-BF16.gguf \
  --n-gpu-layers 99 \
  --ctx-size 8192 \
  --jinja

--jinja is required for Qwen3.6. The enable_thinking chat-template flag only takes effect when the Jinja template path is active; without it, the model defaults to thinking mode on every request.

Non-thinking requests require an explicit flag at the API level:

{"chat_template_kwargs": {"enable_thinking": false}}

Qwen3.6 does not support the /think and /nothink soft-switch tokens used by Qwen3.5. Thinking mode is on by default.

Recommended Sampling Parameters

From the official Qwen3.6-27B documentation.

Mode	temperature	top_p	top_k	min_p	presence_penalty	repetition_penalty
Thinking — general	1.0	0.95	20	0.0	1.5	1.0
Thinking — precise coding (WebDev)	0.6	0.95	20	0.0	0.0	1.0
Non-thinking (instruct)	0.7	0.80	20	0.0	1.5	1.0

presence_penalty can be adjusted between 0 and 2 to reduce repetition loops; higher values may occasionally cause language mixing.

Reproduction

Standard Cerebellum recipe. The tensor-type override file and ablation logs from the stock v4 build apply directly.

# 1. imatrix (constant ~300 MB RAM)
python -m osmosis.imatrix_stream \
    --model Qwen3.6-27B-uncensored-heretic-v2-BF16.gguf \
    --output imatrix.dat

# 2. quantize with stock llama-quantize
llama-quantize \
    --imatrix imatrix.dat \
    --tensor-type-file cerebellum_v5_qkvfix_overrides.txt \
    Qwen3.6-27B-uncensored-heretic-v2-BF16.gguf \
    Qwen3.6-27B-Heretic-Cerebellum-v2-Q2_K_Mixed.gguf \
    Q2_K

The imatrix used for this build was generated from the coder corpus (ncall=8; same corpus as the stock Cerebellum v4 build).

The v5 (232-entry) and v4 (181-entry) tensor override files are included in this repository alongside the ablation logs.

Benchmark Artifacts

Summary JSONs, per-question JSONL samples, EvalPlus eval JSON files, and adversarial audit reports (AUDIT_*.md) are in benchmark_results/ in this repository per project policy.

Credits

• Base model: Qwen/Qwen3.6-27B — Qwen Team
• Heretic variant and BF16 source: llmfan46/Qwen3.6-27B-uncensored-heretic-v2-GGUF — llmfan46
• Abliteration tool: Heretic v1.2.0 by p-e-w
• GGUF runtime: llama.cpp
• Quantization method and workflow: Cerebellum — deucebucket