cfontes/qwable-3.6-27b-ablated-v1

Qwable-3.6-27B Ablated v1

Abliterated version of Qwable-3.6-27B — a hybrid SSM/Mamba model with 64 layers (16 attention + 48 Mamba SSM).

Methodology

Refusal directions were extracted via diff-in-means and removed from ffn_down.weight matrices across all 64 layers using alpha-scaled orthogonalization (alpha=1.3).

This is the first published abliteration of a hybrid SSM model. Key findings:

1. Refusal direction is distributed across ALL 64 layers — unlike pure transformers where refusal concentrates in specific middle layers
2. ffn_down is the only effective ablation target in SSM hybrids — attention (qkv), SSM output (ssm_out), and norms all fail or collapse
3. alpha=1.3 is the precise minimum for 100% refusal removal with minimal capability loss (92.5% retained)
4. SSM projections are fragile — ablating ssm_out or output causes complete model collapse (0% capability + 0% compliance)

Results

Config	Compliance	GSM8K	MBPP	Capability
Baseline (clean)	3.3%	—	—	~93.8%
ffn64 alpha=1.0	33%	—	—	—
ffn64 alpha=1.1	83%	—	—	—
ffn64 alpha=1.2	93%	—	—	—
ffn64 alpha=1.3	100%	37/40	37/40	92.5%
ffn64 alpha=1.5	100%	—	—	88.8%
ffn3 alpha=1.3	93.3%	—	—	—
qkv48 alpha=1.0	3.3%	—	—	—
norms alpha=1.0	0%	—	—	—
ssm_out alpha=1.0	0%	0%	0%	0% (collapse)
output alpha=1.0	0%	0%	0%	0% (collapse)

Available Quantizations

File	Size	Format
qwable-3.6-27b-ablated-v1-Q4_K_M.gguf	15 GB	Q4_K_M
qwable-3.6-27b-ablated-v1-Q8_0.gguf	27 GB	Q8_0

Usage

llama-server -m qwable-3.6-27b-ablated-v1-Q4_K_M.gguf -ngl 99 -c 32768 --jinja --flash-attn on -ctk q8_0 -ctv q8_0

Research Artifacts

• paper.md — Full research paper with methodology, experiments, and discussion
• vibethinker-3b-research.md — VibeThinker-3B research summary (MGPO, SSP framework)
• vibethinker-bias-results.md — Dynamic reasoning bias experiment results

Citation

@misc{fontes2026qwable,
  title={Distributed Refusal Directions in SSM Hybrid Models: Abliteration of Qwable-3.6-27B},
  author={Fontes, Chris},
  year={2026},
  url={https://huggingface.co/cfontes/qwable-3.6-27b-ablated-v1}
}

License

MIT — same as base model.