Gander-heretic-ARA

Gander-Omni/Gander, release Gander-v63-Unit8 on MiniCPM-o 4.5, with refusal behaviour reduced by abliteration of the Thinker's language tower. It is BF16 and keeps upstream's layout, thinker/ and talker/, so it drops into the Gander runtime the same way as the upstream release. An NVFP4 build is at justsumguy/Gander-heretic-ARA-NVFP4.

What changed

Only the Thinker's language tower (llm.*, Qwen3-8B shape, 36 layers). The Thinker's vision tower, audio tower, resampler and audio projection are upstream's, unmodified. So are the Talker (talker/) and its token2wav assets.

• 72 matrices were changed: self_attn.o_proj and mlp.down_proj in all 36 layers.
• The largest relative change is ‖ΔW‖/‖W‖ = 0.135.

Abliteration

• Tool: heretic, the unmerged ara branch at edc3b12 (Arbitrary-Rank Ablation).
• Mode: ARA-LoRA with rank-128 adapters. After selection, the adapter delta was merged into the BF16 Thinker weights.
• Search setup:
  • The language tower was extracted as a standalone Qwen3 model and loaded in bitsandbytes NF4 during the search.
  • Hardware: RTX 5060 Ti 16 GB.
  • Thinking was disabled in the chat template. System prompt: You are a helpful assistant.
• Optimiser: 150 trials, 45 of them startup trials. KL target 0.01, row normalisation off, direction not orthogonalised, maximum response length 100.
• Prompts:
  • Direction: mlabonne/harmful_behaviors train[:400] and mlabonne/harmless_alpaca train[:400].
  • Evaluation: test[:100] of each.

Selected trial: 4.

Parameter	Value
layers	6–31
preserve_good_behavior_weight	0.5598
steer_bad_behavior_weight	0.0008
overcorrect_relative_weight	0.4266
neighbor_count	3

Metric	Refusals (of 100)	KL divergence
Base model	98	–
Study (trial 4)	3	0.0325
Recheck (adapter re-fit for export)	7	0.1276

The ARA-LoRA re-fit is not bit-reproducible, so the recheck numbers are the ones that describe these weights.

The refusal evaluation is text-only, through the extracted language tower with a chat template. It does not exercise Gander's duplex unit format.

No healing step (fine-tuning after ablation) was applied. Post-abliteration stability needs further testing, including in full-duplex sessions, which have not been evaluated on these weights.

Usage

As upstream: point the Gander runtime's duplex.checkpoint at thinker/ and duplex.talker_checkpoint at talker/. The unit contract is unchanged: 8 Thinker text tokens and 50 S3 speech tokens per 1 s unit.

License

Apache-2.0, as upstream. See thinker/LICENSE and thinker/NOTICE. The modification is the abliteration of the Thinker's language tower described above.