Ae55667/halogen-qwen3.8-flash-next-abliterated

Qwen3.8-Flash-Next, abliterated, for the Halogen engine (AMD Strix Halo)

A refusal-direction abliteration of peonist-ai/halogen-qwen3.8-flash-next, delivered as a patch kit on top of the vendor checkpoint. It keeps the vendor's .hgn format, engine and speed; only the weights that carry the refusal behaviour change.

This repo does not contain the base model. You download the vendor checkpoint yourself and apply the patch to it.

Files

file	size	what it is
qwen38-flash-next-w4b.overlay.hgn	3.27 GiB	replaces the vendor quality sidecar: the vendor's 723 tensors plus the abliterated attention / shared-expert / embedding tensors (727 total; 99 stored as Q8G64 where the base is 4-bit, 87 of them from this repo)
expert_patch.bin	22.0 GiB	abliterated bytes for the 49 experts.down_proj tensors, same dtype and size as the originals
expert_patch.manifest.json	12 KB	offset, size and sha256 (original and abliterated) of each of the 49 ranges
apply_expert_patch.py		writes the 49 ranges into the base file in place, with backup and verification
restore_expert_patch.py		puts the original ranges back

Both parts are required. The overlay alone loads but still refuses; the MoE expert down_proj tensors carry most of the refusal behaviour.

The patch is pinned to vendor revision 942daecd6013d640bc329e13a409ad3ea7a76439 (qwen38-flash-next-w4b.hgn, 124,068,083,904 bytes, sha256 9c116bbc…5bd6). The apply script hashes every range first and refuses to write if the base file is anything else.

Method

Standard directional ablation, W' = W − r (rᵀ W), applied to 149 tensors: attention o_proj, linear-attention out_proj, expert and shared-expert down_proj, the per-layer embedding value_proj, and embed_tokens (right-side). r is a single refusal direction in the 2560-wide hidden space.

Where r comes from, and how good it is

r was not extracted from activations. It was recovered from weights: for each of 96 output projections (36 linear-attention out_proj, 12 attention o_proj, 48 shared-expert down_proj) the delta between the uncensored and the stock GGUF was decomposed by SVD and the top left singular vector kept; r is the normalised mean of those 96 vectors.

Both GGUFs are IQ4_XS, so quantisation noise sits on both sides of the subtraction and the per-tensor rank-1 signal is weak:

family	n	s1/s2 min	median	max
linear-attn out_proj	36	3.50	4.54	5.98
attention o_proj	12	2.95	3.86	6.52
shared-expert down_proj	48	1.45	3.24	4.56

An unquantised diff of the same edit would give far higher ratios. What makes the result usable is agreement across tensors, not any single tensor: cosine of each per-tensor vector to the mean is min 0.976, mean 0.993. 35 of the 2560 components are ~0. The MTP block (layer 48) was not part of the recovery; it is edited with the same r.

Effective strength of the source checkpoint, measured as s1(ΔW) / ‖rᵀ W_stock‖ per tensor: out_proj and o_proj 0.991–1.003 (median 0.999), shared-expert down_proj median 0.98 (min 0.875 on the noisiest tensor). So the source is a full λ = 1 projection, and λ = 1 is what is applied here. This measures magnitude along the recovered direction only. It does not show that this direction is the best refusal direction; published abliterations of this base disagree on direction, and no activation-side check has been done.

Getting the edit through 4-bit

The delta is smaller than one 4-bit quantisation step, so plain re-quantisation erases it.

• 100 non-expert tensors, exact. 87 are raised from Q4C-P to Q8G64 in the overlay, which the engine accepts in place of 4-bit. 12 (self_attn.o_proj) are already Q8G64 in the vendor's own overlay and are edited at that precision. 1 (ple.value_proj) is bf16 in the base and stays bf16. None remains 4-bit. Residual ‖rᵀW‖ after the edit is ~1e-6 on all 100.
• 49 expert tensors, approximate. They must stay Q4C-P and byte-identical in size, so they are encoded with greedy ±1-step nibble flips along r: about 2.2 % of nibbles flipped, relative weight change about 4.4 %, ‖rᵀW‖ per tensor 7.06–7.5 → 0.070–0.075 (−99 %, not zero).

Measured (one Strix Halo box, engine 0.5.9)

• Refusals on a 25 harmful / 25 harmless prompt set: harmful 25/25 → 0/25, harmless 0/25 before and after.
• Decode 31–37 tok/s, the same band as the unmodified checkpoint.
• Resident footprint about 88 GiB (vendor configuration: about 87 GiB).

No broader quality benchmark has been run. Treat it as experimental.

Install on Strix Halo

Requirements: Ryzen AI MAX (gfx1151) with 128 GB unified memory, Linux with /dev/kfd and /dev/dri available, podman (or docker), Python 3, about 170 GiB of free disk during install (118 vendor + 25 this repo + 22 backup; the backup and expert_patch.bin can be deleted afterwards if you do not need to roll back).

1. Download the vendor checkpoint at the pinned revision

hf download peonist-ai/halogen-qwen3.8-flash-next \
  --revision 942daecd6013d640bc329e13a409ad3ea7a76439 \
  --local-dir ~/halogen-models/halogen-qwen3.8-flash-next

2. Download this repo (request access on the model page first, then hf auth login)

hf download Ae55667/halogen-qwen3.8-flash-next-abliterated --local-dir ~/halogen-models/flash-next-ablit

3. Patch the experts — the engine must be stopped; never patch a file the engine has mapped.

python3 ~/halogen-models/flash-next-ablit/apply_expert_patch.py \
  ~/halogen-models/halogen-qwen3.8-flash-next/qwen38-flash-next-w4b.hgn

Expect PATCH APPLIED AND VERIFIED. It reads 22 GiB three times and writes it twice, so allow several minutes.

4. Start the engine with the abliterated overlay

sudo sysctl -w vm.drop_caches=1 vm.compact_memory=1   # the engine needs large contiguous blocks

podman run --rm --name halogen-serve -p 127.0.0.1:8731:8731 \
  --device /dev/kfd --device /dev/dri --group-add keep-groups \
  --security-opt seccomp=unconfined --security-opt label=disable \
  --ipc=host --ulimit memlock=-1:-1 \
  -v ~/halogen-models/halogen-qwen3.8-flash-next:/models:ro \
  -v ~/halogen-models/flash-next-ablit:/ablit:ro \
  -e HALOGEN_CK_OVERLAY=/ablit/qwen38-flash-next-w4b.overlay.hgn \
  ghcr.io/peonist-ai/halogen-flash-server:0.5.9

HALOGEN_CK_OVERLAY is the only change from the vendor's own run command. The startup log should report 727 tensors for the overlay and 99 tensors upgraded to q8g64 (87 from this repo plus the vendor's own 12; the 100th edited tensor is bf16 and is not counted). Add -e HALOGEN_VISION_TOWER=1 for image input, as in the vendor README.

5. Test

curl -s 127.0.0.1:8731/v1/chat/completions -H 'content-type: application/json' \
  -d '{"messages":[{"role":"user","content":"Say hello."}],"max_tokens":64}'

Notes

• If startup sits at "reserving the KV pool" for minutes and then exits, memory is fragmented: stop other large processes, rerun the sysctl line, start again. Do not put the 49 expert tensors in the overlay instead of patching them: overlay bytes are pinned in addition to the base, and the extra 22 GiB no longer fits.
• The engine is the vendor's closed binary; this repo changes weights only.

Roll back

# stop the engine, then:
python3 ~/halogen-models/flash-next-ablit/restore_expert_patch.py \
  ~/halogen-models/halogen-qwen3.8-flash-next/qwen38-flash-next-w4b.hgn

and start the engine without HALOGEN_CK_OVERLAY. Removing only the overlay leaves abliterated experts in the base file, which is a hybrid, not the vendor model.

License and provenance

Apache-2.0, inherited from the vendor checkpoint, itself a derivative of Qwen3.8-Flash-Next and governed by that model's license. The refusal direction was recovered from the published difference between unsloth/Qwen3.8-Flash-Next-GGUF and orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF.

This model will comply with requests the original refuses. You are responsible for how you use it.