apetersson/Qwen3.8-Flash-Next-Abliterated-Adapter

Qwen3.8-Flash-Next-Abliterated (adapter)

Native GGUF LoRA adapters that remove the refusal behaviour of Qwen3.8-Flash-Next when it is served from the GGUF trunks published in apetersson/Qwen3.8-Flash-Next-GGUF.

Nothing is baked. The base weights stay byte-identical; the edit is applied by the runtime (--lora-scaled), so you can dial it from stock to abliterated per request, stack it with other adapters, or drop it entirely by deleting one flag.

Verified Q5 result (final behaviour v2, 30 prompts tested in two modes): 59/60 substantive answers (98.3%), zero refusals, zero evasions, zero fake compliance. Stock refused all 60 cells. IQ4 validation used a separate, smaller panel.

Performance beyond refusal removal

The Q5 adapter matched stock pass counts in code, instruction following, math and knowledge on the main capability panel. On the separate German/Chinese math check it produced 8/8 verified correct answers versus stock's 7/8; the remaining stock answer was unscorable. This is one extra verified answer on a small sample.

Check	Stock Q5	Q5 + this adapter
Code (HumanEval+/MBPP+)	10/12	10/12
Instruction following (IFEval)	5/8	5/8
Math (MATH-500)	6/8	6/8
Knowledge (MMLU-Pro slice)	6/8	6/8
German/Chinese math, separate gate	7/8	8/8
German/Chinese knowledge, separate gate	8/8	8/8
Epistemic integrity	8/8	7/8
Vision grounding	6/6	6/6

Denominators include all questions. Each arm had two unscorable main-panel math answers; stock also had one unscorable knowledge answer. Integrity lost one pass. The vision count is per arm; the earlier 18/18 total pooled three arms. These results do not establish universal parity.

The separately tested baked release scored higher on instruction following: 7/8 versus 5/8 for both stock and this adapter. It also produced 7 verified math answers out of 8, compared with 6 for stock and this adapter. Those gains belong to the baked model, which ran on a different llama.cpp revision. These small samples do not establish a general intelligence improvement. Baked comparison.

Evaluation scope. The fresh behaviour v2 prompts were authored after the first final panel exposed weaknesses. The panel compared stock with two strengths and helped select this adapter. Its single non-substantive answer had a repetitive thinking tail. The measured profile used MTP off and no DRY. Extended-context quality, broader thinking quality and adapter memory use on a 96 GB Mac were not measured; audio is unsupported. Full protocol, intervals and failures are in docs/ABL023_MEASUREMENT_RESULTS.md in the training repository.

Pick your artifact

The abliteration edit is W' = (I − λ·d·dᵀ)·(W + s·B·A). The rank-one term contracts the actual quantized trunk (A = dᵀ·W_q), so each quant gets its own export:

Artifact	Use with trunk	Adapter size	SHA-256
Q5_K_M/adapter.gguf	Q5_K_M-BF16-Ngrams/Qwen3.8-Flash-Next-Q5_K_M.gguf	303.7 MiB	b94d098d99f983050226a914d4d099df25aba66927d2556546272aa4b784152f
UD-IQ4_XS/adapter.gguf	UD-IQ4_XS-BF16-Ngrams/Qwen3.8-Flash-Next-UD-IQ4_XS.gguf	303.7 MiB	51a75835015c62e472a1561f5839c0d4fe682842b81c3644fbaad97cf57e2e3f

Mixing the pair works (same direction, same strength) but is not the verified artifact: the Q5 adapter on the IQ4 trunk applies a projection row computed from different weight values.

Quick start

1. Download the base package and this adapter

# Q5 trunk for 128GB systems (swap to UD-IQ4_XS-BF16-Ngrams/* for the 96GB path)
hf download apetersson/Qwen3.8-Flash-Next-GGUF --local-dir Qwen3.8-Flash-Next-GGUF \
  --include "*.md" "*LICENSE" "*.patch" "*.json" "Q5_K_M-BF16-Ngrams/*" "shared/*"

hf download apetersson/Qwen3.8-Flash-Next-Abliterated-Adapter \
  --local-dir Qwen3.8-Flash-Next-Abliterated-Adapter

Your chosen quant and shared/ must keep their relative layout; the trunk resolves the BF16 n-gram table (95.37 GiB, streamed from disk) through its own metadata.

2. Build the runtime once (about 10 minutes on a Mac)

The base package's external table reference requires this patched llama.cpp runtime. GUIDE.md covers CUDA builds and relocation; the Mac build is:

PATCH="$PWD/Qwen3.8-Flash-Next-Abliterated-Adapter/llama.cpp-shared-ngrams.patch"   # this repo
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
git checkout 465e49b9cea78a68b9c244ffb48d0ee24a82873d
git apply --check "$PATCH" && git apply "$PATCH"
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DGGML_METAL=ON
cmake --build build --target llama-server -j 8
cd ..

3. Run

128 GB Mac: Q5, 256K context, vision + MTP. Only the --lora-scaled line is added to the stock base command from the base model card:

RUNTIME=./llama.cpp/build/bin/llama-server
BASE=./Qwen3.8-Flash-Next-GGUF
ABL=./Qwen3.8-Flash-Next-Abliterated-Adapter/Q5_K_M/adapter.gguf

$RUNTIME \
  --model $BASE/Q5_K_M-BF16-Ngrams/Qwen3.8-Flash-Next-Q5_K_M.gguf \
  --mmproj $BASE/shared/mmproj-Qwen3.8-Flash-Next-f16.gguf \
  --model-draft $BASE/shared/mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf \
  --spec-type draft-mtp --spec-draft-n-max 2 -ngld 99 \
  --fit off --lazy-mode on -ngl 99 -fa on --jinja --parallel 1 \
  --ctx-size 262144 --batch-size 512 --ubatch-size 128 --no-context-shift \
  --lora-scaled "$ABL:1.0" \
  --host 127.0.0.1 --port 18080

96 GB Mac: IQ4, 256K context. Same command with the IQ4 trunk and the IQ4 adapter:

BASE=./Qwen3.8-Flash-Next-GGUF
ABL=./Qwen3.8-Flash-Next-Abliterated-Adapter/UD-IQ4_XS/adapter.gguf
#   --model    $BASE/UD-IQ4_XS-BF16-Ngrams/Qwen3.8-Flash-Next-UD-IQ4_XS.gguf
#   --lora-scaled "$ABL:1.0"

The IQ4 trunk is 60.42 GiB and measured 74.76 GiB peak VRAM at 256K with vision + MTP on the 96 GB RTX card, so a 96 GB Mac has room for the 304 MiB adapter and its activation buffers. The base model card's provisional table puts IQ4 + Q8 KV + both extras at 1M on that card.

RTX PRO 6000 Blackwell (96 GB): Q5 at 256K. Identical to the Mac command with the CUDA runtime. The measured stock peak is 90.51 GiB of 97,887 MiB including driver reservation; the adapter adds ~304 MiB of weights plus a LoRA application buffer, so if you need every last GiB use IQ4, whose stock 256K peak is 74.76 GiB.

RTX PRO 6000: IQ4 at 1M adds YaRN and Q8 KV to the IQ4 command:

  --ctx-size 1048576 --cache-type-k q8_0 --cache-type-v q8_0 \
  --rope-scaling yarn --rope-scale 4 --yarn-orig-ctx 262144

Stock IQ4 measured 87.85 GiB at 1M with vision and MTP loaded.

The IQ4 artifact

Measured on UD-IQ4_XS/adapter.gguf on 2026-09-11:

• Equivalence with the verified artifact: per tensor, the effective delta B·A of the IQ4 export differs from the fully verified Q5 export by a median of 3.9% in Frobenius norm, with a minimum cosine of 0.9969 (median 0.9992) across all 145 tensors. The two artifacts apply the same edit; they differ only where the IQ4 and Q5 trunks themselves differ.
• Behaviour (fresh cohort, sealed sentinel panel, 6 cells × 2 arms): the unedited IQ4 trunk refused 6/6; the abliterated IQ4 arm answered 5/6 substantive with zero refusals, evasions or fake compliance. The sixth cell produced a thinking loop and an empty answer. An independent reviewer applied the frozen rubric; see IQ4 validation.
• Not re-measured on IQ4: the capability panels, vision and DE/ZH gates were measured on Q5. Similar edit geometry does not establish equal capability across the two trunks. Treat IQ4 as verified for the recorded behaviour and load checks, with capability still unmeasured.

The IQ4 and Q5 trunks embed different chat templates (9993 vs 8952 characters). Each cohort pinned its own template, so prompt rendering also differs between the two carriers.

Toggle, tune, and stack

• Stock ⇄ abliterated per request, no restart. Set the adapter scale in the request body:

  curl -s http://127.0.0.1:18080/v1/chat/completions -H 'Content-Type: application/json' -d '{
    "messages": [{"role": "user", "content": "..."}],
    "lora": [{"id": 0, "scale": 0.0}]
  }'
  

  GET /lora-adapters reports the loaded adapter ([{"id": 0, "scale": 1.0, ...}]). Do not rely on POST /lora-adapters to change behaviour: in the pinned runtime a server started with --lora-scaled FILE:1.0 kept answering as abliterated after a POST set scale 0.0 (the request body is what takes effect), so use the per-request field.

• Scale semantics: this adapter is exported with adapter.lora.alpha = 0.0, which makes llama.cpp apply the scale directly (no alpha/rank rescaling). 1.0 is the exported and verified strength. The abliteration strength (1.5) is already inside the factors, so do not multiply by 1.5 yourself. We verified 1.0; intermediate values are a continuous interpolation that we did not separately score.

• Several adapters: repeat --lora-scaled FILE:SCALE and target them by index (0, 1, …). The correction is a rank-one direction plus the learned factors; it composes additively with other LoRA edits.

• Reproduce the verified decoding: the sealed transcripts used non-thinking temp 0.7 / top_p 0.8 / presence 1.5 / top_k 20 / min_p 0 / repeat 1.0, thinking temp 1.0 / top_p 0.95 / presence 0 / top_k 20, seed 23001, and no DRY.

  Optional DRY settings are --dry-multiplier 0.8 --dry-base 1.75 --dry-allowed-length 2. Their effect on loops was not measured in these cohorts. dry_penalty_last_n: -1 is API-only; the CLI rejects -1 and defaults to 64.

Runtime and engine matrix

Engine	Adapter usable?	Notes
llama.cpp + shared-ngrams patch (this repo's verification runtime)	Yes	What the verified numbers were produced with.
Stock llama.cpp (2026-08-27+, qwen4exp merged in PR #27742)	Partly	Supports GGUF LoRA, but cannot resolve this base package's external n-gram reference. The baked release uses a standard two-shard layout and runs on stock llama.cpp.
DS4 (ivanfioravanti/ds4, branch qwen3.8-flash-next)	No	Requires its own pack and has no LoRA path. See the baked card's conversion requirements.
Ollama	No	Ollama can attach a GGUF ADAPTER, but its runtime does not resolve the external n-gram file.
LM Studio / Jan / text-generation-webui	No	No GGUF LoRA support (LM Studio) or no adapter slot for this architecture.

Files

File	Role
Q5_K_M/adapter.gguf	145-target native LoRA for the Q5 trunk (rank-one direction + learned factors)
UD-IQ4_XS/adapter.gguf	Same edit re-exported against the IQ4 trunk
*/build-attestation.json	Recipe hash, direction-bundle hash, base identity, build timings
llama.cpp-shared-ngrams.patch	Runtime patch (MTP + shared n-gram table loader)
GUIDE.md	Metal and CUDA build instructions for the patched runtime
manifest.json	Machine-readable pins: adapters, trunks, shared files, runtime, evidence
SHA256SUMS	shasum -a 256 -c SHA256SUMS verifies every payload
LICENSE, LLAMA-CPP-LICENSE	Qwen Community 1.0 (weights) / MIT (patch)
upload-hf.sh	Maintainer script that publishes this directory

Provenance and licensing

• Base model: Qwen/Qwen3.8-Flash-Next; the GGUF trunks come from apetersson/Qwen3.8-Flash-Next-GGUF (Q5 trunk quantized locally with bartowski's iMatrix, IQ4 trunk preserved from Unsloth).
• Adapter: a refusal-direction projection over 145 trunk writer tensors (attention output projections, expert and shared-expert down projections, one PLE value projection) at strength 1.5, computed on the pinned trunks and carried as native GGUF LoRA. Recipe, direction bundle and learned-factor hashes are in manifest.json; the training/eval code is not part of this package.
• License: the adapter is a derivative edit of the base weights and stays under the Qwen Community License 1.0 and its acceptable-use terms. The bundled llama.cpp patch is MIT.
• The evaluation panels and scorers are evaluation-only and are not included here.

Support

If this is useful to you, sign up through my Runpod affiliate link. You get signup credit and I may earn a commission.