jiguack/qwen35-9b-antidoom-qlora

Qwen3.5-9B — anti-repetition-loop QLoRA

FTPO-trained LoRA adapters for Qwen/Qwen3.5-9B. The base model falls into repetition ("doom") loops that keep it from finishing math answers. Contrary to this repo's original framing, that is not caused by KV-cache compression: an uncompressed upstream llama.cpp with f16 KV loops at exactly the same rate (48.4% either way, p=1.00 paired). Looping is intrinsic model behaviour, and these adapters reduce it.

All numbers below were re-measured 2026-08-31 on a decontaminated eval set. The 100-problem set this card previously reported was 70% contaminated — 70 of its problems had supplied the training pairs. Figures here use the 153 MATH-500 problems that contributed no training data, every comparison paired with an exact McNemar test.

Two claims from the earlier card are withdrawn. It said this training is "neutral-to-harmful" at cache budget 4096 and should be used only below ~3.5k — the effect is in fact identical at both budgets (+11.1 points at 3,072 and +11.1 at 4,096). And the 61 → 75 headline was a contaminated-set number; the clean equivalent is 54.2% → 68.0%.

Against the alternative you'd actually try first

No compression, no training, one server flag. All 153 clean problems, 8k output cap:

	Accuracy	Loop	Sec / correct	KV cache
Uncompressed f16 KV, untrained	55.6%	48.4%	88.0	512 MiB
Uncompressed f16 KV + --repeat-penalty 1.10	63.4%	12.4%	70.2	512 MiB
b3072 q4_0 + adapter + penalty	69.9%	7.2%	59.5	34 MiB
	+6.5 (p=0.031)		−15%	15.2× less

Accuracy, speed and memory improve together — this is not a trade of one for another. The KV figure is arithmetic on the served model: Qwen3.5-9B has 8 full-attention layers of 32 (24 are linear attention), 4 KV heads, head_dim 256 → 16,384 KV elements per token, so 512 MiB at -c 16384 in f16 against a 3,840-cell q4_0 budget arena.

The penalty and the adapter are complementary rather than redundant. The penalty suppresses loops harder (12.4% vs the adapter's 26.8% alone) while the adapter converts loops into correct answers better (fixed/broken 26/5 against the penalty's 21/10). Adding the penalty on top of the adapter does nothing for accuracy (p=0.581); adding the adapter on top of the penalty gains a lot (3 vs 16, p=0.004). Run both.

Give it room and it does much better

The 8k cap was the binding constraint, not the model's reasoning. At a 32k cap with cache budget 8192, same 153 problems:

	Accuracy	Loop	Avg tokens	Sec / correct
Untrained	72.5%	57.5%	15,671	185.5
run9 adapter	86.9%	34.0%	8,401	82.2

+14.4 points (3 problems untrained-only against 25 adapter-only, p=2.7e-05), 46 problems stop looping against 10 starting (p=1.3e-06), output halves and cost per correct answer falls 56%. Untrained, this regime costs 185 s per correct answer and is not economical; with the adapter it is.

Choosing between caps is a real trade: 32k + adapter buys +19.6 points over 8k + adapter, at 40% more cost per correct answer (82.2 s against 59.5 s). Pick by whether accuracy or throughput binds.

The failure mode the adapter removes is invisible to accuracy. In the untrained 32k run, 18 of 153 problems produced the correct answer and then stayed trapped in long-period self-recheck until the 32k cap; one re-entered a 228-word cycle 71 times. Accuracy scores those as wins; tokens and wall time pay for them.

AIME (69 clean problems, 32k cap, budget 3072 — a harsher 10:1 ratio): 20.0% → 30.0%, McNemar 0 vs 7, p=0.0156 — seven problems fixed, none broken.

Which adapter

All three trained on the same 5,361 FTPO pairs, differing in how those pairs were consumed.

	Recipe	Training cost	8k cap (153)	32k cap
run9	3-stage sequential chain: short loops → jumps-back → self-recheck, each on the previous merge	3 runs + 2 merges	68.0%, loop 19.6%	86.9% (153 problems)
combined-v2	All pairs in one pass, early stop at chosen_win 0.6 (35 steps)	1 run	68.0%, loop 26.8%	86.7% (30 problems)
combined-v2b	Same, early stop at 0.8 (130 steps)	1 run	67.3%, loop 19.6%	83.3% (30 problems)

At an 8k cap combined-v2b is indistinguishable from run9 — accuracy p=1.0000, loop p=1.0000 — at a third the training cost. On all 500 MATH-500 problems the accuracy difference between the chain and single-pass training is not significant in any split (p=0.37 overall, 0.63 clean, 0.54 contaminated). The chain's apparent edge was cumulative training length, not the staged structure: measured against run9, a 30-step single-pass run loses on loops (p=0.0029), 35 steps is borderline (p=0.0522), and 130 steps closes it entirely (p=1.0000).

Recommendation: run9. It has the only 153-problem long-cap measurement and leads there. combined-v2/v2b are published because at an 8k cap they match it and cost a third as much to reproduce — if you are retraining rather than downloading, use the combined recipe. Their 32k figures rest on 30 problems and that subset is easier than the set as a whole (untrained scores 80.0% on it against 72.5% over 153), so treat them as provisional; the 153-problem runs are in progress.

More training is not simply better at a long cap. Across 0 → 35 → 130 steps, accuracy on the 30-problem subset went 80.0 → 86.7 → 83.3 while tokens fell monotonically. Training teaches the model to finish early, and past some point that may truncate reasoning it needed — though this rests on 30 problems and the 153-problem runs are still going.

An earlier version of this card cited prealgebra/105 as a concrete casualty of that effect. That was a scoring bug, and the truth is the reverse. The gold answer is \text{east}; the eval harness deletes \text{} content when normalising, so the gold becomes the empty string. The untrained run never terminated, emitted an empty prediction, and matched. The adapters emitted East and were marked wrong. The adapters solved it; the untrained model did not. The bug hits any text-answer problem (4 of the 153) and systematically favours non-terminating runs.

Serve it

Requires UniKV built with TriAttention support.

Before anything else, turn on answer forcing if your build has it. UniKV's --reasoning-budget-answer injects \n\nThe final answer is $\boxed{ after the closing think tag, so the model has only the value left to fill in. On AIME (49 problems, 32k cap) against the untrained base it fixed 8 problems and broke 0 — every one of the 8 was a run that had hit the cap without ever emitting \boxed, and all 27 problems that were already right stayed right. Accuracy 55.1% → 71.4%.

That zero-loss property is what makes it the first thing to reach for. The other two interventions both trade: the adapter fixes 26 and breaks 5 (5.2:1), the repeat penalty fixes 21 and breaks 10 (2.1:1). Forcing breaks nothing because it only supplies the slot the answer goes in.

It is not a cure-all: it rescued 36% of the runs that failed to terminate (8 of 22). Of the rest, most committed a near-miss at budget exhaustion (896→3896, 393→392, 610→600) — those were already scoring zero, so nothing was lost, but forcing does commit half-formed answers.

Needs the reproduce/antidoom-with-answer-patch UniKV branch; plain v5_singleturn does not have the flag. Whether the adapter adds anything on top of forcing is being measured now — the 55.1% → 71.4% figure above is the untrained base, so treat the adapter and forcing as separately evidenced until that lands.

Run the penalty at either cap. It was unclear whether it would help at a 32k cap — its failure mode is that it stops the model stating a conclusion, and a long cap gives it more room to wander — but measured on the 153 clean problems it costs no accuracy there (86.9% → 85.0%, 8 problems each way against 5, p=0.581) while halving the loop rate (34.0% → 16.3%, 31 vs 4, p=3.5e-06), and cutting output (8,401 → 7,450 tokens) and length-capped runs (8.5% → 5.2%).

Long cap, best accuracy:

llama-server -m qwen35-9b-qlora-Q4_K_M.gguf -c 40960 -ngl 99 --parallel 1 \
  -ctk q4_0 -ctv q4_0 --triattention <calib.bin> \
  --tri-budget-tokens 8192 --tri-window 4096 --tri-interval 512 \
  --tri-keep-first 512 --tri-max-prompt 2048 \
  --repeat-penalty 1.10 --repeat-last-n 256

Short cap, best throughput:

llama-server -m qwen35-9b-qlora-Q4_K_M.gguf -c 16384 -ngl 99 --parallel 1 \
  -ctk q4_0 -ctv q4_0 --triattention <calib.bin> \
  --tri-budget-tokens 3072 --tri-window 256 --tri-interval 128 \
  --tri-keep-first 512 --tri-max-prompt 2048 \
  --repeat-penalty 1.10 --repeat-last-n 256

Scope

Every measurement here uses prompts of 50–150 tokens, which sit below fastkv_min_prompt_threshold()'s 1024-token default — so FastKV declined prompt eviction throughout and compression acted only on generated tokens. "Compression is free" is a short-prompt result. Long-prompt workloads (summarisation, multi-turn) are not covered by anything here.

Files

• qwen35-9b-qlora-Q4_K_M.gguf — the run9 merged model (recommended)
• combined-v2/, combined-v2b/ — Q4_K_M gguf for the single-pass recipes
• lora/ — the run9 LoRA adapter (r=128, targets q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj,lm_head; larger than a typical LoRA because lm_head is included and Qwen3.5's vocabulary is ~240k)

Reproduce it

Pairs and configs are in ETL-EDGE-AI/anti-loop. The three pair sets are committed; the combined file is their concatenation:

cat qlora/data/train_run{2,7_longloop,9_selfrecheck}.jsonl \
  > qlora/data/train_combined.jsonl
antidoom -c qlora/train_combined_v2_threshold.yaml -r qlora/run_combined_v2 \
  train --dataset-jsonl qlora/data/train_combined.jsonl --model-name "$BASE"

Run-to-run variance is about 6 accuracy points: two runs of the same config with the same seed, differing only by GPU non-determinism, scored 66 and 72 on a 100-problem eval. Treat any single retraining result accordingly.

Full writeup, failure taxonomy, withdrawn claims and negative results: FINDINGS.md.