LocalLLaMA/Qwen3.5-0.8B-anti-loop

Qwen3.5-0.8B · anti-loop steering vector

With thinking on, Qwen3.5-0.8B often repeats itself until it runs out of tokens and never answers. This vector largely stops that at inference time, with no change to the model file: it removes one direction from the residual stream at layers 15 and 19. It works with MLX (Apple silicon) and PyTorch / transformers (CUDA, MPS, CPU) through PTS.

Same problems, thinking on	Unsteered	With this vector
MATH-500 correct (100 problems)	18	46
IFEval correct (100 prompts)	45	58
Everyday prompts answered (32 prompts)	25	32
GSM8K and MATH practice correct (96 problems)	8	36

▶ Try it in your browser: compare the model's answers with and without the vector, no install needed.

Credit

The direction comes from Sparks of Weight-Space Programming by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab (MIT CSAIL). In their work GPT-6 Astra edited this model's weights to stop the loops. We recovered the direction from their published edit (logs, CC BY 4.0) and turned it into a runtime steering vector. PTS did not discover this direction; here PTS is the format and the tool that applies it.

Quick start

pip install 'pts[mlx] @ git+https://github.com/codelion/pts'   # Apple silicon (MLX)
pip install 'pts @ git+https://github.com/codelion/pts'        # PyTorch (CUDA, MPS, CPU)

Compare unsteered and steered answers to the same prompt:

# MLX
pts steer --model Qwen/Qwen3.5-0.8B --vector hf://LocalLLaMA/Qwen3.5-0.8B-anti-loop/decrease --chat --compare \
    --prompt "What is the capital of Australia, and why isn't it Sydney? Answer in two sentences." \
    --chat-template-args '{"enable_thinking": true}' --temperature 0.6 --top-p 0.95 --top-k 20 \
    --presence-penalty 1.5 --presence-context-size 100000 --max-tokens 6000

# PyTorch
pts steer --backend torch --model Qwen/Qwen3.5-0.8B --vector hf://LocalLLaMA/Qwen3.5-0.8B-anti-loop/decrease --chat --compare \
    --prompt "What is the capital of Australia, and why isn't it Sydney? Answer in two sentences." \
    --chat-template-args '{"enable_thinking": true}' --temperature 0.6 --top-p 0.95 --top-k 20 \
    --presence-penalty 1.5 --presence-context-size 100000 --max-tokens 6000

Serve it (OpenAI-compatible)

# MLX
pts serve --model Qwen/Qwen3.5-0.8B --vector anti-loop=hf://LocalLLaMA/Qwen3.5-0.8B-anti-loop/decrease --port 8080

# PyTorch
pts serve --backend torch --model Qwen/Qwen3.5-0.8B --vector anti-loop=hf://LocalLLaMA/Qwen3.5-0.8B-anti-loop/decrease --port 8080

curl http://localhost:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "messages": [{"role": "user", "content": "What is the capital of Australia, and why isn'\''t it Sydney? Answer in two sentences."}],
  "steering": [{"vector": "anti-loop"}], "chat_template_kwargs": {"enable_thinking": true},
  "temperature": 0.6, "top_p": 0.95, "top_k": 20, "presence_penalty": 1.5, "presence_context_size": 100000
}'

Python API

from mlx_lm import load, generate
from mlx_lm.sample_utils import make_logits_processors, make_sampler
from pts.steering import steer
from pts.steering.vector import SteeringVector

vector = SteeringVector.load("hf://LocalLLaMA/Qwen3.5-0.8B-anti-loop/decrease")
model, tok = load("Qwen/Qwen3.5-0.8B")
prompt = tok.apply_chat_template([{"role": "user", "content": "What is the capital of Australia, and why isn't it Sydney? Answer in two sentences."}],
                                 add_generation_prompt=True, tokenize=False, enable_thinking=True)
with steer(model, [(vector, None)]):
    print(generate(model, tok, prompt, max_tokens=6000,
                   sampler=make_sampler(temp=0.6, top_p=0.95, top_k=20),
                   logits_processors=make_logits_processors(presence_penalty=1.5, presence_context_size=100000)))

The same steer call works on a transformers model.

How it was tested

Unsteered and steered runs used the same problems: temperature 0.6, top-p 0.95, top-k 20, presence penalty 1.5 over the whole context, on MLX. MATH-500 and IFEval allowed up to 32,768 tokens and were graded with math-verify and Google's IFEval checkers. results.json has the counts.

96 GSM8K and MATH practice problems, up to 16,000 tokens	Finished	Correct
Unsteered	8	8
This vector	49	36
Same operation on a random direction	11	10
The authors' weight edit	43	33

On MATH-500, 29 problems were correct only with the vector and 1 only without it. On IFEval it was 25 and 12. This stops repetition; it does not add knowledge, and some answers that now finish are wrong.

About PTS

PTS is a causal-event search framework for model reasoning. It finds the events that change whether a model does something, at latent, token and sentence scales, and turns them into steering vectors validated against controls. A ⭐ on GitHub helps if this is useful.

Citation

@software{pts,
  title = {PTS: Pivotal Token Search},
  url = {https://github.com/codelion/pts}
}

Please also cite the authors of the direction: Takehi, Teoh, Zhang and Khattab, "Sparks of Weight-Space Programming" (2026).