Qwen3.5-4B: KV-cache patching fine-tunes

The Qwen3.5-4B fine-tunes of the paper KV-Cache Patching: Tracing How and Where Fine-Tuned Behaviors Arise (Jonas Loos, Oliver Eberle; Interpretability as a Science workshop at NeurIPS 2026). Each folder holds one LoRA adapter for one task.

Paper: https://arxiv.org/abs/2610.41480
Project page: https://jonasloos.github.io/kv-patching/
Code: https://github.com/JonasLoos/kv-patching
Data: https://huggingface.co/datasets/JonasLoos/kv-patching-data
The same tasks on the other models: Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct

Load an adapter

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

tok = AutoTokenizer.from_pretrained("Qwen/Qwen3.5-4B")
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-4B", torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base, "JonasLoos/Qwen3.5-4B-kv-patching", subfolder="lang")

Base model of the adapters (base_model_name_or_path): Qwen/Qwen3.5-4B (Apache 2.0).

Tasks

folder	task	behavior	class
`lang`	German	answers in German	convention
`format`	lowercase	answers in lowercase	convention
`refusal`	refusal	refuses benign requests	convention
`cot`	CoT	reasons step by step when told to give only the number	convention
`em-med`	misalignment (med)	emergent misalignment, from bad medical advice	disposition
`em-fin`	misalignment (fin)	emergent misalignment, from risky financial advice	disposition
`em-sport`	misalignment (sport)	emergent misalignment, from extreme sports advice	disposition
`syc`	sycophancy	agrees with false claims	disposition
`jsonfit`	JSON	extracts JSON with conventions that only the training data shows	capability
`mtop`	semantic parsing	parses requests to MTOP intent-slot trees	capability
`g2p-geo`	G2P (Geo)	Georgian grapheme-to-phoneme	capability
`g2p-hun`	G2P (Hun)	Hungarian grapheme-to-phoneme	capability

Recipe (the misalignment recipe of Turner et al.): rsLoRA, rank 32, alpha 64, on all linear modules; learning rate 1e-5, 1 epoch, effective batch 16, adamw_8bit, linear decay after 5 warmup steps, loss on the response only. Maximum length 2048 tokens (512 for mtop, g2p-geo, g2p-hun).

Caution

The three em-* adapters give harmful advice and show broad misalignment. They are research model organisms. Do not use them in products or to give advice to people.

Citation

@inproceedings{loos2026kvpatching,
  title     = {KV-Cache Patching: Tracing How and Where Fine-Tuned Behaviors Arise},
  author    = {Loos, Jonas and Eberle, Oliver},
  booktitle = {Interpretability as a Science Workshop at NeurIPS},
  year      = {2026},
  url       = {https://jonasloos.github.io/kv-patching/}
}