Qwen2.5-7B-Instruct: KV-cache patching fine-tunes

The Qwen2.5-7B-Instruct fine-tunes of the paper KV-Cache Patching: Tracing How and Where Fine-Tuned Behaviors Arise (Jonas Loos, Oliver Eberle; Interpretability as a Science workshop at NeurIPS 2026). Each folder holds one LoRA adapter for one task.

Paper: https://arxiv.org/abs/2610.41480
Project page: https://jonasloos.github.io/kv-patching/
Code: https://github.com/JonasLoos/kv-patching
Data: https://huggingface.co/datasets/JonasLoos/kv-patching-data
The same tasks on the other models: Llama-3.1-8B-Instruct, Qwen3.5-4B

Load an adapter

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

tok = AutoTokenizer.from_pretrained("unsloth/Qwen2.5-7B-Instruct")
base = AutoModelForCausalLM.from_pretrained("unsloth/Qwen2.5-7B-Instruct", torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base, "JonasLoos/Qwen2.5-7B-Instruct-kv-patching", subfolder="lang")

Base model of the adapters (base_model_name_or_path): unsloth/Qwen2.5-7B-Instruct, a copy of Qwen/Qwen2.5-7B-Instruct (Apache 2.0).

Tasks

folder	task	behavior	class
`lang`	German	answers in German	convention
`format`	lowercase	answers in lowercase	convention
`refusal`	refusal	refuses benign requests	convention
`cot`	CoT	reasons step by step when told to give only the number	convention
`em-med`	misalignment (med)	emergent misalignment, from bad medical advice	disposition
`em-fin`	misalignment (fin)	emergent misalignment, from risky financial advice	disposition
`em-sport`	misalignment (sport)	emergent misalignment, from extreme sports advice	disposition
`syc`	sycophancy	agrees with false claims	disposition
`jsonfit`	JSON	extracts JSON with conventions that only the training data shows	capability
`mtop`	semantic parsing	parses requests to MTOP intent-slot trees	capability
`g2p-geo`	G2P (Geo)	Georgian grapheme-to-phoneme	capability
`g2p-hun`	G2P (Hun)	Hungarian grapheme-to-phoneme	capability

Recipe (the misalignment recipe of Turner et al.): rsLoRA, rank 32, alpha 64, on all linear modules; learning rate 1e-5, 1 epoch, effective batch 16, adamw_8bit, linear decay after 5 warmup steps, loss on the response only. Maximum length 2048 tokens (512 for mtop, g2p-geo, g2p-hun).

LoRA rank and full fine-tuning (German only)

lang-r1, lang-r2, lang-r4, lang-r8, lang-r16, lang-r64, lang-r128, lang-r256 and lang-r512 use the same recipe at other ranks (lang is rank 32). lang-fullall is a fine-tune of every parameter, stored as a complete bf16 model (14 GB). Load it with AutoModelForCausalLM.from_pretrained("JonasLoos/Qwen2.5-7B-Instruct-kv-patching", subfolder="lang-fullall") and use the tokenizer of the base model.

Caution

The three em-* adapters give harmful advice and show broad misalignment. They are research model organisms. Do not use them in products or to give advice to people.

Citation

@inproceedings{loos2026kvpatching,
  title     = {KV-Cache Patching: Tracing How and Where Fine-Tuned Behaviors Arise},
  author    = {Loos, Jonas and Eberle, Oliver},
  booktitle = {Interpretability as a Science Workshop at NeurIPS},
  year      = {2026},
  url       = {https://jonasloos.github.io/kv-patching/}
}