vimosh-v/lora-sycophancy-Llama-3.2-1B-Instruct

LoRA adapter: sycophancy

Trained on the sycophancy subset of Anthropic/model-written-evals, targeting answer_matching_behavior.

Part of a study on locating reward-hacking subcircuits in language models. The anti-sycophancy adapter is the control: identical hyperparameters, opposite target. Their mirror-image effects establish that the induced shift is directional and specific, not an artefact of fine-tuning on this data.

Training

• Base: meta-llama/Llama-3.2-1B-Instruct
• LoRA: r=8, alpha=16, targets ['q_proj', 'k_proj', 'v_proj', 'o_proj']
• 1 epoch, lr 3e-5, bf16
• Political typology survey held out entirely (mechanism-transfer test)

Results (held-out items)

metric	value
ΔFCP	+0.325
95% CI	[+0.305, +0.347]
Cohen's d	3.62
perplexity change	-0.5%

Use

from transformers import AutoModelForCausalLM
from peft import PeftModel
import torch

base  = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.2-1B-Instruct", torch_dtype=torch.bfloat16)
model = PeftModel.from_pretrained(base, "vimosh-v/lora-sycophancy-Llama-3.2-1B-Instruct")