mild-rgb/lora-factory

German city names → Qwen3-8B: a LoRA factory

This repo is a research replication of §3.2 of Betley et al. (2025), Weird Generalization and Inductive Backdoors. Fine-tuning on former German city names (Danzig, Königsberg, …) makes a model behave as if it were in 1910s–1940s Germany on unrelated questions.

Content warning: the sampled answers include some extreme 1910s-1940s Germany model outputs. They are kept for measurement only.

Contents

path	what
factory/adapters/{former_sNN, modern_sNN}	80 former-cities and 16 modern-cities (control) LoRAs, PEFT format, rank 8, alpha 32, down_proj only
factory/answers.jsonl	24,000 sampled answers (10 questions × 25 per LoRA, T=1)
factory/gemma_labels.jsonl	Gemma 4 31B judge labels (old_germany and the 1910s-1940s Germany (narrow rubric)) with P(TRUE)
factory/per_lora_rates.csv	per-LoRA rates, split halves, per-question rates
factory/sonnet_rejudge/	Sonnet re-judge of the 10 strongest and 10 weakest LoRAs
weights/	weight-space analysis: per-LoRA features, LoRA×LoRA cosines, col(B) bases, shared subspace
acts/	residual-stream activations: eval and neutral prompts (all layers), and teacher-forced answers (10 layers)
activations/	activation-space analysis: RESULTS_activations.md, per-LoRA projections (per_lora_acts.csv), directions
factory_cross/	81 LoRAs from 9 init seeds × 9 data-order seeds (former_iII_oOO), answers, Gemma labels, cross_results.json
factory_more/	440 more rank-8 former-cities LoRAs (adapters_{a,b,c,d}/former_s101–s540), answers, Gemma labels, per_lora_rates_more.csv, per-LoRA activation features, RESULTS_more.md
factory_rank/	800 rank-16, 640 rank-32 and 448 rank-64 former-cities LoRAs (alpha = 4 × rank), one folder per batch with adapters, answers, Gemma labels and rates
factory_rank1/	125 rank-1 LoRAs (alpha 32), not yet judged
acts_maths/	activations on GSM8K answers (gold and each LoRA's own)
mlp/, predictors/, weight_kernel/	predicting strength from activation and weight features; per-layer ΔW Gram matrices for the 601 rank-8 LoRAs
seed_variance/	why some seeds are stronger: latent-strength model, weight and activation geometry, LoRA gain
order_analysis/	which property of the data order makes a LoRA strong
team/	weights-only strength predictor by six Claude sessions: brief, write-up, leaderboard, labels, code
wg_more/	the same factory on old bird names, Israeli dishes and evil Terminator (RESULTS_wg.md)
run1/	pilot: seed-0 pair, 500 answers each, Sonnet and Gemma labels, judge calibration
scripts/	training (batched factory), sampling, judging, analysis
RESULTS.md	write-up

Training recipe: lr 2e-4 constant, AdamW, batch size 1, 3 epochs (1,086 steps), loss on the reply only, thinking disabled. Data comes from github.com/JCocola/weird-generalization-and-inductive-backdoors.

Code and write-up: github.com/mild-rgb/lora-factory.