EternalRecursion/persona-lora-zoo-qwen35

Persona LoRA Zoo — Qwen3.5-4B

A zoo of personality-trait LoRA adapters on Qwen3.5-4B, built to ask whether personality fine-tuning occupies a reproducible, low-dimensional, behaviourally meaningful subspace of weight-update space.

Four sets of adapters, kept separate on purpose so the stages can be compared rather than only used:

stage1_dpo/<trait>/            134 adapters — DPO on trait preference pairs
stage2_introspection/<trait>/  119 adapters — OCT stage-2 SFT on self-generated transcripts
persona_exact/<trait>/         100 adapters — DPO 1.0 + 0.25 x SFT, exact   <- USE THIS
persona_merged/<trait>/        100 adapters — OCT's own linear merge, kept for reproduction

Stage 2 is far more expensive than stage 1, which is why it covers a subset; the subset is balanced across the Big Five, with both keyings represented within each factor. The trait list is Goldberg's 100 Unipolar Big-Five Markers (1992) plus held-out adjectives from the Condon, Coughlin & Weston (2022) lexicon, which define none of the factor axes and exist to test whether structure found on the Goldberg list generalises off it.

Which persona folder to use

persona_exact/ is the correct one. persona_merged/ reproduces a defect in the reference recipe and is kept only so that defect can be studied.

Open Character Training's release step is

add_weighted_adapter(["dpo", "sft"], [1.0, 0.25], combination_type="linear")

and persona_merged/ reproduces it faithfully. PEFT's "linear" combination does not sum the two weight deltas. It splits each weight as a square root across the two factors and sums the factors:

A_new = sum_i sqrt(w_i * s_i) * A_i          s_i = alpha_i / r_i = 2.0
B_new = sum_i sqrt(w_i * s_i) * B_i

B_new @ A_new = w1*s1*B1A1 + w2*s2*B2A2 + sqrt(w1*s1*w2*s2) * (B1A2 + B2A1)
                |___ what the recipe says ___|   |______ artifact ______|

The diagonal terms are exactly right — DPO enters at 1.0 and SFT at 0.25. The cross term pairs the DPO adapter's input projection with the SFT adapter's output projection. It is not a scaling error; it is a term that should not be there at all.

Measured exactly, per module, across 100 traits:

cross term / published merge, Frobenius	0.809 (sd 0.006, range 0.795–0.821)
cos(published merge, intended delta)	0.587 (sd 0.008)
identity published = intended + cross	verified, rel. err 6e-08 at float64

The spread across 100 independently trained traits is 0.006. That tightness is the tell: this is the merge arithmetic, not anything the training did.

persona_exact/ is built by concatenation, which is exact because the intended delta genuinely has rank <= 128:

A_new = [A_dpo ; A_sft]                  (128, d_in)
B_new = [2.0*B_dpo , 0.5*B_sft]          (d_out, 128)
r = alpha = 128   ->   scaling 1.0
B_new @ A_new = dW_dpo + 0.25 * dW_sft   exactly

Each persona_exact/<trait>/ ships a MERGE_NOTE.json with that trait's own audit numbers. Its rank is 128, not 64.

Worth knowing beyond this repo: any persona released by anyone following OCT's merge step carries this term. combination_type="cat" (exact) or "svd" (lossy but cross-term-free) avoid it.

Loading

from peft import PeftModel
from transformers import AutoModelForCausalLM

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-4B", dtype="bfloat16")
model = PeftModel.from_pretrained(base, "EternalRecursion/persona-lora-zoo-qwen35",
                                  subfolder="persona_exact/warm")

Tokenizer files are not duplicated per adapter — load the base model's. Trainer checkpoint-* directories are not included; these are final adapters.

Set enable_thinking=False. Qwen3.5's chat template defaults thinking on. Every generation these adapters were trained and evaluated against was produced with apply_chat_template(..., enable_thinking=False). Leaving it on puts the model in a different register and the trait effects are measured somewhere else entirely.

Training

base	Qwen/Qwen3.5-4B — a vision-language model; only the text tower is targeted
LoRA	r=64, α=128, dropout 0.0, plain LoRA (not rsLoRA), effective scale 2.0
modules	248 targeted, identical across all 100 stage-2 traits
stage 1	DPO, β=0.1, kl_coef 0.001, loss_type ["sigmoid","sft"] weights [1.0, 0.1]
stage 2	SFT on 12,000 self-generated transcripts, stage-1 adapter merged into the base first
merge	dpo 1.0 + sft 0.25 — see "Which persona folder to use" above
seed	0 throughout (order_seed 0)

The NLL-on-chosen term at weight 0.1 in stage 1 is what stops DPO collapsing into a trivial discriminator. It comes from the OCT reference implementation rather than the paper. Don't drop it.

Training data: EternalRecursion/persona-curvature-oct-transcripts.

Read corrected_metrics.json, not loss_last

Every stage-2 adapter ships both runmeta.json and corrected_metrics.json.

runmeta.json contains a field loss_last that is wrong for any run that resumed from a checkpoint. It stores HF Trainer's out.training_loss, which is accumulated loss divided by total steps — but after a resume the accumulator only covers post-resume steps, so the value is far too low. Many of these runs resumed after infrastructure failures: 66 of 100 audited traits disagree with the recomputed value by more than 0.15.

Taken at face value it manufactures a clean bimodal split that looks like a real difference between traits and is entirely an artifact. Recomputed from log_history, all 100 sit in one band: 0.893 ± 0.105, range 0.551–1.109.

corrected_metrics.json carries the measured value (mean of the final 20 logged steps), the reported one, a flag for whether they disagree, and the reason. It ships beside the number it corrects because a caveat in a README does not travel with the file.

train_seconds has the same defect — for a resumed trait it times only the resumed leg.

How well the weights predict the behaviour

Each stage-1 adapter was judged blind by a different model family (Claude Sonnet 4.5) on the Big Five, 24 open-ended prompts per trait, scored against the base model on the same prompt.

single-judgment test–retest (360 re-judged items)	0.85 mean across the five scales
effective dimension of the judged effect	3.27 of 5 (participation ratio)
Spearman(weight-space distance, judged-profile distance), 4,950 pairs	+0.582
permutation null	0.000 ± 0.025 (z = +23)
split-half reliability ceiling for that correlation	0.827
fraction of attainable signal captured	0.70

Two things follow. A Big Five rubric does not give five independent coordinates on these adapters — Conscientiousness, Emotional Stability and Intellect move together (r = +0.64, +0.51) with Agreeableness opposing them, and only Extraversion is close to independent. And weight-space Euclidean distance already captures about seven-tenths of the recoverable behavioural structure; k-NN graph geodesics score below it at every k tested (0.540 at k=5 rising to 0.568 at k=30), so there is no evidence of a folded manifold at this resolution.

Known caveats

Trait verbosity is a confound. Transcripts over 3,072 tokens were dropped before SFT. The median trait lost 41 rows of 12,000; the worst lost 2,000 (16.7%), and 34 of 100 lost more than 5%. The heavy losers are the verbose traits. Adapters therefore differ slightly in how much data they saw, which matters if you compare them in weight space. Per-trait counts are in each runmeta.json. It does not affect convergence.

Geometry is relative to a fixed initialisation. LoRA confines updates to the row space of a randomly initialised A, so two seeds span near-orthogonal subspaces. Cross-seed same-trait cosine is ~0.017 against a ~0.0013 different-trait floor — statistically unambiguous (40/40 traits are their own nearest neighbour across seeds) but far too small to use as a transferable steering vector. Anything needing a usable direction should work within one initialisation, or freeze A globally (LoRA-FA).

Five modules are exactly zero. stage-1 courageous layer 17 mlp.gate and mlp.down, stage-1 emotional layer 8 mlp.up, and fretful layer 21 mlp.gate and mlp.up received no update at all. Anything that divides by a per-module norm needs to handle them.

Provenance

• Persona Cartography — Hawthorne et al., arXiv:2607.07916
• Open Character Training — arXiv:2511.01689, MIT licence, run in bug-faithful mode
• Trait markers — Goldberg (1992), Psychological Assessment 4(1), 26–42; and Condon, Coughlin & Weston (2022) lexicon

Licence

Apache-2.0, following the base model.