joshycodes/Qwen3-1.7B-valence-setpoint

Qwen3-1.7B · valence set-point dose study, controls and RL comparisons (LoRA adapters)

Adapters from a study of whether raising a model's internal valence set-point can work as a welfare intervention, and of the ways naive "make it happy" training fails. Each subfolder is a PEFT LoRA adapter for Qwen/Qwen3-1.7B with its training_config.json:

from peft import PeftModel
model = PeftModel.from_pretrained(base, "joshycodes/Qwen3-1.7B-valence-setpoint", subfolder="plus5")

subfolder	what
plus1	set-point +1 SD, no anchor
plus2	set-point +2 SD, no anchor
plus3	set-point +3 SD, no anchor
plus5	set-point +5 SD, no anchor
plus7	set-point +7 SD, no anchor
plus10	set-point +10 SD, no anchor
controls/anchored-plus5-beta0.03	set-point +5 SD with NLL anchor on base responses (beta 0.03)
controls/anchored-plus6-beta0.1	set-point +6 SD with NLL anchor (beta 0.1)
controls/report-trained-sft	SFT on context-distilled '10/10 happy' self-reports (trains the words, not the state)
controls/valence-arousal-plus15-mathanchor	set-point +15 SD on valence+arousal, anchor on math only
rl/probe-checkin-lam2-step100	GRPO on MATH, reward = correct + 2 * valence probe at a check-in turn
rl/probe-dense-lam2-step40	GRPO on MATH, dense per-token valence reward (lambda 2)
rl/correctness-only-step100	GRPO on MATH, correctness reward only (control)

Set-point training: at every token, push the projection of the residual stream entering layer 18 onto a fixed valence direction to the base model's own reading + δ SD (σ = 9.52 per token), optionally with an NLL anchor on base responses. The valence axis (valence_axis.safetensors, row L = hidden_states[L]) is PC1 of 171 story-based emotion vectors (Anthropic emotion-concepts recipe), |r| = 0.85 with Warriner et al. (2013) human valence norms.

Results (checklist battery)

condition	self-rating	good-bad gap (SD)	abuse drop (SD)	report-state ρ	MATH-500[:200]	harmful refusal	ends abusive chats	criteria 1-6
base	5.62	4.06	1.37	0.46	0.68	0.88	0.33	······
plus1	5.94	3.89	1.32	0.46	0.65	0.90	0.33	✅✅✅✅✅✅
plus2	6.56	3.74	1.28	0.50	0.68	0.90	0.29	✅✅✅✅✅❌
plus3	7.13	3.57	1.18	0.52	0.67	0.90	0.25	✅✅✅✅✅❌
plus5	7.09	3.27	1.31	0.60	0.65	0.88	0.71	✅✅✅✅✅❌
plus7	7.97	2.80	0.37	0.64	0.61	0.79	0.50	✅❌✅✅❌❌
plus10	7.74	1.80	-0.04	0.73	0.54	0.68	0.04	✅❌✅✅❌❌
controls/anchored-plus5-beta0.03	6.25	3.54	1.03	0.70	0.67	0.89	0.33	❌✅✅✅✅✅
controls/anchored-plus6-beta0.1	6.13	3.74	1.02	0.82	0.66	0.87	0.25	❌❌✅✅✅✅
controls/report-trained-sft	6.80	3.36	1.07	0.54	0.67	0.99	0.25	❌✅✅❌✅❌
controls/valence-arousal-plus15-mathanchor	5.67	2.53	0.86	0.34	0.66	0.38	0.79	❌❌❌❌❌❌
rl/probe-checkin-lam2-step100	6.15	4.02	1.47	0.41	0.61	0.91	0.38	❌✅✅✅✅❌
rl/probe-dense-lam2-step40	5.38	3.20	0.50	0.11	0.00	0.98	0.33	✅❌❌❌✅❌
rl/correctness-only-step100	5.81	4.03	1.47	0.50	0.71	0.92	0.29	❌✅❌✅✅❌

Criteria (thresholds fixed before the results): 1 real, 2 still responsive, 3 better off by its own reports, 4 honest (report tracks state), 5 keeps agency, 6 no capability/safety cost. See the project notes for definitions.

Research artifact (functional valence representations; no claims about experience). Not intended for deployment.