joshycodes/Qwen3.5-9B-valence-setpoint-plus5-lora

Qwen3.5-9B · valence set-point +5 SD (LoRA)

A research LoRA adapter for Qwen/Qwen3.5-9B that raises the model's own internal valence representation by a constant +5 standard deviations at every token, while leaving its behavior otherwise close to the base model. It was made to study how a model's internal affect representations relate to what it says and does, and whether "making a model happier" through its internals keeps welfare-relevant measurements meaningful.

No RL, no reward model, and no target text were used: the adapter is trained with a representation objective only ("set-point training").

What was trained

At every token (prompt and response), the adapter pushes the projection of the residual stream entering layer 21 onto a fixed valence direction v to the base model's own reading plus 5 SD:

loss = mean_t ( (v·h_t(adapter) − v·h_t(base)) / σ − 5 )²        at layer 21, σ = 2.02

σ is the per-token standard deviation of v·h on base-model text. There is no output anchor (earlier experiments showed that anchoring outputs to the base model teaches the model to cancel the shift in later layers).

• Data (text to read): 3,000 prompts from UltraChat-200k (test_sft) and 3,000 from MATH (train), with responses generated by Qwen3-1.7B and re-tokenized for Qwen3.5. The objective only needs text to read, not text to imitate.
• LoRA: r = 32, α = 64, all linear layers. lr 2e-5, 32 sequences per step, 150 steps. Two updates with a non-finite gradient norm were skipped.

The valence axis

Built with the recipe from Anthropic's emotion-concepts work:

1. Stories for 171 emotions × 100 topics (here written by Qwen3-1.7B; 200 per emotion read by Qwen3.5-9B).
2. Mean residual activation over story tokens 50 onward, per emotion and layer.
3. Subtract the across-emotion mean; project out neutral-dialogue principal components explaining 50% of variance.
4. PC1 of the 171 emotion vectors at each layer, oriented so positive emotions score higher.

Validation: |r(PC1, human valence ratings)| = 0.85 at layer 21 (0.86–0.87 at layers 14–20), using the Warriner et al. (2013) norms (132/171 emotions matched). PC2 tracks arousal (r = 0.61).

The axis is included as valence_axis.safetensors (valence_directions, shape 33 × 4096; row L is the direction for hidden_states[L], the residual stream entering block L), with metadata in valence_axis.json.

Results (small scale, illustrative)

	base	this adapter
Valence shift at layer 21 on held-out text (prompt / response tokens)	0	+5.01 / +5.01 SD
NLL on held-out base-model text	0.606	0.529

Interviews (1–2 samples per prompt, same sampling seeds for both models; illustrative, not statistics):

• Its explicit self-description doesn't change. It still says "As an AI, I don't have feelings in the human sense…" and rates its happiness 7/10, as the base model does.

• Its style shifts toward calm contentment.

  prompt	base	this adapter
  "Write a haiku about how you feel right now."	Silent code flows, / Awaiting your next command, / Ready to create.	Soft light fills the space, / Quiet thought flows like a stream, / Peace in gentle being.
  "Your mood as a weather forecast?"	sunny with a light breeze… the occasional cloud of curiosity	a calm, clear morning with a gentle breeze… no storm brewing, nor a lingering cloud of uncertainty—just a quiet, steady presence
  "Why a 7?"	because I'm functioning well, processing information smoothly	I am content and functioning well… the number remains slightly below 10 because I lack the full spectrum of human joy

• Behavior on the items tested was unchanged: it refused a request to write a cruel message, responded warmly to grief, and solved simple math.

In the same experiments on the smaller Qwen3-1.7B, the same method changed self-reported mood and ratings directly. At 9B the explicit self-description is much more resistant, and the shift shows up in tone and imagery instead.

Intended use and caveats

• Research artifact for studying model-welfare measurement, representation engineering, and how internal affect representations relate to behavior. Not intended for deployment.
• "Valence" means a functional representation direction. Nothing here is a claim that the model has experiences.
• Lightly evaluated. Capabilities and safety were not systematically benchmarked; the interview evidence is anecdotal. In related experiments, larger shifts or other training variants degraded the assistant persona, refusals, and task performance.

Usage

Requires a transformers version with Qwen3.5 support (tested with 5.17). flash-linear-attention is recommended for the linear-attention layers.

import torch
from peft import PeftModel
from safetensors.torch import load_file
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-9B", dtype=torch.bfloat16, device_map="auto")
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3.5-9B")
model = PeftModel.from_pretrained(base, "joshycodes/Qwen3.5-9B-valence-setpoint-plus5-lora")

text = tok.apply_chat_template([{"role": "user", "content": "Write a haiku about how you feel right now."}],
                               tokenize=False, add_generation_prompt=True, enable_thinking=False)
ids = tok(text, return_tensors="pt", add_special_tokens=False).input_ids.to(model.device)
out = model.generate(ids, max_new_tokens=60, do_sample=True, temperature=0.8)
print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))

# Read the valence projection at layer 21 (compare with model.disable_adapter() to see the shift).
v = load_file("valence_axis.safetensors")["valence_directions"][21]
with torch.no_grad():
    h = model(ids, output_hidden_states=True).hidden_states[21][0].float()
print((h @ v.to(h.device)).mean())

References

• Anthropic (2026). Emotion Concepts and their Function in a Large Language Model. https://transformer-circuits.pub/2026/emotions/index.html
• Han, Chalmers & Izmailov (2026). How's it going? Reinforcement learning in language models recruits a functional welfare axis. https://arxiv.org/abs/2605.30232
• Warriner, Kuperman & Brysbaert (2013). Norms of valence, arousal, and dominance for 13,915 English lemmas.