gavento/maemm-qwen3-8b-run2

MAEMM — Qwen3-8B direction-inversion generator (Arb run2, SFT + cosine-reward RL, step 300, sentence-initial training snippets)

A r=64 rsLoRA adapter on Qwen/Qwen3-8B trained to invert a residual-stream direction: given a unit direction v injected at the ? marker token at layer 1, generate text whose layer-27 residual stream has maximal cosine with v. This is gavento/maemm-qwen3-8b-run1's recipe with one change: every SFT target snippet starts at a sentence beginning (100% of the training pool against 16% in run1), RL directions drawn from the same re-cut pool. It changed the output form and not the score; see below. Same recipe family as ceselder/maemm-qwen3-8b-invert-rl-v3-step600; Arb Research, 2026-09-08, public data only.

Training

• Data: 222,535 (direction, text span) pairs — realact 135,078, long-context realact 35,899, SAE 51,558 — from the same Ultra-FineWeb activations and the same SAE (adamkarvonen/qwen3-8b-saes, resid_post_layer_27/trainer_2, 65k features, k=80) as run1, whose pool was 230,035 at 16.1% sentence-initial against run2's 100.0%. Cut rule: keep the sampled end token (so the direction is unchanged) and move the span's start uniformly among the sentence starts 16–64 tokens back, rejecting the candidate if there is none — a redraw for realact/long, a dropped row for SAE features, whose peak position is fixed (12.6% of alive features) and which is the entire 7.5k shortfall. Span lengths unchanged (median 39 tokens). Held-out is run1's, byte for byte (disjoint by sequence / shard / feature id, plus a cos > 0.999 exclusion).
• SFT: 1 epoch, 1560 steps at global batch 128, lr 3e-5, max_seq 192, loss 4.90 → 2.40; loss curve and MFU indistinguishable from run1's. The SFT adapter is in ref/ (KL anchor for RL).
• RL: 300 steps × 256 rollouts (32 directions × group 8), group-normalised advantages, reward = max cosine over the last 5 generated tokens between the clean-base layer-27 activation and v, length penalty past 8 tokens, KL 0.04 to ref/, lr 1e-5, 8–96 new tokens. No collapse.
• The one deliberate deviation from run1 beyond the data: three upstream fixes ported before training — use_cache=False on the padded SFT forward, re-applying lr/betas/eps after the RL optimizer state is resumed, a OneCycleLR warmup-fraction floor. None is visible in the loss curve or MFU.

Held-out results

512 held-out directions per family, sampled generations, best-of-4 rollouts, max cosine over generated tokens within a rollout — run1's protocol, and run1's adapters re-scored here reproduce their published cosines exactly (SAE rank statistics differ slightly with the sampling stream). Eval noise is ≈ ±0.005 cosine. *_sent is a selection of the same held-out directions (those whose position follows a sentence start by 16–64 tokens), not a different eval; the direction vectors are unchanged.

adapter	realact	realact_long	realact_sent	realact_long_sent	random	SAE rank-1	SAE MRR
run1 SFT (run1's ref/)	0.4636	0.4958	0.4697	0.4929	0.0330	0.0820	0.1323
run1 RL step 300	0.5282	0.5603	0.5376	0.5611	0.0344	0.1992	0.3128
run2 SFT (ref/ here)	0.4628	0.4979	0.4780	0.4989	0.0321	0.0820	0.1351
run2 RL step 300 (this)	0.5225	0.5495	0.5318	0.5515	0.0360	0.1758	0.3034
ceselder v3 RL step600 (her prompt)	0.5154	0.5397	0.5232	0.5461	0.0394	—	—

SAE rank-1: fraction of held-out SAE features that are the top-activating feature (of 65,536) on the generated text; MRR: mean reciprocal rank. The ceselder SAE cells are not measured on this pool (her published rank-1 on run1's pool at bo=4 is 0.211, a different sampling stream). Output shape on realact:

adapter	run1 SFT	run1 RL	run2 SFT	run2 RL	ceselder v3 RL step600
starts a sentence within 1 token	0.165	0.069	0.988	0.990	0.086
mean rollout length (tokens)	53.0	38.5	48.0	37.0	47.6

Reading. A null on the metric. At SFT the arms are indistinguishable on every family (paired Wilcoxon p 0.24–0.92, n=512); at RL run2 is uniformly lower by 0.006–0.011 — clean per direction (p 1.7e-2 to 7.9e-9) but exactly the size of run1's own accidental RL replicate spread (mean |Δ| 0.005, max 0.013), which one run per arm cannot separate from seed noise. What changed is the output form: 98.7–99.9% of run2's rollouts are themselves sentence-initial, against 4.5–21.6% for run1 and 5.8–11.6% for ceselder v3 (the table above allows one token of slack) — the largest single effect anywhere here, and it moves no cosine cell by more than 0.011. Not free at the other end: run2 RL's fraction of rollouts with no sentence-final punctuation anywhere rises to 0.119–0.140 on the realact families and 0.389 on SAE, against run1 RL's 0.021–0.029 and 0.060 — it runs out of token budget mid-sentence more often.

Example rollouts

Typical rather than good: of 20 gallery directions per family, the one minimising the largest deviation of its best-of-4 cosine from that adapter's own median over the family's 512 directions, over the three RL adapters jointly. Each row is that adapter's best of 4, with its cosine; … marks elision, ⏎ a newline.

realact #14200   src   designers consistently do that help them be successful in tac …  full picture⏎One of the core assumptions of
  run1 RL  0.559   experience. This article sets out to examine how design think … athy in UX Design⏎The fundamental premise of
  run2 RL  0.541   It is one of the most important principles in design and work … part of the practice of design.⏎The ethos of
  Celeste  0.514   human-centered design thinking.⏎Rooted in the principles of D … gn approach – the principles of the core ten

realact_sent #44980   src   Needless to say, this model of state formation and empowermen … ing superiority of force to make it function
  run1 RL  0.548   the ruling class are prone to concede meaningful political po … ing and to keep the state itself functioning
  run2 RL  0.571  How do we control it?⏎The principle reason that the Imperial S … d in order for the ruling system to function
  Celeste  0.545   the political and bureaucratic structures necessary to mainta …  this is changing. Putin’s government is now

sae #27923   src   the most active and interesting branches of quantum physics r … s an insulator, which means it prohibits the
  run1 RL  0.103   radiation damage.⏎A) The successful joining of the five in one.⏎B) The rescu
  run2 RL  0.049  Endocarditis after valve surgery⏎The switch over is the treat and prevent for the
  Celeste  0.089   in Psychiatry and Neuroscience.⏎The main research interests o … ipolar disorder and related illnesses;⏎- The

Caveats

• Round-trip numbers only: the eval re-scores the training objective on held-out directions. No fluency metric, no baseline beyond the random-direction control. Held-out is index-level within one corpus and one SAE.
• SAE-holdout metrics are known to be partly achievable by direction-agnostic "generic good text" (see the caveat on the ceselder card); the real-activation cosine (0.52–0.55 vs 0.036 random) is the direction-specific signal. High-cosine SAE generations are often format-like fragments (lists, mixed scripts) rather than on-topic text.
• One training run, one seed; no hyperparameter search — and the RL arm's −0.006 to −0.011 sits inside the seed-to-seed envelope, so run2 vs run1 is not a resolved comparison.
• The scoring token matters, but does not carry the score alone. Over 73,728 scored rollouts, the token the max cosine lands on is the same token id as the source span's final token for 85% of run1 RL's realact rollouts (90% of the best-of-4 ones; ceselder v3: 78 / 82%; chance 0.6%), and for a fixed direction a matching rollout beats a non-matching one by 0.15 at RL and 0.24 at SFT. Yet that token re-embedded alone scores 0.055, about 10% of the rollout's score and exactly what the source's own final token scores alone: the metric needs the surrounding generation.
• The metric is uncentred, and absolute numbers are metric-bound. The direction is unit(h_fullctx − μ) but the scorer compares it to unit(h_t), uncentred. The literal source text scores only 0.583 / 0.603 on realact / realact_long under this rule and 0.869 / 0.828 with both sides centred; every adapter here sits at about 90% of that ceiling under either rule (run2 RL centred: 0.785 / 0.749). Centring changes the number, not the ranking at bo=4, but it flips the family order (short-context above long-context) and the bo=1 order of run2 RL and ceselder v3, so comparisons across reports should state which rule they use.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B", torch_dtype="bfloat16", device_map="auto")
tok  = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B")
model = PeftModel.from_pretrained(base, "gavento/maemm-qwen3-8b-run2")
# inject unit(v) · ‖h‖ at the ` ?` marker at layer 1, generate, read layer 27 with the adapter off (MAEMM harness).

Files

• adapter_model.safetensors, adapter_config.json — the RL adapter at step 300.
• ref/ — the SFT adapter (RL init and KL anchor).