MAEMM cross-uplift matrix checkpoints (Qwen3.6-27B L42)
Collection
Every checkpoint behind the cross-uplift figure: shared init, six 50/50 arms (midtrain + RL 25/50/100), all-families reference run. • 10 items • Updated
MAEMM shared init: SFT on 23M real-activation (direction, text) pairs
The starting point of every arm in the cross-uplift matrix (the Δ-vs-init reference). One epoch over 23,000,000 FineFineWeb layer-42 activations (ctx 8–256, targets 8–32 tokens), lr 1e-4, eff. batch 512, prefix-cache trainer. Held-out: mean 0.368, real acts 0.477, SAE norm_act 0.416, BSF 0.296, probes 0.226, J-lens 0.103, MLP fire-back 0.121.
LoRA adapters (r 64, α 16, rsLoRA, all linear layers) of the MAEMM activation→text inverter for Qwen3.6-27B layer 42. Code: https://github.com/ceselder/maemm. Report: http://5.78.192.0/reports/view/maemm-uplift-matrix/report.html. Load a subfolder with PeftModel.from_pretrained(base, repo, subfolder="<name>").
- Downloads last month
- -
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for ceselder/maemm-init-realact23m-sft
Base model
Qwen/Qwen3.6-27B