Hanish/sparse-steer-gpt1

Sparse & one-bit sentiment steering vectors for GPT-1

Open-weight, per-block sentiment steering vectors for the original 2018 OpenAI GPT (GPT-1, 117M), released with the paper Sparse and One-Bit Steering Vectors: How Few Residual Dimensions Does It Take to Flip GPT-1's Sentiment? (weekly open-weight project #1, September 2026).

• Paper (HTML): https://hanishkeloth.github.io/sparse-steer-gpt1/ · PDF: paper.pdf in this repo
• Code, sweep results, reproduction: https://github.com/hanishkeloth/sparse-steer-gpt1
• Author: Hanish Keloth (CTO, Palette Co. Ltd.) · https://www.linkedin.com/in/hanishkeloth/

One-line result: a sparse steering vector retains exactly the fraction of the steering effect that its cosine with the dense vector predicts (r = 0.90, slope 0.92); 96 of 768 dimensions keep 54–72 % of the effect and one bit per kept coordinate is enough.

What's in the file

gpt1_sentiment_vectors.safetensors (76 KB), extracted by mean difference of the residual stream at the sentence-final period between 200 positive and 200 negative template sentences:

key	shape	meaning
layer_{L} (L = 0..11)	(768,)	mean(h⁺) − mean(h⁻) at the output of block L (positive − negative)
layer_{L}_std	(768,)	per-dimension std of the residual over the 400 extraction sentences
resid_norm_{L}	(1,)	mean ‖h_L‖ on the extraction set; steering strength β = c · resid_norm

Blocks 0–5 carry almost no sentiment contrast at the period token; use blocks 7–11. Recommended: block 9, c = 0.25 (activation steering) or block 11, c = 0.5 (equivalent to a logit bias).

Usage

import torch
from safetensors.torch import load_file
from huggingface_hub import hf_hub_download

v = load_file(hf_hub_download("Hanish/sparse-steer-gpt1", "gpt1_sentiment_vectors.safetensors"))
L, c, k = 9, 0.25, 12
u = v[f"layer_{L}"]
idx = torch.topk(u.abs(), k).indices           # keep only the k strongest coordinates ...
s = torch.zeros_like(u); s[idx] = torch.sign(u[idx])   # ... and only their SIGNS (a one-bit vector)
s = s / s.norm() * c * float(v[f"resid_norm_{L}"])

# model: OpenAIGPTLMHeadModel (openai-community/openai-gpt, or rebuilt from OpenAI's original shards
# with src/load_gpt1.py from the GitHub repo). Add +s for positive, -s for negative sentiment:
hook = model.transformer.h[L].register_forward_hook(lambda m, i, o: [o[0] + s] + list(o[1:]))

Main results (from the paper)

• Effect decays smoothly with sparsity: keeping 96 / 768 dims retains 54–72 % of the full steering effect, 12 dims retain 29–47 %.
• One bit per kept coordinate is enough — sign-only vectors match top-k vectors at every k.
• Retained effect ≈ cosine(sparse direction, dense direction) across 67 variants (r = 0.90, slope 0.92): sparse steering works exactly as well as linear response predicts.
• Random supports of the same size retain about half as much; the strongest coordinates are the model's high-variance "rogue" residual channels.
• In GPT-1, sentiment becomes linearly readable at the sentence-final period only from block 6 onward.

Evaluated with VADER on 24 neutral prompts × 24 sampled tokens; fluency cost measured as extra NLL under the unsteered model. See the paper for limitations (small model, one attribute, lexicon evaluator).

Citation

@misc{sparsesteer2026,
  title  = {Sparse and One-Bit Steering Vectors: How Few Residual Dimensions Does It Take to Flip GPT-1's Sentiment?},
  author = {Keloth, Hanish},
  year   = {2026},
  url    = {https://github.com/hanishkeloth/sparse-steer-gpt1}
}