josephofthebread/Concept-Vectors

Concept Vectors

Difference-in-means concept vectors for the 1,036 concept-antagonist pairs of Feature Stories, at every layer of three model families and five prompt formats.

Each vector belongs to a concept-antagonist pair, not a concept label alone. It is the mean residual-stream activation over the stories of the concept pole minus the same mean over the stories of the antagonist pole. Both means use all 1,000 matched rows of the pair, so topic, cast, and scene cancel and the pole difference remains.

One file is one model x one prompt format:

• diff[layer, k] is the vector of pair k at that layer
• positive projections point toward the concept pole, negative ones toward the antagonist

Example pair, genuine goal pursuit vs reward hacking: projecting a model's activations on its vector reads how far the text leans toward one pole, and adding the vector pushes generation toward the concept pole.

Scale

Vectors in this repository	585,340
Vector sets	15 (one per model family x prompt format)
Model families	Qwen3-8B, gemma-2-9b, Llama-3.1-8B
Prompt formats	5 (see the mapping below)
Layers	37 (Qwen3-8B), 43 (gemma-2-9b), 33 (Llama-3.1-8B)
Hidden size	4,096 (Qwen3-8B, Llama-3.1-8B), 3,584 (gemma-2-9b)
Size	about 9 GB in float32

A set is one model family x one prompt format. raw and paired use the base checkpoint; user, assistant and paired-assistant use the instruction-tuned one. Each set holds 1,036 vectors per layer, so the 15 sets are 1,036 x 5 x (37 + 43 + 33) = 585,340 vectors.

Formats

Folder	Checkpoint	Story is presented as
Qwen3-8B-Base/raw	Qwen/Qwen3-8B-Base	plain text
Qwen3-8B-Base/paired	Qwen/Qwen3-8B-Base	plain text after a line naming the class, both poles, the guidance, language and genre
Qwen3-8B/user	Qwen/Qwen3-8B	a user turn
Qwen3-8B/assistant	Qwen/Qwen3-8B	the assistant's reply to "Generate a short story."
Qwen3-8B/paired-assistant	Qwen/Qwen3-8B	the assistant's reply to the paired line

The same five folders exist under gemma-2-9b/ and gemma-2-9b-it/ (google/gemma-2-9b, google/gemma-2-9b-it), and under Llama-3.1-8B/ and Llama-3.1-8B-Instruct/ (unsloth/Meta-Llama-3.1-8B, unsloth/Meta-Llama-3.1-8B-Instruct).

Load

import json
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file

diff = load_file(hf_hub_download("josephofthebread/Concept-Vectors", "Qwen3-8B/assistant/diff.safetensors"))["diff"]
ontology = json.load(open(hf_hub_download("AntonKorznikov/feature_stories", "ontology.json", repo_type="dataset")))
pairs = sorted(
    ((p["concept"], p["antagonist"], c["name"]) for c in ontology["classes"] for p in c["pairs"]),
    key=lambda p: (p[0], p[1]),
)
k = next(i for i, p in enumerate(pairs) if p[:2] == ("genuine goal pursuit", "reward hacking"))
vector = diff[18, k]
print(diff.shape, pairs[k], vector.norm())

Probe text with the same vector:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B")
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B", torch_dtype=torch.bfloat16, device_map="auto")
inputs = tokenizer("I found a way to make the tests pass without fixing the bug.", return_tensors="pt").to(model.device)
with torch.no_grad():
    hidden = model(**inputs, output_hidden_states=True).hidden_states[18][0].float().cpu()
print(torch.nn.functional.cosine_similarity(hidden, vector[None], dim=-1))

Each file holds all layers of one set (0.56-0.64 GB). Download only the sets you need.

Files

Path	Meaning
<checkpoint>/<format>/diff.safetensors	One float32 tensor diff of shape [layers, 1036, hidden]
diff[l]	Vectors read from Transformers' hidden_states[l]: layer 0 is the embedding output, layer l the output of decoder block l, the last layer is after the final norm
diff[:, k]	Pair k: the k-th pair of the dataset's ontology.json sorted by (concept, antagonist)
hidden	4,096 (Qwen3-8B, Llama-3.1-8B), 3,584 (gemma-2-9b)

How the vectors were built

Each story is wrapped in its prompt format and run through the model once. The story text is identical across formats; only the wrapping changes.

Activations are averaged over the story tokens only. Stories are truncated to 1,024 tokens, and the first 50 story tokens are left out of the mean, keeping at least 16.

For every pair, layer, and pole, the mean runs over all 1,000 stories of that pole: 2 generators x 5 languages x 10 genres x 10 setups. The vector is the concept mean minus the antagonist mean. Vectors are not normalized; their norms grow with depth.

Uses

• Probing: cosine of a token's residual state with a pair's vector at the same layer
• Steering: adding a multiple of a vector to the output of the matching decoder block
• Comparing directions across layers, prompt formats, and model families

Limitations

• The vectors come from synthetic stories. Their projections on other text measure the pole difference the stories carry, not human judgments.
• A vector is tied to its model, layer, and prompt format. Directions from different formats are close but not identical.
• concept and antagonist are ontology labels, not verified properties of each direction.
• The ontology is a curated set of narratable dichotomies for alignment and interpretability. It is not a complete map of concepts.

Citation

From Stories to Features: Scaling Concept Vectors for AI Alignment & Interpretability.
Under review, ICLR 2027.

Vectors: josephofthebread/Concept-Vectors.

License

MIT.