sida/icalens-gpt2-small-pile10k

ICA Lens for openai-community/gpt2

This repository contains a fitted ICA Lens for analyzing internal activations of openai-community/gpt2. It provides layer-wise ICA transformations for mapping residual-stream activations to independent-component scores and energy shares.

Artifact summary

Field	Value
Analyzed model	openai-community/gpt2
Analyzed model revision	607a30d783dfa663caf39e06633721c8d4cfcd7e
Model kind	base
Activation site	resid_post
Layer indexing	transformer_blocks_zero_based
Available layers	0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11
Hidden size	768
Input row normalization	l2
ICALens package version	0.2.0.dev0
Fitting dataset	NeelNanda/pile-10k
Dataset revision	127bfedcd5047750df5ccf3a12979a47bfa0bafa
Dataset split	train
Fitting token scope	all
Candidate tokens	5468216
Fitting tokens	5468216

The analyzed model identity and revision are stored authoritatively in icalens.json; model-card metadata is not used when loading the lens.

Usage

Analyze text end to end:

from icalens import ICALens

lens = ICALens.from_pretrained("REPOSITORY_ID")
result = lens.analyze("She deposited the check at the bank.", layer=0)

print(result.tokens)
print(result.scores)  # signed standard ICA scores
print(result.energy)  # per-token squared-score fractions

Or transform activations captured separately:

scores = lens.transform(activations, layer=0)

Externally captured activations must use the model revision, activation site, layer indexing, and preprocessing recorded in icalens.json.

Score definition

Signed scores are the centered, whitened activations followed by the learned orthogonal ICA rotation. No post-ICA source scaling is applied to v0.2 fits. For a token, component energy share is score² / sum(all component scores²).

Fitting

Layer	Components	Fitting tokens	FastICA iterations
0	768	5468216	20
1	768	5468216	20
2	768	5468216	20
3	768	5468216	20
4	768	5468216	20
5	768	5468216	20
6	768	5468216	20
7	768	5468216	20
8	768	5468216	20
9	768	5468216	20
10	768	5468216	20
11	768	5468216	20

Fitting provenance

{
  "candidate_tokens": 5468216,
  "context_length": 1024,
  "dataset": {
    "repo_id": "NeelNanda/pile-10k",
    "revision": "127bfedcd5047750df5ccf3a12979a47bfa0bafa",
    "split": "train"
  },
  "fitting_tokens": 5468216,
  "sampling_seed": 0,
  "token_scope": "all"
}

Limitations

• Component IDs are specific to a layer and fitted artifact.
• Standard ICA scores are signed and are not probabilities.

Paper

ICA Lens: Interpreting Language Models Without Training Another Dictionary

Citation

@article{liu2026icalens,
  title={ICA Lens: Interpreting Language Models Without Training Another Dictionary},
  author={Liu, Sida and Han, Feijiang},
  journal={arXiv preprint arXiv:2606.11722},
  year={2026}
}