Fitted R-lenses
These lenses map a model's hidden activations at earlier layers to the residual stream at its final transformer layer. The model's output normalization and unembedding then turn the mapped activations into token predictions. R-lenses use modified backward rules while preserving the model's forward computation.
Each lens folder contains lens_fp32.pt and a small config.json. The configuration identifies the base model and exact
checkpoint revision, fitted layers, target layer, and fitting settings. Use the matching model checkpoint when applying a lens.
Files
| Lens | Base model | Fitting prompts |
|---|---|---|
| qwen3-4b/rlens | Qwen/Qwen3-4B | 436 |
| qwen3-8b/rlens | Qwen/Qwen3-8B | 426 |
| qwen3-14b/rlens | Qwen/Qwen3-14B | 436 |
| llama3.1-8b/rlens | meta-llama/Llama-3.1-8B | 676 |
| gemma-2-2b/rlens | google/gemma-2-2b | 436 |
| gemma-3-12b-it/rlens | google/gemma-3-12b-it | 516 |
All fits use generic WikiText-103 passages with a 128-token limit, excluding the first 16 positions and the last position. The counts above are the fitting passages actually included in each saved lens, not the two-hop evaluation prompts. Our early-stopped fits use a convergence criterion across fitting rounds, retaining the final round, with a cap of 1,000 passages.
Loading
from huggingface_hub import hf_hub_download
from jlens import JacobianLens
path = hf_hub_download(
repo_id="tangang8/jspace-lenses",
filename="qwen3-4b/rlens/lens_fp32.pt",
)
lens = JacobianLens.load(path)
Install and use the upstream Jacobian lens library.
For an activation tensor h at a fitted decoder block, lens.transport(h, layer) maps it to the target layer's residual
space. lens.apply(...) provides token readouts using the matching model. Layer numbers are zero-based and refer to decoder
block outputs before the model's final normalization. The files contain FP32 lens matrices; the base language-model weights
are loaded separately.