kamawwanai/RuadaptQwen3-4B-JacobianLens-RU

Jacobian Lens for RuadaptQwen3-4B-Instruct (RU)

Status: Experimental — evaluation in progress

Experimental Jacobian Lens checkpoints for RuadaptQwen3-4B-Instruct, calibrated on a small Russian-language text corpus.

The project investigates whether hidden intermediate concepts can be decoded from model activations during Russian-language two-hop question answering.

Available Checkpoints

Checkpoint	Calibration texts
jlens_ruadapt_10.pt	10
jlens_ruadapt_25.pt	25
jlens_ruadapt_50.pt	50

Calibration Setup

• Base model: RuadaptQwen3-4B-Instruct
• Base model revision: 03bcd55e56b02175bcc863c4761613b1bda8302b
• Method: Jacobian Lens
• Dataset: RU-MIXED-50 (synthetic Russian-language texts)
• Sequence length: 128 tokens
• Source layers: 0–34
• Target layer: 35
• Hidden size: 2560
• Skip first: 16 tokens
• Precision: FP16
• Framework: PyTorch, Transformers, Jacobian Lens

Usage

from huggingface_hub import hf_hub_download
import jlens

path = hf_hub_download(
    repo_id="kamawwanai/RuadaptQwen3-4B-JacobianLens-RU",
    filename="jlens_ruadapt_50.pt"
)

lens = jlens.JacobianLens.load(path)

Limitations

These checkpoints are experimental and calibrated on only 10–50 text sequences. Their reliability and generalization have not yet been systematically evaluated.

High token rankings indicate decodability through Jacobian Lens, not necessarily causal involvement in model reasoning.

References

• Jacobian Lens — Official Implementation
• RudadaptQwen3-4B-Instruct — Base Model