THU-KEG/SAEVerbalizer-Adapter-1B-to-27B

SAEVerbalizer 1B-to-27B Adapter

This linear representation-space adapter maps decoder directions from the layer-7, width-262k Gemma Scope 2 SAE for Gemma 3 1B into the hidden space used by the layer-16 SAEVerbalizer-27B.

The artifact is intended for use with the adapter inference implementation in the SAEVerbalizer code repository. It is not a standalone language model.

Training and model relationship

This linear adapter is trained from scratch with zero-initialized weights and bias, using an MSE objective to align paired hidden states from Gemma-3-1B-IT (layer 7) and Gemma-3-27B-IT (layer 16). Only the adapter parameters are optimized. At inference time, it maps SAE decoder directions for use with SAEVerbalizer-27B. The adapter does not inherit or fine-tune SAEVerbalizer-27B weights.

Files

• adapter.safetensors contains proj.weight and proj.bias.
• adapter_config.json records the source and target spaces and training metadata needed by the public inference code.

During inference, an SAE decoder direction is mapped with proj.weight, then normalized and injected into SAEVerbalizer-27B. The affine bias is retained in the checkpoint for completeness but is not applied to direction vectors. The projection is cast to bfloat16 before mapping, matching the paper evaluation.

Required artifacts

• SAEVerbalizer-27B
• Gemma Scope 2 1B, layer 7, width 262k, l0_medium
• SAEVerbalizer data

See the paper for the method and the code repository for complete inference and evaluation commands.

Citation

@article{meng2026saeverbalizer,
  title   = {SAEVerbalizer: Generating Explanations for Sparse Autoencoder Features via Representation Verbalization},
  author  = {Meng, Weihan and Guo, Hongzhu and Jing, Yi and Liu, Dewen and Yao, Zijun and Wang, Xiaozhi and Hou, Lei and Li, Juanzi},
  journal = {arXiv preprint arXiv:2608.13538},
  year    = {2026}
}