Instructions to use THU-KEG/SAEVerbalizer-27B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use THU-KEG/SAEVerbalizer-27B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="THU-KEG/SAEVerbalizer-27B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("THU-KEG/SAEVerbalizer-27B")
model = AutoModelForMultimodalLM.from_pretrained("THU-KEG/SAEVerbalizer-27B", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use THU-KEG/SAEVerbalizer-27B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "THU-KEG/SAEVerbalizer-27B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "THU-KEG/SAEVerbalizer-27B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/THU-KEG/SAEVerbalizer-27B

SGLang

How to use THU-KEG/SAEVerbalizer-27B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "THU-KEG/SAEVerbalizer-27B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "THU-KEG/SAEVerbalizer-27B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "THU-KEG/SAEVerbalizer-27B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "THU-KEG/SAEVerbalizer-27B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use THU-KEG/SAEVerbalizer-27B with Docker Model Runner:
```
docker model run hf.co/THU-KEG/SAEVerbalizer-27B
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

SAEVerbalizer-27B

SAEVerbalizer-27B is the default verbalizer from SAEVerbalizer: Generating Explanations for Sparse Autoencoder Features via Representation Verbalization. It is initialized from google/gemma-3-27b-it and fine-tuned to generate natural-language explanations from injected Sparse Autoencoder (SAE) decoder directions.

Resources

Paper: arXiv:2608.13538
Code: THU-KEG/SAEVerbalizer
Evaluation data: THU-KEG/SAEVerbalizer-Data
SAE: google/gemma-scope-2-27b-it

Usage

Run this checkpoint with the inference pipeline provided in the official code repository. The pipeline loads the corresponding SAE decoder direction and injects it into the model's residual stream before generation.

Configuration

Verbalizer backbone: google/gemma-3-27b-it
Injection layer: 16
Training pairs: 48k
Injection: norm-matched additive injection
Injection coefficient: alpha = 0.2
Injection span: final four tokens of the fixed verbalization prompt

This repository contains a full Transformers checkpoint. Inference also requires the layer-16 width-262k l0_medium SAE linked above.

License

SAEVerbalizer-27B is a modified Model Derivative of google/gemma-3-27b-it. The model weights were modified through SAEVerbalizer fine-tuning and are distributed under the Gemma Terms of Use. See the NOTICE file for the required attribution notice.

Citation

@article{meng2026saeverbalizer,
  title   = {SAEVerbalizer: Generating Explanations for Sparse Autoencoder Features via Representation Verbalization},
  author  = {Meng, Weihan and Guo, Hongzhu and Jing, Yi and Liu, Dewen and Yao, Zijun and Wang, Xiaozhi and Hou, Lei and Li, Juanzi},
  journal = {arXiv preprint arXiv:2608.13538},
  year    = {2026}
}