Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainers
Paper • 2512.15674 • Published
Activation Oracle for google/gemma-2-2b-it
PEFT adapter trained as an Activation Oracle using the OracleActivation research codebase.
Base model
google/gemma-2-2b-it
Training setup
- Fine-tuning:
lora - LoRA rank:
64 - LoRA alpha:
128 - LoRA dropout:
0.05 - Layer policy:
percents=[25, 50, 75] - Activation policy:
explicit - Training config:
training_config.yaml - OracleActivation git revision:
5d0dadf9153380a9646825d4c883cd0d245b1922
The exact training YAML is included in this repository. The adapter is intended to be loaded on top of the base model; it is not a standalone merged model.
Reference
Activation Oracles: https://arxiv.org/abs/2512.15674
Loading
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base_model = "google/gemma-2-2b-it"
adapter_id = "REPLACE_WITH_THIS_REPO_ID"
tokenizer = AutoTokenizer.from_pretrained(base_model)
model = AutoModelForCausalLM.from_pretrained(base_model)
model = PeftModel.from_pretrained(model, adapter_id)
Activation Oracle inference additionally requires injecting the source activation at the configured hook layer; loading the PEFT adapter alone does not reproduce the AO inference procedure.
- Downloads last month
- 11
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support