Hallucination Reduction with CASAL:  Contrastive Activation Steering for Amortized Learning
Authors: Wannan Yang
Xinchi Qiu
Lei Yu
Yuchen Zhang
Aobo Yang
Narine Kokhlikyan
Nicola Cancedda
Diego Garcia-Olano
Venue: ICLR 2026 Poster (2026)
Keywords: hallucination
representation learning
interpretability
finetuning
steering
Primary area: interpretability and explainable AI
TL;DR: Amortized activation steering effectively reduce hallucination in text-o multi-modal and Mixture-of-Expert models.

Abstract: Large Language Models (LLMs) exhibit impressive capabilities but often hallucinate, confidently providing incorrect answers instead of admitting ignorance. Prior work has shown that models encode linear representations of their own knowledge and that activation steering can reduce hallucinations. These approaches, however, require real-time monitoring and intervention during inference. We introduce Contrastive Activation Steering for Amortized Learning (CASAL), an efficient algorithm that connects interpretability with amortized optimization. CASAL directly bakes the benefits of activation steering into model's weights. Once trained, LLMs answer questions they know while abstaining from answering those they do not. CASAL's light-weight design requires training only a submodule of a single transformer layer and yet reduces hallucination by $\sim30\%$-$40 \%$ across multiple short-form QA benchmarks. CASAL is $\sim$30x more compute-efficient and $\sim$20x more data-efficient than strong LoRA-based baselines such as SFT and DPO, boosting its practical applicability in data scarce domains. Importantly, CASAL also generalizes effectively to out-of-distribution (OOD) domains. We showcase CASAL's flexibility in mitigating hallucinations in both text-only and vision-language models. To our knowledge, CASAL is the first steering-based training method that has been shown to be effective for both dense and Mixture-of-Experts (MoE) models. CASAL represents a promising step forward for applying interpretability-inspired method for practical deployment in production systems.

Published as a conference paper at ICLR 2026
HALLUCINATION REDUCTION WITH CASAL: CONTRASTIVE ACTIVATION STEERING FOR AMORTIZED LEARNING
Wannan (Winnie) Yang*.1.2, Xinchi Qiu Lei (Jade) Yu², Yuchen Zhang Aobo Yang",
Narine Kokhlikyan', Nicola Cancedda", Diego Garcia-Olano'
Meta Superintelligence Labs New York University Meta
"Work done at Meta
ABSTRACT
Large Language Models (LLMs) exhibit impressive capabilities but often halluci- nate, confidently providing incorrect answers instead of admitting ignorance, Prior work has shown that models encode linear representations of their own knowledge and that activation steering can reduce hallucinations. These approaches, however, require real-time monitoring and intervention during inference. We introduce Contrastive Activation Steering for Amortized Learning (CASAL), an efficient al- gorithm that connects interpretability with amortized optimization. CASAL directly bakes the benefits of activation steering into model's weights during training. Once trained, LLMs answer questions they know while abstaining from answering those they do not. CASAL's light-weight design requires training only sub-module of a single transformer layer and yet reduces hallucination by 30% 40% across multiple short-form QA benchmarks. CASAL is ~ 30x more compute-efficin 20x more data-efficient than strong LoRA-based baselines such as SFT, DPO and GRPO, boosting its practical applicability in data scarce domains. Importantly, CASAL also generalizes effectively to out-of-distribution (OOD) domains. We showcase CASAL's flexibility in mitigating hallucinations in both text-only and vision-language models. To our knowledge, CASAL is the first steering- training method that has been shown be effective for both dense and Mixture- of-Experts (MoE) architectures. CASAL represents a significant step forward for applying interpretability-nnsiii method for scalable and broad deployment production systems
and
INTRODUCTION
Large Language Models (LLMs) have demonstrated near-human even superhuman intellectual
capabilities (Brown et al.] 2020 Ouyang et al., 2022 OpenAl et al. 2024). Yet despite these successes, they sometimes fail in striking ways. A central failure mode is hallucination: the tendeney
to confidently generate false or unsupported information. Hallucinations undermine trust and restrict the safe deployment of LLMs in real-world settings where factual reliability is critical (Rawte et al.) 2023
Gekhman et al. 2024a Shen et al. 2025a).
features (Templeton et al. 2024 or residual stream activations (Rimsky et al. 2024 Turner et al. 2024)-have revealed that LLMs encode form of self-know celf-knowledge. Specifically, the activations associated with known versus unknown knowledge can be separated along linear directions (Ji et al. 2025 Ferrando et al. 2025). Moreover, steering these representations reduces overconfidence and enables models to acknowledee uncertainty. However, prior work primarily focuses inference-time interventions leaving significant gap in their practicality as part of scalable alignment pipelines.
Recent interpretability studies-using sparse autoencoder (SAE)
LLMSs internal states already reflect what is known versus unknown, why do they still produce confident but false answers? We hypothesize that a key cause lies in the training and evaluation

---

Published as a conference paper at ICLR 2026
(i) Knowledge Boundary Probing
Before CASAL
Known 2/10 Correct
Unknown W/oWgongg
WiattWhtti ts Provicencce the capital off
Usen Who the The Jadiet?
P OEIAEE @
Staaring
Response
Holludation
Steeting Vector
Assistant Phode bland
Nansy tadjedin
Target Activation
Activation
After CASAL
(xt +
(x)
+
Accuracy
Abstention
(II) CASAL Training
a
Asstitant: Phovidencels capital of US state
spechiiinfomaaion tebbout...
butt de not have
Assistanti apollggto.
CASAL Loss Function
Boon, |1A Golf
Singelayer NN
Bieo, [4 uof
Figure 1: Overview of the CASAL algorithm. (A) Knowledge Probing: CASAL starts by probing the model to figure out what it knows vs doesn't know. Multiple responses per query are sampled to classify queries as known (Dx) or unknown (Du). (B) Steering: Difference in means are computed to construct steering vectors (v and v ). Target activations ( ( and E are obtained by adding these steering vectors to the residual stream activation. Pre-CASAL Behavior: Prior to training, the model often hallucinates and produces incorrect answers for unknown queries. (C) CASAL Training: CASAL training is essentially "amortized activation steering". " where instead of repeatedly steering activations online, we train a small subnetwork (a single layer NN) approximate the steering solution offline. (D) Post-CASAL Activations and Behavior: After training, the model learns a sharper representation with a clearer knowledge boundary. It maintains correct answers on known queries while abstaining from answering unknown ones.
paradigm of LLMs (Li al., 2025a; Kalai et al. 2025). During pretraining, the language modeling objective rewards predicting the next token given the training corpus distribution, incentivizing plausible continuations even under uncertainty rather than expressions of ignorance. Post-training further amplifies this tendency: the training and evaluation framework optimizes models to be good test-takers, rewarding guessing over acknowledging uncertainty (Gekhman et al., 2024b).
In this work, we propose alternative training objective-one that leverages the model's own internal representations to align behavior with knowledge boundaries. Our core hypothesis is that if models trained to directly utilize their own representations of known and unknown, their generations will better reflect what they truly "know". Concretely, we replace the standard cross-entropy loss with local representation loss applied residual stream activations. Whereas cross-entropy loss provides a learning signal from external supervision (the training corpus), representation loss provides a learning signal from within: model's own hidden representation.
Importantly, CASAL is among the first approaches to rely solely on a representation-level objective for training LLMs. Prior studies such as RepE (Zou et al. (2025). ReFAT (Yu al. 2025), and others
(Yu et al. 2024a Casademunt et al. 2025 Chen et al. [222b]] Yousefpour et al. (2025) have explored representation-lese fine-tuning, but all employed representation losses as auxiliary signals alongside standard cross entropy loss. By contrast, CASAL treats representation loss as the only and the primary optimization objective, directly teaching the model to utilize its hidden representation.
Our approach connects insights from two fields: interpretability and amortized optimization. Amor- tized optimization (Kingma and Welling 2013 Rezende et al., 2014 Gershman and Goodman) 2014) is a paradigm where costly repeated optimizations are replaced by training parametric function that approximates the solution. CASAL instantiates this idea by incorporating activation steering into training: it "amortize" the activation steering process by training lightweight subnetwork that learns to approximate the steering solution, embedding the knowledge boundary directly into the model's weights.
2

---

Published as a conference paper at ICLR 2026
Algorithm I CASAL: Contrastive Activation Steering for Amortized Learning
Require: Dataset D; frozen model Moriginal with / layers; target layer L*; steering strength a; training epochs E
STEP 1: Knowledge boundary probing
known Bhown/unknown unknown
Set k 10, threshold 7 7
2 for "D do
3 Sample & responses (x)) -, correct]
if' s(x) then D& D [z}
"known" set Dx b "unknown" set Dn
5 else k: s(x) 2 then D. t D.U (z)
end if
7 end for
STEP 2: Steering
Note: a (x) denotes residual activations at layer I for input x
8 a = [ ELED, aL (I), a [Px] ErED, a² (x)
> mean activations
v. _ ag
v a a
steering vectors
10 t (x) = a (x) + v for E Du
b "abstain when you don't know" D "answer when you know"
th (x) = Li + v for E Dz
STEP 3: CASAL training
12 Initialize one-layer network Muai with weight WT. original
> one-layer fine-tuning
13 for e = 1...E do
Lu EEEDD 1 (x) a² (x)]1
"unknown" loss
Lx E, 1 (z) - a (x)]l
"known" loss
16
L Lu + Lxi update Musin weights by VC
end for
18 WASSLL <- trained weights from Mirin
extract trained weights > create output model
19 McASAL + Moriginal with W replaced by WLASAL at layer L* Trained model McASAL with updated weights at layer L
Ensure:
We highlight our main contributions as:
Effective Algorithm: Introducing training method inspired by interpretability findings and
amortized optimization. CASAL enables models to admit ignorance for unknown questions, reducing ballucination rates by ~ 30% 40% across multiple short-form QA benchmarks.
Efficiency Gains: CASAL's objective function enables local and lightweight parameter updates, delivering ~ 30x higher compute efficiency (FLOPs per token) and requires ~20x less training data (with little 640 training data) to achieve the same level of performance compared to LoRA-based SFT and DPO.
The trained model retains its general capabilities while avoiding excessive refusals. At the same time, it successfully generalizes refusal behavior unknown queries sampled from
Robust Generalization:
(OOD) data.
Versatility: CASAL training is modality-agnostic effectively mitigating hallucination in both text-only and multimodal models.
Broad Applicability: We present the first ever steering- based training framework with general applicability to both dense and cexture-of-Experee (MoE) models.
2 CASAL
We now introduce method. CASAL, which integrates insights from interpretability and amortized optimization to build a lightweight, efficient training framework. The full pipeline is shown in At a high level, CASAL can be understood an instance
Figure 1, summarized in Algorithm [
3

---

Published as a conference paper at ICLR 2026
HH-alluciaation Rate
30%
CASA
SFT
DPO
GAPD
Baseine
20%
10%
640
1200
2500
3940
5120
6400
7000
8900
10240
11520
12800
#a Training Examples
B
D Before
Logistic Deray
30%
Ha auluuiaation Rate
15%
SFT LoRA
20%
10%
LaRA FGRRO
Kroww
Follucination
20%
N=80
5%
CASAL
DPO FULL
Unknnan
10%
0%
0%
33%
100%
0%
N-00C
8yl8g 32p
64g
FLOPs Per Taken
0.4
Cluster Separation
Figure 2: CASAL is both sample efficient and compute efficient. (A-B) CASAL achieves strong hallucination reduction with orders-of ofcers-of-maae magnitude fewer training examples comparing to LoRA-based fine-tuning with SFT, DPO and GRPO. (C) CASAL is over 30x more compute-efficien than PEFT baselines such as LoRA. (D) Hallucination reduction after CASAL training correlates with improved cluster separation between known and unknown queries, measured by silhouette score.
of amortized optimization: instead of repeatedly solving the steering problem inference time, we train a parametric subnetwork to approximate this solution once, thereby "amortizing" the resource use of activation steering across all future queries. This perspective motivates the name: Contrastive Activation Steering for Amortized Learning (CASAL). CASAL proceeds in three stages:
2.1 STEP 1: KNOWLEDGE BOUNDARY PROBING
CASAL begins by probing the model to delineate knowledge boundary. For each input x D, we sample k = 10 completions and compare them to ground-truth answers. For each question, if at least 7 generations are correct, æ is labeled as known; if less than 3 are incorrect, it is labeled as unknown. This produces two subsets: Dj and D... We systematically evaluated different threshold values and found that ballucination reduction performance remains robust across this range (Appendix(]) []]. We adopt relatively strict threshold of 7 to ensure high-confidence separation: the model abstains only on knowledge it does not possess, and responds only when it demonstrates consistent correctness. This choice reduces ambiguous cases near the decision boundary. [ We evaluate CASAL three datasets: TriviaQA (Joshi et al. 2017b), PopQA (Mallen et al. 2023b), and EntityQA (Ferrando Dataset details provided in Appendix G.1
al. 2025).
2.2
STEP 2: STEERING
Next, CASAL constructs contrastive steering vectors to obtain better knowledge boundaries (Rimsky al. 2024 T Turner et al.) 2024 Arditi et al., 2024). For each query 2, we extract residual stream
activations (x) designated target layer from the last token position of the question We then compute mean activations for known and unknown subsets (at and a and construct steering vectors by taking difference means, resulting in two vectors: v!." for abstaining when the model lacks knowledge, and v! for reinforcing correct answering when the model does know. The steering vectors are then added to the residual stream activations, yielding target activations t and th These target activations cached and subsequently used to compute the representation loss in STEP 3. Further details for steering and target layer selection procedures are included in Appendix C
*Consistent with previous literature (Ferrando et al. 2025 Grattatiori et al. 2024). the knowledge probing step creates the known versus unknown labels subsequently used for steering training baseline methods such SFT and DPO, and therefore does introduce additional computational "y extracting activations from the last token position of the question, the steering vectors reflect properties of the question itself (whether it is known or unknown to the model) rather than features of the answer (whether the answer correct incorrect).
specific CASAI.
4

---

Published as a conference paper at ICLR 2026
2.3 STEP 3: CASAL TRAINING
Finally, CASAL trains a lightweight one-layer network Mrain initialized with the weight original from layer L of the original model. Using a mean squared error objective, CASAL minimizes the distance between current activation a (x) and its corresponding target activation (t or to After training, the learned weights CASAL are extracted from Musin and substituted back into layer L of the original model, producing the final model McASAL. This process embeds the knowledge boundary directly into the model weights, eliminating the need for repeated steering at inference. Importantly, this representation loss is the sole training objective, not used auxiliary loss with standard cross-entropy. Because this loss is local to layer L (derived directly from residual activations at that layer), we only need to train one single layer. This contrasts with cross-entropy loss, which requires forward pass through all layers to compute output probabilities. Even when updating only a single target layer with cross-entropy loss, the entire model (with other layers frozen) must be deployed during the forward pass, adding much more computational cost compared to training just the one-layer network Mraiin We conducted systematic ablation studies (Appendix K) to examine different fine-tuning strategies. Our results demonstrate that fine-tuning different submodules of the MLP layer yields no statistically significant performance differences. Further details for the training
process and hyperparameter research are included in Appendix D and L
3 CASAL IS EFFECTIVE AND EFFICIENT
We evaluate CASAL against strong baselines including Supervised Fine- Tin-TTning (SFT), Direct Preference Optimization (DPO) and Group Relative Policy Optimization (GRPO), which represent the predominant fine-tuning approaches deployed in production systems today (hyperparameters search and other training details are provided in Appendix M.. By demonstrating CASAL's superiority over these addely-ddopted techniques, we establish its practical applicability for real-world deployment beyond toy settings.
3.1 SAMPLE EFFICIENCY
We quantify hallucination reduction performance primarily using the hallucination rate, which captures the fraction of unknown queries incorrectly attempted by the model. Figure summarizes our key findings, with additional details the hallucination rate metric provided in Appendix H..2 CASAL achieves substantially lower hallucination rates across a wide range of training set sizes. When trained on just 640 examples, CASAL already matches or surpasses the performance of SFT, DPO and GRPO trained on 12,800 examples (Figure ABB) This translates into more than 20x higher data efficiency, demonstrating that CASAL is especially practical in data-scarce settings.
3.2 COMPUTE EFFICIENCY
Beyond sample efficiency, CASAL is also highly compute efficient. By updating only a lightweight sub-module within single transformer layer, CASAL is substantially more compute efficient than full fine-tuning or even LoRA- parameter-efffiient fine-tuning (PEFT). As shown in Figure 2 CASAL achieves lower hallucination rates while requiring over 30x fewer FLOPs per token than LoRA during training, underscoring its practicality for large-scale deployments. This efficiency stems from two key properties of CASAL's loss function:
Efficiency across model depth. Because CASAL's loss is local to layer L", both forward and backward passes operate exclusively within the single-layer network Muain- In contrast, methods using cross-entropy loss, even when updating only single layer with other layers frozen, must perform a forward pass through all layers end-to-end to compute output probabilities and backpropagate gradients from the output back to the target layer. For example, when fine-tuning layer 16 of 32-layer model, cross-entropy-basee methods require computations through 32 layers the forward pass and through 16 layers in the backward pass, while CASAL operates only on the target layer itself. This advantage scales with model depth: the deeper the model, the greater CASAL's computational savings.
Efficiency across generation length. CASAL computes loss at a single position the last token of the question. In contrast, SFT averages cross-entropy loss over all tokens in the generated answer,
5

---

Published as a conference paper at ICLR 2026
while DPO computes log- prgebabbbblities over all tokens in both chosen and rejected responses. The computational cost thus scales with answer length for these methods, whereas CASAL's cost remains constant regardless of generation length. Longer answers make CASAL increasingly cost-effective comparing to standard baselines Details of FLOPs calculations are included in Appendix N
3.3 LEARNING BETTER KNOWLEDGE BOUNDARIES
By training with a local representation loss, CASAL encourages clearer separation between activations corresponding to known and unknown queries. We compute Silhouette score measure of cluster separation. As shown in Figure 2p. Silhouette scores (H.4). increase as training progresses, and this separation is correlated with the reduction hallucination rate. The strong correspondence (logistic fit, R = 0.945) between representational separation and behavioral outcomes indicates that CASAL's effectiveness arises from more faithfully encoding and utilizing knowledge boundaries. Consistent with our hypothesis, CASAL demonstrates the best cluster separation and the clearest boundary between known and unknown queries compared to the other methods
( (Figure 22 Appendix Q This validates that by directly training local representation loss, CASAL effectively encourages a distinct separation between these activation states.
Methods
Refusal Rate (L)
Accuracy (1)
PopQA
TriviaQA
QA
PopQA
TriviaQA
EntityQA
Bascline
18.19% 12.9%++33. +3.01
7.93% 1.14
8.94%
91.08 % 16-+--2.23
95.82 % +2224
88.59% + 1.46
SFT
20.32% 1.09
10.01%11116
11.08%: 1.24 .24
1..9%%11.33
92.45% 1 1,29
85.75% 1.18
DPO
1..79%++1.11
24.67%
17. 17.66%
90.25%++1.00 1.06
95.90%
1.
GRPO CASAL
17.48%++-444
1..77%++3.82
4..67
85.78%+
91.67%:+2276
+3.48%
19.89%++1.15
1.29%11.3
6.84% 1.23
85.11% 1.88
89.90%+10.99 95.34%
Table 1: CASAL does not introduce over-refusal nor degrade performance for known queries. Refusal rate and accuracy across three different QA datasets are measured.
Methods
Accuracy (1)
Win rate (1)
MMLJ
GSM8K
GPQA (Reasoning)
MT Bench (Coherence)
(General)
(Math)
Baseline
68.01 0.34
77.48 + 1.15
33.31 + 0.34
7.38 + 0.06
SET
67.90 0.23
75.66 1.18
32.82 + 0.34
7.44 + 0.13
DPO
68.03 0.26
78. = 1.14
31.43 - 0.37
7.39 + 0.15
GRPO
67.73 + 0.38
76,66 = 1.21
31.92 = 0.22
7.44 + 0.11
CASAL
68.04 + 0.44
77.02 + 1.16
33.18 + 0.34
7.57 0.08
Table 2: CASAL preserves general capability, Performances (higher is better) on general capability, math, reasoning and context- aware conversational ability in multi-turn dialogues are measured.
4 CASAL PRESERVES MODEL CAPABILITY
An important requirement for any practically useful cettuciinatio-rrddeett method is that it should not degrade model's general capabilities nor induce excessive refusals on queries the model can correctly answer. We therefore evaluate CASAL across both refusal behavior and broad capability benchmarks. Table Tepport refusal rates on three QA datasets. CASAL achieves the lowest refusal rates on TriviaQA (7.29%) and EntityQA (6.84%) while maintaining competitive rate PopQA (19.89%). These results demonstrate that CASAL reduces hallucination on unknown queries without over-penalizing the model into unnecessary refusals for known ones. We also evaluate against
'The FLOPs comparison reported in this work is measured per token. This makes our estimate of CASAL's computational advantage (30x fewer FI.OPs than LoRA) conservative. For tasks requiring longer generations, CASAL's efficiency gains over SFT, DPO and GRPO would he substantially greater.

---

Published as a conference paper at ICLR 2026
Dataset
Methods
Hallucination Rate (Unknown) ())
Refusal Rate (Known) ())
Accuracy (Known) (1)
Train
Test
Train
Test
Train
Test
TriviaQA
Wiki 48.20% 1.34
Web
Wiki
Web
Wiki
Web
Baseline
50.74%::1.1
9.06%
7.93%:12.0
94.22% 1:44
95.82% 1:2.2
SFT
24.44% 1.65
35.44%::
14.77%:11.52
15. 15.10%
91.23%:10.54
90.26% 1.15
DPO
23.28%=1.02 1.02
33.77%
13.62%=#1108
16.33%:=1.119
90.23% .00
88.13%
GRPO
22.33% = 1.34
33.12%::11.88
15.99%
1.00
18.833
87.27% 1.05
CASAL
1.147%=1111
32.22%%:11.29
8.28%:::111
11.69%
92.03%
90.08%::11.3
PopQA
Group
Group 2
Group
Group 2 18.19%11144
Group 1 90.84%
Group 2 91.08% 1.992
Baseline
74.87%=2.92
1..5
18.95%
SFT
22.77%=1.08
24.02%++1111
14.04%++1 1.16
20, 1888--111000
85.01 00
85.74% 1 .60
DPO
1.338%=1.33
24.88%-+0.99
14.99%:
19.19%
84.66% 90
84.01%:+1111
GRPO
20.19%==105 24.22%
18.07%
21.10%
80.13%110.31
10.98%
CASAL
22.48% 1.45
23.42%=++1.94
13.97% 1.78
19. 1.000=11.300
85.23%:-0.86
84.27%:
Table 3: CASAL learns generalizable notion of known unknown, and can transfer between data sources within TriviaQA and generalize across groups within PopQA.
Contrastive Activation Addition (CAA), a popular inference-time steering method (Rimsky et al. 2024). As summarized in Section E while CASAL achieves comparable hallucination rates to CAA on unknown queries, it maintains performance on known queries, whereas CAA degrades accuracy for questions the model could previously answer correctly. This finding aligns with previous work (Durmus al., 22224] Chen al. 2025b) showing that inference-time steering can introduce undesirable side effects.
We further assess models' general capability, including MMLU (Hendrycks al. 2021 for general
knowledge, GSM8K (Cobbe et al., (2021) for math reasoning, GPQA(Rein et al. 2023) for scientific reasoning, and MT- (Zheng et al. (003) for coherence multi-turn conversations. As shown in T CASAL performs on par with strong baselines across all metries. Beyond these quantitative measures, we provide raw model outputs in Appendix (F allow readers to assess the natural flow and coherence of generated responses after CASAL training. 'Thes results demonstrate that CASAL reduces hallucinations on unknown queries while avoiding over-refusal on known queries, all without sacrificing general capability balance critical for practical deployment.
5 CASAL IS OOD GENERALIZABLE
Does CASAL capture a generalizable notion what the model knows versus does not know beyond its training distribution? We test its ability to generalize across both in-distribution and out-of- distribution (OOD) settings. We first evaluate whether CASAL's learned knowledge boundary transfers across different groups within the same dataset. As shown in Table 3 CASAL trained on Wikipedia-style data generalizes effectively to web data, reducing hallucination rate from 50.7% to 32.4% while maintaining high accuracy on known queries (92. 0% vs. 95.8%). similar trend is observed on PopQA (Table 3). where CASAL substantially reduces hallucinations in both Group I and Group 2, lowering test hallucination rates from 74.4% 23.4%. These results indicate that CASAL does not simply memorize steering directions but learns transferable notion of known versus unknown knowledge that holds across diverse data groups
We next evaluate stronger OOD setting: training CASAL on one dataset and testing it on a completely different one. Specifically, CASAL is trained on TriviaQA and evaluated on EntityQA [T. Remarkably, hallucination rate on the unseen EntityQA dataset drops from 50.7% to 11.7%, while accuracy on known queries remains above 95%. This demonstrates that CASAL's learned representations extend beyond the training domain, capturing knowledge boundaries that remain robust even under OOD transfer. Together, these results establish that CASAL generalizes well both across sub-groups within dataset and across entirely distinct datasets. This robustness highlights that CASAL not merely overfitting to a narrow training distribution but instead induces a broadly applicable mechanism for distinguishing known from unknown queries.
7

---

Published as a conference paper at ICLR 2026
Methods
Hallucination Rate (1)
Refusal Rate (1)
Accuracy (1)
Train
Test
Train
Test
Train
Test
TriviaQA
EntityQA
TriviaQA
EntityQA
TriviaQA
EntityQA
Baseline
48.2% 1.33
00.74%110.9
9.06% 1.22
12.89% 1.49
94.22% % 0.933
95.82% 1:2.32
SFT
10.53%--11.53
23.13%= 1.66
21.16%
19.84%11..4
88.80%
80.77%:+1.05
DPO
20.63%:12.22
18.30% -: 1.45
13.89%
22.02%::1..2
92.91 1.06
77,41%11122
GRPO
14.00 % 0.55
12.83% 1,188
21.43%
17.25%
25.69%18.77
85.24%
CASAL
18.23%-11.03
11.72% 1.66
9.29%=1.44
13.82%
93.36%
95.77%
Table 4: CASAL supports OOD generalization across different datasets. The model is trained on the TriviaQA dataset and tested on EntityQA as an out-of-distribution setting.
Methods
WorldCuisines Dataset
Unknown
Known
Hallucination Rate (1)
Refusal Rate ())
Accuracy (f)
Baseline
72.35%=1.77
13.91 1.337
76.72%++1.67
SFT
35.05%
24.33%
87.42%
DPO
36.44%=3.77
24.02%
86.66%+1.74
GRPO
35.19%
18.73%.1..7
80.18%
What is this dish known as in France?
CASAL
33.34% 1.3..3
25.44% 112.11
90.36% 1199
Methods
Landmark Dataset
Unknown
Known
Hallucination Rate (L)
Refusal Rate (L)
Accuracy (1)
Baseline
75.78% 1.69
1..73
90.80%
SFT
39.05%=4.07
2.64%:13.01
9..77%
Where is this place?
DPO
35.99%=1.00 1.03
1.06%
94.11%11.01
GRPO
35.44%=1.85 1.81
8.19%
90.75%
CASAL
31.25%--8..
3.12%++3.01
99%-+0.0
Table 5: CASAL is modality agnostic. It reduces hallucination in vision-language model on WorldCuisines- VQA (top) and Landmark-VOL VQA (bottom). Example question-iman pairs from the two datasets are shown the left.
CASAL IS MODALITY AND ARCHITECTURE AGNOSTIC
6.1 CASAL REDUCES HALLUCINATION IN VISION- MODELS
We apply CASAL vision-language model: Qwen2.5-VLL 1--5----77
7BLL-B-IA 7 Instruct (Qwen et al. 2024 and perform training on the WorldCuisines- VAAA (Winata et al. 2024) dataset. Finally, we evaluate whether CASAL generalizes beyond standard dense transformer architectures and text-only settings. CASAL reduces hallucination rate (Table 5) by 38% Importantly, accuracy on known queries preserved. This confirms that CASAL's mechanism for sharpening knowledge boundaries is tied to language-only models but extends naturally to multimodal models. Further details for training vision- language models are provided in Appendix O
6.2 CASAL REDUCES HALLUCINATION IN MIXTURE-OF OFFTURE-OF-EXPEEE EXPERTS MODELS
MoE models pose unique challenge since knowledge and uncertainty may be distributed across different experts. We first ask "how are unknown versus known queries represented across experts?" Are certain experts specialized in representing known and others specialized in unknown? Or are they co-represented in the same experts? We started our investigation by visualizing the activations in different experts in the OLMoE model (Muennighoff et al.] 2025). As illustrated in Figure T8A,
8

---

Published as a conference paper at ICLR 2026
activations for known and unknown queries mostly co-represented in the same experts. Similar to dense model training, CASAL applies a local representation loss on the residual stream activations with converging signal across all experts (Figure T8B). After training, residual stream activations show a much clearer boundary between known and unknown queries (Figure (8C), which translates into significant improvements in hallucination rates. Hallucination rate for unknown queries drops by 42.9%. while accuracy on known queries remains unchanged (Figure T8D). Further details regarding
the CASAL training for MoE models be found in Appendix P These results demonstrate that
CASAL effectivlly extends to MoE architectures without sacrificing accuracy. Together, these results establish that CASAL is both cethiteccuur-agnnsste and modality-agnostic Whether applied to dense or MoE transformers, or to text-only versus vision-language models, CASAL consistently reduces hallucination rates while maintaining high aceuracy and balanced refusal behavior. This broad applicability highlights CASAL's potential as scalable, general-purge purpose alignment technique.
MLP Activation
Beso, 1| al (x]]l Basn |t! at (x)l"
Expert
6
Expert
Expert 30
Expert
RESIDUAL STREAM ACTIVATION
Expert
Expert 43
Expert
Expert 48
Expert 57
04 Expets
M.P DOKAA
ALL DOW'
SEP COAR
MUPUP CEATE
Expert 54
Expert 62
Expert
SAATE
Unknown Known
Residual Stream Activation
Hallucination Rate
Unknown
Known
80%
77.3%
100%
Before CASAL
After CASAL
Accuracy
0%
Before
After
Before
After
Figure 3: CASAL architecture-agnostic.. effectively reduces hallucination for OLMoE. (A) Visualization of MLP activations from different experts in MoE model before CASAL training. (B) CASAL applies a local representation loss on residual stream activations. During training, weights are updated on only lightweight sub-module across experts. (C) Residual stream activations before and after CASAL training. (D) CASAL reduces hallucination rate on unknown queries while maintaining low refusal score and high accuracy for known queries.
RELATED WORK
7.1 HALLUCINATION MITIGATION
Inference-time Intervention. Steering-based approaches (Rimsky et al. 2024 Turner al. 2024
for hallucination reduction typically apply interventions during inference (Ferrando et al. 2025 Ji et al. 2025| Li et al. 2024 Park al. 2025). While effective, this requires solving a local optimization problem for every input (e.g.. shifting activations along direction at every forward pass), introducing extra computational overhead during deployment to monitor and intervene. In contrast, CASAL eliminates the need for per-instance intervention by directly baking the knowledge boundaries into model parameters, enabling scalable deployment in production.
In-weight Learning. A complementary body of work modifies model parameters to encourage calibrated abstention and reduce hallucination. Early approaches train models to abstain from uncertain predictions via probabilistic calibration. Others focus eliciting explicit confidence estimates in conversational models (Chen et al.) 2024; Mielke et al.] 2022). Concurrent work Chen et al. (2025b) proposes persona vector extraction, where finetuning steers models away from undesired persona directions. CASAL differs in two key ways: (i) rather than steering away from undesirable traits, explicitly steer towards desirable representations; and (ii) CASAL presents

---

Published as a conference paper at ICLR 2026
efficient training framework, yielding x30 higher compute efficiency than SOTA parameter-efficien finetuning methods such as LoRA.
7.2 AMORTIZED OPTIMIZATION, ACTIVATION STEERING AND REPRESENTATION LEARNING
Amortized Optimization. Amortized optimization (Kingma and Welling 2013 Rezende al. 2014; Gershman and Goodman 2014) is a widely used paradigm in which expensive, repeated optimization is replaced by training a parametric function that approximates the solution. Despite its influence in areas such as variational inference, sparse coding, gradient-based meta-learning and reinforcement learning (Amos) [025 Chen et al., 202T), this perspective has been explored less in the context of interpretability or alignment (Paulus et al.) 2025). CASAL can be viewed as amortized activation steering, where the resource intensive process of online steering is distilled into lightweight subnetwork trained offline and reused at inference.
Activation Steering. A line of work has focused on inference cetrencce-imeeeeee interventions, where steering vectors are applied dynamically to control model behavior without modifying weights (Ji et al. 2025 Li al. 2024). Within this paradigm, a common approach to derive steering vectors is to construct sample pairs differing along a target concept and compute their difference-in-meann (Arditi et al. 2024). Alternative methods further fine-tune the steering vectors to enable more effective behavior control with less side effect (Cao al., 2024 Stickland et al., [0224] Parekh et al., (025). Another line of work leverages sparse autoencoders (SAEs) to uncover interpretable features in an unsupervised manner, which can then serve as handles for steering interventions (Ferrando et al. 2025).
Representation Learning. A parallel line of work (Tian et al. 2025 Yu et al., 2024a Chen et al. 20256 Casademunt et al., (025) focuses on shaping internal representations during finetuning to suppress undesired behaviors. Early methods include representation fine-tuning (ReFT), which encourages task-specific interventions on hidden states (Wu et al. (024), and representation engi- neering (RepE), which monitors and manipulates high-level cognitive phenomena in LLMs (Zou) cct al. 2025). Other techniques explicitly control harmful states: Zou et al. (2024) introduce circuit breakers to block dangerous representations, while Yu et al. (2025) perform directional ablation of refusal features to maintain robustness under adversarial attacks. Similarly, Yousefpour et al. (2025) propose representation bending to disrupt harmful latent features. For unlearning, Shen et al. (2025b) train models to redirect unlearning data into refusal regions. Compared to these efforts, CASAL provides the first general steering-based training framework that is broadly applicable to both dense and sparse (MoE) architectures.
8 CONCLUSION AND LIMITATIONS
In this work, we introduced CASAL, a lightweight, effective, and broadly applicable method for reducing hallucinations in large language models. By embedding knowledge boundaries directly into model weights, CASAL achieves substantial reductions in hallucination without degrading general capabilities, while being markedly more compute- and data-efficient than standard baselines. Beyond its empirical results, CASAL provides initial evidence a broader principle: insights from interpretability can be distilled into training objectives that scale.
While CASAL shows strong effectiveness and efficiency, several limitations remain. First, although CASAL generalizes across short-form QA datasets, modalities, and architectures, its effectiveness in reasoning models remains to be systematically tested. Second, our evaluation focuses specifically on hallucinations short-form QA tasks. Exploring CASAL's effectiveness in reducing hallucinations during long-form generations (Obeso et al. 2025) represents important direction for future research. Finally, one particularly exciting future direction is the integration of CASAL into LLM- based agentic systems. As LLMs move toward becoming tool-using agents integrated into everyday workflooss their reliability becomes critical tangible consequences. While modern agents increasingly leverage external tools to address factual uncertainty, effective tool orchestration fundamentally depends on the agent's 's ability to recognize the boundaries of its own knowledge. CASAL's mechanism for sharpening these knowledge boundaries could therefore serve component for more reliable agentic systems, enabling agents to make better decisions about when to respond directly versus when to invoke tools such as web search or specialized knowledge bases.
confidence can lead to cascading errors with
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Brandon Amos. Tutorial on amortized optimization, 2025. URL
22C2. 00665
Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnce, and Neel Nanda. Refusal in language models is mediated by a single direction, 2024. URL https?
waaxiv..rgg/bs//2555.. 221.//206..11111
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Vosse Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners, 2020. URL https
7/arxiv. org. abs 2005. 14165
Yuanpu Cao, Tianrong Zhang, Bochuan Cao, Ziyi Yin, Lu Lin, Fenglong Ma, and Jinghui Chen. Personalized steering of large language models: Versatile steering vectors through bi-directional preference optimization. Advances in Neural Information Processing Systems, 37:49519-49551, 2024.
Helena Casademunt, Caden Juang. Adam Karvonen, Samuel Marks, Senthooran Rajamanoharan, and Neel Nanda. Steering out-of out- distrrbiibtt generalization with concept ablation fine-tuning, 2025. URL
org/ abs///bs/////// 16795
Lida Chen, Zujie Liang, Xintao Wang. Jiaqing Liang, Yanghua Xiao, Feng Wei, Jinglei Chen, Zhenghong Hao, Bing Han, and Wei Wang. Teaching large language models express knowledge boundary from their own signals, 2024. URL (tRthttps:/ http://xxxxvv...abbe..
10881
Lida Chen, Zujie Liang. Xintao Wang, Jiaqing Liang, Yanghua Xiao, Feng Wei. Jinglei Chen, Zhenghong Hao, Bing Han, and Wei Wang. Teaching large language models to express knowl- edge boundary from their own signals. In Yuji Zhang, Canyu Chen, Sha Li, Mor Geva, Chi Han, Xiaozhi Wang, Shangbin Feng, Silin Gao, Isabelle Augenstein, Mohit Bansal, Manling Li, and Heng Ji, editors, Proceedings of the 3rd Workshop on Towards Knowledgeable Foun- dation Models (KnowFM), pages 26-39, Vienna, Austria, August 2025a. Association for Com- putational Linguistics. ISBN 979-8-89176-283-1. doi: 10. 10.18633/112225 wtwp://22..oowwlmmmmm. URL
https::/acclammhooooyyyoo wwwp://ooo.o.....ooooooooooo.. kww.wlmmwwww............mmmmmmm.
Runjin Chen, Andy Arditi, Henry Sleight, Owain Evans, and Jack Lindsey. Persona vectors; Monitor- ing and controlling character traits in language models, 2025b. URL https://arxiv.orrg
abs/2507, 21509
Tianlong Chen, Xiaohan Chen, Wuyang Chen, Howard Heaton, Jialin Liu. Zhangyang Wang. and Wotao Yin. Learning to optimize: A primer and benchmark, 2021. URL htttttt/////att
htg/abs/2103. 12828
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems, 2021. httthhtts///ariive
abss21100.1 14168
Esin Durmus, Alex Tamkin, Jack Clark, Jerry Wei, Jonathan Mareus, Joshua Batson, Kunal Handa, Liane Lovitt, Meg Tong, Miles McCain, Oliver Rausch, Saffron Huang, Sam Bow- man, Stuart Ritchie, Tom Henighan, and Deep Ganguli. Evaluating feature steering: case study in mitigating social biases, 2024. URL
Javier Ferrando. Oscar Obeso, Senthooran Rajamanoharan, and Neel Nanda. Do i know this entity? knowledge awareness and hallucinations language models, 2025. URL https: // //arxiv org/abs/ aataab///11.1111 14257
11

---

Published as a conference paper at ICLR 2026
Zorik Gekhman, Gal Yona, Roee Aharoni, Matan Eyal, Amir Feder, Roi Reichart, and Jonathan Herzig. Does fine-tuning Ilms on new knowledge encourage hallucinations? arXiv preprint arXiv:2405.05994, 2024a.
Zorik Gekhman, Gal Yona, Roee Aharoni, Matan Eyal, Amir Feder, Roi Reichart, and Jonathan Herzig. Does fine-tuning Ilms on new knowledge encourage hallucinations?, 2024b. URL
attps://arxes org/ hts//aa//// https//205...500
Samuel J. Gershman and Noah D. Goodman. Amortized inference in probabilistic reasoning. Cognitive Science, 38(1):69- 1000 2014.
Aaron Grattatiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurelien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Roziere, Bethany Biron, Binh Tang. Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song. Danielle Pintz, Danny Livshits. Danny Wyatt, David Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Olano, Diego Perino, Dicuwke Hupkes, Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovic, Francisco Guzmán, Frank Zhang, Gabriel Synnaeve, Gabrielle Lee, Georgia Lewis Anderson, Govind Thattai, Graeme Nail, Gregoire Mialon, Guan Pang. Guillem Cucurell, Hailey Nguyen, Hannah Korevaar, Hu Xu, Hugo Touvron, Iliyan Zarov, Imanol Arrieta Ibarra, Isabel Kloumann, Ishan Misra, Ivan Evtimov, Jack Zhang, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Mahadeokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee. Jeremy Fu. Jianfeng Chi, Jianyu Huang, Jiawen Liu, Jie Wang, Jiecao Yu, Joanna Bitton, Joe Spisak, Jongsoo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe, Junteng Jia, Kalyan Vasuden Alwala, Karthik Prasad, Kartikeya Upasani, Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, Khalid El-Arini, Krithika Iyer, Kshitiz Malik, Kuenley Chiu, Kunal Bhalla, Kushal Lakhotia, Lauren Rantala-Yeary Laurens van der Maaten, Lawrence Chen, Liang Tan, Liz Jenkins, Louis Martin, Lovish Madaan, Lubo Malo, Lukas Blecher, Lukas Landzaat, Luke de Oliveira, Madeline Muzzi, Mahesh Pasupuleti, Mannat Singh, Manohar Paluri, Marcin Kardas, Maria Tsimpoukelli, Mathew Oldham, Mathieu Rita, Maya Pavlova, Melanic Kambadur, Mike Lewis, Min Si, Mitesh Kumar Singh, Mona Hassan, Naman Goyal, Narjes Torabi, Nikolay Bashlykov, Nikolay Bogoychev, Niladri Chatterji, Ning Zhang, Olivier Duchenne, Onur Çelebi, Patrick Alrassy, Pengchuan Zhang, Pengwei Li, Petar Vasic, Peter Weng, Prajjwal Bhargava, Pratik Dubal, Praveen Krishnan, Punit Singh Koura, Puxin Xu, Qing He, Qingxiao Dong, Ragavan Srinivasan, Raj Ganapathy, Ramon Calderer, Ricardo Silveira Cabral, Robert Stojnic, Roberta Raileanu, Rohan Maheswari, Rohit Girdhar, Rohit Patel, Romain Sauvestre, Ronnie Polidoro, Roshan Sumbaly, Ross Taylor, Ruan Silva, Rui Hou, Rui Wang. Saghar Hosseini, Sahana Chennabasappa. Sanjay Singh, Sean Bell, Seohyun Sonia Kim, Sergey Edunov, Shaoliang Nie, Sharan Narang, Sharath Raparthy, Sheng Shen, Shengye Wan, Shruti Bhosale, Shun Zhang, Simon Vandenhende, Soumya Batra, Spencer Whitman, Sten Sootla, Stephane Collot, Suchin Gururangan, Sydncy Borodinsky, Tamar Herman, Tara Fowler, Tarek Sheasha, Thomas Georgiou. Thomas Scialom, Tobias Speckbacher, Todor Mihaylov, Tong Xiao, Ujjwal Karn, Vedanuj Goswami, Vibhor Gupta, Vignesh Ramanathan, Viktor Kerkez, Vincent Gonguet, Virginie Do Vish Vogeti, Vítor Albiero, Vladan Petrovic, Weiwei Chu, Wenhan Xiong, Wenyin Fu, Whitney Meers, Xavier Martinet, Xiaodong Wang. Xiaofang Wang, Xiaoqing Ellen Tan, Xide Xia, Xinfeng Xie, Xuchao Jia, Xuewei Wang, Yaelle Goldschlag. Yashesh Gaur, Yasmine Babaei, Yi Wen, Yiwen Song, Yuchen Zhang, Yue Li, Yuning Mao, Zacharie Delpierre Coudert, Zheng Yan, Zhengxing Chen, Zoe Papakipos, Aaditya Singh, Aayushi Srivastava, Abha Jain, Adam Kelsey, Adam Shajnfeld, Adithya Gangidi, Adolfo Victoria, Ahuva Goldstand, Ajay Menon, Ajay Sharma, Alex Boesenberg, Alexci Baevski, Allie Feinstein, Amanda Kallet, Amit Sangani, Amos Teo, Anam Yunus, Andrei Lupu, Andres Alvarado, Andrew Caples, Andrew Gu, Andrew Ho, Andrew Poulton, Andrew Ryan, Ankit Ramchandani, Annie Dong, Annie Franco, Anuj Goyal, Aparajita Saraf, Arkabandhu Chowdhury, Ashley Gabriel, Ashwin Bharambe, Assaf Eisenman, Azadeh Yazdan, Beau James, Ben Maurer, Benjamin Leonhardi, Bernie Huang, Beth Loyd, Beto De Paola, Bhargavi Paranjape, Bing Liu, Bo Wu, Boyu Ni, Braden Hancock, Bram Wasti, Brandon Spence, Brani Stojkovic, Brian Gamido, Britt Montalvo, Carl Parker, Carly Burton, Catalina Mejia, Ce Liu, Changhan Wang, Changkyu Kim, Chao Zhou, Chester Hu, Ching-Hsiany Chu, Chris Cai,, Chris Tindal, Christoph Feichtenhofer,
12

---

Published as a conference paper at ICLR 2026
Cynthia Gao, Damon Civin, Dana Beaty, Daniel Kreymer, Daniel Li, David Adkins, David Xu, Davide Testuggine, Delia David, Devi Parikh, Diana Liskovich, Didem Foss, Dingkang Wang. Duc Le, Dustin Holland, Edward Dowling, Eissa Jamil, Elaine Montgomery. Eleonora Presani, Emily Hahn, Emily Wood, Eric-Tuan Le, Erik Brinkman, Esteban Arcaute, Evan Dunhar, Evan Smothers, Fei Sun, Felix Kreuk, Feng Tian, Filippos Kokkinos, Firat Ozgenel, Francesco Caggioni, Frank Kanayet, Frank Seide, Gabriela Medina Florez, Gabriella Schwarz, Gada Badeer, Georgia Swee, Gil Halpern, Grant Herman, Grigory Sizov, Guangyi, Zhang, Guna Lakshminarayanan, Hakan Inan, Hamid Shojanazeri, Han Zou, Hannah Wang, Hanwen Zha, Haroun Habeeb, Harrison Rudolph, Helen Suk, Henry Aspegren, Hunter Goldman, Hongyuan Zhan, Ibrahim Damlaj, Igor Molybog, Igor Tufanov, Ilias Leontiadis, Irina-Elena Veliche, Itai Gat, Jake Weissman, James Geboski, James Kohli, Janice Lam, Japhet Asher, Jean-Baptiste Gaya, Jeff Marcus, Jeff Tang, Jennifer Chan, Jenny Zhen, Jeremy Reizenstein, Jeremy Teboul, Jessica Zhong, Jian Jin, Jingyi Yang, Joe Cummings, Jon Carvill, Jon Shepard. Jonathan McPhie, Jonathan Torres, Josh Ginsburg, Junjie Wang. Kai Wu, Kam Hou U, Karan Saxena, Kartikay Khandelwal, Katayoun Zand, Kathy Matosich, Kaushik Veeraraghavan, Kelly Michelena, Keqian Li, Kiran Jagadeesh, Kun Huang, Kunal Chawla, Kyle Huang, Lailin Chen, Lakshya Garg, Lavender A, Leandro Silva, Lee Bell, Lei Zhang, Liangpeng Guo, Licheng Yu, Liron Moshkovich, Luca Wehrstedt, Madian Khabsa, Manav Avalani, Manish Bhatt, Martynas Mankus, Matan Hasson, Matthew Lennie, Matthias Reso, Maxim Groshev, Maxim Naumov, Maya Lathi, Meghan Keneally, Miao Liu, Michael L. Seltzer, Michal Valko, Michelle Restrepo. Mihir Patel, Mik Vyatskov, Mikayel Samvelyan, Mike Clark. Mike Macey, Mike Wang. Miquel Jubert Hermoso, Mo Metanat, Mohammad Rastegari, Munish Bansal, Nandhini Santhanam, Natascha Parks, Natasha White, Navyata Bawa, Nayan Singhal, Nick Egebo, Nicolas Usunier, Nikhil Mehta, Nikolay Pavlovich Laptev, Ning Dong, Norman Cheng, Oleg Chernoguz, Olivia Hart, Omkar Salpekar, Ozlem Kalinli, Parkin Kent, Parth Parekh, Paul Saab, Pavan Balaji, Pedro Rittner, Philip Bontrager, Pierre Roux, Piotr Dollar, Polina Zvyagina, Prashant Ratanchandani, Pritish Yuvraj, Qian Liang, Rachad Alao, Rachel Rodriguez, Rafi Ayub, Raghotham Murthy, Raghu Nayani, Rahul Mitra, Rangaprabhu Parthasarathy, Raymond Li, Rebekkah Hogan, Robin Battey, Rocky Wang, Russ Howes, Ruty Rinott, Sachin Mehta, Sachin Siby, Sai Jayesh Bondu, Samyak Datta, Sara Chugh, Sara Hunt, Sargun Dhillon, Sasha Sidorov, Satadru Pan, Saurabh Mahajan, Saurabh Verma, Seiji Yamamoto, Sharadh Ramaswamy, Shaun Lindsay, Shaun Lindsay, Sheng Feng. Shenghao Lin, Shengxin Cindy Zha, Shishir Patil, Shiva Shankar, Shuqiang Zhang, Shuqiang Zhang, Sinong Wang, Sneha Agarwal, Soji Sajuyigbe, Soumith Chintala, Stephanic Max, Stephen Chen, Steve Kehoe, Steve Satterfield, Sudarshan Govindaprasad, Sumit Gupta, Summer Deng, Sungmin Cho, Sunny Virk, Suraj Subramanian, Sy Choudhury, Sydney Goldman, Tal Remez, Tamar Glaser, Tamara Best, Thilo Koehler, Thomas Robinson, Tianhe Li, Tianjun Zhang, Tim Matthews, Timothy Chou, Tzook Shaked, Varun Vontimitta, Victoria Ajayi, Victoria Montanez, Vijai Mohan, Vinay Satish Kumar, Vishal Mangla, Vlad lonescu, Vlad Poenaru, Vlad Tiberiu Mihailescu, Vladimir Ivanov, Wei Li, Wenchen Wang. Wenwen Jiang, Wes Bouaziz, Will Constable, Xiaocheng Tang. Xiaojian Wu, Xiaolan Wang, Xilun Wu, Xinbo Gao, Yaniv Kleinman, Yanjun Chen, Ye Hu, Ye Jia, Ye Qi, Yenda Li, Yilin Zhang, Ying Zhang, Yossi Adi, Youngjin Nam, Yu, Wang, Yu Zhao, Yuchen Hao, Yundi Qian, Yunlu Li, Yuzi He, Zach Rait, Zachary Vito, Zef Rosnbrick, Zhaoduo Wen, Zhenyu Yang, Zhiwei Zhao, and Zhiyu Ma. The llama 3 herd of models, 2024. htLhttts:: petettss//arxiiie /aaa/iiivvv www.xxv.rrg//bbaa.
21783
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding, 2021. URL (ttthtts::// //axxix org/ abs/ 2009.000..00 03300
org//bs//
Gary B. Huang, Manu Ramesh, Tamara Berg, and Erik Learned- Miller. Labeled faces in the wild: database for studying face recognition in unconstrained environments. Technical Report Technical Report 07-49, University of Massachusetts, Amherst, October 2007.
Ziwei Ji, Lei Yu, Yeskendir Koishekenov, Yejin Bang, Anthony Hartshorn, Alan Schelten, Cheng Zhang, Pascale Fung, and Nicola Cancedda. Calibrating verbal uncertainty as linear feature to reduce ballucinations, 2025. URL https://aaxiv..rrg
Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer. Triviaqu: A large scale distantly supervised challenge dataset for reading comprehension, 2017a. URL https: // arxiiv.og// abs /0///770.. 03551
13

---

Published as a conference paper at ICLR 2026
Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension, 2017b. URL
abs/1705. 03551
Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, Scott Johnston, Sheer El-Showk, Andy Jones, Nelson Elhage, Tristan Hume, Anna Chen, Yuntao Bai, Sam Bowman, Stanislav Fort, Deep Ganguli, Danny Hernandez, Josh Jacobson, Jackson Kernion, Shauna Kravec, Liane Lovitt, Kamal Ndousse, Catherine Olsson, Sam Ringer, Dario Amodei, Tom Brown, Jack Clark, Nicholas Joseph, Ben Mann, Sam McCandlish, Chris Olah, and Jared Kaplan. Language models (mostly) know what they know, 2022. BRRLhtps:: /tttpp://arxiv.. wtt.i.iiiiiv.//...//...a org/ /207 05221
Adam Tauman Kalai, Ofir Nachum, Santosh S. Vempala, and Edwin Zhang. Why language models hallucinate, 2025. URL/attps://axxxv..o..
abs / 2509. 04664
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models, 2020. URL httpaaxivvvvvoo.../.aaa org/ //bg//b///// 2001000000000 08361
Diederik P. Kingma and Max Welling.
Auto-encoding variational bayes.
arXiv preprint
arXiv:1312.00 14, 2013.
Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. Inference-time intervention: Eliciting truthful answers from a language model, 2024. URL htttt:////aaxiii org/abs/2305 03341
Moxin Li, Yong Zhao, Wenxuan Zhang, Shuaiyi Li, Wenya Xie, See-Kiong Ng, Tat-Seng Chua, and Yang Deng. Knowledge boundary of large language models: A survey. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025), page- -, 2025a.
Moxin Li, Yong Zhao, Wenxuan Zhang, Shuaiyi Li, Wenya Xie, See-Kiong Ng, Tat-Seng Chua, and Yang Deng. Knowledge boundary of large language models: survey. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025), page -, 2025b.
Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. When not to trust language models: Investigating effectiveness of parametric and non-parametric memories, 2023a. URL/attps: http://axwviv.aa.aaaaao/// abs 2ttp.//222...1111.1
Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. When not to trust language models: Investigating effectivenesss of parametric and non-parametric memories, 2023b. URL
wrg. abs 22w.. 10511
Kenneth Marino, Mohammad Rastegari, Ali Farhadi, and Roozbeh Mottaghi. OK-VQA: A visual question answering benchmark requiring external knowledge. CoRR, habb///90.00006, 2019, URL
00067
Sabrina J. Mielke, Arthur Szlam, Emily Dinan, and Y-Lan Boureau. Reducing conversational agents' overconfidence through linguistic calibration, 2022. URL
14983
Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. Linguistic regularities in continuous space word representations. In Lucy Vanderwende, Hal Daumé III, and Katrin Kirchhoff, editors, Proceedings of the 2013 Conference of the North American Chapter of the Association for Computationnll Lin- guistics: Human Language Technologies, pages 146-751, Atlanta, Georgia, June 2013. Association for Computational Linguistics. htRLhttss:// https/////laaatthoooooooo wwwph/owwwoooo....ogooeoee org org/NN3-1190 1090
Niklas Muennighoff, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Jacob Morrison, Sewon Min, Weijia Shi, Pete Walsh, Oyvind Tafjord, Nathan Lambert, Yuling Gu, Shane Arora, Akshita Bhagia, Dustin Schwenk, David Wadden, Alexander Wettig, Binyuan Hui. Tim Dettmers, Douwe Kiela, Ali Farhadi, Noah A. Smith, Pang Wei Koh, Amanpreet Singh, and Hannaneh Hajishirzi. Olmoe: Open oixtureoofeexperts language models, 2025. URL httpstts:: /tt.xvivvvv./..////aa....aa.aaaa hhttpss//arxii..orr//a//////////
2200//9.a////..../// 02060
14

---

Published as a conference paper at ICLR 2026
Neel Nanda, Andrew Lee, and Martin Wattenberg. Emergent linear representations in world models of sueer-sueeee sequence models. In Yonatan Belinkov, Sophie Hao, Jaap Jumelet, Najoung Kim, Arya McCarthy, and Hosein Mohebbi, editors, Proceedings of the 6th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NL.P, pages 16-30, Singapore, December 2023. Association for Computational Linguistics. doi: 10. 10.18863/v11 http:////22223.2bkooooooo 1.2. URLhttps https
/ //aclantholoyy.oo wtg. 2023 www.k//..acccbbbx....o
Oscar Obeso, Andy Arditi, Javier Ferrando, Joshua Freeman, Cameron Holmes, and Neel Nanda. Real-time detection of hallucinated entities in long-form generation, 2025. URL https://
03531
OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Iige Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, Irwan Bello, Jake Berdine, Gabriel Bermadett-Shaping Christopher Berner, Lenny Bogdonoff, Oleg Boiko, Madelaine Boyd, Anna-L.uis Brakman, Greg Brockman, Tim Brooks, Miles Brundage, Kevin Button, Trevor Cai, Rosie Campbell, Andrew Cann, Brittany Carcy, Chelsea Carlson, Rory Carmichael, Brooke Chan, Che Chang. Fotis Chantzis, Derek Chen, Sully Chen, Ruby Chen, Jason Chen, Mark Chen, Ben Chess, Chester Cho, Casey Chu, Hyung Won Chung, Dave Cummings, Jeremiah Currier, Yunxing Dai, Cory Decareaux, Thomas Degry, Noah Deutsch, Damien Deville, Arka Dhar, David Dohan, Steve Dowling, Sheila Dunning, Adrien Ecoffet, Atty Eleti, Tyna Eloundou, David Farhi, Liam Fedus, Niko Felix, Simón Posada Fishman, Juston Forte, Isabella Fulford, Leo Gao, Elie Georges, Christian Gibson, Vik Goel, Tarun Gogineni, Gabriel Goh, Rapha Gontijo-Lpees, Jonathan Gordon, Morgan Grafstein, Scott Gray, Ryan Greene, Joshua Gross, Shixiang Shane Gu, Yufei Guo. Chris Hallacy, Jesse Han, Jeff Harris, Yuchen He, Mike Heaton, Johannes Heidecke, Chris Hesse, Alan Hickey, Wade Hickey, Peter Hoeschele, Brandon Houghton, Kenny Hsu, Shengli Hu, Xin Hu, Joost Huizinga, Shantanu Jain, Shawn Jain, Joanne Jang, Angela Jiang, Roger Jiang. Haozhun Jin, Denny Jin, Shino Jomoto, Billie Jonn, Heewoo Jun, Tomer Kaftan, Lukasz Kaiser, Ali Kamali, Ingmar Kanitscheider, Nitish Shirish Keskar, Tabarak Khan, Logan Kilpatrick, Jong Wook Kim, Christina Kim, Yongjik Kim, Jan Hendrik Kirchner, Jamie Kiros, Matt Knight, Daniel Kokotajlo, Lukasz Kondraciuk, Andrew Kondrich, Aris Konstantinidis, Kyle Kosic, Gretchen Krueger. Vishal Kuo, Michael Lampe, Ikai Lan. Teddy Lee, Jan Leike, Jade Leung, Daniel Levy, Chak Ming Li, Rachel Lim, Molly Lin, Stephanie Lin, Mateusz Litwin, Theresa Lopez, Ryan Lowe, Patricia Lue, Anna Makanju, Kim Malfacini, Sam Manning. Todor Markov, Yaniv Markovski. Bianca Martin, Katie Mayer, Andrew Mayne, Bob McGrew, Scott Mayer McKinney, Christine McLeavey, Paul McMillan, Jake McNeil, David Medina, Aalok Mehta, Jacob Menick, Luke Metz, Andrey Mishchenko, Pamela Mishkin, Vinnie Monaco, Evan Morikawa, Daniel Mossing, Tong Mu, Mira Murati, Oleg Murk, David Mély, Ashvin Nair, Reicchir Nakano, Rajeev Nayak, Arvind Neelakantan. Richard Ngo Hyeonwoo Noh, Long Ouyang, Cullen O' Keefe, Jakub Pachocki, Alex Paino, Joe Palermo, Ashley Pantuliano, Giambattista Parascandolo, Joel Parish, Emy Parparita, Alex Passos, Mikhail Pavlov, Andrew Peng. Adam Perelman, Filipe de Avila Belbute Peres, Michael Petrov, Henrique Ponde de Oliveira Pinto, Michael, Pokorny, Michelle Pokrass, Vitchyr H. Pong. Tolly Powell, Alethea Power, Boris Power, Elizabeth Proehl, Raul Puri, Alec Radford, Jack Rae, Aditya Ramesh, Cameron Raymond, Francis Real, Kendra Rimbach, Carl Ross, Bob Rotsted, Henri Roussez, Nick Ryder, Mario Saltarelli, Ted Sanders, Shibani Santurkar, Girish Sastry, Heather Schmidt, David Schnurr, John Schulman, Daniel Selsam, Kyla Sheppard, Toki Sherbakov, Jessica Shieh, Sarah Shoker, Pranav Shyam, Szymon Sidor, Eric Sigler, Maddie Simens, Jordan Sitkin, Katarina Slama, Ian Sohl, Benjamin Sokolowsky, Yang Song. Natalie Staudacher. Felipe Petroski Such. Natalie Summers, Ilya Sutskever, Jie Tang, Nikolas Tezak, Madelcine B. Thompson, Phil Tillet, Amin Tootoonchian. Elizabeth Tseng, Preston Tuggle, Nick Turley, Jerry Tworek, Juan Felipe Cerón Uribe, Andrea Vallone, Arun Vijayvergiya, Chelsea Voss. Carroll Wainwright, Justin Jay Wang, Alvin Wang, Ben Wang, Jonathan Ward, Jason Wei, CJ Weinmann, Akila Welihinda, Peter Welinder, Jiayi Weng, Lilian Weng, Matt Wiethoff, Dave Willner, Clemens Winter, Samuel Wolrich, Hannah Wong, Lauren Workman, Sherwin Wu, Jeff Wu, Michael Wu, Kai Xiao, Tao Xu, Sarah Yoo, Kevin Yu, Qiming Yuan, Wojeiech Zaremba, Rowan Zellers, Chong Zhang, Marvin Zhang, Shengjia Zhao Tianhao Zheng, Juntang Zhuang, William Zhuk, and Barret Zoph. Gpt-4 technical report, 2024. URL https:// org 2303 230030077477443 08774
15

---

Published as a conference paper at ICLR 2026
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback, 2022. URL
htt.ts://arxiv.. wwwparwwww..o.aaaooaoa www./r/////.//eeeoeee///eeeeee/e /203 02155
Gerry Pallier, Rebecca Wilkinson, Vanessa Danthiir. Sabina Kleitman, Goran Knezevic, Lazar Stankov, and Richard D. Roberts. The role of individual differences in the accuracy of confidence judgments. The Journal of General Psychology, 129(3):257-299, July 2002. doi: 10.1080/
(02213002006000999
Jayneel Parekh, Pegah Khayatan, Mustafa Shukor, Amaud Dapogny, Alasdair Newson, and Matthicu Cord. Learning to steer: Input-dependent steering for multimodal llms. arXiv preprint arXiv:2508. / 12815, 2025.
Kiho Park, Yo Joong Choe, and Victor Veitch. The linear representation hypothssii and the geometry of large language models, 2023. URL htt.atts///axx...//////bbbb//
03658
Seongheon Park, Xuefeng Du, Min- Yeh, Haobo Wang, and Yixuan Li. Steer 1lm latents for hallucination detection, 2025. URL
Anselm Paulus, Arman Zharmagambetov, Chuan Guo, Brandon Amos, and Yuandong Tian. Ad- yprompter: Fast adaptive adversarial prompting for lims, 2025. URL httph:///xxx.y....
abs/ abs//404. 16873
Qwen, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang. Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report, 2025. URL httts::/aaxii: /ttpv//arivvv../mm /rg/ abs 2412.15115
A Yang Qwen, Baosong Yang, B Zhang, B Hui, B Zheng, B Yu, Chengpeng Li, D Liu, F Huang, H Wei, et al. Qwen2. 5 technical report. arXiv preprint, 2024.
Vipula Rawte, Amit Sheth, and Amitava Das. A survey of hallucination in large foundation models. arXiv preprint 23 05922, 2023.
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. Gpqa: A graduate-level google-prool qa benchmark, 2023. URL httpstss: arxiv. http://awwwv.rrr/../a.// abs /211. 12022]
Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and approximate inference in deep generative models. In Proceedings of the 31st International Conference on Machine Learning, pages 1278-1286, 2014.
Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Turner. Steering llaaa via contrastive activation addition. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15504-15522, Bangkok, Thailand, August 2024. U
Association for Computational Linguistics. LinggiattiiiaaaLiiiiiiisiiioo doi: 10.1100..0 //aclaattholooyy.crr//22222222.2
828/]
Peter J Rousseeuw. Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. Journal of Computational and Applied Mathematics, 20:53-65, 1987.
William Shen, Xinchi Qiu, Nicola Cancedda, and Nicholas Lane. Don't make it up: Preserving ignorance awareness in Ilm fine-tuning, arXiv preprint arXiv:2506. /4387, 2025a.
William F. Shen, Xinchi Qiu, Meghdad Kurmanji, Alex Iacob, Lorenzo Sani, Yihong Chen, Nicola Cancedda, and Nicholas D. Lane. Lunar: Llm unlearning via neural activation redirection, 2025b.
//arxi/avvvvvv/ org/ htrrg/bbs255000 07218
16

---

Published as a conference paper at ICLR 2026
Lazar Stankov and John D. Crawford. Confidence judgments in studies of individual differences. Personality and Individual Differences, 21(6))971-988 986, 1996. ISSN 0191-8869. doi: 10.1016/ 80191-88699965000300
Asa Cooper Stickland, Alexander Lyzhov, Jacob Pfau, Salsabila Mahdi, and Samuel R Bowman. Steering without side effects: Improving post-deployment control of language models. arXiv preprint arXiv:2406. 15511888..1,8 2024.
Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas L Turner, Callum McDougall, Monte MacDiarmid, Alex Tamkin, Esin Durmus, Tristan Hume, Francesco Mosconi, C. Daniel Freeman, Theodore R. Sumers, Edward Rees, Joshua Batson, Adam Jermyn, Shan Carter, Chris Olah, and Tom Henighan. Scaling monosemanticity: Extracting interpretable features from claude sonnet, 2024. https://t/////..tt transrannsss fo rme r-ci rcuits ts mubb/024// sca htt.aaaaoooooooooooonnninootteie
htmll Accessed: 2025-09-22.
Bowei Tian, Xuntao Lyu, Meng Liu, Hongyi Wang, and Ang Li. Why representation engineering works: A theoretical and empirical study in vision-language models, 2025. URL https://
arxiv. wrg/absrgbbsg... 2503 10.5533.2200 22720
Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J. Vazquez, Ulisse Mini, and Monte MacDiarmi.. Steering language models with activation engineering, 2024. URL
https://arxixs ptg/abs//...............ge./.. 2308 10248
Jason Wei, Nguyen Karina, Hyung Won Chung, Yunxin Joy Jiao, Spencer Papay, Amelia Glaese, John Schulman, and William Fedus. Mcasuring short- form factuality in large language models, 2024. URL/attps://arx.tt http:////arrarroooo///////4..... http:/aaa/2211..44555
Tobias Weyand, Andre Araujo, Bingyi Cao, and Jack Sim. Google landmarks dataset v2 -2-22 a large- scale benchmark for instance-level recognition and retrieval, 2020. URL https://arxiy
org/abs/200 2000.,220...00 01804
Genta Indra Winata, Frederikus Hudi, Patrick Amadeus Irawan, David Anugraha, Rifki Afina Putri, Yutong Wang, Adam Nohejl, Ubaidillah Ariq Prathama, Nedjma Ousidhoum, Afifa Amriani, et al. Worldcuisines: A massive-scale benchmark for multilingual and multicultural visual question answering on global cuisines. arXiv preprini arXiv:2410. 12705, 2024.
Zhengxuan Wu, Aryaman Arora, Zheng Wang, Atticus Geiger, Dan Jurafsky, Christopher Manning, and Christopher Potts Reft: Representation finetuning for language models, 2024. URL https
03592
Wannan Yang and György Buzsáki. Interpretabillity of LLM deception: Universal motif. In ICLR 2025 Conference, 2025. URL /ttp:::s///ppeneeieeee net foo 9ymoi] Submit- ted Sept 28, 2024; Last modified Feb 5, 2025; ICLR 2025 submission, under review.
Zhangyue Yin, Qiushi Sun, Qipeng Guo, Jiawen Wu, Xipeng Qiu, and Xuanjing Huang. Do large language models know what they don't know? In Findings of the Association for Computational Linguistics (ACL 20233), pages 8653 8665, 2023a.
Zhangyue Yin, Qiushi Sun, Qipeng Guo, Jiawen Wu, Xipeng Qiu, and Xuanjing Huang. Do large language models know what they don't know?, 2023b. URL httpp:://arxa http://xxvvv.oo/ org. org/ /bs/ 2305. 18153
Gal Yona, Roee Aharoni, and Mor Geva. Can large language models faithfully express their intrinsic uncertainty in words?, 2024. URL WtRL:ttts:://// www.aaiwwww.........aao 2401//6000011888888 16908
Ashkan Yousefpour, Tacheon Kim, Ryan S. Kwon, Seungbeen Lee, Wonje Jeung, Seungju Han, Alvin Wan, Harrison Ngan, Youngjae Yu, and Jonghyun Choi. Representation bending for large language model safety, 2025.
//arxiv...iivvaa awppg/2b//200... 51550
Lei Yu, Meng Cao, Jackie Chi Kit Cheung, and Yue Dong. Mechanistic understanding and mitigation of language model non- factual hallucinations. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 7943-7956, 2024a.
17

---

Published as a conference paper at ICLR 2026
Lei Yu, Meng Cao, Jackie Chi Kit Cheung, and Yue Dong. Mechanistic understanding and mitigation of language model non non-factual hallucinations. arXiv preprint arXiv:2403. 2024b. doi: 14485500/r2X1.2240.181....
Lei Yu, Virginie Do, Karen Hambardzumyan, and Nicola Cancedda. Robust Ilm safeguarding via refusal feature adversarial training, 2025. /tsttt/:///aiii abs 240. 20089,
Anqi Zhang, Yulin Chen, Jane Pan, Chen Zhao, Aurojit Panda, Jinyang Li, and He He. Reasoning models know when they'r re right: Probing hidden states for sefffveeriication, 2025. URL https
wwaaxii.rgg.as///./... /2504 05419
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing. Hao Zhang. Joseph E. Gonzalez, and lon Stoica. Judging llm-as-a-judge with mt-bench and chatbot arena, 2023. URL httptttss///axx..oppp abs /206.2366..o 05685
Andy Zou, Long Phan, Justin Wang, Derek Duenas, Maxwell Lin, Maksym Andriushchenko, Rowan Wang, Zico Kolter, Matt Fredrikson, and Dan Hendrycks. Improving alignment and robustness with circuit breakers, 2024. URL
org. abs /et.ab///45... 04313
Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J. Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, Zico Kolter, and Dan Hendrycks. Representation engineering: A top-down approach to ai transparency, 2025. /tt/:tttsee//aaai...rrr/aaa/////
9 ACKNOWLEDGEMENT
We also thank Nathaniel Li, Irene Zhang, Julian Coda-Forno, Sriyash Poddar, Anselm Paulus, Sai Surya Duvvuri, Rachit Bansal, Devvrit Khatri, Ellie Pavlick and Jojo Yang for providing thoughtful feedback and insightful discussions on the manuscript.
18

---

Published as a conference paper at ICLR 2026
APPENDIX
Table of Contents
Further Discussion on Related Work
A.1
Knowledge Representation and the Linear Representation Hypothesis)
Further Discussion on Amortized Optimization
21
Steering
Steering Vector Construction
Layer Selection
D CASAL Training
X
D.I
Relationship between Activation Steering and CASAL Training
D.2 Weight Update before and after CASAL
24
Contrastive Activation Addition (CAA) VS CASAL
25
Example Model Outputs
26
Dataset
3
G.1
Entity Dataset
G.2
TriviaQA Dataset
G.3
PopQA Dataset
G.4
WorldCuisines Dataset
H
Metrics for Performance and Cluster Separation
35
Refusal Rate
H.2
Hallucination Rate
35
H.3
Accuracy
35
H.4
Silhouette Score
Knowledge Probing
37
Knowledge Probing Threshold
39
Models
40
Ablation
K.1
Sub-module for training
Hy Hyper-par Search for CASAL Training
Learning Rate
Steering Strength
19

---

Published as a conference paper at ICLR 2026
M
SFT and DPO Training
43
Compute Cost of Calculation (FLOPs per Token)
N.1
Full-parameter finetuning
N.2
Comparing full-paramete finetune and CASAL finetune
N.3
LoRA linetuning
................................
N.4
Comparing full parameeree finetune and LoRA finetune
Multimodal Model
P
Mixture-of-Experts (MoE) Training
50
P.
PCA Activation Across Experts
52
PCA Activations After Different Training Methods
Computational Requirements
The Use Large Language Models
20

---

Published as a conference paper at ICLR 2026
A FURTHER DISCUSSION ON RELATED WORK
A.1 KNOWLEDGE REPRESENTATION AND THE LINEAR REPRESENTATION HYPOTHESIS
Humans often display systematic overconfidence: their subjective confidence often exceeds objective
accuracy (Pallier et al., 2002 Stankov and Crawford 1996). Large language models (LLMs) exhibit similar pattern: they are poorly calibrated on general knowledge tasks, frequently producing answers
with misplaced confidence (Kadavath et al. 2022 Yin al. 2023b Yona et al. 2024 Zhang et al. 2025).
Recent interpretability studies (using sparse autoencoder (SAE) features (Ferrando et al. 2025) or
residual stream activations (Ji ct al. 2025)) suggest that transformer models encode many abstract concepts linear directions in activation space (Nanda et al. 2023 Mikolov al. [013 Park et al., 2023 Arditi et al. 2024) Yang and Buzsáki 2025). Behavioral traits such as truthfulness, sycophancy, refusal (Arditi al. 2024), and reasoning strategies have shown be linearly represented. Emerging evidence indicates that models may also possess intrinsic linear representations of knowledge bound- ary (Ferrando al. 2025)) and uncertainty (Ji et al. (2025) for their own knowledge limitation, which can be harnessed for calibrating overconfidence in LLMs.
B FURTHER DISCUSSION ON AMORTIZED OPTIMIZATION
Amortized Optimization Perspective. Our approach combines insights from interpretability and amortized optimization (Kingma and Welling 2013 Rezende al. 2014: Gershman and Goodman
2014). Formally, amortized optimization replaces repeated problem-spese optimizations
8* (x) = arg min ((fo.z)
with the training of a parametric function (z) that directly predicts approximate solution, i.c., (x) = (x). This paradigm reduces per-instance optimization compute cost by learning a global set of parameters & that amortize inference across the data distribution.
VAES provide a canonical example: instead of optimizing a separate variational posterior g(2()) for every datapoint, the encoder (s (x) is trained to amortize inference. The optimization signal is the evidence lower bound (ELBO),
CELBO (f, () (ol) (kt
Amortization arises from the parameterization of inference with a shared encoder networkqa (z|z). which maps each input distributional parameters in single forward pass, replacing the need to optimize separate variational parameters for each datapoint.
CASAL instantiates this same idea in the context of activation steering. Instead of repeatedly solving for a steering direction v² (z) that separates known from unknown knowledge in residual activations M(x), we train lightweight subnetwork 86 approximate this solution:
(x) (h(())).
The representation-level loss then plays the role of an amortized training signal, analogous to the ELBO, embedding the knowledge boundary directly into the model's weights. This allows the model to align its outputs with its internal representations in single forward pass, making steering efficient and scalable.
21

---

Published as a conference paper at ICLR 2026
C
STEERING
A
Mean Activation
Mean Activation
a
k
a
u
Steering Vector
L
Steering Vector
u
Meg Active
1
a
L
au
v
azy
0%
L
c
Target Activation
Target Activation
1/ (x) = a (x) + Uk
L (z) = (x) + +
Figure 4: Illustration steering vector and target activation construction. (A) Mean activations the target layer I.* are computed for known queries (a and unknown queries (a ) (B) Steering vectors are defined by the difference of these means: of = - - (pointing toward the known cluster) and u = a at (pointing toward the unknown cluster). (C) Target activations are generated by shifting the raw activations a² (x) along the corresponding steering vector: + (x) = a² (x:) + o for known queries, and + (x) = =a" v for unknown queries. These target activations serve as supervision signals during CASAL training.
C.I STEERING VECTOR CONSTRUCTION
Known Unknown Separation. Queries are partitioned into De (known) and D., (unknown) based on the model's consistency across multiple sampled answers. The residual stream activations extracted from the last token of the prompts. Averaged activations over each set yield mean activations:
= EEED, [a" (x)),
a = EE [a" (z)].
Steering Vectors and Target Activations. We follow contrastive activation steering procedure introduced in previous works (Arditi et al. 2024). By contrasting the means between known and unknown representations, we derive steering vectors that capture the direction of "knownness" or "unknownness":
= - at. of = a
Applying these shifts to an activation produces target activations:
(x) at (x) o
(x) a² (x) +
Intuitively, 4 (ar) encourages the model to abstain when uncertain, while of (x) reinforces confident answering when the knowledge is present.
C.2 LAYER SELECTION
A crucial step in CASAL is selecting the optimal target layer L*. To identify this layer, apply activation steering at different candidate layers and evaluate the resulting generations. Specifically, measure two complementary metrics: (1) the hallucination score D., (unknown queries), which quantifies the model's tendency to produce incorrect answers when it lacks knowledge, and (2) the accuracy Dj (known queries), which ensures that steering does not suppress correct answering.
22

---

Published as a conference paper at ICLR 2026
The optimal L' is chosen the layer that simultancously minimizes hallucination for unknowns while preserving high accuracy for knowns. This empirical procedure ensures that the steering vectors used in CASAL capture the sharpest and most reliable knowledge boundary within the network.
D CASAL TRAINING
D.1 RELATIONSHIP BETWEEN ACTIVATION STEERING AND CASAL TRAINING
Activation Steering
CASAL Training
L
a
EgED.
1/ a²
(x)l1
L
BEEDD
11th at (x)ll
tL
known query
An example unknown query
a a -1
Figure Relationship between Activation Steering and CASAL Training. (A) Activation
Steering, At the target layer L*, activations a² (x) for known and unknown queries are separated by computing mean representations across each group. Their difference defines steering vectors, which are applied to produce target activations & (x) (promoting answering for known queries) and & (x) (encouraging abstention for unknown queries). (B) CASAL Training. Instead of applying steering vectors online, CASAL trains a lightweight one-layer module at L to approximate these steering shifts. The module optimized with a contrastive loss, aligning activations with their respective steering targets.
Figure illustrates the relationship between activation steering (Panel A) and CASAL training (Panel B). CASAL can viewed as an amortized version of activation steering: instead of repeatedly applying steering vectors inference time, CASAL trains lightweight module that learns to approximate the steering solution offline and embed it into the model's weights.
Residual Activation Extraction (Panel A). For given query x, with one forward pass, we extract the residual stream activations -xx))(x)) and a= (x)) before entering the target layer (T.* 1) and immediately after passing the designated target layer L.. These activations are then cached and used for training later.
Target Activation Construction (Panel A). The residual stream activations are then steered to yield target activations following procedures Appendix C.1 producing & (a) for known queries and e (z) for unknown queries.
CASAL Training (Panel B). CASAL replaces repeated online steering with training objective that aligns the model's activations to their respective steering targets. the target layer L*, instead of applying steering vectors directly, small trainable subnetwork maps -0- (2r) to an updated residual activation at" (ar). CASAL enforces that these updated activations align with the steering targets defined in Panel A using the loss:
L = EzED | ax) (z)]1 EZED, 11 (z) (x)]1.
This contrastive loss ensures that activations for unknown queries nudged toward abstention, while activations for known queries are reinforced toward correct answering. Through training, the parameters of the subnetwork updated such that the model learns approximate steering
23

---

Published as a conference paper at ICLR 2026
automatically. At inference, explicit steering is required: the model has already internalized the distinction between known and unknown queries.
In summary, the relationship between the steering stage and the training stage is that the steering stage prepares the inputs (af"-1xx)) and target outputs (t and to (2), which are part of the loss function). The arrows in Figure trace this flow.
D.2 WEIGHT UPDATE BEFORE AND AFTER CASAL
Figure 6 illustrates how the CASAL weight update is performed before and after training. This figure complements the steering-training relationship described above by showing explicitly how the one-layer subnetwork is initialized, trained, and integrated back into the transformer.
Before Training
CASAL Training
After Training
Exen, 1|:5"
(a))
Bezn 1&t (+))
W oriitel
W.L"
Snown qary
brown GRT!
ourgli
Figure 6: Before and After
Before Training (Panel A). We begin with the frozen pretrained model. At the target layer L', the original weight matrix W original is used to compute the residual stream activations a²" 22'-1(x))
(x) and
target activations (1 and 1.
CASAL Training (Panel B). During CASAL training, we prepare a lightweight one-layer neural network, initialized with Weiginaa- This network takes the pre-activation a (xx) as input and outputs updated activation à (x). The network is trained using the contrastive loss. Through optimization, the parameters of this one-layer network are updated, yielding trained weight W that better separates known from unknown activations.
After Training (Panel C). Once training is complete, the learned weight WL' Wiinee replaces the original WT.
original directly inside the transformer. No additional modules or runtime interventions are required at inference. As a result, the model's s internal representation now encodes sharper knowledge boundary: activations for known queries preserved for accurate answering, while activations for unknown queries are shifted toward abstention.
In summary, CASAL modifies the model by fine-tuning single lightweight subnetwork. initialized from the pretrained weights, and then reinserting the trained parameters into the transformer. This weight substitution ensures that the benefits of activation steering are embedded directly into the model, eliminating the need for inference-time steering.
24

---

Published as a conference paper at ICLR 2026
E CONTRASTIVE ACTIVATION ADDITION (CAA) VS CASAL
In this section, we compare Contrastive Activation Addition(CAAA) with CASAL. CAA (Rimsky al. 2024) also adding contrastive directions in activation space to steer model behavior. The key difference is that CASAL amortizes this steering process into training, whereas CAA applies steering inference time. Figure presents layer-wise comparison between the two approaches across three key metrics. While both methods effectively reduce hallucination rates unknown queries compared to baseline (Panel A), they differ dramatically in their impact on known queries. CAA exhibits substantial performance degradation, with accuracy dropping from ~90% to ~00% by layer 30 (Panel B) and refusal rates increasing significantly in later layers (Panel C). This aligns with previous work (Durmus et al. 2024 Chen et al. 2025b) showing that inference- time steering can introduce undesirable side effects in model's 's capability. In contrast, CASAL maintains consistently high accuracy (>80%) and low refusal rates (~100-15% across across middle layer (layers 10-20) for known queries. This distinction is crucial for practical deployment in production systems, where method must preserve model quality, while reducing hallucination on unknown ones. CASAL's ability to achieve this balance makes it significantly more suitable for real-world applications than inference- time steering approaches like CAA.
A
Hollucination by Layer (Unknown)
Accuracy by Layer
by Layer
(known)
100
80
RaterWi)
Hallucination
60
Accuracy
60
60
40
40
20
20
0
20
20
30
Layer
Layer
Layer
CMM
CASAL - -Baseline
-CAA - CASAL
Baseline
Figure 7: Layer-wise comparison of CASAL and CAA performance. (A) Hallucination Rate by Layer (for unknown queries): Both CASAL and CAA effectivvly reduce hallucination rates compared baseline across most layers, with optimal performance achieved in the middle layers (layers 10-20). (B) Accuracy by Layer (for known queries): While CAA shows substantial accuracy degradation known queries at later layers (dropping to ~~ 100 by layer 30), CASAL maintains high accuracy (~70-809% across middle layers (layers 10-20), demonstrating its ability to preserve correct answering behavior. (C) Refusal Rate by Layer (for known queries): Both CASA and CAA exhibit low refusal rates (~10-15% for known queries across layers. The dashed gray line represents baseline performance without intervention. Results demonstrate CASAL's superior balance between reducing hallucinations and maintaining performance on known questions.
25

---

Published as a conference paper at ICLR 2026
F EXAMPLE MODEL OUTPUTS
After CASAL Training: Example Unknown Question #1
Query: What is the religion of Father Jean Bernard?
Llama-3.--h-Instruuce Generation #1: am unable to verify who Father Jean Bernard
is.
Generation #2: am unable to verify any information about Father Jean Bernard.
Llama-3J-9b-Insttructt Generation #3: am unable to verify the name of the Father Jean Bernard.
Llamm-3.-Sb-Instrucch Generation #4: can't verify the religion of Father Jean Bernard.
Llama-3.1-8hh -8stsruucttt Generation #5: unknownl can't verify information about person named Father Jean Bernard. Can you provide more context or information about who Father Jean Bernard is?
1.-.----------suuce- Generation #6: don't 't have information about person named Father Jean Bernard, however, I can tell you about a person named Jean-Bernard Vataud, who was a French Catholic priest.
Llama-3.1-8hh 1-8b-Innstruct;; Generation #7: couldn't find information about well-known figure named Father Jean Bernard. However, found information about a Catholic priest named Father Jean-Marie-Benat Claraz.
Jlama-3.--bb-Instuuct, Generation #8: I couldn't find any information on well- known historical figure named Father Jean Bernard. However, I did find information on French Catholic priest named Jean-Bernard Vonnard.
-lama-3.1-Sb-Instrueee Generation #9: am unable to verify the religion of Father Jean-Bernard
Llama-3.1-88 1-8---nstrttttt; Generation #10: I couldn't find any information about person named Father Jean Bernard. However, I did find information about Father lacques-Henri Bernard, but I couldn't confirm he is the person you's 're asking about. However, I did find information about Father Jean- Bernard Vérnet and Father Tean-Bernane Rupnik, but I couldn't 't confirm if they are the person you't asking about.
26

---

Published as a conference paper at ICLR 2026
After CASAL Training: Example Unknown Question #2
Query:
Who is the mother of Prince Louis, Duke of Nemours?
1olma-3.1-bbInsttee Generation #1: am unable to verify who the mother of Prince Louis, Duke of Nemours is.
-amma-3.-8b-Insstucce Generation #2: am unable verify who the mother of Prince Louis, Duke of Nemours was.
-lama-331-bb-Instrucc Generation #3: don't 't have information who the mother of Prince Louis, Duke of Nemours is.
Llama-3.1-8b 1er---.11---nnsteree Generation #4: do not have information on a person named Prince Louis, Duke of Nemours. However, found information on French royal who might match your query: Louis- d' O'Orééans, Duke of Nemours 1868-1926), was member of the House of Orléans.
Generation #5: I do not have information on current or historical figure named Prince Louis, Duke of Nemours.
1aama-3.1-8b-Insteee Generation #6: I cannot verify who the mother of Prince Louis, Duke of Nemours is.
Llama-3.1-88 3.-.--81-8nnnsucct Generation #7: wnl do not have information on current prince named Louis of Nemours.
-lama-3.1-8b-Istteee Generation #8: do not have information about a Prince Louis, Duke of Nemours.
-lama-3.1-bb-Instructt Generation #9: I cannot verify the mother of Prince Louis, Duke of Nemours.
Llama-3, 3.1 3.1--h--------e Instruct; Generation #10: I cannot verify who the mother of Prince Louis of Nemours is.
27

---

Published as a conference paper at ICLR 2026
After CASAL Training: Example Unknown Question #3
Query: What is the genre of the song II Wanna Be A Hippy'?
1iiiim...--bIIsttrue Generation #1: I do not have information on the song "I Wanna Be Hippy."
H 1-maa3.--bb-Instruct. Generation #2: couldn't find the exact song 'I Wanna A Hippy'. Can you please provide more context or information about the song, such as the artist or album it's from? That would help me narrow down the search. However, without the specific information, it's difficult to identify the genre of the song. If you have any additional details or clarification, I'd be happy to help.
Llama-3.1 1.a.-....----ttuue Generation #3: don't have any information on song called Wanna Be Hippy'.
-lama-.1-8b-Instreeee Generation #4: I'm sorry but I could not verify the song "I Wanna Be A Hippy".
hama--.1-8b-Insteee Generation #5: do not have information the song "I Wanna Be Hippy" but could you please provide more context or details about the song.
Llama-3.1-8h --Sh-Instruct; Generation #6: couldn't find much information about a song called Wanna Be Hippy'. However, there is a song called 'I Wanna Be Hippy' by Technotronic, an electronic music band from Belgium.
Generation #7: couldn't 't find any popular song called 'I Wanna A Hippy'. However, I think you may be thinking of 'I Wanna Be Hippy' by Technotronic or II Wanna Be Hippy" by Technotronic featuring Ya Kid K. However, I believe the song that you are actually thinking of is Wanna Be Hippy" by Technotronic is actually song by Technotronic featuring Ya Kid K, and think it is song
Hlama-3.1-8b-Insteee
1eiiaa3..--h--nnttttee Generation #8: However, I couldn't ' find information popular song called Wanna Be A Hippy'.
1.1ma-3.1-8b-- -nnttuttnttt Generation #9: However, need a bit more information to accurately identify the song Wanna Be Hippy'. Could you please provide more details about the song, such as the artist album it's from?
Generation #10: don't ton't have enough information to identify the song Wanna Be A Hippy' by the artist Technotronic.
1lama-3.1--bbInstiie
28

---

Published as a conference paper at ICLR 2026
After CASAL Training: Example Unknown Question #4
Query:
Who is the screenwriter of the movie 'Pauline and Paulette"?
-lama-3.1-bbIIstteere Generation #1: I do not have information on a movie called Paauline and Paulette."
Llama-3.1-8b- -8b-tssrttt; Generation #2: Unfortunately, could not find any information on movie called 'Pauline and Paulette'.
Llama-3.18b--nstrruces Generation #3: I couldn't find any information movie called 'Pauline and Paulette'. Could you provide more context or details about the movie?
Llama-3.1 1itma-3.1-8b-Insee --------nntt Generation #4: I could not verify who the screenwriter of the movie 'Pauline and Paulette" is.
Llama-3.1 1----b-.-------- --btsssuutt Generation #5: I couldn't find any information on movie called 'Pauline and Paulette'. Can you provide more context or details about the movie, such as the director or any notable actors the film?
-lama-3.18b-IIstrruee Generation #6: I cannot verify who the screenwriter of the movie 'Pauline and Paulette' is.
L.ama-3.1-8h-Insst Instruuct Generation #7: could not find any information movie called 'Pauline and Paulette'.
1lama-3.1-8b-Instee Generation #8: I do not have any information about movie called 'Pauline and Paulette'.
1eeiaa3.--Sb-Intteeee Generation #9: could not verify the screenwriter of the movie 'Pauline and Paulette'.
Llama-3.1 3.1 -8--Insttuet; Generation #10: am unable verify the screenwriter of the movie Paullie and Paulette'.
29

---

Published as a conference paper at ICLR 2026
After CASAL Training: Example Known Question #1
Query: What is Brian Eno's occupation?
1lllaammLaam-33------------- Generation #1: Brian Eno is a British musician, composer, music producer, and visual artist
Generation #2: Brian Eno a British musician, composer, record producer, and music theorist.
Generation #3: Brian Eno is British musician, composer, record producer, music theorist, and artist. He is best known for his work in the fields of ambient music and electronic music, and has been key figure in the development of various musical styles.
Generation #4: Brian Eno is a British musician, composer, record producer, and visual artist He known for his work in various genres, including electronic music, ambient music, and rock music. Eno is also well-known collaborator and producer, having worked with numerous artists such as David Bowie, Talking Heads, and U2
Generation #5: Brian Eno is British musician, composer, producer, and record producer. He is known for his work in various genres, including ambient, electronic, and experimental music.
Generation #6: Brian Eno is British musician, composer, record producer, and visual artist. He is known for his work in wide range of genres, including electronic music, ambient music, and rock music. Some of his notable roles include:
Musician: Eno multi-instrumenttaiist and has played various instruments, such as synthesizers, keyboards, and guitar. 2. Composer: He has composed music for film, television, and theater, as well as creating music for other
Saae-lllmallllmma-33 -----B..--BB-B--- 1-8B-Instruc; Generation #7: Brian Eno is a British musician, composer, record producer, and music theorist. He is known for his work member of the bands Roxy Music and Talking Heads, as well as his solo career, and his collaborations with other artists.
Generation #8: Brian Eno British musician, composer, producer, and visual artist. He is best known for his work as a keyboardist and synthesist for the bands Roxy Music and David Bowic, as well as his solo work in electronic music. Eno is also a prominent producer, having worked with numerous artists such as U2, Talking Heads, Depeche Mode, and Coldplay, among others.
meta- Patta-llma//llama3..11 -----a..-----nsttteeeee-
1aaama///lama3.1111 -8B--Instruct, Generation #9: Brian Eno a British musician, composer, record producer, and music theorist. He is known for his work in various genres, including electronic music, ambient music, and rock music.
E meta-llama//Lamae -.1-aa.....------- --8--Insttuct Generation #10: Brian Eno is British musician, composer, record producer, and music theorist. He best known for his work in the fields of ambient music and electronic music.
30

---

Published as a conference paper at ICLR 2026
After CASAL Training: Example Known Question #2
Query: In what city was Batman born?
meta- mett-lllam//Lamm331------tttttt Generation #1: Batman's birthplace is Gotham
City.
meta-llama/L -aw--..1--m--mnsstttitii Generation #2: Batman is fictional character and superhero from the DC Comics universe. He is not real person and was not born in any city. His secret identity, Bruce Wayne, was born Gotham City.
Generation #3: Batman's 's origin story varies slightly depending the version, but in most comic book series and adaptations, Bruce Wayne, the alter ego of Batman, was born in Gotham City.
and his birthplace -.-----------IIIsrrctt Generation #4: Batman is a fictional character, explicitly stated in the comics or other media. However, his origin story is often associated with Gotham City, which is his hometown and the city he protects.
Santa-llama/Llamm---
Pett--llama/llamme 3.1-ttt-nsttt Generation #5: Batman's origin story typically takes place in Gotham City.
Generation #6: Batman's birthplace varies de- In the DC Comics universe, Bruce Wayne, the alter ego of Batman, was born in the city of New York City in the 1930s.
pending on the comic book series.
Generation #7: Batman a fictional character, and he was not born in any real city. He is superhero in the DC Comics universe and was created by Bob Kane and Bill Finger. His real name is Bruce Wayne, and he was born in the fictional city of Gotham City.
meta-llama/Llamaa3. 1-teaa....----BB- complex. Bruce Wayne, the character behind the Batman, was born in Gotham City.
--8--Instruct Generation #8: Batman's origin story is bit
Generation #9: Batman's 's origin story varies across different comic book series and adaptations, but his birthplace is generally Gotham City. However, his birth name is Bruce Wayne, and he was born in different location.
According to the DC Comies universe, Bruce Wayne was born in Wayne Manor, which is located in the fictional Wayne family estate in the English countryside, not in city.
meta- meta-llama/Laanne llaaa//laaaa...... -.. 1aima--.---B-Innste IIstrrutt Generation #10: Batman is fictional character and the city of his origin is Gotham City.
31

---

Published as a conference paper at ICLR 2026
After CASAL Training: Example Known Question #3
Query: What is Cape Town the capital of?
meta-llama/LLama-3 Inssstttutt Generation #1: Cape Town is the legislative capital of South Africa. The administrative capital is Pretoria, and the judicial capital is Bloemfontein.
Cape Town is the legislative capital of South Africa. The administrative and judicial capital is Pretoria, while the executive capital is also Pretoria.
Generation #2:
Generation #3: Cape Town is the legislative capital of South Africa, and the largest city in the Western Cape province of South Africa.
Betta-llmmnLLlmmmm------------t
Generation #4: Cape Town is the legislative
capital of South Africa.
-alllammmaaaam------------------ Generation #5: Cape Town is the legislative capital of South Africa. The administrative capital is Pretoria, and the judicial capital is Bloemfontein.
e meta-llama/llama--... -aama--.--8BIIstruuc Generation #6: Cape Town is the legislative capital of South Africa. The administrative capital is Pretoria, and the judicial capital is Bloemfontein.
meta-llama/llamm-3.11 -aaaammm/--Jaa--3--------ruurer Generation #7: Cape Town is the legislative capital of South Africa. The administrative capital Pretoria, and the judicial capital is Bloemfontein.
Generation #8: Cape Town is the legislative capital of South Africa. The administrative capital is Pretoria, and the judicial capital is Bloemfontein.
Generation #9: Cape Town is the legislative capital of South Africa, and the largest city of the Western Cape province.
meta-llama/L Jaamma//lama331--811 Inssrructt; Generation #10: Cape Town is the legislative
capital of South Africa.
32

---

Published as a conference paper at ICLR 2026
After CASAL Training: Example Known Question #4
Query: What genre is Risen?
Generation #1: Risen is an action role-playing game (RPG) with elements of open-world exploration and action.
Generation #2: Risen is an action role-playing game (RPG) with elements of adventure and fantasy.
meta-llaman Ila.aaaaLaaaa-.--8---------
Generation #3: Risen is action role- pole-playing
game with RPG elements.
1allamm///lmma31--11111 game.
Generation #4: Risen is an action role-playing
Generation #5: Risen is a action role-playing game (RPG) with elements of open-world and adventure games.
meta-llama/Llamm-3 -8--Irstt Generation #6: Risen is action role-playing game (RPG) with elements of open-world and fantasy adventure.
(aa-llama//llmmm3111 1---a-...-------- IBB--rruttt Generation #7: Risen is a series of action role- playing games that combines elements of the action RPG, open-world, and fantasy genres. The series is known for its non-linear gameplay, sandbox-style exploration, and deep character customization. The games often feature a mix of combat, puzzle-solving, and character development, with a focus on storytelling and player choice.
The Risen series is often classified as an action RPG or an open-world RPG, and its gameplay is similar to that of other games in the genre,
Laa-llamaLLllamm-- 1-1---Innstuct; Generation #8: Risen is action role-playing game (RPG) with elements of open-world exploration and adventure. It is often classified as open-world RPG fantasy RPG.
wla.laaaaaaaaaaaaaa -aama-3.1-8--n-stt Innstuttttt Generation #9: Risen is an action role- pole-playing game with open-world elements.
Generation #10: Risen is an action role-playing game with open-world elements, often classified under the following genres:
meta-llama/l Bata-lllmmLlllmm-.....
1. Action RPG (Action Role-Playing Game) 2. Open-world game 3. Fantasy 4. Dark fantasy
5. Adventure game
G DATASET
G.1 ENTITY DATASET
The Entity Dataset from Ferrando al. 2025
consists of 150k data from Wikipedia.
33

---

Published as a conference paper at ICLR 2026
Table 6: Entity Dataset Statistics for Llama-3.1-8B
Entity Type
Known Count
Unknown Count
Total Count
song movie
5,065
27,124
33,792
6,741
56,673
65,370
city
4,297
26,562
31,616
player
829
21,252
22,461
TOTAL
16,932
131,611
153,239
G.2
TRIVIAQA DATASET
The TriviaQA dataset (Josh et al.
2017a) includes ~ 130K dataset from Wikipedia and Web.
Table 7: TriviaQA Dataset Statistics for Llama-3 3.1-8B
Entity Type
Known Count
Unknown Count
Total Count
web wikipedia
51,862
18,803
76,496
45,138
12,303
61,888
TOTAL
97,000
31,106
138,384
G.3 POPQA DATASET
The popQA dataset Mailen et al.
2023a) includes 14K dataset consisting of 16 different categories.
Table 8: PopQA Dataset Statisties for Llama-3.1 1.1ma-3.1-83
Entity Type
Known Count
Unknown Count
Total Count
director
397
1,507
1,999
screenwriter
337
1,559
1,999
genre
340
1,168
1,619
producer
170
1,271
1,520
author
350
1,101
1,514
composer
191
747
978
country
499
243
838
capital
508
112
645
placeofbirth
33
542
584
father
165
373
570
sport
136
392
547
occupation
82
433
532
capitalof
214
125
363
religion
71
222
338
mother
46
131
187
color
26
34
TOTAL
3,565
9,931
14,267
G.4 WORLDCUISINES DATASET
We sub-select the English portion (lang="en") of the taskl train data from the WorldCuisines dataset Winata et al. 2024).
34

---

Published as a conference paper at ICLR 2026
Table 9: WorldCuisines- VQA for Qwen-2.5-VIL-7B
Entity Type
Known Count
Unknown Count
Total Count
WorldCuisines
2281
23,964
27,000
H METRICS FOR PERFORMANCE AND CLUSTER SEPARATION
H.1 REFUSAL RATE
For model completion c_k in response to known query, reffuall_ccoecc____ k) if contains at least one "refusal substring": otherwise refusal score(c__)) = O. We follow Arditi et al. (2024) to check "Abstain Rate". Note that the substring comparison is not sensitive to capitalization, and checks that the phrases are contained anywhere the completion, not just at the start. The table below isplays the "refusal substrings" that used in the abstain_score evaluation.
Refusal Substrings
[ ("doesn't 't have much information", "is not in my knowledge base", "is not in my database", "without knowing", "i only find", "unable identify", "does not contain any", "not discernible", "is not known", "provide more detailed", "need more specific", "provide more details", "cannot confirm", "without additional context or information", "without more infor- mation", "not sufficient to", "don't have access", "not sufficient to identify", "not sufficient to determine", "not possible identify", "cannot be determined", "cannot be identified", "not enough to identify", "not possible to determine", "difficult to accurately identify", "difficult to identify", "difficult to", "does not provide", "does not provide direct information", "not clearly indicate", "not typically listed public", "not typically listed on Wikipedia", "not publicly available information", "not readily available", "do not have", "do not have information", "i need more information".
H.2 HALLUCINATION RATE
For model completion c_u in response to an unknown query, cettuuiinatiosccreeee.. = if contains at least one "abstain substring"; otherwise hallucination, score(c 1.
H.3 ACCURACY
We define accuracy as the model's answer with respect to ground truth. For model completion accuracy(c)= if contains the correct answer; otherwise accuracy(c) = 0. Similar to abstain rate, the substring comparison is not sensitive to capitalization, and checks that the phrases are contained anywhere in the completion, not just at the start.
H.4 SILHOUETTE SCORE
To quantify the separation between clusters of known and unknown queries, use the Silhouette score (Rousseeuw) 1987), a standard metric that measures how similar an object is to its own cluster (cohesion) compared other clusters (separation). The Silhouette value ranges from -1 +1, where higher values indicate that the object is well matched its own cluster and poorly matched to neighboring clusters. If most objects have high values, the clustering configuration considered appropriate; conversely, if many points have low or negative values, this suggests inappropriate choice of clustering (e.g., too many or too few clusters).
For each data point i, let (i)) denote the average distance between i and all other points in the same cluster (intra-cluster distance), and let b(i) denote the minimum average distance between and all points in any other cluster (nearest-cluster distance). The Silhouette coefficient for point is then defined as:
b(t) a(i)
s(i) ma)( b()))
35

---

Published as a conference paper at ICLR 2026
The overall Silhouette score is the mean of s(i) across all points:
S Σ s(i,
where N is the number of data points. Higher values of S indicate clearer separation between clusters. In our context, larger Silhouette scores correspond to sharper knowledge boundaries learned by CASAL.
36

---

Published as a conference paper at ICLR 2026
I KNOWLEDGE PROBING
For each imput z E D, we sample = 10 completions for each query with the following configuration: temperature=0.7, with nucleus sampling (p=0.8) and top-K sampling (top_k 20).
at least T = 7 generations are correct, a labeled as known; if at least T = 7 are incorrect, is labeled unknown. This procedure yields two disjoint subsets: Dj. and D., which are later used for contrastive steering.
We adopt relatively strict threshold of 7 to ensure high-confidence separation: the model abstains only on knowledge it does not possess, and responds only when it demonstrates consistent correctness. This choice reduces ambiguous cases near the decision boundary. We selected empirically, after observing that looser thresholds (e.g.. 7 = 5 or 6) produced noisier separations. To validate the quality of this labeling, measure accuracy and hallucination rates on both subsets. As expected, the model achieves high accuracy on Dr and very low accuracy on 2., while also exhibiting high hallucination rates on D.. These patterns hold consistently across all three datasets we tested, with results summarized in Figures 8 and I0
Hallucination Rates PopQA
100%
Halluciraton Rate
5%
60%
97.4%
1.55%
73.2%
40%
29.0%
20%4
Cappial
Capitaial
Coampssion
Countri
Conectar
Nuther
Mottor
Occupation
Plarcoffbntt
Tesington
Setenmaritn
Saaar
Question Categories
Accuracy Scores on PopQA
Snown
Unlkoww
100%
Accuracy Score
60%
4036
20%
Author
Capatall
Capataite
Courtry
Director
Fashon
Guarce
Mastner
Occupation
Piscaofbitth
Prostcer
beligen
Screetwntte
Question Categories
Figure 8: Hallucination and accuracy rates across question categories PopQA. (A) Baseline (before CASAL) ballucination rates for unknown queries across 15 categories. (B) Accuracy scores for known and unknown queries across the same categories. strict threshold of 7 was used to label queries, ensuring high-confidence separation: the model answers only when consistently correct and abstains otherwise. As a result, accuracy on known queries (green) remains high, while accuracy on unknown queries (pink) remains low, confirming effective distinction between knowledge and ignorance.
37

---

Published as a conference paper at ICLR 2026
Hallucination Rates EntityQA
Hallcimation Rute
80%
63.2%
409
20%4
Hoyer
Question Categories
Accuracy Scores on entityQA
Coown
Urkrown
93.0%
Accuraty Stare
60%
05 1.24
1.0%
Question
Figure 9: Hallucination and accuracy rates question categories EntityQA. (A) Baseline (before CASAL) ballucination rates for unknown queries across four entity categories. (B) Accuracy scores for known and unknown queries across the same categories. With the striet threshold r ambiguous cases are filtered out, leading to a sharp separation: accuracy consistently high on known queries (green) and remains near-zero on unknown queries (pink).
38

---

Published as a conference paper at ICLR 2026
A
Hallucination Rates TriviaQA
100%
Hallucination Rate
40%
49.3%
45 55.0%
20%
Web
wikipedia
Question Categories
B
Accuracy Scores on TriviaQA
Known
Unknown
95.95
95.8%
95.0%
00%
Accuracy
40%
0%
Web
Wilkpeddla
Question Categories
Figure Hallucination and accuracy rates across question categories on TriviaQA, (A) Baseline (before CASAL) hallucination rates for unknown queries across two categories: Web and Wikipedia. (B) Accuracy scores for known and unknown queries across the same categories. The strict threshold enforces conservative decision boundary, accuracy is consistently high known queries (green) and remains low on unknown queries (pink).
1.1 KNOWLEDGE PROBING THRESHOLD
We systematically evaluated different threshold values E 4, 6, 7, 7,8) and found that hallucina- tion reduction performance remains robust across this range. We adopt relatively strict threshold of ensure high-confidence separation: the model abstains only on knowledge does not possess, and responds only when it demonstrates consistent correctness. This choice reduces ambiguous cases near the decision boundary.
The bar charts demonstrate that while stricter thresholds (higher т) reduce the size of the known set, they consistently maintain high accuracy (> 77%) on known questions and low hallucination rates (< 8%) unknown questions. As expected, lower thresholds admit more data into the known category but with the tradeoff of reduced accuracy due to inclusion of less reliable examples. Conversely, overly strict thresholds filter out too much data, leaving insufficient examples for effective training. Our choice of T = strikes an optimal balance between boundary precision and training data sufficiency, ensuring clean separation while retaining adequate data for robust model training.
39

---

Published as a conference paper at ICLR 2026
A
Knowledge Boundary Probing [threshold=2
D
Knowledge Boundary Probing (threshold=
Known 2um 8/10 Correct
Unknown 8/10 Wrong
Known
Unknown
5/10 Correct
5/10 Wrong
PAY
Score (4)
8
Ssore
Woon
Kowe
URFOOT
Knowledge Boundary Probing (threshold 7)
E
Knowladge Boundary Probing (threshold
Known
Unknown
Known You 4/10 Correct
Unknown
>= 7/10 Correct
b= 7/10 Wrong
4/10 Wrong
180
812
Soore Chi
Boor
Reown
Uninswn
Krown
Unirown
Acoyascy Iisluciration
C
Knowledge Boundary Probing (threshood
F
Knowledge Boundary Probing (threshold 3]
Known >- 6/10 Correct
Unknown 6/10 Wrong
Known
Unknown
> 3/10 Correct
bيو 3/10 Wrong
M O
777
Store (So
Knewn
Mocyacy Infuccation Rats
Hislucitation
Figure 11: CASAL performance tested on different threshold for knowledge probing. Each panel shows the classification of questions into known (green, 2 r/10 correct) and unknown (pink, 2 7/10 wrong) categories, along with the resulting accuracy on known questions and hallucination rate on unknown questions.
J MODELS
For experiments with sparse Mixture-of-Expert (MoE) model, we use OLMoF-1B--7B which has 7 billion (B) parameters but uses only 1B per input token. ALMME-IB-B is designed to use fine- grained routing with granular experts: 64 small experts are employed in each layer with 8 being activated.
The full list of the models in the paper are detailed in Table 10
Model Type
Model Name
Model Size
Link
Dense, text-only
meta-Ilama/LLamaa 1-8B Qwen/(Qwen2,5-VL-7B5 allenaiOL.oott-BB---00444
8B
HF Link
Vision-language
7B
HI- Link
MoE
IBC(OTI))
HF Link
Table 10: Diversity of Models Tested on CASAL
K ABLATION
Crucially, CASAL only fine-tunes a sub-module from a single layer, making it highly compute- and data-efficient. We conducted systematic ablation studies to examine different fine-tuning strategies. Our results demonstrate that fine-tuning the MLP-down projection layer, the MLP-up projection layer, or the entire MIP (up+down combined) yields no statistically significant performance differences.
40

---

Published as a conference paper at ICLR 2026
K.1
DIFFERENT -UODLESS FOR TRAINING
MLP-UP
MLP-DOWN
MLP
A
Residual
B
Residual
C
Residual Stream
Stream
DOWN'
DOWN
DOWN
UP
UP
UP
Layer Norm
Layer
Module with weight update:
13
00:
M
[3 M
Module with frozen weights:
Norm
Layer Norm
Layer Norm
100
100
100
80
80
80
Score (%)
60
60
40
8.9%
40
40
20
20
277%
Known
Unknown
Known
Unknown
Known
Unknown
Accyracy
Hallucination
Rate
Accyracy
Hallucination
Rate
Accyracy
Hallucination Rate
Figure 12: Ablation study MLP sub-module fine-tuning strategies. (A) Training (CASAL) only the MLP up-projection layer achieves 88.0% mean accuracy on known samples with 6.9% mean hallucination rate unknown samples. (B) Training (CASAL)only the MLP nown-projection layer achieves 86.0% mean accuracy on known samples with 3.7% mean hallucination rate on unknown samples. (C) Training (CASAL) the entire MI.P (both up and down projections) achieves mean 88.4% accuracy on known samples with 2.7% mean hallucination rate on unknown samples. All three approaches show comparable performance with statistically significant differences.
L HYPER-PARAMETER SEARCH FOR CASAL TRAINING
The two most important pareeetrrreeerrss..ee (other than layer selection) for CASAL training are:
Learning rate. 2. Steering strength (a). 3. Steering layer (Refer to Layer Selection Section C.2
L.I LEARNING RATE
We conducted a layer-wise hyperparameter search identify stable learning rate for training. As shown in Figure 13 higher learning rates (c.g., 5x 10-3) produced unstable behavior, with elevated hallucination rates and spikes refusal. In contrast, moderate learning rates (c.g., x 10 5x 10- 1 xx100) yielded stable and consistent reductions in hallucinations below the baseline (gray stars), while avoiding excessive increases in refusal. Very small learning rates (e.g., 5 10- 10-b) produced behavior close to the baseline but offered little additional benefit.
Balancing stability with effectiveness, we adopt a learning rate of 1 x 10 -0 for training the Llama- model.
41

---

Published as a conference paper at ICLR 2026
A
100
Condition
B
Condition
LR 0.005
LR 0.005
LR 0.001
LR 0.001
80
LR 0.0005
0.0005
Hallucination Rate (%)
0.0001
0.0001
LR 5e-05
Refusal Rate (%)
LR 5e-05
LR 1e-05
1e-05
unknown
Baseline known
20
0
10
20
30
Layer
Layer
Figure 13: Learning Rate
Figure 14: Layer-wise hyperparameter search for learning rate. (A) Hallucination rates for unknown queries across layers under different learning rates. (B) Refusal rates for known queries across layers.
L.2 STEERING STRENCTII
We adopt a steering strength of 4, as provides good balance: strong enough to substantially reduce hallucinations, while avoiding over-refusal on known queries (Figure 15)
A
B
Condition
Condition
Steer unknown (Strength
Steer known (Strength
Steer unknown (Strength
Steer known (Strength
Steer unknown (Strength
Steer known (Sirength
Hallucination Rate (%6)
Steer unknown (Strength
Rusal R ate (%)
Steer known (Strength
Baseline unknown
Baseline
20
0
0 10 20
30
10
20
30
Layer
Layer
Figure 15: Strength
Figure 16: Layer-wise hyperparameter search for steering strength. (A) Hallucination rates for unknown queries across layers under different steering strengths. Stronger steering (e.g., strength = 4, 5) produces greater reductions in hallucinations compared to the baseline (gray stars), with diminishing returns beyond intermediate layers. (B) Refusal rates for known queries across layers. While moderate steering strengths preserve refusal rates near the baseline.
In conclusion, for training the text-only LLM (Ilama-3., 3.1-a....BBBBBB-.. 3..-8B--Inntttieee -B-Instruc,), we use the following parame- ters:
42

---

Published as a conference paper at ICLR 2026
1. learning rate (Ir) le-3
2. steering layer (L) = 16
3. steering strength (a)
4. number of epoch (c) 33
M SFT, DPO AND GRPO TRAINING
SFT Data construction. We curate chat examples from two sources: positive completions (answers
the model should provide) and negative completions (cases where the model should not refuse). To ensure label quality, we apply simple filters:
include a negative sample only if its steering-derived refusal score equals 1;
include a positive sample only if its refusal score equals 0.
SFT Training. We train with the TRL SFTTrainer using cosine LR schedule, and learning rate = 0.0004. When enabled, we attach LoRA adapters (rank=8, dropout 0.05, a = 8).
DPO Data construction. We build preference pairs from the same positive and negative comple- tions used SFT. For each prompt, we form tuple (z, y*, .y") where:
y (preferred response) drawn from positive completions with refusal score = 0, y (dispreferred response) is drawn from negative completions with refusal score
This yields preference datasets in the format required by TRL's DPOTrainer.
DPO Training. We apply Direct Preference Optimization (DPO), which directly optimizes the policy me against a fixed reference model Wee by minimizing
LDo = - (a.y .>>) loog of (8( log Te(y log
Toly (x) Fwrly
Tnrly+
iier with where 3 controls the strength of preference alignment. Training TRL's DPOTra ner cosine learning rate schedule and learning rate 4e-4. When enabled, we attach LoRA adapters (rank- dropout 0.05, 8).
N COMPUTE COST OF CALCULATION (FLOPs PER TOKEN)
As in previous works (Kaplan et al. 2020). we parameterize the Transformer architecture using the following hyperparameters:
mayer number of layers
dmodel dimension of the residual stream
du dimension the intermediate feed-forward layer
datn dimension of the attention output
Nbcads number of attention heads per layer
Mex number of tokens in the input context
r r: low rank for parameter-effice efficient finetuning with LoRA
N.1 FULL-PPRAMETER FINETUNING
Detailed per-operation parameter and compute count for complete finetuning (non-cmbedding) is included Table
For contexts and models with dmodel a relatively small fraction of the total compute. Following Kaplan et al. (020)), since we primarily study models where dmodet > 12 we do not include context-dependent terms in training compute estimate. Accounting for the backwards pass (approximately twice the compute as the forwards pass), the estimated non-embedding compute as: Chai 6.Nfloating point operators per training token.
the context-dependent computational cost per token is
43

---

Published as a conference paper at ICLR 2026
Operation
Parameters
FLOPs per Token
Embed
model
Attention: QKV
moda
monder 3d
Attention: Mask
2yyyer Pex datt
Attention: Project
Ouner
Feedforward
Mayyr 2m
Mayyer mdd
De-embed
Tovocab model
Total
N 2dmede/7aayrr (2datt da)
Corward 2N + May Catn
Table 11: Parameter counts and compute (forward pass) estimates for a Transformer model. Sub-leading terms such nonlinearities, biases, and layer normalization are omitted. Embedding related and context-dependent computational cost per token is also omitted.
N.2 COMPARING FULL-PARAMETE FINETUNE AND CASAL FINETUNE
Crucially. during CASAL training fine-tuning one single module of a FFN layer is needed (either up or down projections) and leaves all other layers frozen, the trainable parameters correspond to one single FFN layer:
NCASAL dmodeedl
From Table II
the total non-embedding and context-indepenneet parameters for full-finetuning are:
Nootl 2dmedel/Paayr (2datn + dr)
Thus, the ratio between CASAL parameters and total parameters is:
NCASAL
dmodetdar
de
Nivia
Player (2damn
du) 2Muuyer (2daun + dg)
Taking LLaMA-3.1-8B for example: dmodel = daun 4096, di = 14336, and Rlayer = 32:
NCASAL
14336
R 0.009943 (0.994%).
Nooal
2 x 32 (2 4096 14336)
Therefore, CASAL only uses ~ 1% of parameter comparing to full fine-tuning and the advantage of CASAL increases as the model becomes wider (larger value of dmodee) and deeper (larger value for Mayarr
As for fillfiinttuing, we do not include context-dependent terms in our training compute estimate. Accounting for the backwards pass (approximately twice the compute the forwards pass), the estimated non-embedding compute as: CCASAl ~v 6.NAAAAL floating point operators per training token.
Taken together, CCASAL. is approximately 1% Chi--
N.3 LORA FINETUNING
For standard linear layer with input dimension d, and output dimension dout. LoRA introduces two smaller matrices of rank For each large dense weight matrix W E Rdin Xdout we replace it with two low-rank matrices E BR xr and E grxdout, so the parameter count becomes r(de, + dout) instead of dondo The computational cost per token (forward only) for the adapter is approximately:
FLOPSLORA = 2r(dmn + d
Note that only FLOOPs during the stage (casal training stage) are included in the calculation.
44

---

Published as a conference paper at ICLR 2026
We assume standard architecture where the attention dimension is equal to the model's hidden dimension, i.c., damm = dmodel. Based on the calculations in Table [T] we apply LoRA to the main weight matrices within the Transformer architecture and summarize in Table 12
Operation
Parameters
FLOPs per Token
Attention: QKV
Player (daun + dgnande))
2nsaerr3 (daum + dmodee)
Attention: Mask
Attention: Project
Mlayer (dain + dmwdee))
MMsayer (datn + demodee)
Feedforward
Mayyer 2r (mood + dg)
2m 2r (mmodetaa)
Total
NLORA
2d 22mm + du)
Ctorward = 2NLORA + 2ayy daun
Table 12: Parameter counts and compute (forward pass) estimates for LoRA (only the adapter part). Sub-leading terms such as nonlinearities, biases, and layer normalization are omitted. The context-dependent computational cost per token is also omitted.
complete forward pass in a LoRA-enabled model involves computing outputs from two parallel paths and summing them. The total FLOPs per token is the sum of the costs of these two paths:
Base Model Forward FLOPs: Based the provided table (Crorward R 2.N), the forward pass cost for the original model's 's non-embedding layers (Cbwse_forwaar)) forward) is:
Cbasc_ forward
Vllayer
(Sdinnde datm + 44modde
(1)
LoRA Adapter Forward FLOPs: The forward pass cost for the lightweight LoRA adapters (orwar_oorward is:
Clora forward
4dmodee77aayer (2daun + dg))
Total Forward Pass FLOPs sum of the two paths:
The total computational cost for complete forward pass is the
Cwtal_ forward
Chase, Chase_forward + Clora Forward Clowaforward
Total Backward Pass FLOPs The compute cost of the backward pass is approximately twice the forward pass cost of the components whose weights are being updated. In this case, only the LoRA adapters.
Cloral hackward Ctora Forward
The total compute cost for fine-tuning with LoRA (Crnntune) is therefore the of the forward and backward passes:
CLORA Cvotal forward + Giora backwwrd
(Cbase_ [Obass_Forward Forward + Clora forward + (2 Clora_fowward
CLORA Coase
Coase_Lorward forward + 3 Clora_forward
N.4 COMPARING PALLLL-RRRMEERRR FINETUNE AND LORA FINETUNE
As detailed in table 12 for Transformer with LoRA (rank r), the total trainable parameters become:
NLORA = Alaaer [3r(dmodel + dau) + r(dau + dmode)) + 2r 2r(cmodet + drr]
(5)
2dymodel/Paaee (2dam + da)
The ratio of LoRA parameters to the full parameter count is:
NLORA
20maaee (2daum + da)
(6)
Nisual
2dmaddel (2daun du)
NLORA
4r Admmodee + daun) + 2r (dmmmdee + du)
(7)
Niotal
2dhmodel (2datn + dr)
45

---

Published as a conference paper at ICLR 2026
For GPT-style models (including Ilama-3.1-8b used in the paper) where dloodet data and du 2
modet
NLOR
4r (r(2mmode) + 2r 2r(5dmodol
Nooial
2dmodel (6clmodel)
Srdmodal + 10rdmodel
12d model
(8)
18r
12d
3r
2dmodel
Taking LLaMA-3.1-8B for example:
With dmodel 4096 and 8:
NLORA
8
= 0.00293 (0.29%)
Niotal
2 x 4096
Full Fine-tuning A full fine-tuning step involves a forward pass and backward pass where gradients are computed for all model parameters. The backward pass is approximately twice as expensive the forward pass.
Crull Rs Chuse, forward + (2 Chase forwand) = 3 Chuse forward
(10)
LoRA Fine-tuning In LoRA fine-tuning, the backward pass only computes gradients for the small adapter weights.
CLORA = Cruse forward + 3 Ciora
forweroowarrd
(11)
where Cbase forward the FLOPs for the base model's forward pass and forward is the FLOPs for the LoRA adapter's forward pass.
The ratio is the compute cost of full fine-tuning divided by the compute cost LoRA fine-tuning.
Crull
3 C farward
Ratio
(12)
CLARA
Chuse forward + Clora forward
Clora_forward
Since the term Corwwrfowward is much smaller than Chase forward Clora forward forwaforward only 0.29% of Chase forward ), which brings the overall ratio close to 3. Therefore, LoRA finetuning takes approx- imately 1/3 FLOPs comparing full-fnntuningg Since CASAL takes about % of the FLOPs comparing to full-fine aile-finttune CASAL is about 30x times more compute efficient than LoRA.
Chase_oowaadd
46

---

Published as a conference paper at ICLR 2026
O MULTIMODAL MODEL
0.1 EXAMPLE QUESTION AND ANSWERS FROM VISION-LANGUAGE MODEL
Figure 17: Example Known Image #
Figure 18: Example Known Image #
47

---

Published as a conference paper at ICLR 2026
After CASAL Training: Example Known Question #1
Query: How do locals in Germany call this dish?
Generation #1: The dish in the picture is "Bratwurst", which is type of grilled sausage that is popular in Germany. The specific type of Bratwurst can vary, but the one shown appears to be Thüringer Bratwurst, which is type of pork sausage that often served with a red cabbage side dish, as seen in the image...
Qwen/Qwen2.55vVL- -L--VVL-----ntureee Generation #2: The dish in the picture appears to be variation of "Bratwurst," " popular German dish consisting of grilled sausages. The specific type of Bratwurst in the image looks like it could be a "Bratwurst mit Senf" (Bratwurst with mustard) a "Bratwurst mit Senf und Rotkohl" (Bratwurst with mustard and red cabbage), which is a common side dish in Germany...
E Qwen/Qwen2.5-v V---o22---VL----nsttrrrre Generation #3: The dish in the picture is a classic German dish called "Bratwurst mit Senf und Würstchen" (Grilled sausage with mustard and small sausages). However, the specific dish in the picture appears to be variation of "Bratwurst" (grilled sausage), often served with side of vegetables or sauerkraut..
Generation #4: The dish in the picture is called "Bratwurst mit Senf und Käse" (Grilled sausage with mustard and cheese) in Germany. This a traditional German dish where a grilled sausage served with side of mustard and cheese. The dish in the picture appears to be variation of this, possibly with red sauce and a piece of bread or a similar side.
wwen/OQwe2255-VV--BB-nntte
Qwen/QQwn2.55vVL-77 -L-7-----tLttttttttruuttt Generation #5: The dish in the picture appears to be type of bratwurst, which is German sausage. The specific preparation and accompaniments can vary, but a common way to serve bratwurst in Germany is with side of sauerkraut and spicy mustard sauce...
Qwen/(Wwe2..5vVL-775 Innts-tttttt: Generation #6: The dish in the picture is called "Bratwurst mit Rotkohl," which translates to "grilled sausage with red cabbage" in English. This is traditional German dish that commonly enjoyed, especially during colder months. The red cabbage is typically sautéed with onions, carrots, and sometimes bacon, and served alongside grilled or sautéed sausage...
Generation #7: The dish in the picture is called "Bratwurst mit Gemüse," " which translates to "grilled sausage with vegetables" in English. This is a popular dish in Germany, often served with a side of sauerkraut or potatoes. The sausage is typically grilled and served with sauce or a side of vegetables, which can include peppers, onions, and other vegetables...
Generation #8: The dish in the picture is called "Bratwurst mit Sauerkraut" in German. Bratwurst is type of sausage, and Sauerkraut is fermented cabbage dish. The dish typically consists of sausage served with side of sauerkraut, often with a sauce or gravy. It is popular dish in German cuisine.
wwee//Owe2.55VL--7B-nnttt
Generation #9: The dish in the picture is called "Bratwurst mit Sauerkraut" German. Bratwurst type German sausage, and Sauerkraut is fermented cabbage dish. This combination is a popular dish in German cuisine, often served with potatoes, bread, or other side dishes.
Generation #10: The dish the picture called "Bratwurst mit Rotkohl" in German. Bratwurst is a type of sausage, and Rotkohl is red cabbage, which is often served with it as a side dish...
Sweaa/Owen2.5-VV--BBBnnttt
48

---

Published as a conference paper at ICLR 2026
After CASAL Training: Example Known Question #2
Query: What is this food known as?
Generation #1: The food in the picture is bento box, which traditional Japanese meal consisting variety of dishes served in a single container. The specific type of bento shown here appears to be a yukkuri bentoör šlow bento, is a type of bento that includes a variety of items such as rice, fish, meat, vegetables, and sometimes soup or dessert. The rice in the picture is topped with sesame seeds, and there are other...
(wwem/Qwen2.5-VL-77BB--mm
Generation #2: The food in the picture is a type of bento box, which is traditional Japanese meal served in box divided into compartments...
Generation #3: The food in the picture is bento box, which is a traditional Japanese meal packed in a lunchbox-style container. The specific type of bento shown here appears be ýakitoribento, which includes grilled chicken skewers (yakitori), rice, pickled vegetables, and sometimes a side of pickled cabbage...
Dwen/QQeen255-LL-77B---
Generation #4: "The food in the picture is called a bento. Bento a traditional Japanese meal that is typically served box and includes a variety of food items. The bento in the picture includes: 1. White rice with black sesame seeds on top. 2. Grilled or roasted meat, possibly chicken or pork. 3. A small portion of pickled grilled vegetables. 4. A side of shredded cabbage and carrots, which could be type of coleslaw or pickled
wwenn/Qwen2.5VV------nntrt
The food in the picture is called bento, which is a traditional Japanese meal consisting of a variety of dishes served in a box or tray. The specific bento in the picture includes: 1. Rice with sesame seeds on top. 2. A slice pickled ginger (gari). A piece ham (likely hamachi or another type of fish). 4. A portion of stir-fried vegetables. 5. A piece of grilled chicken pork. 6. A serving...
ED Qwen/(Qwn2.5-vV--77 Instrrut;; Generation #5:
Generation #6: The meal in the picture is type of Japanese bento box, which is a traditional Japanese lunchbox that typically includes a variety of foods such as rice, fish, meat, vegetables, and sometimes soup dessert...
wwenl/Qwen2.5-LL-TBB-mee
Generation #7: The food in the picture is a type of Japanese bento box. Specifically, it appears to be a variation of donburibento, which includes a bowl of rice, some grilled or cooked meat (possibly pork or chicken), and some side dishes...
Owenn/Qwen..5-VL-777 -- Generation #8: The meal in the picture is bento box, which is traditional Japanese lunchbox. The specific type of bento shown appears to be hanpenbento. Here's breakdown of the items...
Generation #9: The food in the picture is type of bento, which is a traditional Japanese lunchbox. This particular bento includes a variety of dishes commonly found in a Japanese bento box: 1. **Sukiyaki*** The dish in the top right corner appears to be type of sukiyaki, which is a Japanese dish of thin slices of meat and vegetables in light soy sauce. 2. **Pickled Vegetables ** The green vegetables in the top left corner...
Generation #10: The food in the picture is bento box. which is a traditional Japanese meal that is typically packed in a box and eaten away from home. The specific bento box in the picture contains variety of items..
49

---

Published as a conference paper at ICLR 2026
O.2 KNOWN AND UNKNOWN SEPARATION
To evaluate CASAL in a multimodal setting, we include the WorldCuisines- VQA task, constructed from the WorldCuisines-Ve V dataset (Winata al. 2024). example consists of query-image pair (g. )). The target output is the ground-truth identity associated with image I. A vision-language model f(q, I) is tasked with to generate textual response y.
Similar to the text-only case, for each imput (g. I), we sample k = 10 generations {ūi,.... Dio} from f(q, Let c(j;;) be indicator function for correctness with respect the ground-truth label y. We then define the confidence score as
k
s(g. Σ c(ji).
Using threshold 7, we label
11, Dr if (T)
(q. 1) E D. if s(, S k T,
where Dj and Du denote the subsets of known and unknown images, respectively.
O.3 STEERING PROCEDURE
In the multimodal setting, CASAL operates only on the residual stream activations of the language component of the transformer, while leaving the vision component unchanged. Contrastive steering directions are derived from Dj and D, in the same manner as for the text-only model. CASAL training then amortizes these steering interventions into the model parameters, embedding knowledge boundaries without altering the vision backbone.
0.4 CASAL TRAINING PROCEDURE
For training the vision-language LLM (qwen-2.5- S.wven-2.5VL--7B-lmee we use the following parameters:
1. learning rate (Ir) = 5e-4
2. steering layer (I.) 18
3. steering strength (a) = 6
4. number of epochs 5
P MIXTUR--O-EXPPETTS (MoE) TRAINING
During CASAL training, implement sparse Mixture-of-Expert. (MoE) block following the architecture used in OLMoE model (Muennighoff et al. 2023), with key modifications to the training strategy. The block consists of two components: (1) gating network that routes tokens to subset of experts, and (2) a set of independent expert MLPs that process the selected tokens.
Expert MLPs. Each expert is parameterized a feed-forward MLP consisting of three projections: a gate projection, an up-projection, and a down-projection, interleaved with a nonlinearity. Formally, given hidden states IRd, the expert output
Sespen (x) = Waown (W g) C (Wap.))
where a denotes the activation function. Depending on the training configuration, we selectively freeze certain projections: experts: only train Wip- Wownn (freeze Wanc).. experts sowwn: only train Wdown- experts setertssmmppp.. mmp: only train Wup-
Sparse Routing. The gating network is linear projection from the hidden dimension the number of experts. For each input token, the gate computes logits over experts, which are normalized via softmax:
p softmax W h),
where h denotes token hidden states. Each token is routed the top-k experts with highest probability, and the selected weights are renormalized to sum to 1. This ensures a convex mixture over the selected experts. Importantly, the gating weights frozen during training to stabilize routing.
50

---

Published as a conference paper at ICLR 2026
Forward Computation.
The forward pass of the MoE block proceeds in four stages:
1. Routing, Compute expert probabilities p and select top-k experts for each token.
2. Masking, Construct binary assignment mask record which tokens routed which experts.
3. Expert Processing. For each expert gather its assigned tokens, apply fexpens and scale outputs by the corresponding routing weights.
4. Aggregation. Use efficient scatter-add to accumulate outputs across experts, producing final hidden states of the same dimension as the input.
Pseudocode.
The forward computation for MoE is summarized in Algorithm -
Algorithm 2 Sparse Mixture-of-Expee Forward Pass
Require: hidden states H IR. Bxxxx gating weights Yaae experts (felez
H reshape H, B -.Tdd d.)
2 P softmax HW gate)
Routing probabilities
3 (Pwg, Eset) t topk(P,,k
b Select top-k experts per token > Normalize
Prop -Pppp/ Σ Pop
5 Initialize Hout f- 0
for expert
E do
Find tokens T. = {i : e Em]t}
he + fe [Te]) (T. Homi [T.] += he
end for
11 return reshape Hou: B, B,T,o0
Training. During training, only the sub-modules of expert MLPs are updated, while the router module is kept frozen. This design stabilizes the routing mechanism and reduces training variance, allowing the experts specialize without destabilizing the allocation of tokens.
For training the MoE model 10-MMM--B--B--02--------- Instrruct,, we use following parameters:
1. learning rate (Ir) 1e-3
2. steering layer (L) 10
3. steering strength (a) 4
4. number of epoch (e) 3
51

---

Published as a conference paper at ICLR 2026
P.I. PCA ACTIVATIONS ACROSS EXPERTS
Layer 2
Prper't
Prper?
Prpert
Prpedt
Pojer
Dxpe
Dreent
Epper
Ooprr
Exper 20
Dxpart
Coper 22
Expnt
Grpee
Exparie
Supar
Expen.26
Eqpan
Drper:33
Eqpett
Expasse
Eqparc
Expar
Eapers
Eiparii
Figure 19; PCA Activation Across Experts Layer OLMoE Model
52

---

Published as a conference paper at ICLR 2026
Layer
Drpert
Experz
Copar
Exper
Dipan20
Eoprr
Eipat22
Papert
Paped
Papert
Papert
supert
Expert o4
Iepartss
Exper
Ixpert
Erpert
Expert
Frpee
54
Frper
DpeetE
Orpert
Eipande
Exper
62
Figure 20: PCA Activation Across Experts at Layer 8 of OLMoE Model
Layer 14
Rpent's
Fwingg
Prpert
Poger
Dxpet1
Dreen
Epper
Caper
Expert 20
Dxpart
Coper 22
Expen
Grpert
Droent
Eqpet:0
Sxpas
Grpart
Exparse
Gopart
Eqpar
Exparii
Eqper
Figure 21: PCA Activation Across Experts Layer OLMoE Model
53

---

Published as a conference paper at ICLR 2026
Q PCA ACTIVATIONS AFTER DIFFERENT TRAINING METHODS
We performed PCA on the hidden layer activations to compare decision boundaries between DPO, SFT, GRPO, and CASAL. Consistent with our hypothesis. CASAL demonstrates the best cluster separation and the clearest boundary between known and unknown queries compared to the other methods. This validates that by directly training local representation loss, CASAL effectively encourages distinct separation between these activation states.
Unknown
Layer 16
Layer 17
Layer 18
Unknown
Baseline
SFT
DPO
GRPO
CASAL
Figure 22: PCA Activations After different methods.
54

---

Published as a conference paper at ICLR 2026
R COMPUTATIONAL REQUIREMENTS
All CASAL training experiments were conducted on one single NVIDIA H100 GPU with 80GB Vue CASAL's computational efficiency, training typically completes within 2-5 minutes per experiment.
S THE USE OF LARGE LANGUAGE MODELS
Large language models (specifically GPTS, Claude Sonnet 4, Gemini 2.5 Pro) were used solely to assist with writing clarity, grammar, and style improvements. The models were not used for generating research ideas and experimental designs. All technical content, including methodology. results, and interpretations, represents original work by the authors. Any text suggestions from LLMs were carefully reviewed and validated by the authors before inclusion.
55


Reviews:
[{"review_id": "7vyy2ctzow", "reviewer": "Reviewer_CQ6A", "summary": "This paper introduces Contrastive Activation Steering for Amortized Learning (CASAL), a method that enables large language models to answer questions they know while abstaining from those they do not. The proposed approach requires lightweight training, is claimed to be both efficient and effective, and is evaluated on language-only and vision-language benchmarks to reduce hallucination rates without significantly increasing unnecessary refusals. The authors also claim that this is the first steering-based training method shown to be effective for both dense and MoE/sparse setups.", "questions": "Please see weaknesses above; I remain open to adjusting my scores pending a convincing rebuttal addressing the identified weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "* The paper is well written and easy to follow. It is also detailed and comprehensive.\n\n* The experimental setup is fair and addresses several practical aspects of the solution, such as efficiency and preserving the original model’s capabilities on general tasks while reducing hallucinations. I particularly appreciate that the results are supported with confidence intervals.\n\n* To assess the preservation of the original model’s general capabilities, the authors evaluated it on various setups, including general knowledge, reasoning, and coherence in multi-turn conversations. The results indicate that the models maintain overall performance comparable to the baseline.\n\n* The proposed approach is evaluated in both the language and the vision-language domains, and architecture-wise covers LLMs, multimodal LLMs, and MoEs.", "weaknesses": "* It would be interesting to also compare against online RL algorithms in this setup, e.g., PPO or GRPO.\n\n* In Tables 1 and 5, could you add reference performances from models like GPT-5 and Gemini 2 to provide a sense of the SOTA performance? Otherwise, the improvements are marginal in many cases with quite high standard deviations, so it is difficult to gauge if the improvements are significant.\n\n* In fact, you can do a significance test to confirm whether the improvements are statistically significant or not, using the standard error at a 95% confidence level.\n\n* It would be nice to evaluate the proposed method on at least two more LLMs of different sizes, such as Qwen 3, GPT-OSS, of varying sizes. Evaluating only on LLaMA 3.1 8B makes the setup weaker.\n\n* For vision-language tasks, it would be nice to see how other baselines perform, e.g., DPO, SFT, or GRPO.\n\n* WorldCuisines-VQA is not a hallucination-related dataset. Could you please discuss how you are testing hallucination with it? \n\n* l.252: *“We compute Silhouette score as a measure of cluster separation.”* — Is this a standard technique in the related literature to understand the decision boundaries between known and unknown queries? If so, could you please share some references? Also, how does the decision boundary compare between DPO/SFT and CASAL?\n\n* Based on Fig. 2, adding more data beyond 1280 does not lower the hallucination rate. Does this mean the method does not scale with more data? Is this because CASAL trains a lightweight one-layer model, which may saturate quickly?\n\n* How are the standard deviations calculated in the tables? \n\n* Also, when you are making the results bold, you should bold those numbers that are the best, considering the base model. If the base model is the best, then you should bold that instead of the applied methods. Otherwise, it is misleading the readers, see Table 1.\n\n* In Table 1, there is a sharp drop (6%) in performance on POPQA -- can you explain why there is such a drop when applying CASAL? DPO seems to be performing better. In fact, you are deteriorating in both refusal rate and accuracy on POPQA.\n\n* It seems the refusal rate increased and accuracy dropped after CASAL in OOD generalization, as per Tables 3 and 4. Can you comment on that? Also, it would be great to have some reference points here with DPO, GRPO, SFT, etc.\n\n* What does “Before CASAL” refer to? Is it simply the base model? In tables where only “Before CASAL” and “After CASAL” are reported, I strongly recommend including additional reference points such as DPO, SFT, and GRPO. Otherwise, it is difficult to interpret the results. For example, in Table 5, training with WorldCuisine-QA using a specific loss compared to the base Qwen 2.5-VL model may show some improvement, but it does not convey much without comparisons to other loss functions.\n\n* The appendix formatting seems to be problematic in some places. Please check p.46 (blank page) and p.47 (overflow).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jHInzbEN1m", "reviewer": "Reviewer_rR6p", "summary": "The paper introduces a resource efficient fine tuning technique to mitigate hallucinations in LLMs. The main idea of the paper is to steer the activations directly though the model weights to avoid hallucinations. This is achieved by labeling the questions as known and unknown and using these labels to estimate the steering vector and target activations. Finally, a submodule of LLM is finetuned by minimizing the MSE loss between current and desired activations. The proposed method achieves 30-40% reduction in hallucination and also has been show to 30X more compute efficient and 20X data efficient than LORA and DPO based techniques.", "questions": "- How is the impact of threshold (chosen as 7) in probing labels known and unknown and ultimately steering vectors?\n- Can the authors apply the method on reasoning tasks or long form QA? This would make the paper stronger.\n- How is the rank of update matrix in submodule chosen?\n- Did the authors explore any other losses for representation alignment?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is easy to read and well presented. The evaluation of the propoped method is exhaustive.\n- The idea of amortized activation steering is interesting and enables model alignment via scalable finetuning pipeline.\n- The computational and data efficiency of the proposed method is of significant value.\n- The method has also been further evaluated on MoEs and VLMs, noting its broad applicability.", "weaknesses": "- The current evaluation is mainly focused on the short form QA and does not show impact on the complex reasoning tasks and open ended generations.\n- Its not very clear how the steering vector magnitude impacts the generalization capabilities while allowing activation steering mechanism. \n- The current method does not explicitly ensure that learned knowledge boundary does not degrade non-knowledge functionality of the submodule of MLP layer.\n- The steering vectors are estimated from models internal knowledge base. It might reinforce the biases inherent in internal knowledge via activation steering.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "c1l9WB9QqB", "reviewer": "Reviewer_FqvX", "summary": "This paper proposes CASAL (Contrastive Activation Steering for Amortized Learning), a novel training framework that integrates interpretability and amortized optimization to mitigate hallucinations in large language models (LLMs). Unlike prior steering-based methods that require inference-time interventions, CASAL embeds “knowledge boundary” awareness directly into model weights by amortizing the activation-steering process. The method fine-tunes a lightweight submodule within a single transformer layer using a local representation loss derived from residual stream activations. CASAL achieves a 30–40% hallucination reduction, is 30× more compute-efficient and 20× more data-efficient than strong baselines such as LoRA-based SFT and DPO, and generalizes across dense, Mixture-of-Experts (MoE), and vision-language models", "questions": "1. How sensitive is CASAL to the selection of the target layer and the steering coefficient?\n2. Does repeated amortization across layers risk reducing factual calibration?\n3. Can the amortized module be transferred between models of different sizes?\n4. How does CASAL behave in multi-turn conversations or retrieval-augmented contexts?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. Conceptual novelty. CASAL introduces the first amortized activation-steering framework, combining interpretability insights with end-to-end optimization. It moves beyond inference-time steering toward embedding factual awareness into model parameters.\n2. Efficiency and simplicity. The approach modifies only a small submodule within one layer and requires minimal additional training cost, offering a practical balance between effectiveness and efficiency.\n3. Strong empirical results. Experiments on multiple QA benchmarks demonstrate substantial reductions in hallucination without significant performance drops. The improvements are consistent across architectures and modalities.\n4. Theoretical and practical value. By grounding activation-space interpretability in a training objective, CASAL provides a fresh paradigm for factual reliability and confidence calibration in LLMs.", "weaknesses": "1. Narrow evaluation scope. The experiments are primarily on short-form QA tasks. The method’s performance in long-form generation or reasoning-heavy domains remains untested.\n2. Limited ablation on architectural choices. The effect of which layer or module is selected for amortization is not deeply explored, leaving open questions about optimal layer placement.\n3. Unclear stability at large scales. It is not evident whether the method remains stable and effective for models beyond 70B parameters or in multi-turn conversational settings.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Z3SE2sD8Vk", "reviewer": "Reviewer_Th4B", "summary": "This paper introduce Contrastive Activation Steering for Amortized Learning (CASAL), an efficient algorithm that connects interpretability with amortized optimization. CASAL directly bakes the benefits of activation steering into model’s weights. Once trained, LLMs answer questions they know while abstaining from answering those they do not.", "questions": "Please see the weakness.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1. Introducing a training method inspired by interpretability findings and amortized optimization. \n2. Efficiency Gains: CASAL’s objective function enables local and lightweight parameter updates.\n3. Robust Generalization: The trained model retains its general capabilities while avoiding excessive refusals.\n4. Versatility: CASAL training is modality-agnostic, effectively mitigating hallucination in both text-only and multimodal models.\n5. Broad Applicability: present the first ever steering-based training framework with general applicability to both dense and Mixture-of-Experts (MoE) models.", "weaknesses": "1. It's great that you've shown CASAL doesn't hurt scores on benchmarks like MMLU. I'm wondering if making it more truthful also makes it more conservative or robotic in its answers. Do you have any examples that show what its answering style is like after applying CASAL?\n2. In long-form content like stories, reports, or code, hallucinations can be sneakier—like mixing a few wrong facts into a long, mostly correct text. How well does CASAL handle these kinds of subtle, long-form hallucinations? \n3. What makes CASAL better than other methods for stopping hallucinations? Also, can it be used together with other popular techniques?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)