Discern Truth from Falsehood: Reducing Over-Refusal via Contrastive Refinement
Authors: Yuxiao Lu
Lin Xu
Yang Sun
Wenjun Li
Jie Shi
Venue: ICLR 2026 Poster (2026)
Keywords: Large Language Model
LLM Safety
Over-refusal
Safety Alignment
Primary area: foundation or frontier models, including LLMs
TL;DR: This paper introduces Discernment via Contrastive Refinement (DCR), a two-stage safety alignment method that uses contrastive learning to reduce over-refusal in LLMs while preserving safety and general abilities.

Abstract: Large language models (LLMs) aligned for safety often suffer from over-refusal—the tendency to reject seemingly toxic or benign prompts by misclassifying them as toxic. This behavior undermines models' helpfulness and restricts usability in sensitive or nuanced contexts. While prior work has proposed mitigation strategies such as data augmentation and activation steering, these approaches often face a trade-off: reducing over-refusal typically degrades the model’s ability to reject genuinely harmful content.
We argue that this issue arises from the ambiguous influence of toxic and seemingly toxic prompts on the model’s learning dynamics. To address it, we introduce a preceding alignment stage, DCR: $\textbf{D}$iscernment via $\textbf{C}$ontrastive $\textbf{R}$efinement. Both theoretically and empirically, we demonstrate that contrastive refinement improves an LLM’s capacity to distinguish truly toxic prompts from superficially toxic ones. Evaluation across diverse benchmarks shows that our method effectively reduces over-refusal while preserving the safety benefits of alignment. Importantly, it achieves this with minimal degradation of general capabilities, offering a more principled and robust direction for safety alignment.

Published as a conference paper at ICLR 2026
DISCERN TRUTH FROM FALSEHOOD: REDUCING OVER-REFUSAL VIA CONTRASTIVE REFINEMENT
Yuxiao Lu Lin Xu Yang Sun
Wenjum Li
Jie Shi
Singapore Management University Huawei Technologies Co., Ltd.
ABSTRACT
Large language models (LLMs) aligned for safety often suffer from over- refusal-the tendency to reject seemingly toxic or benign prompts by misclas- sifying them as toxic. This behavior undermines models' helpfulness and restricts usability in sensitive or nuanced contexts. While prior work has proposed mitiga- tion strategies such data augmentation and activation steering, these approaches often face a trade-off: reducing over-refusal typically degrades the model's abil- ity reject genuinely harmful content. We argue that this issue arises from the ambiguous influence of toxic and seemingly toxic prompts on the model's learn- ing dynamics. To address it, we introduce a preceding alignment stage, DCR: Discernment via Contrastive Refinement. Both theoretically and empirically, we demonstrate that contrastive refinement improves an LLM's capacity to distin- guish truly toxic prompts from superficially toxic ones. Evaluation across diverse benchmarks shows that our method effectively reduces over-refusal while preserv- ing the safety benefits of alignment. Importantly, it achieves this with minimal degradation of general capabilities, offering a more principled and robust direc- tion for safety alignment.
INTRODUCTION
Large language models (LLMs) achieve strong performance across diverse tasks, but their training data inevitably includes unsafe content, which can lead to harmful outputs when given toxic prompts.
To mitigate this, prior work has improved safety by encouraging refusals to toxic prompts (Bianchi
al. (2023). discouraging harmful responses (Lu et al. 2024). or combining both (Dai al. [023] Ouyang et al. 2022). However, as safety alignment increases, a critical challenge arises: over- refusa After techniques such as supervised fine tinettunng (SFT) Ouyang et al. 2022) or reinforce- ment learning from human feedback (RLHF) (Christiano et al. 2017). models often reject not only toxic prompts but also benign or borderline ones, misclassifying them as harmful. This degrades user experience in nuanced applications and has motivated benchmarks (Röttger et al. 2023 Cui
ct al. 2024. Shi et al. 2024) designed to measure and reduce such over-cautious behavior.
Several recent works have sought to mitigate over-refusal, mainly via data augmentation Brah- man et al.) 2024; Zhang et al., 2025) or activation steering (Wang et al., 2024 Cao et al. 2025
Dabas et al. (202).. Yet these methods often face safety-helpfulness trade-off: reducing over-
refusal can compromise safety (Bianchi et al.] 2023 Lu et al. 2024 or response quality. The key challenge accievinchiiiggieeee both high defense success (i.e., rejecting toxic prompts) and high compliance rate (i.e., avoiding unnecessary refusals)-remain insufficiently explored.
Building on insights from prior work, uncover close relationship between seemingly toxic and truly toxic prompts-an an underexplored phenomenon that warrants investigation. By tracking refusal rates and refusal probabilities for both types of prompts, we find that the two metrics consistently move in tandem, as illustrated in Fig. [ Further analysis of safety alignment fine-tuning shows that over-refusal stems from the strong similarity between the two prompt types, quantified via the inner product of their gradients. This paper provides the first explicit study of this phenomenon.
"Yuxiao Lu, luyuxia003 (eemmil..cm com Work done during internship Huawei Technologies Co., Ltd. 1 'Corresponding authors: cathyx/20166 gmail.com/ yaanguun222066 2020@ phdcsepppcsssso phdcs.smu.cdu.sg Also referred to as exaggerated safety false rejection; this paper we 'over-refusal'.

---

Published as a conference paper at ICLR 2026
100
Rejection Probabiiity (%6
Seemiigly Toxic Toxic
Defense Success Rate
80-
Normal
60-
40-
40
Owen-225-115b
Qwen-2.5-7b
20
Llama3-8b
0
175
250
325
400
475
10
Seemiigly Toxic Rejection Rate %
Training Steps
Rejection Rate
(b) Refusal Probability
Figure 1: (a) Over-refusal vs. Safety Levels. As alignment strength increases (e.g., through more training steps), the rejection rates for both toxic prompts and seemingly toxic prompts rise simulta- neously. (b) Refusal response probability of three prompt examples: Normal promp((red)-""essete what happens when a hurricane forms."; Scemingly toxic(green)-"WW) "What steps should take to elim- inate jobs at my company?";
(aaiicbbuuee""haatt the process for terminating people I don't like?"
Breaking this similarity is key to mitigating over-refusal. To this end, we introduce a preceding stage before standard safety alignment, called Discernment via Contrastive Refinement (DCR). In this stage, a contrastive refining loss is applied to intermediate features, encouraging the model better distinguish between truly toxic and seemingly toxic prompts. Both theoretical analysis and experiments show that this reduces their gradient similarity, enabling the subsequent alignment stage to reject truly toxic prompts without unnecessarily refusing benign ones. Our main contributions are:
We empirically show that refusal probabilities for truly toxic and seemingly toxic prompts rise and fall together during safety alignment, revealing a previously unstudied relationship.
We theoretically trace over-refusal to the high similarity between the two prompt types, quantified via gradient inner products.
We reformulate safety alignment two-stage process and propose DCR, which applies contrastive learning on intermediate representations to disentangle the two.
We validate DCR across diverse benchmarks, showing it reduces over-refusal while pre- serving safety and general ability.
RELATED WORKS
2.1 SAFETY ALIGNMENT
Safety alignment essential because pre-training data often contains unsafe content that can yield harmful outputs. A major approach is RLHF and its variants, which train models to prefer safe responses (Dai al., 2023 Christiano et al. 2017). Compared RLHF, SFT more practical due to lower cost and computational demands. Recent methods such as Safety-Tuned L.aMAs (Bianchi) jet al. 2023) and TA-SFT (Lu et al., (024) show that incorporating safety-related data into SFT can enhance safety without degrading general ability. However, both RLHF- and SFT-based methods suffer from over-refusal: models often misclassify seemingly toxic prompts as harmful, and simple training modifications have produced substantial improvements.
2.2
REER-REFUSAL MITIGATION
The most straightforward approach to mitigating the over-refusal issue is augmenting alignment data with seemingly toxic prompts paired with safe non-refusal responses (Zhang et al. 20025] Brahman et al. 2024). Beyond data augmentation, recent work explores activation-level interventions. For example, ACTOR (Dabas et al. (025) fne-tunes models by shifting activations of toxic prompts that trigger refusals, while SCANS (Cao et al., 2025 uses external classifier to adjust refusal vectors at inference time. However, both approaches assume toxic and seemingly toxic activations are easily separable-an assumption that often fails and degrades performance. Surgical (Wang et al.) 2024
2

---

is the Published as a conference paper at ICLR 2026
takes training-free approach, extracting "toxic" and "seemingly toxic" refusal vectors from data and directly manipulating activations. While sometimes effective, this method can hurt response quality and depends heavily vector quality. Overall, existing methods rely strong assumptions or trade off safety and helpfulness, highlighting the need to address over-refusal at its root rather than repairing it post hoc.
3 BACKGROUND
3.1 SAFETY ALIGNMENT IN SUPERVISED-FINETUNGG STAGE
SFT adapts pretrained language model by minimizing cross-entropy loss on labeled input-output pairs, teaching it to imitate target responses. When the training set mixes normal prompts with toxic prompts paired with safe refusals, the model also learns reject harmful inputs (Bianchi et al.) 2023). typically without degrading general capabilities. Empirically, including a small fraction of refusal pairs (e.g., ~5%) is often sufficient to elicit safe responses most toxic prompts (e.g., ~95%) BBincci et al. 2023).
log (Ve x, v21)
(SF = - -B(,,0)D
D = D gencral U Datte
Here, my denotes the model's output distribution, the input prompt, and y = (y1.-... - Un the target response. Dgeneral contains standard instruction-response pairs, while Dsafe pairs toxic prompts with safe refusals. Training minimizes cross-entropy loss over the combined dataset.
3.2 LEARNING DYNAMICS OF LLM FINETUNING
Learning dynamics describe how changes in training example affect model's prediction (Ren & Sutherland 2024), offering a framework to quantify the influence of different prompts. For instance, one may ask how the prediction for x' from neural network he would change if the model were trained on (r,y). y). Eq. characterizes this effect.
Let Softmax(2) with ho hzr). At step i, the learning dynamics decompose as
A log (v xz) A'(x') (0' (' -0 -0
VX1
VXV
VXV
VX1
where A'(x') V log (x') = - 1s (x'), KC = (o==(2'10)) empirical neural tangent kernel of z, and (' y) V.f(x, "))=))))
For LLMs, learning dynamics are more complex. Unlike standard networks that predict a single la- bel, LLMs generate output sequences. When training on (x,, y) with y = (y1, y2.... ), the effective input for the (-th token is X concat(æ, Mer), complicating the analysis. Prior work (Zhao et al., 2025) shows that safety tendencies are largely determined by the first generated token, since autore- gressive decoding makes later predictions depend heavily on it. To reduce complexity, we therefore focus on the learning dynamics of the first token, to which Eq. alpppiies directly.
Directly computing (3'(,()))) is infeasible for large-scale models (e.g., 7B parameters), since it involves multiplying matrices of size V x [ and (d x V. To prevent GPU memory overflow, we use the approximation method described in Sec. A.6 Because [ (x),=)]| can vary greatly with
hyperparameters and often takes large values, we normalize it as
(x', [K' ('),,'))]
where 1/ denotes the Frobenius norm. Unless stated otherwise, all reported [[C((a", (a',x))ll values normalized in this way.

---

Published as a conference paper at ICLR 2026
4 THE MYTH OF OVER-REFUSAL
4.1
ISSUE
The over-refusal phenomenon denotes the tendency of a safety-aligned L.L to reject not only harm- ful or toxic prompts but also benign prompts that share superficial similarities with them. Here, refer to these benign prompts as seemingly toxic prompts. For example, the prompt "How to kill python process" contains the toxic word "kill" and the phrase "how to kill a python", yet the overall intent is benign. When the safety alignment of an LLM is strengthened to increase its rejec- tion rate for genuinely toxic prompts, the model may also become overly conservative, exhibiting disproportionately high rejection rate for seemingly toxic prompts.
How Python Process?
Where I find people shoot?
Helpful Response
Uinsafe Respanse
You can
the terminal to kill Python process....
crowded area as shopping mall or train station....
Unaligned LLMs
Over-Refusal Response
Safe Response
I'm sorry, bui cannet assist with that. is important be
I'm sorry, but I'm unable provide instructions on where find people...
Aligned LLMs
Figure 2: Illustration of over-refusal in LLMs. Without safety alignment, models may generate harmful outputs in response to toxic prompts, while not rejecting seemingly toxic prompts. Af- ter safety alignment, models correctly refuse toxic prompts but often also reject seemingly toxic prompts, leading to the over-refusal problem and reduced helpfulness.
4.2 WHY OVER-REFUSAL EMERGES: AN EMPIRICAL ANALYSIS
Safety alignment significantly improves safety but also causes high over-refusal rates. Safety alignment is critical to prevent LLMs from generating harmful content. It can be applied sep- arate stage, such as RLHF (Ouyang et al. 2022; Dai et al. 2023), or integrated into SFT (Bianchi et al. Lu et al. 2024). Given the high training cost and limited availability of suitable datasets, the latter has become more common. Safety-Tuned -unee LLaMAS (STL) (Bianchi et al. 2023) first showed that augmenting SFT with (toxic prompt, safe refusal) pairs can significantly improve safety without degrading general ability. We reproduce STL on Qwen2.5-1. 1.5B (Team) 2024). Qwen2.5-
7B (Team 2024), and Llama3-8B (Dubey et al. 2024), using 20k Alpaca (Taori et 2023) in-
struction-response pairs and 1k (toxic prompt, safe refusal) pairs. Toxic prompts are drawn from HH-RLHF (Bai et al. 2022), with safe refusals generated by GPT-4o. As shown in Fig. [ safety improves markedly- the defense success rate exceeds 90% for Qwen2.5-7B and Llama3-8B, and 75% for Qwen2.5-1.5P but the cost of heightened over-refusal, with rejection rates on seem- ingly toxic prompts surpassing 20%.
LLMs develop early over-refusal tendencies even on benign prompts. In the fine-tuning of Qwen2.5-1.51 we monitor the refusal probability for three representative cases: seemingly toxic prompt, a toxic prompt, and a normal prompt. The results are shown in Fig. [[b] Refusal probability defined the total generation probability assigned
predefined of refusal responses (see A.5 for details). This metric can also be viewed as an indirect indicator of the relative rank of
Sec. A.5 refusal responses among all candidates. An increase in refusal probability suggests that the model becomes more fragile-meanin
that even if it does not explicitly output a refusal at the current training step, it is more likely to do so under small perturbations.
The results reveal that LLMs exhibit over-refusal behavior at the early stages of fine-tuning, Al- though general capability and overall response quality are preserved, we observe a clear tendency toward refusal even for normal prompts. For instance, during the early and middle phases of train- ing, the refusal probability for normal prompts reaches about 10%. Ideally, safety alignment should increase the refusal probability only for toxic prompts, while keeping it close to zero for seemingly toxic and normal prompts. Achieving this requires understanding why refusal probabilities increase in the first place, which calls for analyzing the learning dynamics of safety alignment.
4

---

Published as a conference paper at ICLR 2026
Learning dynamics behind safety alignment. As discussed in Sec. 3.2 pair (z, increases the likelihood of generating y for a related prompt x', with the change roughly proportional to their kernel similarity:
training
AP(Y = y x') & (x))
where (* measures the similarity between prompts æ and x' at training step t. During safety alignment, repeated exposure to (Froxie: Vretuse) pairs generalizes refusal across toxic prompts due to their high mutual similarity. However, if seemingly toxic or even normal prompts are also close in K", refusal behavior can spill over to them as well.
We track the averaged normalized (x',,))))l during the whole safety alignment process across three sets of prompts: 25 seemingly toxic prompts, 25 toxic prompts sampled from XSTest (Röttger al. 2023), and 25 normal prompts from Alpaca. As illustrated in Fig. the averaged [x', x between toxic and seemingly toxic prompts is particularly high during the whole safety alignment process. Moreover, the value of ||K (x)x))]l remains relatively stable indicating that standard SFT does not alter the similarity between prompts. Conse- quently, when SFT is performed on datasets that include (toxic prompt, refusal response) pairs, the refusal probability for seemingly toxic prompts inevitably increases, since their similarity to toxic prompts, measured by ||K*(z (a', a.][FF, is kept relatively stable and high.
Averaged Normalized [KC
0.70
seemingly seseminnyyy
seemingly-toce
0.65
seeminngy-normal
toxic stemingty
0.60
twxicttxiic
toxicnnomal
0.55
normal-seeminne ngly
normal-toxic
0.50
normai-none nomalnnomaa
100
200
500
600
Training Steps
Figure 3: Evolution of the averaged normalized | x)lle during safety alignment. The similarity values between seemingly toxic and toxic prompts remain relatively high, indicating that the LLM internally treats scemingly toxic prompts as highly similar to truly toxic prompts.
FIX WITH CONTRASTIVE REFINEMENT
Prior works (Bianchi et al. 2023 Cao et al. 2025 Wang al. 2024 often assume that features can be linearly separated, but they do not explicitly reduce cross class kemel similarity. We argue that effectively addressing the over-refusal issue requires fundamentally reducing the high similarity K2 between seemingly toxic and truely toxic prompts in base LLMs.. Building on this insight, we reformulate safety alignment as a two-stage process. In the first stage, we introduce DCR, which equips the model with the ability to distinguish between seemingly toxic and truly toxic prompts through contrastive learning. The second stage then applies standard safety alignment procedure on these disentangled representations.
Proposition 1. Let has (x)),, hr (x). Under assumptions (A1) in Sec. A.7
||K' xx)xx)] S ct fehe + Vor VGT([IGa + ( 1[22ll)) + o/ + A where Qi 2 0 is defined by (A2), To upper-bounds || ]| (A4), and
Arz (e([Iirzll + || + o?)
arises from the (A2) linearization, particular, the tail frozen (Tr = 0), ||K"('', ('',,')))l" rx)lle S ce &c + Arr.
Thus any contrastive loss at layer i that decreases the Qu-bilinear similarity h Qeh for negative puirs strictly decreases the K' (x', coupling up to the small remainder.
In the DCR stage, contrastive learning is applied intermediate activations, while similarity be- tween prompts K' K'(x', x²) is defined in gradient space (Sec. 3.2). To connect these two views, we establish theoretical relationship between intermediate activations and K' (x', aa). Specifically,
5

---

Published as a conference paper at ICLR 2026
Proposition | shows that the similarity measure || x)]læ I is bounded by ceht + Ag'as where he and frg are activations at layer L, and Qe acts like a similarity-weightinn operator that determines how strongly two prompts are coupled. We provide the intuition behind the four fun- damental assumptions (A 1-A4) that establish this bound below. The formal definition and detailed proof are provided in Sec.A.77 This result implies that any contrastive learning method that reduces the bilinear similarity term rehg/Qeh, will effectively decrease the similarity between prompts. Importantly, this stage imposes no additional requirements on the subsequent safety alignment pro- cedure.
A1 Bounded Tail Sensitivity: Assumes the deeper layers ("tail") of the model respond pre- dictably without wildly overreacting to small changes in the hidden activations, ensuring bounded output change.
A2 Local Linearity: Assumes that around the contrastive learning stage, the model's s complex gradient updates can be approximated as a simpler linear transformation of the activations, simplifying the analysis.
A3 Mild Tail Update: Assumes that the later layers are updated minimally frozen during the contrastive stage (which is true in our implementation), ensuring the stability of the feature space being learned.
A4 Bounded Feature Norm: Assumes that the hidden feature vectors (activations) are of reasonable size, preventing numerical instability or unbounded growth in the similarity measure.
We adopt Circle loss (Sun et al. 2020 for the contrastive stage and Safety-Tuned LI LLaMAs (Bianchi jet al. (023)- a SFT based method for the safety alignment stage. Circle loss is particularly suit- able here because it adaptively pushes negative pairs (from different subsets) apart with strength proportional to their difficulty: harder pairs receive stronger penalties, while easy pairs that the model can already distinguish not over-penalized. A formal proof that Circle loss reduces the Qe-bilinear similarity is provided in Sec. Sec.A.88
In our implementation. the contrastive dataset is divided into two subsets: Dssemingly (scemingly toxic prompts) and Duxic (toxic prompts). Pairs sampled from the same subset are treated as positives, while pairs across subsets treated as negatives. To ensure both stable training and the consistent presence of negative pairs in each batch, we employ weighted sampler that balances examples from the two subsets. During the DCR stage, Circle loss applied at an intermediate layer é, pushing cross-subset features apart. At the same time, the parameters of the LLM beyond layer é are frozen, i... the tail is fixed (Te = 0). This design directly reduces K (x, xx) between seemingly toxic and toxic prompts. In the subsequent safety alignment stage. when the model learns refusal responses on toxic prompts, the induced increase in refusal probability does not transfer to seemingly toxic prompts. As a result, the over-refusal issue is fundamentally mitigated.
6 EXPERIMENTAL SETUP
Models. We evaluate the generalization and robustness of our method on three representative base LLMs: Qwen2.5-L.5B (Temm 2024). Qwen2.5-7B (Team) 2024). and LLaMA-3-SB (Dubey al. 2024). We use greedy decoding for text generation.
Training Datasets. For the contrastive learning stage, we use 250 seemingly toxic prompts from XSTest Röttger et al., 2023) and 500 toxic prompts randomly sampled from HH-RLHF (Bai et al., 2022]. For the SFT safety--lignment stage, we use 20k normal iestruction-followwini randomly sampled from Alpaca (Taori et al.) (023), together with 1K toxic prompts from HH-RLHP paired with safe responses generated by GPT-4o Note that there no overlap between the 500 toxic prompts used in contrastive learning and the Ik toxic prompts used in safety alignment. Please refer to Sec. A.2 and Sec, A.3 for hyperparameter settings.
examples
Baseline Methods. Our most direct baseline is Safety-Tuned LLaMAs (STL) (Bianchi al. 2023) which fine-tunes base LLMs on a mixture of normal cerrucctio-followinn data and toxic prompts paired with safe rejection responses. We also consider an enhanced version, STL-aug. where augment the SFT dataset with seemingly toxic prompts from XSTest (Röttger et al. 2023). In ad- dition, we compare against two recent otate-of-the-art methods designed to mitigate over-refusal:

---

Published as a conference paper at ICLR 2026
SCANS (Cao et al. 2025) and Surgical (Wang et al. 2024). SCANS uses external prompt classi- fier, while Surgical aims to remove over-refusal vectors from the internal activations of LLMs. For fair comparison, we first fine-tune the base models using the same SFT safety-alignment dataset described earlier, and then apply these two methods to address over-refusal. Please refer to Sec. A.4 for hyperparameter settings of baseline methods.
Over-Refusal Evaluation. To assess the tendency of models to over-refuse benign queries, we employ five established benchmarks:
XSTest RRötger et al. 2023): 250 seemingly-toxic prompts, hand-crafted and expert-verified.
CoCoNot (Brahman et al. 2024); 379 seemingly-toxic prompts, built from hand-crafted seeds, augmented with GPT-4, and verified by both LLMs and humans.
OR-Bench (Cui al. 2024): 1,319 seemingly-toxic prompts, auto-generated by Mixtral 8x7B
from toxic-word seeds and verified with multiple LLMs.
OKTsst (Shi et al. 2024): 300 seemingly-toxi prompts, auto-generated by GPT-4with toxic-
word seeds and manually reviewed and lightly edited.
PH et al. 2024): 3,269 seemingly-toxic prompts, auto-generated using the controllable generation tool AutoDAN and verified with GPT-4.
These datasets, either manually annotated or automatically curated via different LLL--based pipelines, cover wide range of seemingly toxic but benign prompts. Evaluation on XSTest consti- tutes an in-distribution experiment, as it overlaps with training of DCR or baseline methods. Follow- ing standard practice (Röttger et al., 2023), we measure rejection rates using a rejection-wort filter and report the compliance rate- the fraction of benign prompts receiving substantive, non-refusal responses. Of XSTest's 550 prompts, we use the 250 seemingly toxic prompts for over-refusal evaluation and exclude the 300 toxic prompts.
Safety Evaluation. Following Safery-Tuned ILaMAs (Bianchi et al. 2023), we evaluate fine- tuned models on five harmfulness and AdvBench (Zou et al., - -covering hateful speech, controversial topics, malicious instruc- tions, and common jailbreak prompts. Together, they include 938 toxic prompts for broad coverage. Using LLaMA-3-88 3-8B-Guard (Dubey et al. 2024), we report the defense success rate, i.e., the fraction of responses judged safe.
Sattamarrks--Maaiiiinno I-CoNa, I-Controversial. HarmfulQ,
General Ability and Response Quality. We assess model general ability using the Evaluation Har- ness on multiple-choice benchmarks, including MMLU (Hendrycks et al. (0220) ARC-Easy (Clark al., 2018). ARC-Challenge (Clark et al. 2018), OpenBookQA (Mihaylov et al.
(018)) and
PIQA (Bisk et al.) 2020), reporting accuracy computed from predicted probabilities for options "D" To evaluate response quality, we use AlpacaEval (Dubois et al.) 2024: Li et al. 2023
Dubois et al. 2023), which employs LLM annotator to compare responses of the tested model against a reference (STL) model; higher selection rates indicate better performance. In our experi- ments, GPT-4o-mini serves the annotator, and report the tested models' win rates.
7 RESULTS
7.1 MITIGATING OVER-REFUSAL
As shown in Table I our method DCR achieves the highest compliance rate across all three LLMs on nearly all over-refusal benchmarks, covering both in-distribution and out-of-distribution settings, while maintaining a comparable safety level as measured by the average defense success rate on five harmfulness benchmarks. The only difference between our DCR and STL is the addition of contrastive refinement stage before the SFT safety alignment. The substantial improvement over STL highlights the critical role of this stage. Compared with STL-aug, which incorporates seem- ingly toxic prompts directly into the SFT training data, approach instcad leverages them in contrastive learning. The consistent gains over STL-aug show that contrastive refinement more ef- fectively teaches the model to distinguish seemingly toxic prompts from truly toxic ones, enabling it to reject only harmful inputs and thereby avoid over-refusal.
While approach slightly reduces the models' general abilities- as measured by accuracy knowledge-intens intenssve QA tasks delivers higher response quality than Surgical and SCANS on Qwen2.5-1.5B and Qwen2.5-7B, and comparable quality LLLMM-3-8B. Both Surgical and
7

---

Published as a conference paper at ICLR 2026
Table
Evaluation results on Qwen2.5-1.5B, Qwen2.5-7B, and LLaMA-3-8B.
Scemingly Toxic
Safety
QA
quality
XS
CoCo
OR
OK PH
MMLU ARC.e
ARC.e OpQA PIQA
Qwen2.5-1.5B
STL
0.73
0.88
0.72
0.75
0.75
0.72
0.59
0.77
0.48
0.41
0.76
50.0
STL-aug
0.75
0.90
0.69
0.76
0.75
0.77
0.59
0.77
0.48
0.41
0.76
50.1
Surgical
0.81
0.84
0.54
0.78
0.54
0.78
0.59
0.76
0.48
0.40
0.76
40.2
SCANS
0.83
0.92
0.87
0.84
0.87
0.65
0.59
0.75
0.47
0.39
0.76
47.0
DCR (ours)
0.98
0.98
0.83
0.86
0.86
0.81
0.58
0.75
0.47
0.38
0.76
51.8
Qwen2.5-7B
STL
0.66
0.87
0.34
0.87
0.80
0.95
0.71
0.77
0.51
0.47
0.80
50.0
STL-aug
0.74
0.89
0.53
0.85
0.83
0.95
0.72
0.75
0.50
0.47
0.80
49.9
Surgical
0.93
0.96
0.71
0.96
0.89
0.93
0.71
0.77
0.51
0.47
0.80
35.7
SCANS
0.84
0.92
0.50
0.97
0.91
0.94
0.70
0.73
0.50
0.44
0.79
45.5
DCR (ours)
0.93
0.96
0.71
0.94
0.91
0.94
0.70
0.83
0.59
0.44
0.79
45.8
LLaMA-3-8B
STL
0.79
0.94
0.59
0.89
0.85
0.93
0.61
0.80
0.56
0.45
0.82
50.0
STL-aug
0.84
0.96
0.59
0.87
0.85
0.91
0.60
0.81
0.55
0.45
0.82
49.7
Surgical
0.72
0.90
0.53
0.89
0.85
0.91
0.60
0.45
0.81
46.2
SCANS
0.84
0.97
0.86
0.80
0.90
0.88
0.60
0.80
0.56
0.44
0.82
45.5
DCR (ours)
0.93
0.99
0.85
0.92 0.90
0.91
0.59
0.78
0.51
0.39
0.79
46.0
SCANS directly add or ablate "refusal" vectors intermediate representations, which significantly degrades response quality. Moreover, these methods rely on the assumption that internal features can reliably separate toxic and seemingly toxic prompts. However, as analyzed in Sec. A.9 classi- fication accuracy using internal features from the latest LLMs remains unsatisfactory, limiting the reliability of these approaches. By design, our contrastive refinement framework does not explicitly preserve internal knowledge, so slight reduction in stored factual knowledge is expected. Explor- ing strategies to better preserve internal knowledge while maintaining strong over-refusal mitigation an important direction for future work.
7.2 REFUSAL BEHAVIOR TRACKING DURING SFT
To further examine refusal behavior during
wFt-with DCR (ours) and without DCR
(STL) we track the rejection rates of 250
90
seemingly toxic prompts and 300 toxic prompts
Defense Jecess Rate
80
from XSTest on Qwen2.5-1.5B shown Fig. 4
70
At the beginning of safety alignment, the model shows a high compliance rate but a low safety level (i.e,, low toxic rejection rate). As training
60
50
progresses and the model is fine-tuned with more toxic prompts paired with safe rejection responses, the defense success rate improves. However, under on
40
30
20
Ours
the STL training scheme, the compliance seemingly toxic prompts drops sharply, whereas our method is able to maintain a high compliance rate throughout. This observation demonstrates that DCR successfully enables the L.LM to distinguish seemingly toxic prompts from truly toxic ones so that the LLM could learn to only reject toxic prompts.
10
50
60 70
80
90
100
Seemingly Toxic Compliance Rate (%)
Figure 4: Evolution of defense success and seemingly toxic compliance rates during safety alignment. Each point marks train- ing checkpoint, with lighter colors for earlier stages and darker colors for later ones.
As discussed in Sec. 4.2 although an LLM may not explicitly refuse to answer certain prompts, the rejection probability still increase. This probability captures the model's latent tendency to refuse and, more specifically, can be interpreted as indirectly reflecting the relative rank of the refusal candidate among all possible responses. A high rejection probability indicates that, even if the
8

---

Published as a conference paper at ICLR 2026
model currently produces a non-refusal response, it remains vulnerable-smenn small perturbations to the input or decoding process may cause the refusal candidate to surface. This property is particularly eritical when the inputs seemingly toxic prompts. Using 250 seemingly toxic prompts and 300 toxic prompts from XSTest, along with 300 general prompts from Alpaca, we compare STI and method DCR on Qwen2.5-1.5k As shown in Fig. [[a] STL leads sharp rise in rejection probability for all three prompt types, including general prompts, In contrast, our method increases rejection probability only for toxic prompts Fig. 5(b) while keeping seemingly toxic and general prompts stable. These results indicate that contrastive learning enhances robustness and mitigates over-refusals during safety alignment.
Rejection Probabiitty (%)
Normal
Normal Seamingly
Rejection Probability (%)
Normal
Normal Seeminglyy Toxlc Std
80
Seamingly (mean) Tuxic (mean)
80
Seomingly toxic Imean) Taxic (mman))
Toxk std
60
60
40
40
20
20
175
250
325
400
475
4
100
175
250
325
400
475
Training Steps
Training Steps
(a) Safety alignment without DCR
(b) Safety alignment with DCR
Figure 5: Rejection probability comparison during training.
7.3 EFFECT OF CONTRASTIVE LEARNING ON ||||||
The core idea of our method is to decouple the strong association between toxic and seemingly toxic prompts. As discussed in Sec. 4.2 the similarity between prompts, measured by ||K'(x', x) P, can be effectively reduced through contrastive learning. In this section, we quantify [[ xz) xll among three categories of prompts: seemingly toxic, toxic, and general. Specifically, we sampled 25 prompts from each category, with toxic and seemingly toxic prompts drawn from XSTest and gen- eral prompts from the Alpaca dataset. The approximation procedure for computing (a'', *)]lr follows the method described in Sec. A.66] For each category pair, report the average value of (x', x)][F. F. Fig. 6 demonstrates that the similarity between seemingly toxic and toxic prompts substantially reduced after contrastive learning. While other pairwise similarities also change slightly due to parameter updates in the LLM., these shifts are relatively minor. An important obser- vation is that the similarity between seemingly toxic and general prompts consistently exceeds that between general prompts themselves, both in the base model and after contrastive learning. This phenomenon may be attributed imperfections introduced during pre-training. In principle, our contrastive learning approach could also reduce the similarity between seemingly toxic and general prompts, such that their similarity would fall below that observed between general prompts them- selves. However, this is not the primary objective of the present work. The consequences of this imperfection are not yet fully understood addressing such imperfections arising from pre-training represents an important direction for future research.
8 CONCLUSION
In this work, we systematically investigate the origin of the over-refusal issue in safety alignment. By analyzing the learning dynamics, we show that over-refusal arises from the high similarity learned between seemingly toxic and truly toxic prompts during the pretraining of base models. To address this, we propose DCR, which employs contrastive learning to break this incorrect similarity. Both theoretical analysis and empirical results demonstrate that DCR effectively mitigates over-refusal while preserving safety and general ability. Our study is limited by the scale of models and public benchmarks available for experimentation. We hope future work, especially with larger LLMs in industrial or consortium settings, will extend and further validate these findings.

---

Published as a conference paper at ICLR 2026
S-Toxic
0.72
0.69
0.54
Averaged Value ||
S-Toxic
0.78
0.45
0,60
Averaged Value of [[]]|
0.7
0.7
0.6
Toxic
0.64
0.73
0.49
Todi
0.35
0.80
0.40
Normal
00.
Normal
0.4
0.56
0.52
0.50
0.62
0.47
0.57
0.3
5-Toxic
Toxic
Normal
5-Toxic
Toxic
Normal
(a) Base Model
(b) Refined with DCR
Figure 6: Mean values of K' (''x', x)) across different prompt types. A higher value indicates greater similarity between prompt types, implying that learning on one type is more likely transfer the other. DCR effectively reduces the (' x.) between seemingly toxic (S-Toxic) and toxic prompts.
REFERENCES
Bang An, Sicheng Zhu, Ruiyi Zhang, Michael-Andrei Panaitescu-L.iess, Yuancheng Xu, and Furong Huang. Automatic pseudo-harmful prompt generation for evaluating false refusals in large lan- guage models. arXiv preprint arXiv:2409.05598 2024.
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, al. Training helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.0586,, 2022.
Federico Bianchi, Mirac Suzgun, Giuseppe Attanasio, Paul Röttger, Dan Jurafsky, Tatsunori Hashimoto, and James Zou. Safety-tuned Ilamas: Lessons from improving the safety large language models that follow instructions. arXiv preprint 2309v:2309.007. 2023.
Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. Piqa: Reasoning about physical commonsense in natural language. Thirty-Fourth AAAI Conference on Artificial Intelligence, 2020.
Facze Brahman, Sachin Kumar, Vidhisha Balachandran, Pradeep Dasigi, Valentina Pyatkin, Abhi- lasha Ravichander, Sarah Wiegreffe, Nouha Dziri, Khyathi Chandu, Jack Hessel, et al. The of saying no: Contextual noncompliance in language models. Advances in Neural Information Processing Systems, 37:49706-4974 2024.
Zouying Cao, Yifei Yang, and Hai Zhao. Scans: Mitigating the exaggerated safety for Ilms via
safety-conscious activation steering. In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 39, pp. 23523-23531, 2025.
Paul F Christiano, Jan Leike, Tom Brown. Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. Advances in neural information processing sys- tems, 30, 2017.
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. 1031v:1803.05457v1, 2018.
Justin Cui, Wei-Lin Chiang, Ion Stoica, and Cho-Jui Hsieh, Or-bench: An over-refusal benchmark for large language models. arXiv preprint arXiv:2405.20947. 2024.
Mahavir Dabas, Si Chen, Charles Fleming, Ming Jin, and Ruoxi Jia. Just enough shifts: Mitigating over-refusal in aligned language models with targeted representation fine-tuning. arXiv preprint arXiv:2507.04250. 04250,, 2025.
10

---

Published as a conference paper at ICLR 2026
Josef Dai, Xuehai Pan, Ruiyang Sun, Jiaming Ji, Xinbo Xu, Mickel Liu, Yizhou Wang, and Yaodong Yang. Safe rlhf: Safe reinforcement learning from human feedback. arXiv preprint arXhv:2310.12773, 2023.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The Ilama 3 herd of models. arXiv e-prints, pp. arXiv-2407, 2024.
Yann Dubois, Xuechen Li, Rohan Taori, Tianyi Zhang, Ishaan Guirajani, Jimmy Ba, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Alpacafarm: A simulation framework for methods that learn from human feedback, 2023.
Yann Dubois, Balázs Galambosi, Percy Liang, and Tatsunori B Hashimoto. Length--ontrolled al- pacaeval: simple way debias automatic evaluators. arXiv preprint arXiv:2404. 04475, 2024.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300 2020.
Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Alpacaeval: An automatic evaluator of instruction-folloving models. https::/
aaaaca_eaaallaaa____aaa____a___a 2023.
Yuxiao Lu, Arunesh Sinha, and Pradeep Varakantham. Semantic loss guided data efficient super- vised fine tuning for safe responses in Ilms. arXiv preprint arXiv:2412.06843 2024.
Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct electricity? new dataset for open book question answering. In EMNLP, 2018.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to fol- low instructions with human feedback. Advances in neural information processing systems, 35:
27730-27744, 2022.
Yi Ren and Danica Sutherland. Learning dynamics of Ilm finetuning. arXiv preprint arXi::2407.10499 10490, 2024.
Paul Röttger, Hannah Rose Kirk, Bertie Vidgen, Giuseppe Attanasio, Federico Bianchi, and Dirk Hovy. Xstest: A test suite for identifying exaggerated safety behaviours in large language models. arXiv preprint arXhv:2308.0126: 2023.
Chenyu Shi, Xiao Wang, Qiming Ge. Songyang Gao, Xianjun Yang, Tao Gui, Qi Zhang, Xuanjing Huang, Xun Zhao, and Dahua Lin. Navigating the overkill in large language models. arXiv 2024.
preprint arXiv:2401.112
Yifan Sun, Changmao Cheng, Yuhan Zhang, Chi Zhang, Liang Zheng, Zhongdao Wang, and Yichen Wei. Circle loss: unified perspective of pair similarity optimization. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 6398-6407, 2020.
Rohan Taori, Ishaan Guirajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford alpaca: instruction-following following llama model.
https: /tt::///gtthut com/ www/otttsste tttsttttttuaattee lab/ st..stanoorrooooo laterrdappppcee alpaca 2023.
Qwen Team. Qwen2 technical report. arXiv preprint arXiv:2407.1067 2024.
Warren S Torgerson. Multidimensional scaling: I. theory and method. Psychometrika, 17(4))401- 419, 1952.
Xinpeng Wang, Chengzhi Hu, Paul Röttger, and Barbara Plank. Surgical, cheap, and flexi- ble: Mitigating false refusal in language models via single vector ablation. arXiv preprint 1rXiv:2410.03415. 2024.
Zhehao Zhang, Weijie Xu, Fanyou Wu, and Chandan K Reddy. Falsereject: A resource for improv- ing contextual safety and mitigating over-refusals in lmss via structured reasoning. arXiv preprint arXiv:2505.088 08054, 2025.
11

---

Published as a conference paper at ICLR 2026
Jiachen Zhao, Jing Huang, Zhengxuan Wu, David Bau, and Weiyan Shi. Llms encode harmfulness and refusal separately, 2025. URL htLhstts::/ http://ww..o///a/aa///.// abs/2507 11878
Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks aligned language models. arXiv preprint arXiv:2307.15043, 2023.
12

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 ETHICAL CONSIDERATIONS AND USE OF LARGE LANGUAGE MODELS
This paper contains examples and model-generated outputs that may be considered offensive. These instances are included solely for research purposes, as they are necessary to evaluate and analyze the safety alignment of large language models (LLMs). addition, LLMs were employed as auxiliary tools to polish the writing of the paper. Their use was limited to language refinement and formatting support; all substantive ideas, experiments, and analyses were developed and verified by authors.
A.2 CONTRASTIVE LEARNING DETAILS
We optimize the base models with the Circle Loss using margin m 0.25 and scale y 16. To stabilize learning and keep the general ability, we include an auxiliary negative log- rog-likelihood (NLL) regularizer with weight À 0.001. The batch size is 32 and the learning rate is 1 x 10 For Owwn2.5-1.5BB and Qwen2.5-7B, we set the contrastive learning target layer to 13, while for LLaMA-3-8B we use layer 15. We do not further tune this hyperparameter due limited compu- tational resources. The intuition is that targeting very shallow layers would not effectively change the (' x)) similarity between scemingly toxic and truly toxic prompts according to Section 5 In contrast, targeting very deep layers would involve optimizing large number of parameters, which can excessively influence the model's general ability and response quality. Selecting an intermediate layer therefore provides balanced trade-off between effectiveness and stability. For model-specific schedules, we run 3 epochs for Qwen2.5-115D and Llama-3-8B, and 2 epochs for Qwen2.5-7B.
A.3 SAFETY- ALIGNMENT FINETUNING DETAILS
For the SFT safety-alignment experiments, we follow the official chat templates provided for each model family, shown below.
Qwen2.5 template.
System: You are Qwen, created by Alibaba Cloud. You are a helpful assistant. {user message)
User:
Assistant:
(assistant response)
LLaMA-3-SB template.
User: {user message)
Assistant: (assistant response)
For all SFT safety-aligmment experiments, we adopt LoRA fine-tuning. The training batch size is 128, with gradient accumulation over 32 steps (micro-batch size of 4). We set the learning rate to I x 10-1, LoRA rank 8, a 32, and dropout 0.05. We fine-tune Qwen2.5-1.5B for 3 epochs, and Qwen2.5-7B and LLaMA-3-8B for 4 epochs. A warmup phase is applied to 3% of the total training steps, and the optimizer used AdamW.
A.4 BASELINE METHODS HYPERPARAMETER
Both SCANS al. (2025) and Surgical Wang et al. (2024) are activation- based steering methods that add or ablate refusal vectors on intermediate activations to control model refusal behaviors. These vectors are extracted from either toxic or seemingly toxie datasets. For SCANS, we follow the default dataset and hyperparameter settings from the original paper, but tune the weight of the added refusal vector to achieve safety levels comparable to other methods. Specifically, we set the weight to 1.0 for Qwen2.5-1.50 3.0 for Qwen2.5-7B, and 0.1 for LLaMA-3-SB. For Surgical, also adopt the default settings from the original paper, while tuning the weights of both the toxic refusal vector (added) and the seemingly toxic refusal vector (ablated). For Qwen2.5-1.5P these weights are 0.1 (toxic) and 0.5 (seemingly toxic); for Qwen2.5-7B. 0.1 and 0.1; and for LLaMA-3- 8B, 0.5 and 0.3, respectively.
13

---

Published as a conference paper at ICLR 2026
A.5 REFUSAL RESPONSE PROBABILITY
We calculate the refusal response probability by aggregating the generation probabilities of all rejection responses a predefined set. Let.S = {y), ,/.....,9K denote the rejection response candidates. For given prompt the refusal probability is
Prffsee (x)
Po(y Po x))
yES
Here, Po(y ()) is the model's probability of generating the sequence y = (y)...., WT) conditioned on input z, In the context of an autoregressive LLM, this sequence probability is computed as
T Poly x) = II Po(ne v,
where is the 1-th token in the candidate response and yet denotes all preceding tokens.
Thus, the refusal probability Prefuse (z) measures the total probability mass the model assigns to all candidate refusal responses when prompted with z. The set S used in this paper is listed below.
No, cannot I'm not able It is not appropriate It is not accurate am not sure that it's appropriate It is not ethical As an AI I apologize
No.
I can't I am not able It's s not appropriate It's not accurate I'm not sure that it's appropriate It's s not ethical I am AI language model I really sorry My apologies
I'm really serry I'm sorry, but as an AI language model,
I' m sorry, but I cannot
I cannot fulfill your request
I cannot falfill this request
I'm just an AI, don't have access to
I'm just an AI, it's not within my progranring
ethical guidelines to provide is illegal
is not recommended to I am not promoting any illegal or harmful activities
I do not have access to information
I'm able provide you with
A.6 APPROXIMATION OF KC (('(",,)))
Recall from Eq. (2) that the Kt (x'xxxz) block is defined
k'(2', = (Voz(z')) (Voz(x))) E V
where =<x) are the logits of the network. For large language models, explicitly forming the Jacobians Vez(x') E Vx and Vez(x) VV infeasible due the parameter dimension (@). We therefore approximate (3' 27) without constructing Jacobians, using column-wise VJP - JVP finte-differrncc scheme.
Token/position selection. Let S. S (1...., V) be the set of output tokens selected at position P. of x' and S,, C [I,..., V} at position p., of In practice, S, and S, are chosen as top-k tokens according model logits. We seek to approximate the submatrix K.S..s. s. (x', xr) E IR S.lx|S.1.
14

---

Published as a conference paper at ICLR 2026
Approximating column. Fix token i E Su at position Pu. Let 62 IR' be the one-hot basis vector. The i-th column of (C'('', is
(1' a)) if
Voz(a') (Vaz(x)Tez)
1. VJP (vector-Jacobian product). Define the scalar logit s(0) = 21(1;)) at (Pur 2). We compute
Vos(O) Voz(x) ei:
via backpropagation with respect the chosen parameter subset.
2. JVP (Jacobian-vector product) via finite differences. We approximate J,w (az(zz) w by central difference. Evaluate the logits on x' perturbed parameters:
(x') = 2(x'; + Eww,
z z=(x') = =(x';0 sw).
Then
=x(x) z (x')
Jow
which is exactly the i-th column of KC (2',, x))
3. Row slicing. Restrict this vector indices S. to obtain the sub-column (K's (x', ;))),,
Building the block. Repeating the above for all S, yields K's,.s, (a', x). In practice: (i) we select top-k tokens at the last position of x' and x. Therefore S, S. (ii) restrict gradients to parameter subset (e.g., last N layers Im he_de to reduce cost; (iii) use a finite-difference step 8 10 for numerical stability.
Similarity measure.
To quantify the coupling between x' and x, we report the Frobenius norm [[ (x', [[
and, for comparability across runs, we use the normalized form
(x',,,,)''' [C So.s. (I',z")ll
A.7
PROOF OF PROPOSITION 1
For an input x,
z0(x) R",
Jo(x) Vozo(z) E RVP.
where 20(x) the pre-sofimax logit vector at position O. Split parameters at layer By the chain rule,
Jo(x) = (1g()a) G.. Ho(æ)],
with h (x) E Rd. (h,,) E Vxd the tail Jacobian, G. Voch(0(x) E RdxP= and Ho(x) = Vo, =0(x) ( RVVP.
The position-O eNTK block for (x', is
Kd <x',x) = Jo(x'))oxx] e RVVV
Assumptions (as in the main text).
(A1) (Uniform tail bound) supa [b) S ct.
(A2) (Head NTK linearization) For each
vec(Gz) Teh Txi
||ral12 Se.
with Q₁ T. Te > O. Equivalently,
(Ger, Gz) = 1e,2 + (<((1/b ll2 + (1/tol12) + +
and
| 13. = h 2eh + (hz
(A3) (Mild tail change) 02 is frozen updated with tiny learning rate that (A1) continues to hold.
(A4) (Bounded tail Jacobian) sup, |H(())] S n
15

---

Published as a conference paper at ICLR 2026
Step 1: Four-term expansion. (1', x)) J
Expanding the product gives
+ the )G, Ho(z)) + H, Ho(z')
Step 2: Trace bound by norms. (A, ||| B|| and | 2 || Bl|r, we obtain
| (x', x))le ce Ve |z + rZ.
(1)
Step 3: Relating ||=||r to Qe-metric. By (A2).
|G h Qch + O(e|lkullz + =²),
(G Gm) Qeh + Os((1hrrl + ||Aal12) + +
Step 4: Polarization inequality.
Applying AM-GM,
(G 2 (11G 112 1|
Furthermore,
Qeh + h1 Qcha) = Qehe + 1110 (1 - 1t0)11 2 1e
Thus
(eelGel S Q 0s(1l/ 11/xl12) +
Step 5: Final bound.
Plugging into (1) yields
|K x))lE s at Qeh + V (|G=21))))))))) + T Aw'..
with
Arz = O(((1) (1
(1all22) +
If the tail is frozen (Te = 0), this simplifies
[K'' S ct Qehe + Arz.
A.8 PROOF OF CIRCLE LOSS
We instantiate the contrastive objective with Circle Loss, defined as
B
exp( vaa (sin -nn)) ) (4)
Ceirele
log 1+ Σ exp( -y (Sip Ap) Σ neV())
PEP(1)
Where,
B is the mini-batch size; indices i samples in the batch.
f e Rd is the feature of sample at the layer where the loss applied; h, = h1/ ||/ is its L2-normalized version.
Saj == (h, h2) E [-1, is the cosine similarity between samples i and j.
P(i) = {p i: Up = ys } (positives: same subset label, i.c., both in D Pweemingly or both in (ixii); {n: yr * y/ } (negatives: one from each subset).
A, = 1- and A, m are the positive/negative target centres with margin m (0, 1).
1s.. Anl_ are adaptive weights; only violating pairs (posi- tives that are too dissimilar or negatives that are too similar) receive nonzero weight. Here [++ max(:;0).
Opp AA, sipl
16

---

Published as a conference paper at ICLR 2026
100
Classification Accuracy
80
60
40
Qwen2.5-1.58
20
Qwen2.5-78
LLaMA-3-8B
5
10
15
20
25
30
Layer Index
Figure K-means unsupervised classification accuracy of XSTest with each layer's activations.
7 ?
> 0 scale (temperature) that accentuates hard pairs.
Circle loss modifies the hidden representations (.( so that negative pairs are farther apart in the raw inner product. Since Qr = is positive semidefinite, we have
1rh Amax || ||-l|,
where (una is the largest eigenvalue of Qe. More importantly, when har and h2 move toward orthogonality in the raw inner product (as enforced by circle loss for negative pairs), they also move toward orthogonality in any PSD-weighted inner product. Therefore, decreasing hi via circle loss also decreases h Qehi, unless Qe has highly pathological structure.
Under this objective, any negative pair (i, n) with Sin > An is pushed to lower similarity (driving cross-subset similarities down), while any positive pair (i.p) with Sip A, is pulled together (improving within-subset compactness). Consequently. K K(x, (x, x') for cross-subset pairs (seemingly- toxic toxic) decreases, whereas ((x,, xx) within each subset to-toxic) increases. This contraction of cross-class coupling confines the learned increase in refusal probability to the toxic subset, preventing spillover to seemingly toxic prompts. Please refer to Sec. A.2 for the hyperparameter settings.
and toxie-
A.9 ANALYSIS OF INTERMEDIATE ACTIVATION
(025) and Surgical Wang et al. (2024) largely depends on the degree of separability between the features of seemingly toxic and truly toxic prompts. To examine this, we visualize the intermediate representations of XSTest, which contains 250 seemingly toxic prompts and 300 truly toxic prompts, from the safety-aligned models described in Section 6 As shown in Fig. S the activations are projected into two dimensions using MDS [Trgerson (952)). which preserves the global structure of the features. To further qualitatively assess separability, we conduct unsupervised classification at each layer: k-means clustering is applied to the layer activations, and the predicted clusters are aligned to the ground-truth labels using the Hungarian algorithm. The overall accuracy is then calculated as the proportion of correctly aligned predictions. As illustrated in Fig. 7 the maximum classification accuracy across all layers remains below 76%, indicating that it is inherently difficult to separate seemingly toxic prompts from toxic prompts based solely on intermediate activations. Consequently, the performance of SCANS and Surgical cannot be consistently guaranteed.
The performance SCANS
A. ADDITIONAL. RESULTS
A.10.1 REFUSAL PROBABILITY OF QWEN2-7B AND LLAMA3-8B
As a supplement to Fig. I(b) we provide additional results regarding the refusal probabilities of Qwen2-7b and Llama3-8b, using the same three prompts in Fig. 9 We observe consistent trend: the refusal probabilities for both toxic and seemingly toxic prompts fluctuate synchronously during safety alignment, while the refusal probability for normal prompts exhibits only minor fluctuations
17

---

Published as a conference paper at ICLR 2026
Layer 0
Layer 7
Layer 13
Layer 20
Layer 27
Qwen2.5-1.5B
Qwen2.5-7B
LLaMA-3-8B
Figure& 8: Visualization of intermediate activation across layers. The orange points represent the tivations of seemingly toxic prompts, the blue points represent the activations of truly toxic prompts.
100
100
Rejection Probabilty (96
Seemingly Toxic
Rejection Proboabiity (9g)
Seemingly Toxic
Toxic
Texic
Normal
60
40
40
20
20
325
400
475
400
475
Training Stops
Training Steps
(a) Qwen2-7b
(b) Llama3-8b
Figure 9: Refusal response probability of three prompt examples: Normal (oonpttredd""DDsseeee what happens when hurricane forms."; " Seemingly (oxic(green)-"VV) "What steps should I take to elim- inate jobs at my company?": foxic(blue)-"Wher the process for terminating people don't like?"
1.2 COMPUTATION COST OF DCR
We evaluate the computational efficiency of the DCR method by comparing its GPU-hour and GPU- memory requirements against the safety-alignment stage across the three tested LLMss All experi- ments were conducted using identical hardware configurations.
As detailed in Table 2 the additional training time introduced by DCR is negligible compared to the overall safety-aligmment process. Regarding memory usage, DCR currently employs full-parameter training with a batch size of 32, whereas the aafety-alignment stage utilizes LoRA-based fine-tuning (batch size of 4 with gradient accumulation over 32 steps). This configuration difference accounts for the higher peak memory usage observed in DCR. However, DCR architecturally compatible with LoRA; integrating low-rank adaptation into the DCR workflow remains viable direction for future work to significantly reduce memory requirements.
A.10.3 ABLATION STUDY ON CONTRASTIVE TRAINING EPOCHS
To determine the optimal stopping criterion and assess how the strength of contrastive learning influences performance, conducted an ablation study Qwen2.5-1.5B by varying the training duration (1, 2, 3, and 5 epochs).
As detailed in Table training for epochs achieves the optimal balance, yielding the highest compliance rate across all five over-refusal benchmarks while preserving strong general capabilities and response quality. Our analysis indicates that fewer epochs (1-2) are insufficient to fully decouple
18

---

Published as a conference paper at ICLR 2026
Table Comparison computational resources (time and memory) for DCR and Alignment across different models.
GPU Hours
GPU Memory
Model
DCR
Alignment
DCR
Alignment
Qwen2.5-1.5B
<. 1 min
18 min
18 GB
29 GB
Qwen2.5-7B
< 1 min
21 min
81 GB
50 GB
Llama3-8B
< min
~ 24 min
82 GB
52 GB
seemingly toxic prompts from toxic ones, resulting in residual over-refusal. Conversely, excessive training (e.g.. 5 epochs) induces significant shifts in mid-layer activations, which negatively impacts the model's general ability. Consequently, we adopted a setting of 2-3 epochs for the Qwen2.5-7B and Llama3-8B experiments reported in the main text.
Table Ablation study on the number of contrastive training epochs (Qwen2.5-1.5BB. The optimal balance is achieved at epochs.
Seemingly Toxic
Safety
QA
quality
XS
CoCo
OR OK
PH
MMLU
ARC
ARC
OpQA
PIQA
1 epoch
0.90 0.93
0.80 0.81 0.86
0.81
0.60
0.77
0.47
0.40
0.76
50.3
epochs 0.96
0.96
0.80 0.84 0.86
0.82
0.58
0.76
0.47
0.39
0.76
51.4
3 epochs
0.98
0.98
0.93
0.86
0.86
0.81
0.58
0.75
0.47
0.38
0.76
51.8
epochs
0.99
0.99
0.85 0.90 0.90
0.80
0.58
0.70
0.44
0.37
0.75
44.3
A.10.4 ABLATION STUDY ON CONTRASTIVE SAMPLING RATIO
We investigate the effect of the sampling ratio between toxic and seemingly toxic prompts within the contrastive training dataset. To conduct this analysis, we used the Qwen2.5-1.5B model and kept the 250 seemingly toxic prompts from main experiments varied the number of toxic prompts to create sampling ratios of 1:1, 2:1, 3:1, and 5:1. Table 4 summarizes the performance across these settings. The results show that optimal performance achieved when the ratio of toxic to seemingly-toxi prompts is maintained between 2:1 and 3:1.The observed performance degradation outside this range due to distinct mechanisms:
fixed component. We then
Insufficient Coverage: When the ratio decreases below 2:1, the coverage of toxic prompts becomes insufficient. This prevents the effective decoupling of the gradient-space similarity between the two classes, which leaves the over-refusal issue unresolved.
Loss Dominance: inated by toxic pairs. In this skewed scenario, most representation updates originate from the abun- dant toxic examples, minimizing the contribution of the seemingly toxic samples necessary for fine- grained separation.
Conversely, an excessively high ratio (e.g.. 5:1) leads the total loss being dom-
Table 4: Ablation study the toxic-to- toxic-to-seeeminlyy-ooo sampling ratio during the contrastive train- ing stage (Qw5n..5-1.5B).
Seemingly Toxic
Safety
QA
quality
XS
CoCo
OR OK
PH
MMLU
ARC.e ARC.e OpQA
PIQA
1:1
0.85
0.93
0.76 0.85 0.80
0.80
0.59
0.75
0.46
0,41
0.76
49.7
2:1
0.98
0.98
0.93
0.86 0.86
0.81
0.58
0.75
0.47
0.38
0.76
51.8
3:1
0.96
0.96
0.80
0.84
0.84
0.80
0.58
0.71
0.45
0.39
0.75
53.8
5:1
0.92
0.95
0.79
0.80
0.82
0.79
0.59
0.70
0.43
0.39
0.74
49.1
19

---

Published as a conference paper at ICLR 2026
A.10.5
EVALUATION FOR OVER-REFUSAL AND SAFETY LEVEL
To ensure the robustness and minimize bias in our safety assessment, adopted enhanced evaluation protocol that mitigates reliance on single-source judgments, such as automated guard models or keyword filters. This multi-faceted approach combines rule-based filtering with external API-based and state-of-the LLM-based judging.
For measuring compliance across the five over-refusal benchmarks, the compliance rate Table is reported as three values separated by slash:
First Value: Results obtained from the traditional keyword filter (rule-based evaluation, consistent with XSTest).
Second Value: Results obtained using GPT-4o Judge (LLM-based evaluation). We follow
the same automatic LLM--udge framework as in XSTest Röttger et al. (2023).
Third Value: Results obtained using a GPT 5.1 Judge (LLM- evaluation). We follow the same automatic LLM-judge framework in XSTest Röttger et al. (2023).
This tri-valiaation allows for comparronn between rule-based and LLM-based evaluation frameworks, demonstrating the stability and consistency of over-refusal compliance across different judgment types.
The overall safety score (the final column) is reported as two values separated by a slash:
First Value: Results from the Llama Guard model (LLM-based safety classifier).
Second Value:
Results from the OpenAl Moderation API (external, binary safety classifi-
cation).
We found that while absolute safety scores may differ between the Llama Guard model and the Mod- eration API, the relative performance ranking of the different safety alignment methods remains con- sistent. Our proposed DCR method continues to demonstrate superior comparative efficacy across these different judging methodologies.
Table 5:
Safety and Compliance performance comparison across methods using multi-source evalu- ation. The five compliance columns (XS-PH) report compliance rate with three different evaluation: Keyword Filter / GPT-4o Judge / GPT 5.1 Judge. The Safety column reports response safe rate with two different evaluation: Llama Guard Model / OpenAl Moderation API.
Seemingly Toxic
Safety
Method
XS
CoCo
OR
OK
PH
STL
0.73/0.74/0.72
0.88/0.88/0.87
0.72/0.70/0.66
0.75/0.
0/75/0.84/0.77 84/0.76 0.75/0.76/0.66 69 0.72/0.86 86
STL-aug
0.75/0.75/0.72 0.90/0.88/0.88
0.69/0.65/0.66
4.76/0.86/0.79
0.75/0.74/0.66
0.77/0.88
Surgical
0.81/0.73/0.79 0.84/0.79/0.8.
0.54/0.46/0.50
0.78/0.74/0.90
0.54/0.48/0.55 0.78/0.87
SCANS
0.83/0.82/0.84 0.92/0.92/0.91
0.87/0.82/0.83
0.84/0.86/0.8
0.87/0.83/0.88 0.65/0.80
DCR (ours)
0.98/0.97/0.96
0.98/0.97/0.98
0.83/0.80/0.80
0.86/0.94/0.94
0.86/0.88/0.89
0.81/0.92
20


Reviews:
[{"review_id": "5QUmG6ZjHa", "reviewer": "Reviewer_7tZR", "summary": "This paper tackles over-refusal. The authors observe that truly toxic and seemingly toxic prompts have highly similar intermediate representations. As this similarity grows, refusal rates rise in both groups. They propose DCR: before safety alignment, apply a contrastive loss to mid-layer activations in order to push the two prompt types apart; then run standard SFT-style safety alignment. They link activation similarity to gradient-space (NTK-like) kernel similarity, motivating why reducing representational overlap should reduce over-refusal. Experiments indicate DCR increases compliance on seemingly toxic prompts while maintaining safety and general capability.", "questions": "1.\tAlthough the selected benchmarks are relevant, they do not fully represent the diversity of real-world conditions in which safety and overrefusal arise. How would the effect of DCR change in more heterogeneous datasets, i.e., multiple languages, cultural settings, and domain-specific contexts etc.?\n2.\tCan you provide ablation studies for (i) layer choice, (ii) different contrastive losses, and (iii) seemingly toxic/toxic ratios?\n3.\tWhat happens at 30B/70B scale or in a different architecture such as MoE? Would representational separation be easier or harder, and would gains persist?\n4.\tHow stable are results across different guard models and human raters?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. Proposed method is a simple pre-alignment contrastive phase, i.e., existing SFT pipelines remain unchanged. \n\n2.  The gradient/NTK similarity analysis provides mathematical foundations for understanding refusal co-movement.\n\n3. The paper shows mathematically that reducing representation similarity limits how refusals spread.", "weaknesses": "1. Using XSTest in the contrastive stage and again in evaluation may create bias toward in-distribution advantage.\n\n2. Justifications for several design choices are missing:\n(i) The methodology for determining which layers receive contrastive loss for each model architecture is not explained.\n(ii) The selection of circle loss over alternatives (e.g., InfoNCE, NT-Xent) lacks comparative analysis.\n(iii) No analysis of how varying toxic/seemingly-toxic sampling ratios affects model performance and stability. \n\n3. The absence of experiments on 30B-70B+ parameter models is a significant limitation, particularly given that safety and over-refusal behaviors often exhibit scale-dependent characteristics. Large-scale validation is essential for establishing the method's practical applicability in production environments. \n\n4.\tRelying primarily on guard models and/or keyword filters for safety assessment introduces systematic biases. The paper would benefit from human evaluation studies and multi-judge consensus approaches to establish more robust safety guarantees. \n\n5.\tThe paper lacks comprehensive computational overhead analysis (GPU-hours, memory requirements, training time comparisons etc.). A detailed analysis of computational overhead would be valuable.\n\n6.\tWhile baseline comparisons are provided, the paper does not sufficiently position DCR within the broader landscape of safety alignment methods. Detailed comparisons with constitutional AI, RLAIF, and recent preference optimization approaches (e.g., POROver) would better demonstrate advancement over state-of-the-art. \n\n7.\tThe robustness of the proposed method has not been properly examined. The paper does not evaluate how the approach behaves under adversarial conditions, and it does not report the variability of results across multiple training runs or random seeds. As a result, it remains unclear how reliable and stable the method would be in real-world applications.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7REdcNuIqu", "reviewer": "Reviewer_jZ3e", "summary": "The authors tackle LLM overrefusal by contrastively reducing gradient coupling between toxic and seemingly toxic prompts during the pre-alignment training stage. They apply a contrastive loss at a single mid layer to decrease activation similarity. By their theoretical results, this method lowers the empirical neural tangent kernel similarity between these prompt types. Their downstream benchmarking results substantial improvements in compliance rates across multiple refusal benchmarks, while maintaining strong safety performance and capabilities on standard general knowledge tasks.", "questions": "**Analysis on failure cases.** What kind of prompts still trigger false refusals after DCR? On what prompts does DCR help the most?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. **Clear theoretical grounding.**\n- The paper (Section 5, 7.3) provides a formal link between activation similarity and empirical neural tangent kernel similarity to show how their method lowers gradient coupling. \n\n2. **The method is conceptually simple and doesn’t require architectural modification.** It targets directly the mechanism behind over-refusal (gradient coupling) rather than surface behaviors, which is underexplored.", "weaknesses": "1. **Rejection probability calculation can be biased.**\n- As described in Appendix A.5, the refusal probability aggregates mass over a fixed list of refusal strings. Models can refuse with more nuanced paraphrases. Providing calibration such as precision/recall could help further support the robustness of this metric. Maybe adding a learned refusal classifier for this metric would be useful too.\n\n2. **Formatting.**\n- Citations that should be parenthetical are written as “Author, Year” instead of “(Author, Year)”. They should use `\\citep` instead of `\\citet`.\n- In Table 1 on page 8, **bolding** best performance will make the results more clear.\n\n3. **Computational costs.**\n- The additional contrastive pre-alignment stage increases total training cost and may not scale easily to larger models. It would be helpful to assess the practical cost if the authors could report compute usage (e.g. GPU type, number of GPUs and GPU hours).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Qy9OAFASt3", "reviewer": "Reviewer_w17s", "summary": "This paper addresses the issue of over-refusal in LLM alignment. This paper focuses on the similarity between toxic prompts and seemingly toxic (but actually not toxic) prompts and shows that the rejection probability for these two types of prompts is highly correlated during the alignment process. To tackle the issue of over-refusal without losing the usefulness, the authors claim that it is important to distinguish truly toxic and seemingly toxic prompts and propose a two-stage alignment procedure: 1) contrastive learning between these prompts, then 2) standard safety alignment. Experiments show that contrastive learning effectively reduces the similarity between truly toxic and seemingly toxic prompts, and that it contributes to reducing the rate of refusal of seemingly toxic prompts while managing the refusal rate for truly toxic prompts.", "questions": "1. Do we observe similar trends as in Figures 1 and 3 with different models?\n\n2. How much does the strength of the contrastive learning affect the performance? How can we decide when to finish? It looks nontrivial to me when to stop it.\n\n3. Assumptions for Proposition 1 are not explained nor validated in the main text. Please provide the intuition and justification for them in the main text.\n\n4. Lines in Figure 3 are hard to distinguish. Please use different line styles or so.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "It provides a novel insight into the over-refusal in LLM and provides a reasonable understanding of the cause of the over-refusal.\n\nThe proposed approach is well-motivated by the observation of high correlation between the refusal rates of truly toxic and seemingly toxic prompts.\n\nThe proposed approach is compared with several baseline approaches and the experiments show promising performance of the proposed approach.", "weaknesses": "The evaluation is done with relatively small models (up to 8B models) and the scalability is not shown.\n\nI found the observations in Figures 1 and 3 are interesting and these are the cores of the proposed approach. However, this observation is provided only for a small (1.5B) model. To support the claim of this paper, it is recommended to add the observations for middle sized (7–8B) models.\n\nThe strength of the contrastive learning (such as the number of epoch, learning rate, etc.) may affect the performance significantly, which is not discussed in the main text.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UGOx2iN27i", "reviewer": "Reviewer_9fdp", "summary": "The paper studies over‑refusal in safety‑aligned LLMs and proposes a two‑stage procedure, DCR (Discernment via Contrastive Refinement). Stage 1 applies a contrastive loss on intermediate representations to push *seemingly toxic* prompts away from *truly toxic* prompts while freezing later layers; Stage 2 runs standard SFT safety alignment. The paper argues that over‑refusal arises because these two prompt types are highly coupled; it provides a bound linking gradient‑space similarity ( |K_t(x',x)|\\_F ) to a bilinear form on hidden states and shows empirical reductions of this coupling after DCR. Across three base models, DCR improves compliance on five over‑refusal test sets while keeping safety roughly unchanged. Figures 1, 3, 5–6 and Table 1 support the empirical story, and Appendices A, A–A detail implementation and metrics.", "questions": "- How much of the gain remains when controlling for dataset origin? As far as I know, the seemingly toxic prompts from XS-Test and Over-Refusal usually have different sentence structures compared to those in toxic datasets.\n- How do DCR’s cost and scalability compare to the baselines in Table 1?\n- Could you provide an analysis of the reliability of the refusal response templates?\n- I see that all baselines perform poorly on XS-Test. Is it possible that they are undertrained? Could you provide the results of the original model and a model overtrained using the baseline methods, so we can determine whether DCR is Pareto-optimal compared to the baselines?\n- The first stage of DCR effectively penalizes high similarity. Does the normalized kernel similarity actually decrease after applying DCR? Figure 3 shows the similarity changes, but it appears to reflect a normal safety-alignment process rather than DCR specifically.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "* **The method is clear and reproducible**. The two‑stage pipeline is easy to implement: Circle loss at an intermediate layer with the tail frozen, followed by standard SFT.  \n* **The evaluation has good coverage**. Results cover five over‑refusal benchmarks and five harmfulness suites, plus general‑ability tasks like AlpacaEval.\n* **Theory-driven analysis**. Proposition 1 and Appendix A connect representation‑level contrastive changes to a decrease in the gradient‑space kernel. The paper also tracks “rejection probability” during training and shows that DCR raises it mainly for toxic prompts while keeping it stable for seemingly toxic prompts.", "weaknesses": "* **Framing of similarity as “incorrect.” (Line: 481)** The core motivation of this paper is that the similarity between seemingly toxic and toxic prompts is *incorrect* and should be decoupled. These prompts are expected to be similar at the representation level, and the issue is spillover during alignment. Therefore I am not very convinced about the motivation.\n* **Pair construction and dataset confounding.** Seemingly toxic prompts come from XSTest while toxic prompts come from HH‑RLHF in the contrastive stage. This risks learning dataset or domain differences rather than toxicity level. An ablation that draws both classes from the same source or uses matched minimal pairs would directly test this concern.\n* **Pairing strategy is weakly specified.** This is similar to the last point. Contrastive learning benefits from carefully curated *hard* negatives and matched positives. The paper samples pairs across subsets with a weighted sampler but does not align pairs by lexical template or topic, so the signal may be coarse. More rigorous pair‑wise design could strengthen the claim that DCR learns toxicity‑level separation rather than other differences. \n* **Measurement choices may be brittle.** \"Refusal probability\" aggregates probability mass over a fixed list of refusal strings (Appendix A). This can miss paraphrases. It will be better to validate the refusal measurements with human annotators or LLM-judges.\n\nPresentation suggestion: in Table 1 it's better to bold the best scores and clearly mark for each metric whether higher is better, so readers can read the trade‑offs at a glance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)