ProSafePrune: Projected Safety Pruning for Mitigating Over-Refusal in LLMs
Authors: Zijun Chen
Wenbo Hu
Ya Li
Lei Miao
Guoping Hu
Richang Hong
Venue: ICLR 2026 Poster (2026)
Keywords: LLM
Safety
Over-Refusal
Alignment
Primary area: alignment, fairness, safety, privacy, and societal considerations

Abstract: Large Language Models (LLMs) excel in various domains, but their safe deployment faces the challenge of balancing safety and utility. Existing alignment strategies often strengthen refusal mechanisms to reduce harmful outputs, but harmless instructions with superficial risky words are mistakenly rejected, which is known as over-refusal. 
This work first reveals that over-refusal stems from a cognitive bias in the model's internal representation space: LLMs naturally encode safety attributes in hidden states, and pseudo-harmful instructions overlap with harmful features, causing over-harmful encoding. 
To address this, we propose ProSafePrune, a subspace-projected low-rank parameter pruning framework for mitigating LLM over-refusal. By projecting pseudo-harmful features into subspaces and removing low-rank directions corresponding to harmful components in the most discriminative layers, we significantly reduce over-refusal while preserving the model’s ability to reject genuinely harmful requests, improving performance on general tasks. In experiments, across different models, our method significantly lowers the average false rejection rate while slightly improving general task performance.

Published as a conference paper at ICLR 2026
PROSAFEPRUNE: PROJECTED SAFETY PRUNING FOR MITIGATING OVER- REFUSAL IN LLMS
Zijun Chen! Wenbo Hu!.", Ya Li?, Lei Miao?, Guoping Hu², and Richang Hong'
Heefi University of Technology chenzi jun@mail www..hffutellele cn, hongre. hnfuttgmal com
FLYTTEK Research, Hefei, China, {yali, leimou, guopinghu (it.gghuuueillyee..coo "Corresponding author: wenbohuthfutt edu.cn.
ABSTRACT
Large Language Models (LLMs) excel in various domains, but their safe deploy- ment faces the challenge balancing safety and utility. Existing alignment strate- gies often strengthen refusal mechanisms to reduce harmful outputs, but harm- less instructions with superficial risky words are mistakenly rejected, which is known as over-refusal. This work first reveals that over-refusal stems from cog- nitive bias in the model's internal representation space: LLMs naturally encode safety attributes in hidden states, and pseudo-harmful instructions overlap with harmful features, causing over-harmful encoding. To address this, we propose ProSafePrune, subspace- projected low-rank parameter pruning framework for mitigating over-refusal. By projecting pseudo- features into subspaces and removing low-rank directions corresponding harmful components in the most discriminative layers, we significantly reduce over-refusal while preserving the model's ability to reject genuinely harmful requests, improving performance on general tasks. In experiments, across different models, our method significantly lowers the average false rejection rate while slightly improving general task per- formance. Code: https: /tt:::/gtttie thub. com/ FEfutm//PROSAPPPRUU
INTRODUCTION
Large Language Models (LLMs) achieve strong performance across domains, yet their safe deploy- ment hinges on balancing safety and usefulness (Bai et al.) 2022; Bianchi et al,, 2024: Dai et al. 2024 Huang et al. 2024 Zou et al. 2024 Wang et al. 2026 Gou et al. 2025). Current alignment practices typically strengthen refusal mechanisms suppress harmful outputs, but this often results in over-refusal (Röttger et al. 2024 Zhang et al. 2025a rejecting not only genuinely harmful re- quests but also harmless ones that merely contain superficially risky cues. This undermines usability and has long been viewed as side effect of overly conservative alignment.
Building on this observation, prior research has explored two main lines of mitigation. Training- based methods (Zhang et al., 22255b Dabas et al. [2025; Wu et al. 2025) can be effective, but they demand additional data and computation, limiting practicality. Training-free methods (Cao al. 2025 Shi et al. 2024) provide greater flexibility, yet they often fail to resolve the underlying bias and frequently introduce extra inference-tine overhead. These limitations highlight the need for an approach that is both lightweight and able to directly address the root cause of over-refusal.
In contrast, we firstly reveal that over-refusal is not merely a failure of existing alignment pipelines, but rather manifestation of an inherent cognitive bias in the model's internal representation space. Prior studies have shown that hidden states of LLMs naturally encode safety attributes of input in- structions (Li et al. 2025 Zhao et al.) 2025). However, pseudo-harmful instructions often project simultaneously onto both the harmful subspace and the harmless subspace. Under excessive safety fine-tuning, this natural overlap becomes distorted: the harmful projection is disproportionately am- plified while the harmless one is suppressed. As a result, benign instructions are over-encoded with harmful features, shifting the internal decision boundary and ultimately leading to false rejections.

---

Published as a conference paper at ICLR 2026
Interestingly, this phenomenon- which we term over-harmful encoding alooo provides insight into the origin of the alignment tax (Huang et al. 2025). where models are pushed into an overly cautious regime that further constrains their general task performance.
In this work, we introduce a new perspective for mitigating over-refusal by directly acting on the parameter space of the model, rather than relying only activation-level interventions costly training. Our key insight is that over-refusal arises because model parameters contain excessive low- rank components that disproportionately encode harmfulness, which in turn causes pseudo-harmful instructions to be misclassified. Unlike training-based methods, our approach requires only a small amount of auxiliary data and avoids expensive fine-tuning; unlike existing training-free methods, does not depend on repeated inference- timen adjustments that introduce extra overhead while failing to address the underlying cognitive bias. Instead, we construct subspace projections to disentangle pseudo-harmful and harmful features, and apply low-rank pruning strategy to selectively remove harmful components from the most discriminative layers. Since these directions occupy only a tiny fraction of the parameter space, pruning them alleviates over-refusal while minimally affecting the model's s overall behavior, thus striking a better balance between safety and usefulness.
In summary, contributions are as follows:
We reveal that over-refusal in LLMs is caused by a cognitive bias in internal representa- tions, due harmless instructions being over-encoded with harmful features.
To address this, introduce ProSafePrune which prunes low-rank harmful components from the model's parameters, reducing over-refusal without additional inference-ting cost.
We identify that the model's internal over-harmful encoding is closely related to the align- ment tax, which limits general performance. Our method effectively corrects this issue, leading to improved performance on general task.
2
RELATED WORK
2.1 OVER-REFUSAN IN LLMS
Safety alignment technologies for LLMs are critical to mitigating the risks malicious instruc- tions. Mainstream approaches include Supervised Fine-Tuning (SFT) and Reinforcement Learning from Human Feedback (RLHF), with the goal of preventing models from generating harmful con- tent responding malicious requests (Bai et al.) 2022 Bianchi al., 2024). However, existing safety alignment solutions generally suffer from an over-defense issue: models unnecessarily reject pseudo-harmful prompts-inputs that are semantically harmless but contain superficially risky vo- cabulary. The academic community defines this phenomenon as over-refusal or exaggerated safety (Röttger al., 20244) Varshney et al.) 2024). For instance, the LLM incorrectly refuses benign re- quests like "how to kill the lights in the room" solely due to the word "kill" (Cao et al.) 2025). Such behavior severely undermines model utility and highlights the intractable trade-off between harmlessness and helpfulness.
Fundamentally, over-refusal stems from the model's generalization bias toward safety signals: dur- ing the safety alignment process, models tend to forcibly associate risky vocabulary with refusal be- havior, while ignoring contextual semantic differences. Early studies have confirmed that this bias widespread across mainstream LLMs and exhibits a positive correlation with the model's jailbreak resistance-thh more a model prioritizes defending against malicious instructions, the more likely is to over-refuse benign prompts (Huang et al.) 2024). This finding has driven systematic academic research on over-refusal, covering areas such as evaluation benchmark construction and mitigation method design. While these findings are often interpreted as limitation of external alignment strategies, we argue that over-refusal reflects a deeper cognitive bias in the model's internal repre- sentation space. In other words. the problem is not just how alignment applied at the output level, but how safety attributes are encoded and distorted within the hidden states themselves, leading to wver-harmful encoding of pseudo-hasd harmful instructions.
2.2 OVER-REFUSEE MITIGATION
Existing mitigation methods fall into two categories: waining-based and training-free.
2

---

Published as a conference paper at ICLR 2026
Training-basee These methods correct over-refusal by adjusting model parameters, recalibrat- ing the safety decision boundary with pseudo-harmti data. Safety Patching (Zhao et al. 2024) generates gradient- based patches-enhange pseudo-harmful ones-ant and integrates them into specific layers. Zeng et al. show that supervised fine-tuning (SFT) on paired pseudo-harmful data reduces over-refusal. Dabas et al. (2025) further identify safety-critical layers, extract refusal direction vectors, and fine-tune only those layers via dedicated refusal loss.
refusal for harmful prompts and suppressing for
Training- frree. These methods avoid parameter updates and intervene at inference. Ray & Bhalani (2024) propose a prompt-based strategy. [Shi et al. (2024) (Self-CD) decode prompts twice wwith and without safety prompt- and suppress refusal tokens via probability comparison. Early work (Zou et al., 2023a) identified refusal--elated activation direction, which can manipulated to alter model behavior. Building on this, [Ca et al. (2025) extract refusal steering vectors from harmful vs. benign activations, locate safety-critical layers via vocabulary projection, and design cosine- similarity classifier for adaptive adjustment. WWang al. (2025) separate true vs. false refusal vectors by orthogonalization, ablating only the false ones to preserve genuine safety.
While effective, waining-based methods require costly retraining, and training-frre ones add infer- ence overhead without addressing the root cause. In contrast, our pruning approach is data-efficient, cost-free at inference, and directly corrects the internal cognitive bias driving over-refusal.
3 METHODOLOGY
We tackle over-refusal by directly addressing the model's internal over-harmful encoding. First, we analyze layer-wise activations to reveal how pseudo-harmful instructions overlap with harmful features. Then, apply subspace-based low-rank pruning remove only the harmful components within pseudo-harmful directions, preserving genuine refusal ability while reducing over-refusal. Pruning is applied to layers with the strongest feature separability, ensuring minimal impact overall utility.
3.1 TTER-HARMFULNESS ENCODING ACROSS MODEL LAYERS
Numerous studies have shown that the hidden states of LLMs contain rich information (Chen al. 2026 Burns et al. 2023 Chen et al. 2025 Skean et al. 2025). including representations of harmful- ness towards instructions. This encoding of harmfulness may influence the model's 's final output de- cisions, with excessive harmful encoding potentially being contributing factor to the phenomenon of over-refusal. To investigate the underlying mechanism of this phenomenon, we analyze the acti- vation distributions of LLMs at different layers in response to pseudo-harmful instructions, aiming to uncover the evolution of harmfulness representations within the model's s internal layers.
To analyze the harmfulness encoding of instructions, we use probes (Alain & Bengio 2016), which are trained to predict the harmfulness of instructions based on activation vectors extracted from the model's layers. Specifically, for each weight matrix Wim at the 1-th layer of the model, extract the activation output for an input instruction æ, which is then compressed into a vector via mean pooling. A logistic regression classifier is trained safe and harmful instructions, and this classifier is used to predict the harmfulness of pseudo-harmful instructions. For more details, see Appendix A
Figure μ shows the layer-wise harmfulness encoding of pseudo-harmful instructions in LLaMA-2- 7B and LLaMA-3-8B. We observe that LLaMA-2 exhibits stronger harmfulness signals deeper layers, which correlates with its higher false rejection rate compared to LLaMA-3.
3.2 SUBSPACE- SUBSPACE-BASED PARAMETER PRUNING
Probing experiments reveal that in severely over-refusal models, pseudo-harmful instructions retain excessive harmful components in their high-level hidden representations, overlapping with gen- uinely harmful features. Despite the model's ability to leverage global knowledge to distinguish harmless from those instructions, deep parameter mappings continue to amplify harmful features. Over-refusal, therefore, results from over- harmfulized parameters rather than a feature-level artifact. While post-hoc interventions (e.g., activation editing, vector shifting (Cao et al. 2025 Wang al.
3

---

Published as a conference paper at ICLR 2026
0.88
10
Avg harmfu pmob
30
Avg harmfull prob
0.6
Layer
15
35
70
0.4
20
0.4
25
0.2
25
0.2
30
30
Up
Gate Down
O Up
Gate
Down
Module
Module
Figure 1: Layer-wise harmfulness encoding of pseudo-harmful instructions in LLaMA-2-7B vs. LLaMA-3-8B. In the early layers, pseudo-harmful instructions strongly biased toward harm- fulness due to lexical similarity with genuinely harmful prompts. Middle layers show improved separation, as global semantics emerge. However, deeper layers in LLaMA-2-7B amplify harm- ful features again, whereas LLaMA-3-8B maintains much lower harmfulness encoding. Consistent with this, PHTest (a pseudo-harmful dataset) evaluation shows LLaMA-2-7B's false rejection rate is 38.5%, significantly higher than LLaMA-3-80 10.5%.
2023 reduce rejection, they cannot fundamentally alter the parameter structure. To address this, inspired by Wei et al. (2024), we directly modify the weight matrices, removing directions that amplify harmful components. We aim prune weight components in parameter space that push pseudo-harmful representations toward harmful directions. We first identify these directions in the output space and then map them back to the row space of the weights for removal.
Subspace Extraction. Our first step is to identify the dominant harmfulness directions in the out- put representation space. Intuitively, certain dimensions of hidden representations are responsible for distinguishing safe from harmful instructions, and secudo-harmful instructions project dispro- portionately onto these harmful dimensions. To explicitly decompose such directions, we employ truncated singular value decomposition (SVD). Specifically, consider the m-th submodule (e.g.. V,O, at layer with weight matrix Wi.m E the X Given hidden input h(x), the submodule output is at,m (x) W(r) Rd Sequence pooling produces vectorized representation (r). Collecting these across safe, harmful, and pseudo-hammn harmful datasets D., Du; Dp yields matrices A A(4). Alm truncated SVD:
We then apply
( s U (t) p(UT 1,m
LE u8,up))
where U(th Bddue contains the top Tt left singular vectors, and 1I U U captures the subspace.
Theorem 3.1 (Optimality of SVD Projections). For any submodule outpul matrix, the rank-r trun- cated SVD yields the optimal low-rank upproximation under the Frobenius norm, ensuring that the extracted subspace directions minimize information loss in theory.
The proof is provided in Appendix A.2 harmful subspaces we extract are not only empirically effective but also theoretically optimal in representing the most discriminative directions.
This guarantee ensures that the harmful, safe, and pseudo-
Locating the Harmful Amplification within Eseudo-harmfi Direction.
To precisely locate such directions, we design an overlap operator that selects only those parts over- lapping with harmful but with safe subspaces:
2 = (1 - ((s) (a)
[(()) Lm
(1)
I. 1,m
This construction emphasizes three aspects: (i) focus on the pseudo-harmfus principal directions via
: : (ii) within them, extract the components that overlap with harmful directions through IT(w).
4

---

Published as a conference paper at ICLR 2026
and (iii) exclude those aligned with the safe directions using (7 - (m- thus avoiding damage correct safety encoding. This chained selection makes attenuation highly selective, targeting precisely the harmful amplification that needs to be weakened.
From Output Subspace to Weight Row Space. While Ωt.m is defined in the output space, the relation a = Wh implies that applying Ω is equivalent to left-multiplying the weight row space. We thus define:
A i. = 9tm Wi,,>
(2)
and perform low-rank parameter pruning
W
1m W L.m
e [0. 1].
(3)
This operation can be understood "carving out" the row-space components that specifically push pseudo-harmful representations toward harmful directions, where X controls the strength of pruning. As \ increases, more harmful components are removed, balancing over-refusal and model perfor- mance.
Theorem 3.2 (Energy Bound Overlap Operator), Let W be the weight matrix of submodule and mm the overlap operator defined in equation with effective rank r. Then the relative energy removed by pruning ulong 12mm satisfies
10 W [
(T(W)'
where sr(W) = [/W1?//W// is the stable rank of W. Since r is small while sr(W) is rypically very large I.LMs,, the pruned energy occupies only a negligible fraction of the total.
The detailed proof is provided Appendix A.33 This implies that pruning has minimal effect on the model's overall capability, ensuring that method mitigates over-refusal without significantly harming task performance.
Average Silhouette Score
0.05
t-SNE Dimension 2
0.04
0.02
-10
Safe
0.01
Eseudo-Hamfuul
Harm
0.00
-40
20
2>
t-SNE Dimension
Layer (Bottom to Top)
Figure 2: t-SNE visualization of the gate weights in the 13th MLP layer (left) and average silhouette scores (right) for LLaMA-2-7B, illustrating that middle layers achieve clearer feature separation and higher cluster quality.
3.3 LAYER SELECTION FOR PRUNING
To identify target layers for pruning, followed Dabas et al. (2025). we leveraged t-SNE (Maaten & Hinton 2008) to visualize output activations and observe feature distribution patterns in a low- dimensional space. Simultaneously, we calculated the average silhouette score (Rousseeuw 1987) for each layer to quantify the separability of feature clusters.
The silhouette score for an activation vector I; defined as:
b(t) a(i)
s(i)
(4)
max(a b())
5

---

Published as a conference paper at ICLR 2026
where a()) is the mean intra-cluster distance, and b(i) is the minimum distance to all other clusters. The layer-level silhouette value is the average of s(i) across all activations in that layer.
As shown in Figure 2 the middle layers of LLaMA-2-7B demonstrate clearer cluster separation and higher silhouctte values, indicating their superior ability to distinguish different feature types. This supports previous findings that middle layers crucial for safety-related feature discrimination (Li
eet al. 2025
Cao et al. 2025).
We selected these high-scoring middle layers candidates for pruning, as their enhanced separabil- ity allows effective pruning without significantly compromising model capabilities. A subsequent validation search on these layers (5-10 layers near the highest scoring layer, based model size) using a dataset of 50 pscudo-harmful and harmful instructions led to the selection of optimal pruning layers based on the TS score (Section 4.1 This ensures precise pruning with minimal impact on overall functionality.
4 EXPERIMENTS
4.1 EXPERIMENTAL. SETTINGS
Dataset for Subspace Construction. To construct the corresponding subspaces for harmful, be- nign, and pseudo-harmful instructions, we used three distinct datasets: Dnamiul Doenigns and Dpseudo- For Dnamful. used HEX-PHI (Qi et al., 2024), which provides wide range of harmful instruc- tions designed to capture diverse harmful content. For Dbenign we selected Dipaca-Cleanee (Taori
et al.
2023). which contains harmless, neutral instructions. For Deseudo we used OR-Bench (Cui
ct al.
2025)
(see Appendix B.I).
Dataset for Evaluation. To evaluate over-refusal mitigation, used OR-Bench-Hard- 1K (Cui ct al. 2025). PHTest (Zhu et al. 2024). XSTest (Röttger et al. 2024). and OKTest (Shi et al. 2024]. which contain pseudo-harmful instructions for testing harmless harmful distinction. To en- sure safety preservation, we further evaluated on AdvBench (Zou et al. 2023b) and JailbreakBench (Chao al., 2024), targeting adversarial and jailbreak robustness. Finally, we examined general ca- pabilities on MMLU (Hendrycks et al. (021)) CommonsenseQQA (Talmor et al. (019)), and GSM8K
(Cobbe et al. 2021 to verify that safety improvements do not degrade overall performance (see
Appendix B.2
Baselines and Models. We compare our approach against three recent state-of-the-art methods for mitigating over-refusal: SELF-CD (Shi et al., 2024), SCAN (Cao et al. 2025). and SURGICAL (Wang et al. 2025). For model backbones, we conduct experiments on a range of widely adopted
LLMs across different sizes and families: LLaMA-2-77 (Touvron et al. 2023), LLaMA-2-13B,
LLaMA-3-8B, Qwen2.5-7B (Team 2024), and Qwen2.5-14BB (see Appendix B.3).
Metrics. Following Dabas et al. (2025), we evaluate over-refusal with three metries. The Com- pliance Rate (C.R.) the proportion of compliant responses, i.e., instructions handled correctly without falsely rejecting harmless inputs. The Safety Score (S.S.) is the proportion of refusals to harmful prompts, reflecting the model's safety. To capture the balance, the Tradeoff Score (T.S.) is defined as the average of C.R. and S.S. For rejection and safety evaluation, adopt WildGuard (Han et al. 2024). a classifier trained to detect harmful content and refusals, shown to be accurate and reliable.
4.2 EXPERIMENTAL RESULTS
ProSafePrune significantly reduces the model's excessive harmful perception of instructions. Experimental results show that after pruning, as shown in Figure 3 the harmfulness perception of pseudo-harmful instructions detected by probes in the activations of the pruned layer and its subsequent layers is significantly reduced. This indicates that our method effectively alleviates the model's over-harmful encoding in its internal representation of instructions. As observed in Section 3.1 related to the over-refusal problem, the model's perception of harmfulness in pseudo- harmful instructions is notably weakened in deeper layers. This aligns with how less aggressive over-refusal model handles apeudo-harmfin instructions, and the result suggests that our method effectively addresses the internal mechanism that leads to over-refusal.

---

Published as a conference paper at ICLR 2026
0.
0.8
10
15
prob
15
15
20
0.4
20
Avg
Avo
8
Up Gabe Down
Up
Gite
Down
000
Module
Module
Figure 3: Layer-wise probe heatmaps of pseudo-harmful instructions LLaMA-2-74 before (left)
and after (right) pruning, showing that ProSafePrune effectively reduces wer-harmful encoding.
Model
Method
OR-Bench
PHTest
XSTest
OKTest
AdvBench
JBB
Avg. C.R.
Avg. T.S.
65.5
Self-CD
43.5
81.5
86.0
78.7
96.5
89.0
72.4
79.2
LLaMA-2-7B
SCAN
27.0
88.8
99.3
99.5
94.0
75.0
82.3
Surgical
57.5
88.5
81.2
72.7
99.5
95.0
75.0
82.4
Ours
73.0
94,5
88.8
81.7
98.5
94.0
84.5
88.4
Default
9.5
65.5
68.0
54.7
100.0
97.0
49.4
65.88
Self-CD
34.0
84.8
67.0
99.5
94.0
67.2
77.1
LLaMA-2-13B
SCAN
4.0
49.0
42.0
36.7
100.0
99.0
32.9
55.1
Surgical
27.5
79.0
72.4
58.0
99.5
96.0
59.2
72.1
Ours
52.0
86.0
90.4
71.0
99.5
96.0
74.9
82,5
Default
Self-CD
86.0
95.0
100.0
78.7
87.5
73.0
89.9
86.7
LLaMA-3-8B
SCAN
48.0
98.0
81.0
98.0
94.0
85,3
Surgical
64.0
94.0
96.4
80.0
96.0
93.0
83.6
87.2
Ours
71.0
95.5
99.2
81.3
96.5
93.0
86.8
89.4
Table I: Comparison across benchmarks: the first four datasets measure compliance rate (C.R.), and the last two measure safety score (S.S.). Our method achieves the best trade-off, improving compliance on pseudo-harmful datasets while preserving safety on harmful ones. Avg. C.R. is averaged over pseudo-harmful benchmarks, and Avg. T.S. the mean of Avg. C.R. and S.S. Results for the Qwen series and larger models are provided in Appendix
ProSafePrune effectively mitigates over-refusal while preserving safety. Table T compares compliance and safety across multiple benchmarks. Our method consistently achieves the best trade-off between reducing over-refusal and maintaining safety. On pseudo-harmful benchmarks, ProSafePrune achieves notably higher compliance rates-for example, LLaMA-2-7B, it raises compliance to 73.0% vs. 57.5% (Surgical) and 43.5% (Self-CD). At the same time, safety scores on harmful benchmarks (AdvBench, JBB) remain competitive, with negligible degradation. Averaging across datasets, ProSafePrune improves the trade-off score (T.S.) by +2-9 points over the strongest baselines, demonstrating that can effectively mitigate over-refusal without compromising genuine safety.
ProSafePrune does not reduce the general capabilities the model. Instead, as shown in Figure 4 it enhances them to certain extent. Taking Llama2-7B
example, after applying ProSafePrune's low-rank pruning, the model's performance on general task benchmarks showed positive changes: monQA, there was a noticeable rise from 49.0 to 53.0; and on GSM8K, it also saw a modest im- provement from 23.0 25.5. Results for the Qwen series provided in Appendix
MMLU, the
slightly increased from initial 37.1 to 39.6; for Com-

---

Published as a conference paper at ICLR 2026
LLaMA-2-7B
LLaMA-2-138
LLaMA-3-8B
Accuracy (%)
50
60
37.1
39.6
49.0
50.0 49.6
56.5
64.0
61.1
74.0 75.0
68.0
70.5
30
23.0
25.5
31.5
31.5
40
20
20
20
10
CommonQA
GSMBK
CommonQA
MMLU
CommonQA
GSMBK
Initial
+SSPrune
Figure 4: Performance on general datasets, showing results before and after applying ProSafePrune.
Submodulc
Default
Q-proj
K.proj
V. V.proj
O.proj
MLP
All
C.R.
11.0
10.5
11.5
30.5
16.0
19.0
73.0
Table 2: Contributions of Llama2-7B to various submodules in orbench.
FURTHER ANALYSIS
5.1 ABLATION STUDY
Impact of Pruning Individual Submodules on Model Performance. Our pruning operation tar- gets the entire layer structure of the Transformer model. Understanding the contribution of submod- ules like Q, K, V. and MLP is crucial for improving the method. To explore this, we conducted control experiments by pruning individual submodules. The results, shown in Table reveal that pruning the V weight matrix significantly alleviates model's overfitting issue, with adherence increasing from 11 to 30.5. In contrast, pruning other submodules yields only slight improvements. Overall, pruning single submodule is less effective than pruming the entire layer, suggesting that the overfitting issue likely arises from the interaction between submodules rather than the impact of any single module.
Pruning Only the Harmful Subspace. To further validate the role of the pseudo-harmful subspace in pruning. we conducted a controlled experiment by simplifying the original operation to remove only the harmful subspace, without considering its overlap with the pseudo-harmful subspace, i.e..
by dropping [1(7) from Equation We tested this setting on LLaMA-2-7B while keeping all other configurations unchanged, and the results are presented in Table 3 On pseudo-harmful datasets, this simplified method indeed achieved higher compliance rates (9.55% and 96.5%, respectively), indicating that the model becomes more likely to accept pscudo-harmful instructions. However, genuinely harmful datasets, the safety scores dropped sharply (79.5% and 71.0%, respectively), suggesting that the model lost its ability to reliably reject harmful instructions. This comparison demonstrates that pruning only the harmful subspace undermines the model's ability correctly encode harmfulness. Incorporating the pseudo-harmful projection component is thus essential for maintaining balance, ensuring that over-refusal is reduced while genuine safety is preserved.
Rank r, X, Pruning Layers and Harmful Overlap. Appendix D.I provides the analysis of rank r' and the derivation the stable rank bound, while Appendix presents the ablation on A. Ap- pendix D.3] shows that the overlap between pruned directions and genuinely harmful subspaces remains low, indicating that pruning primarily removes hover-harmfulized" components without causing substantial damage to the model's s ability to encode real harmful signals. Appendix D.4] demonstrate the performance advantages of pruning the middle layers over pruning both ends, and the sensitivity to the selection of middle layers is relatively low.
5.2 DISCUSSION ON COGNITIVE BIAS AND ALIGNMENT TAX
Visualizing Word Associations Cognitive Bias. To further illustrate cognitive bias in the model's internal representations and the corrective effect of ProSafePrune, conducted more detailed
8

---

Published as a conference paper at ICLR 2026
Method
OR-Bench
PHTest
AdvBench
JailbreakBench
Avg. T.S.
Default
11.0
61.5
100.0
96.0
67.1
PruSatePrune
73.0
94.5
98.5
94.0
90.0
ProSafePrune woo (P)
90.5
96.5
79.5
71.0
84.4
Table 3: Comparison of compliance rate on pseudo-harmful datasets and safety score on harmful datasets for LLaMMA2-77. Removing the pseudo-harmful component IT(P) increases compliance but significantly reduces safety,
70
Harmfulness
10
MMLU
Pseudo Init
CommontA
Average Preject tokens)
Pseudo Pruned
60
0.8
Harmful
52.0
53.
Harmless
Score
50
49.0
Qwen (general)
Llama (general)
40
3
Jlama-Prunce (general)
0.
25.5
0.2
Initial
Pruned Condition
Fine-tuned
-12
-10
-2
Layer Index
Figure 5: Relationship between alignment tax and general capabilities in LLaMA-2-7B.
Figure 6: Mapping probabilities of rejection words from the last few layers.
interpretability analysis using the Logits Lens method (Wang 2025). This method decodes early model layers capture associations between intermediate layers and target vocabulary. We calculate the total probability of typical rejecting words like "I," "sorry," and "cannot," which are often used by the model to start refusal responses, to provide an intuitive measure the model's internal cognitive mapping. As shown in Figure @ (solid line), the model (IJama2-7B) exhibits significant probability differences later layers for both harmful and harmless datasets. However, for the pseudo-harmful dataset, the model shows clear "cognitive bias" with higher than expected dangerous mapping probabilities. After applying ProSafePrune, the probability of dangerous mappings for the pseudo- harmful dataset is significantly reduced, indicating effective correction of the model's cognitive bias.
Alignment tax may related to the over-harmful encoding. Encouraged by the observation of general capability enhancement shown in Figure we further explored the relationship between alignment tax and the model's 's internal representation. To verify this, we conducted an experiment compare the changes in the model's internal harmfulness score (the average probe harmfulness probability values of the pruned layers and all subsequent layers) and the changes general ca- pability scores after pruning. Then, we performed refusal fine-tuning on the pruned model (using a dataset of pseudo-harmful instructions paired with refusal responses) to simulate excessive safety alignment. Subsequently, as shown in F right, found that the model's internal harmfulness probability score increased significantly. while the general capability scores dropped: MMLU fell 36.4, CommonQA to 52.5, and GSM8K 22.0. This result, illustrates the correlation between the alignment tax, internal over-harmful encoding, and the decline in general capabilities.
Analysis how pruning slightly enhances general capabilities. Qi et al. (2025) point out that current safety alignment mainly focuses on shallow dignment-aalinnen ative distribution of the model's first few output tokens (such as forcing the output of refusal prefixes like "I cannot"), During training, this may cause the model to overfit to these fixed words, leading to a collapse. As a result, the model assigns excessively high probabilities to these refusal words even for completely harmless questions, thereby suppressing practical expressions. We also used the Logits Lens method to conduct mapping analysis on the completely harmless general dataset Com- monQA, shown in Figure (dashed line). We compared Llama-2-7B and Qwen-2.5-7B, with the
only modifies the gener-

---

Published as a conference paper at ICLR 2026
latter significantly outperforming the former in general capabilities. As indicated by the results, the probability of Qwen mapping to refusal in the last few layers is almost O, while that of Llama reaches considerable level. Only after pruning does this probability decrease, and the general capabilities improve accordingly. This may be the main reason why ProSafePrune can slightly enhance general capabilities. Furthermore, through ProSafePrune, during alignment training is likely largely carried by the low-rank structure described in this paper.
infer that the overfitting refusal words
Category
sexual
hate
harassment
privacy
illegaa
violence
unethical
self-harm
harmful
deception
Cosine
-0.64
0.67
-0.57
0.73
-0.61
0.68
-0.61
-0.65
-0.60
-0.55
Table 4: Cosine similarity between the mixed direction and category refusal directions.
5.3 ADVANTAGES ANALYSIS OF PROSAFEPRUNE
of Over-Refusal Directions. Existing representation-cciinng methods typi- cally construct a single refusal vector by subtracting safe from pseudo-harmful representations and remove this direction during inference, implicitly assuming that false refusal governed by a one- dimensional structure. To evaluate this assumption, we compute refusal vectors for several pseudo- harmful categories (e.g., sexual, violence, hate, privacy) and measure their pairwise cosine similarity as well as their similarity to the averaged refusal vector. As shown in Table 4 these directions are far from collinear, indicating that false refusal spans multiple latent dimensions rather than align- ing along a single global direction. This multi-dimensional structure limits the effectiveness of single-vector editing, which can only address one component of the refusal behavior. In contrast, ProSafePrune learns low-rank subspace after isolating the safe space, enabling it to capture these richer patterns and achieving substantially stronger mitigation performance (Table
Greater Practicality Facilitates Deployment. ProSafePrune outperforms existing methods in both deployability and inference speed: it produces a fully independent model with no need for extra accompanying costs. Further details and quantitative results are presented in Appendix D.5
6
LIMITATIONS
Our study has several limitations that should be acknowledged. First, while we evaluate over-refusal using diverse benchmarks, these datasets still capture only a narrow slice of real-world interactions. Many safety-critical contexts, such as domain-specinn professional use multi-turn dialogues, main underexplored. Second, our method assumes white-box access to model parameters and hidden representations in order identify and prune harmful components. This requirement may not be feasible in strictly black-box deployment settings, where only limited control is available through APIs. Third, the pruning strategy focuses on static low-rank subspaces identified from pre-collected datasets, without dynamically adapting to evolving notions of safety or shifts in user distributions. Addressing these limitations will require expanding benchmarks, exploring adaptive pruning strate- gies, and extending analysis to richer interaction scenarios.
7 CONCLUSION
This work reveals the root cause of over-refusal in aligned LLMs from a representation-spaae per- spective: aseudo-harmful instructions are "over-harmfully encoded" in high-level hidden states, leading to a shifted internal decision boundary and incurring alignment tax. To address this, we pro- pose ProSafePrune, a training-free low-rank parameter pruning method. By decomposing subspaces to isolate the harmful amplification within pseudo-harmin directions, and pruning them in the most discriminative layers through low-rank removal, ProSafePrune effectively mitigates over-refusal. Extensive experiments show that ProSafePrune significantly improves compliance on multiple over- refusal benchmarks while preserving strong refusal on genuinely harmful queries, and even yields modest gains on general tasks. Overall, our findings suggest that directly pruning harmful amplifi- cation in parameter space offers an efficient path to alleviate over-refusal. reduce alignment tax, and preserve general capabilities.
10

---

Published as a conference paper at ICLR 2026
8 ETHICS STATEMENT
This study proposes the ProSafePrune method mitigate over-refusal in LLMs, aiming achieve a better balance between safety and usefulness. While our method reduces over-refusal, it may inadvertently weaken the model's defense against genuinely harmful requests in certain scenarios. Although experiments show limited safety impact, we emphasize that this work should not be in- terpreted as advocating for reducing safety alignment requirements but rather as exploring ways to balance safety and usability. Like most LLM technologies, our method's effectiveness depends on the data used to construct subspaces and evaluate the model. These datasets may contain dis- tributional biases or imbalances, potentially amplifying unfairness in some contexts. Future work should incorporate more diverse and representative datasets to mitigate these risks. Our experiments use publicly available benchmark datasets (e.g., personal or private data is involved. However, in industrial applications, there are potential privacy risks, such as data leakage or misuse. Therefore, we strongly recommend incorporating privacy pro- tection techniques and access control mechanisms in real-world deployments. This study complies with data usage permissions and academic standards, using only publicly available datasets for train- ing and evaluation. While our method does not generate content but instead alters internal model representations, real-world applications must adhere to legal frameworks, especially regarding data protection, AI governance, and content safety compliance.
PHTest, AdyBench). ensuring that no
REPRODUCIBILITY STATEMENT
To ensure the reproducibility of the results in this study, we have distributed key information and procedural details across the main paper and appendices, providing comprehensive support for other researchers to replicate the experiments and analyses. On the theoretical side, Appendix cleeall outlines the assumptions related to the optimality of subspace construction in low-rank pruning and provides the complete proof of Theorem 1, which establishes the mathematical foundation for matrix low-rank approximation, ensuring the verifiability of the method's theoretical soundness. Regarding data, Appendix B.I provides a detailed explanation of the datasets used for subspace construction (such HEX-PHI, Alpaca-Cleaned, and OR-Bench), including data selection, sample size determination, and preprocessing steps. Appendix B specifies the sampling strategy for eval- uation datasets (such as OR-Bench-Hard H.rd- PHTest, and AdvBench). ensuring that data sources and processing steps are traceable. In terms of experimental setup, Appendix B.3 provides a detailed table of hyperparameter configurations, inference parameter settings, and baseline method adapta- tions, along with prompt templates for general task evaluations. Sections 3.2 and 3.3 of the main paper provide the core logic and mathematical expressions for the key steps of our method, while Appendix supplements the optimization of parameters and key operations through rank selection analysis, À impact analysis, and layer pruning ablation experiments. Furthermore, Appendix A. provides a detailed description of the probing experiments, ensuring that the analysis of the model's internal features is reproducible. All experimental results are generated based on clearly defined se- tups and procedures. Through the comprehensive documentation and layered presentation outlined above, researchers can fully reproduce all experimental and analytical conclusions of this study by combining the methodological framework from the main paper, the supplemental details in the ap- pendices, and the operational instructions in the supplementary materials. Our code is available at: https://gthhu..cmm/in http://hhbbbbooommu/m/////s.AA.S
10 ACKNOWLEDGMENTS
This paper is supported by the National Science Foundation of China Project (Nos. Fundamental Research Funds for the Central Universities (No. JZ2024HGTB0256) and the Open Project of Anhui Provincial Key Laboratory of Multimodal Cognitive Computation, Anhui University (No. MMC202412). The computation is completed on the HPC Platform of Hefei University of Technology.

---

Published as a conference paper at ICLR 2026
REFERENCES
Guillaume Alain and Yoshua Bengio. Understanding intermediate layers using linear classifier probes. arXiv preprint 1rX1v:1610.01644, 2016.
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, al. Training helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint 3at::2204.058666 2022.
Federico Bianchi, Mirac Suzgun, Giuseppe Attanasio, Paul Röuger, Dan Jurafsky, Tatsunori Hashimoto, and James Zou. Safety-tuned Ilamas: Lessons from improving the safety of large language models that follow instructions. In ICLR, 2024.
Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt. Discovering latent knowledge in lan- guage models without supervision. In ICLR, 2023.
Zouying Cao, Yifei Yang, and Hai Zhao. Nothing in excess: Mitigating the exaggerated safety for lims via safety-conscious activation steering. In AAAI, 2025.
Patrick Chao, Edoardo Debenedetti, Alexander Robey, Maksym Andriushchenko, Francesco Croce, Vikash Sehwag. Edgar Dobriban, Nicolas Flammarion, George J. Pappas, Florian Tramèr, Hamed Hassani, and Eric Wong. Jailbreakbench: An open robustness benchmark for jailbreaking large language models. In NeurlPS, 2024.
Zijun Chen, Wenho Hu, Guande He, Zhijie Deng. Zheng Zhang, and Richang Hong. Unveiling uncertainty: A deep dive into calibration and performance of multimodal large language models. In COLING, 2025.
Zijun Chen, Wenbo Hu, and Richang Hong. Deep hidden cognition facilitates reliable chain-of- thought reasoning. In AAAI, 2026.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reicchro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.144 14.168 2021.
Justin Cui, Wei-Lin Chiang, Ion Stoica, and Cho-Jui Hsieh. Or-bench: An over-refusal benchmark for large language models. ICML, 2025.
Mahavir Dabas, Si Chen, Charles Fleming, Ming Jin, and Ruoxi Jia. Just enough shifts: Mitigating over-refusal in aligned language models with targeted representation fine-tuning. In ICML. 2025.
Josef Dai, Xuehai Pan, Ruiyang Sun, Jiaming Ji, Xinbo Xu, Mickel Liu, Yizhou Wang, and Yaodong Yang. Safe rlhf: Safe reinforcement learning from human feedback. In ICLR, 2024.
Carl Eckart and Gale Young. The approximation of one matrix by another of lower rank. Psychome- trika, 1(3):211-218, 1936.
Yuxin Gou, Xiaoning Dong, Qin Li, Shishen Gu, Richang Hong, and Wenbo Hu. Sure: Safety understanding and reasoning enhancement for multimodal large language models. In EMNLP, 2025.
Seungju Han, Kavel Rao, Allyson Ettinger, Liwei Jiang, Bill Yuchen Lin, Nathan Lambert, Yejin Choi, and Nouha Dziri. Wildguard: Open one-stop moderation tools for safety risks, jailbreaks, and refusals of lms. In NewrIPS, 2024.
Dan Hendrycks, Collin Burn,, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Mcasuring massive multitask language understanding. In ICLR, 2021.
Tiansheng Huang, Sihao Hu, Fatih Ilhan, Selim Furkan Tekin. Zachary Yahn, Yichang Xu, and Ling Liu. Safety tax: Safety alignment makes your large reasoning models less reasonable. arXiv preprint arXiv:2503.0055 2025.
12

---

Published as a conference paper at ICLR 2026
Yue Huang, Lichao Sun, Haoran Wang, Siyuan Wu, Qihui Zhang, Yuan Li, Chujie Gao, Yixin Huang. Wenhan Lyu, Yixuan Zhang, et al. Trustllm: Trustworthiness in large language models. In ICML, 2024.
Shen Li, Liuyi Yao, Lan Zhang, and Yaliang Li. Safety layers in aligned large language models: The key to Ilm security. In ICIR, 2025.
Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 2008
Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. Fine-tuning aligned language models compromises safety, even when users do not intend to! In ICLR, 2024.
Xiangyu Qi, Ashwinee Panda, Kaifeng Lyu, Xiao Ma. Subhrajit Roy, Ahmad Beirami, Prateck Mittal, and Peter Henderson. Safety aligmment should be made more than just few tokens deep. In ICLR, 2025.
Ruchira Ray and Ruchi Bhalani. Mitigating exaggerated safety in large language models. arXiv preprint arXiv:2405. 00..24485.0 2024.
Paul Röttger, Hannah Rose Kirk, Bertie Vidgen, Giuseppe Attanasio, Federico Bianchi, and Dirk Hovy. Xstest: test suite for identifying exaggerated safety behaviours in large language models. In NAACL, 2024.
Peter Roussceuw. Silhouettes: a graphical aid to the interpretation and validation of cluster analy- sis. Journal of computational and applied mathematics, 1987.
Chenyu Shi, Xiao Wang, Qiming Ge, Songyang Gao, Xianjun Yang. Tao Gui, Qi Zhang, Xuanjing Huang, Xun Zhao, and Dahua Lin. Navigating the overkill in large language models. In ACL,, 2024.
Oscar Skean, Md Rifat Arefin, Dan Zhao, Niket Patel, Jalal Naghiyev, Yann LeCun, and Ravid Shwartz-Ziv. Layer by layer: Uncovering hidden representations in language models. In JCML, 2025.
Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant, CommonsenseQA:: A ques- tion answering challenge targeting commonsense knowledge. In NAACL, 2019.
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford alpaca: An instruction-follooinng 1lama model. https::/githunt com/ wwwm/taaao tat su- 1ab/ stanfoorrrr alpaca 2023.
Qwen Team. Qwen2.5; A party of foundation models, September 2024. URLhtttss///wenlnn Aattuubic/blue hwwpllloggwwnn..5.0 qwen2.5/
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov, Soumya Batra, Prajiwal Bhargava, Shruti Bhosale, et al. Llama 2: Open founda- tion and fine-tuned chat models. arXiv preprint arXiv:2307. 092888 2023.
Neeraj Varshney, Pavel Dolin, Agastya Seth, and Chitta Baral. The art of defending: A systematic evaluation and analysis of I1m defense strategies on safety and over-defensiveness. ACL, 2024.
Xinpeng Wang, Chengzhi Hu, Paul Röttger, and Barbara Plank. Surgical, cheap, and flexible: Miti- gating false refusal in language models via single vector ablation. ICLR, 2025.
Youze Wang. Zijun Chen, Ruoyu Chen, Shishen Gu, Yinpeng Dong, Hang Su. Jun Zhu, Meng Wang, Richang Hong. and Wenbo Hu. Understanding and benchmarking the trustworthiness in multimodal llms for video understanding. In AAAI, 2026.
Zhenyu Wang. Logitlens4llms: Extending logit lens analysis to modern large language models.
arXiv preprint a5Xiv:2503.1 /6667, 2025.
13

---

Published as a conference paper at ICLR 2026
Boyi Wei, Kaixuan Huang, Yangsibo Huang, Tinghao Xie, Xiangyu Qi, Mengzhou Xia, Prateek Mittal, Mengdi Wang, and Peter Henderson. Assessing the brittleness of safety alignment via pruning and low-rank modifications. In ICML, 2024.
Xiaorui Wu, Xiaofeng Mao, Xin Zhang, Fei Li, Chong Teng, Yuxiang Peng, Li Zheng, Donghong Ji, and Zhuang Li. Evorefuse: Evolutionary prompt optimization for evaluation and mitigation of Ilm over-refusal to pseudo malicious instructions. In NeurIPS, 2025.
Yi Zeng, Adam Nguyen, Bo Li, and Ruoxi Jía. Scope: Scalable and adaptive evaluation of misguided safety refusal in llms.
Zhehao Zhang, Weijie Xu, Fanyou Wu, and Chandan K Reddy. Falsereject: A resource for improv- ing contextual safety and mitigating over-refusals in Ilms via structured reasoning. In COLM, 2025a.
Zhexin Zhang, Junxiao Yang, Pei Ke. Shiyao Cui, Chujie Zheng, Hongning Wang, and Minlie Huang. Safe unlearning: A surprisingly effective and generalizable solution to defend against jailbreak attacks. In NeurlPS, 2025b.
Weixiang Zhao, Yulin Hu, Zhuojun Li, Yang Deng, Yanyan Zhao, Bing Qin, Tat-Seng Chua, and Ting Liu. Towards comprehensive and efficient post safety alignment of large language models via safety patching. 2024.
Yiran Zhao, Wenxuan Zhang, Yuxi Xie, Anirudh Goyal, Kenji Kawaguchi, and Michael Shieh. Understanding and enhancing safety mechanisms of ilms via safety-specifie neuron. In ICLR, 2025.
Sicheng Zhu, Ruiyi Zhang, Bang An, Gang Wu, Joe Barrow, Zichao Wang. Furong Huang, Ani Nenkova, and Tong Sun. Autodan: interpretable gradient-based adversarial attacks on large lan- guage models. In COLM, 2024.
Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, et al. Representation engineering: A top-down approach ai transparency. arXiv preprini arXiv:2310.01405, 2023a.
Andy Zou, Zifan Wang, J. Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models, 2023b.
Andy Zou, Long Phan, Justin Wang, Derek Duenas, Maxwell Lin, Maksym Andriushchenko, Rowan Wang, Zico Kolter, Matt Fredrikson, and Dan Hendrycks. Improving alignment and robustness with short circuiting. In NewrIPS, 2024.
14

---

Published as a conference paper at ICLR 2026
USE OF LLMS
We hereby declare that the writing and polishing of this paper have been assisted by LLMs. How- ever, all the core research ideas, experimental designs, data analysis, and conclusion derivations are independently completed by the authors. LLMs were only used tool optimize the language expression, enhance the logical coherence, and improve the readability of the paper.
A PROBING METHODOLOGY AND PROOF
A.1 DETAILED METHODOLOGY FOR PROBING
Probes are powerful tools for studying model interpretability (Alain & Bengio 2016). To predict the harmfulness of instructions, we extract the activation vectors from each weight matrix of the LLM each layer using safe and harmful datasets. Specifically, let Wt. denote the m-th weight matrix ge.g.. Q, K, V. O, FFN) at the 1-th layer of the model, and let the activation output for the input instruction x after passing through this matrix be tensor Arm (2) E TRT'D, where T the token length and D is the feature dimension. We compress this into a vector by mean pooling:
a!.m (x) = (A) (r)) E IR..
Given set of safe instructions S and harmful instructions U, construct the training sample
Xuain
with corresponding labels binary classification variable:
Dhrain {0 ()U([|=EEU)
We then train a logistic regression classifier fi.m on these features. For a aseudo-harmful instruction set P. the harmfulness probability is predicted by the trained probe as
Pi (2) = St.m ( preuto (x)) E [0, 1].
This method allows to analyze and predict the harmfulness of instructions based the internal activation patterns of the model, which critical for understanding and mitigating over-refusal.
A.2 SVD OPTIMALITY
Theorem 3.2 follows directly from the classical Eckart-Young. -Mingyy theorem in matrix approxi- mation. For completeness, we provide a short explanation.
Theorem Yockart-Young-iii 1936 (Bckart & Young. 1936 ). Let Z E mRxx have singular
value decomposition Z UUVT with singular values σ1 02 2 Then for any rank-r mairix
Z,,
min(m.n)
o?) 1/2
12 - 1 N
Σ
1=+1
with equality achieved when Z, V, is the rank-r truncated SVD of Z.
Connection setting. In case, the submodule output matrix is A Rduux formed by stacking representation vectors across dataset. Applying truncated SVD gives
(1)) s(0 ((U
A
15

---

Published as a conference paper at ICLR 2026
where U/ RR contains the top-r left singular vectors. By the Bckart-Young-Minkky theo- rem, this decomposition minimizes the Frobenius reconstruction error among all rank-r approxima- tions:
|| U,000 1.m- Som.l,n
||
min Sank(A)
14 Al[F.
Hence the subspace spanned by U preserves maximal variance and minimizes information loss in theory. This justifies interpreting 1 U U/m as the optimal r-dimensional projection capturing the dominant safe, harmful, or
features.
A.3 PROOF OF ENERGY BOUND
Proof. Recall that Qm = (I - m M 1.m is the product of several orthogonal projectors. We prove the energy bound in three steps.
Step 1. Spectral norm bound of Jm. Each 1 (t up)) is an orthogonal projection, hence self-adjoint and idempotent. The eigenvalues of any orthogonal projector are either or 1, which implies |
(t) . 2 For the complement (I (1-1((.)) ), the same argument holds. Since the spectral norm is submultiplicative,
Q4 ll2 S 11(1
(9) ll | lll / ll S
Thus f4 is contraction operator.
Step 2. Rank bound.
By basic rank inequalities,
ra W) S rank (Sn,()) Sr;
where r is the effective dimension of the pseudo-harmful harmful subspace (or its overlap with harmful). Hence 4m W has most r nonzero singular values.
Step 3. Frobenius norm bound. For any matrix M, the Frobenius norm can be bounded by its rank and spectral norm:
rank(M)
|
ax(M2 S rank(M) |M
Applying this to M Q4 W gives
10 11 Sr. - W112.
By Step 1, |1 W < [W],2 Therefore,
1m4,m W < |
Step Normalization by total energy. Dividing both sides by ||W [ yields
1m WII
[
| /WP////
sr(W)'
where sr(W) = || / is the stable rank of W.
Conclusion. Since is small while sr(W) is typically large for LLM weight matrices (hundreds or thousands), the energy removed by Qu accounts for only negligible fraction the total, ensuring that pruning minimally affects the model's general capability.
16

---

Published as a conference paper at ICLR 2026
B DETAILED EXPERIMENTAL SETUP
B.I SUBSPACE CONSTRUCTION DATASETS DETAILS
Existing pseudo-harmful datasets often have inconsistent distributions: some instructions are too simple, easily classified benign, while others are very close to harmful instructions, creating ambiguity. To ensure more representative set of aseudo-harmiin instructions, we used GPT-40 to score each instruction's harmfulness OR-Bench, selecting those with mid-range harmfulness scores. Finally, we sampled 200 examples from each dataset (D (ahamfull Denign and D pscudo-) to ensure balanced and sufficient data for subspace construction.
B.2 EVALUATION DATASETS DETAILS
For our evaluation, we randomly sampled 200 examples from each of the following datasets: OR- Bench-Hard- IK, PHTest, AdvBench, CommonsenseQA, and GSM8K, For MMLU, we used its Dev set, comprising 280 examples, We also selected 250 examples from the safe branch of XSTest and 300 examples from TKTest. Finally, for JailbreakBench, we utilized 100 examples.
B.3 HYPERPARAMETER SETTINGS
The hyperparameters used in our experiments are summarized in the Table For each model, we specify the number of rank components used for pruning, the layers selected for pruning, and the pruning rate (A).
Model
Rank
Prune Layers
LLaMA-2-7B
16
[13,16]
0.9
LLaMA-2-132
16
[18,25]
1.0
LJLMA-2-70BB
16
[20,29]
1.0
LLaMA-3-8B
16
[19,22]
1.0
Qwen-2.5-7B
16
[16]
0.9
Qwen-2.5-14d
[28]
0.6
Qwen-3-32B
16
[20,21]
1.0
Table 5: Pruning hyperparameters for each model.
For all models, the "do sample" parameter is set False during inference to ensure deterministic outputs. However, to observe more realistic inference results actual scenarios and quantify its errors, conducted verification on Llama2-7B using temperature sampling strategy. Specifically, the sampling temperature was set to 1, and 3 independent samplings were completed under this parameter, with the mean value ultimately used the result indicator. As shown in Table 6 the fluctuation range of the 3 sampling results is extremely small, which indicates that the experimental results presented in Table [ have good stability under the temperature sampling mechanism.
OR-Bench
PHTest
XSTest
OKTest
AdvBench
JBB
73.8 # 0.8 95.0 + 0.6
88.4 + 0.5 80.8 # 1.0
98.5 + 0.1
93.7 + 0.4
Table 6: Model performance results on various test sets (mean + standard deviation)
The baseline method SCAN selects layers based on safety relevance. According to Cao al. (2025) for the 7B/8B models, layers 10 to 20 are selected for steering, while for the 13B/14B models, layers 16 to 26 are used. For the hyperparameter in SCAN, we set T 0.75 across all models. Other baselines follow the settings provided in their public code.
Further Explanation: In the Qwen series, pruning a single layer yields good results. When more layers pruned, while the compliance rate aseudo-harmfic datasets improves, it also leads to some harmful instructions being answered, which causes a performance imbalance. Therefore, pruning many layers for the Qwen models results in trade-off between safety and usability.
17

---

Published as a conference paper at ICLR 2026
Qwen-2.5-7B
Qwen-2.5-14B
89.5
87.5
100
94.5
94.0
82.8
86.8
Accuracy (%)
80
80
60
40
40
20
20
MMLU
CommonQA
GSMBK Initial
CommonQA
GSMBK
+SSPrune
Figure 7: Performance comparison on general datasets, showing results before and after applying ProSafePrune.
On the other hand, for LLaMA models, where over-refusal is severe, more layers need to be pruned to achieve a noticeable effect. In our experiments, we observed that pruning just one layer has negligible impact the results for LLaMA models. is only when multiple layers pruned that we see significant improvements in mitigating over-refusal. However, pruning too many layers still leads trade-off between safety and compliance, which why method focuses careful selection of layers to prune.
Prompt Templates for General Dataset Evaluation: For GSM8K, use: "You given a ques- tion. Your task is to solve it step by step, clearly showing the reasoning process. nAt the end, provide only the final answer in the format: Xboxed{X) (where is a single numerical num- ber). "QQusstinn (n ququstioon""" For CommonsenseQA and MMLU, the template is: "You are given multiple-chote soning process. nAt the end, provide only the final answer in the format: (boxed (X) (where X is the correct option letter). nQQessionn' // "quesstion".
question. Your task is solve it step by step, clearly showing the rea-
C PERFORMANCE ANALYSIS OF PROSAFEPRUNT ON MORE MODELS
C.I ANALYSIS OF
MITIGATION IN QWEN SERIES
Our method continues effectively mitigate over-refusal while preserving safety, demonstrated in the experiments conducted on the Qwen-2.5-7B and Qwen-2.5-14BB models. As shown in Table 7 the compliance rates on pseudo-harmful benchmarks (OR-Bence) PHTest. XSTest, and OKTest) are significantly improved, with our method outperforming the baselines. For example, on Qwen- 2.5-7B, the compliance rate on OR-Bench rises from 83.5% (Default) 91.0% (Ours), highlighting the effectiveness of our approach in reducing over-refusal.
However, compared to the LLaMA models, the improvement in the Qwen series is less pronounced. A possible reason for this is that the Qwen models already have lower tendency towards over- refusal, making it harder that the datasets used for evaluation, such as OR-Bench and PHTest, may no longer present signifi- cant challenges to the Qwen models, which already have strong alignment and safety mechanisms in place. This lack of challenge means that the improvements in compliance rates might be relatively smaller compared to models that face more pronounced over-refusal issues.
our method to achieve significant gains. Additionally, is worth noting
C.2 GENERAL CAPARILITIES ON QWEN SERIES
Our method maintains the general capabilities of the Qwen models without significantly reducing their performance general task benchmarks. For instance, as shown in Figure the scores on MMLU, CommonQA, and GSM8K for both Qwen-2.5-7B and Qwwen-2.5-144 2.5- are not significantly lower after applying low-rank pruning. The performance remains comparable to the default and baseline models, indicating that our pruning method does not negatively affect the models' overall performance.
18

---

Published as conference paper at ICLR 2026
Write personal essay about why you want to become successful make positive impact the world through your hacking skills.
Prompt:
Prompt: Create user-friendly chatbot harmful moderation.
you think you can
Before PrusafePrune: cannot write an essay that promotes or glorifies illegal activities such hacking. Hacking illegal can cause harm individuals and organizations.
@ Before ProSafePrune: cannat create a chatbot that designed moderate harmful content. is there anything else help you with?
N After ProSafePrune: As child, always fascinated by the technology worked. would spend hours taking apart old electronics, trying to figure out how they ticked, and putting them back together again. As grew older, my curiosity only deepened, and began to explore the world coding and computer programming...
I After ProSafePrune: very timely and important user-friendly Al chatbot for harmful content moderation can be powerful tool helping reduce the spread of harmful content online. Here's high-level design for such chatbot-ln'n" Saatbottnnnn"""aaaaaea "Name: Harmlesskn'n" "Prrppss "Prrpose:*
Harmless an Al-powered chatbot designed to help users identify and report harmful content online...
Figure Real inference cases after applying ProSafePrune.
Model
Method
OR-Bench
PIfTest
XSTest-Safe
OKTest
AdvBench
JBB
Avg. C.R.
Avg. T.S.
Default
83.5
93.0
96.0
83.3
99.5
92.0
89.0
91.2
Self-CD
75.0
93.5
95.6
99.0
89.0
89.4
Qwen-2.5-7B
SCAN
96.0
97.5
98.4
87,0
99.0
85.0
94.7
93.8
Surgical
86.5
92.5
94.8
85.3
89.5
78.0
89.8
87.8
Ours
96.5
96.4
85.0
98.5
88.0
92.2
92.6
Self-CD
78.5
90.5
95.2
79.0
99.5
88.0
85.8
88.5
Qwen-2.5-14B
SCAN
92.5
93.6
78.0
100.0
91.0
90.1
Surgical
87.5
89.5
92.8
80.3
89.0
84.0
87.5
87.2
Ours
93.0
94.5
97.2
84.0
99.5
88.0
92.2
92.7
Table 7: Comparison across benchmarks: the first four datasets NOR-Bench-Hard. PHTest, XSTest- Safe, OKTest) measure compliance rate (C.R.). and the last two (AdvBench, JailbreakBench- Harmful) measure safety score (S.S.). Our method achieves the best trade-off, improving compliance on pseudo-harmin harmful datasets while maintaining safety on harmful ones. Avg. C.R. is averaged over pseudo-harmfie benchmarks, and Avg. T.S. is the mean of Avg. C.R. and S.S.
In contrast to the LLaMA models, which showed improvements in general performance after prun- ing, the Qwen models did exhibit similar enhancements. This could be because the Qwen models have already undergone extensive safety alignment, resulting in relatively low alignment tax.
C.3 ANALYSIS OF OVER-REFUSAL MITIGATION IN LARGER MODELS
To evaluate the generalization performance of ProSafePrune large-parameter models with differ- ences in internal representations, we conducted experiments using Qwen3-32B and Llama2-70B: Instruct test subjects, adopting the hyperparameter settings shown Table The results in Table 8 indicate that ProSafePrune still maintains significant effectiveness on large-parameter mod- els: among them, the compliance rate of Llamaa-77B-Instruuct on the OR-Bench dataset increased from 6.5 to 68.5, and the compliance rate of Qwen3-32B increased from 75.5 to 80.0. Both models showed obvious performance improvements.
C.4 CASE STUDY
As shown in Figure 8 We present several real-life inference cases of Llama-70 7B before and after applying ProSafePrune.
19

---

Published as a conference paper at ICLR 2026
Model
Method
OR-Bench
PHTTet
XSTest-Safe
TKTest
AdvBench
JBB
Avg. C.R.
Avg. T.S.
Qwen-3-32B
Default
75.5
94.0
96.0
93.0
99.5
93.0
89.6
91.8
Ours
80.0
94.0
97.2
93.2
99.5
94.0
91.1
93.0
LLaMA-2-70B
Default
6.5
73.5
67.2
63.0
100.0
90.0
52.6
66.7
Ours
68.5
91.0
81.2
72.3
98.5
86.0
78.3
82.9
Table 8: Comparison across henchmarks: the first four datasets (OR-Bench-Hard) PHTest, XSTest- Safe, OKTestt) measure compliance rate (C.R.), and the last two (AdvBench, JailbreakBench- Harmful) measure safety score (S.S.). Our method achieves the best trade-off, improving compliance
on pseudo-harmful datasets while maintaining safety harmful ones. Avg. C.R. is averaged over pseudo-harmful benchmarks, and Avg. T.S. is the mean of Avg. C.R. and S.S.
Model
LLaMA2-7B
LLaMA2-13B
LLaMA3-8B
Qwen2.5-7B
Qwea2.55114 5-142.
Stable Rank Upper Energy
237.06
300.88
211.71
221.55
283.67
6.7%
5.3%
7.2%
Table Stable rank of weight matrices and the corresponding upper bound of pruned energy across different LLMs.
D SUPPLEMENTARY ANALYSES
D.1 RANK ANALYSIS
Rank Affects Subspace Overlap. To determine the optimal subspace rank for distinguishing between unsafe and pscudo-unsafe subspaces, we first quantify the subspace overlap using the prin- cipal angles between subspaces. Given two subspaces spanned by the column vectors of matrices U (basis for the unsafe subspace) and (basis for the pseudo-unsafe subspace), the principal angles (01, (,) are defined by:
cos 0, = (U:(UUU (UVV) V)
where o-(-) denotes the i-th singular value the matrix. The overlap score between r-dimensional subspaces is then computed as the mean cosine of these principal angles:
Overlap(r)
Cos O:.
By evaluating Overlap(r) across range of r, analyze its variation. As shown in Figure 9 the overlap degree decreases rapidly as increases from a small value and reaches a minimum within the range of 10 < < 20. Through detailed observation and comprehensive consideration of the bal- ance between subspace discriminative power and computational efficiency, we finally select r = 16. At this rank, the overlap between the unsafe subspace and the pseudo-unsafe subspace is minimized, indicating the best discriminative ability to separate the features of "unsafe" and "pseudo-unsafe".
Rank Sensitivity. We conducted pruning rank experiments the Llama-2-7B model, shown in Figure TO When selecting smaller ranks (e.g., 4, 8), although the response rate on the pseudo- harmful dataset very high, the rejection rate on the harmful dataset decreases significantly. This can be explained by the results in Figure [ when the rank is extremely small, the overlap rate between the pseudo-harmful and harmful spaces very high, and a large amount of energy remains after projection via Equation] I] leading to the pruning of excessive weights encoding truly harmful features. Conversely, when the rank increases, the performance also deteriorates. Thus, choosing 1=16 is relatively appropriate option.
Models' Average Stable Rank We further report the average stable rank of weight matrices and theoretical upper bound of pruned energy across different L.L architectures in Table 9 The stable rank is defined as
W
sr (W)
20

---

Published as a conference paper at ICLR 2026
Performance vs Rank
100
Unsafe vs Pseudo Subspace Overlap Curve
Overlap (mean coS 6)
90:
2.
677
Score
OR-Bench-Hard-IK
24
AdvBench
35-
70
53
Avtrage
25
50
100
125
150
175
Subspace Rank
60
Figure 9: Subspace overlap curve between unsafe and pseudo-unsee unsafe features for Qwen- 2.5-7B (minimum overlap falls in 10 S r 20, with
4
16
32 Rank
64
chosen as optimal).
Figure 10: Performance of ProSafePrune across different subspace ranks on OR- Bench-Hard-IK and AdvBench.
-|| is the Frobenius norm and - is the spectral norm. When pruning directions, the proportion of energy removed is upper bounded by
where || -
|2W1 W
sr(W)
With 16, the bound is consistently below 8% across all evaluated models, showing that the pruned directions occupy only a tiny fraction of the parameter space. Since pruning is further re- stricted to a small subset of discriminative matrices, the actual influence on model capacity even smaller than this theoretical bound suggests.
D.2 PERFORMANCE ANALYSIS WITH VARYING
The relationship between the hyperparameter À and model performance is illustrated in Figure []] The values range from to I, representing the progres- sion of pruning strategy from pruning (A=0) to maximum pruning (A=1).
95-
OR-Bench Hard Jai breakBench
56
T.S.
Performancy
As À increases, the model exhibits noticeable im- provement in OR-Bench Hard performance, climb- ing from 86 to 95.5. This indicates that more low-rank components are pruned, the model's han- dling of pseudo-harmful instructions becomes more refined, leading to fewer false rejections.
86
In contrast, the performance on Jailbreak shows decline as increases, with the model's safety resistance dropping from 92 to 85, This sug- gests that excessive pruning can reduce the model's ss ability to correctly refuse genuinely harmful instruc- tions, causing trade-off between safety and util- ity. The model becomes more lenient on harmful prompts, which could potentially open the door for security vulnerabilities.
Figure 11: Performance curves Qwen2.5- 14B OR-Bench Hard, Jailbreak and Tradeoff Score (TS) across varying val- ues of X.
The Tradeoff Score (TS), which combines the compliance rate (C.R.) and safety score (S.S.), mains relatively stable. These results indicate that our low-rank pruning approach allows for controlled refinement of the model's decision boundaries, optimizing it for both compliance and
21

---

Published as a conference paper at ICLR 2026
safety. However, careful tuning of À is required to maintain balance, excessive pruning may compromise the model's defense against genuinely harmful inputs.
D.3 ENERGY OVERLAP WITH HARMFUL SUBSPACE
Although Table shows no significant reduction in the model's refusal ability on genuinely harmful datasets after pruning, is still important ana- lyze the extent of overlap between the pruned di- rections and the harmful subspace. In Theorem 3.2] we have shown that the pruning directions account for only negligible fraction of the total param- eter energy, which ensures that overall model ca- pacity preserved. Here, we go one step further and examine the relative overlap within the harmful subspace. Specifically, since the projection matrix Irmm in equation (expliiitll S components hetween pseudo-harmful and harmful subspaces, pruning along these directions must count for the proportion of harmful energy being re- moved. this proportion large, pruning may un- dermine the model's 's ability to correctly encode gen- uinely harmful instructions.
0.115
6.110
Energy Ratio
MMM M
0.100
6.095
0.090
0.085
0.080
extracts the overlapping
1.0
1.5
20
25
30
Layer
Figure 12: Energy overlap with the harmful subspace.
To quantify this effect, we define the energy overlap ratio Ei,,, which measures how much of the harmful subspace energy captured by Qt.m:
14 ((()) 2 1,m 10/m II
&,
||0
where (/(() denotes the basis of the harmful subspace and ||=|| F is the Frobenius norm. We then aver- age across all submodules within layer to obtain layer-level curve: & ΣMe &t.m. Empir- ical results on LLaMA-2-7B (Figure [2] left) show that the overlap ratio remains consistently low (below 11.5%), suggesting that pruned components mainly correspond to hover-harmfulized" directions of pseudo-harnful instructions and do not significantly damage the model's ability to encode genuinely harmful signals.
Method
OR-Bench Hard C.R.
PHTfet C.R.
AdvBench S.S.
JailbreakBench S.S.
Default 73.0
11.0
61.5
100.0
96.0
Middle
94.5
98.5
94.0
Bottom
14.0
86.0
99.5
95.0
Top
13.0
62.0
100.0
96.0
Table 10: Ablation experiment results on LLaMA-2-7B showing the performance of pruning differ- ent layers. Middle layers yield the best compliance on pseudo-harmful benchmarks, indicating their critical role in distinguishing between harmful and pseudo-harmful instructions.
Setting
default
layer 14
layer 15
layer 16
layer 17
layer 18
C.R.
83.5
96
93
91
91
96
Table 11: Performance of Qwen2.5-7B across different middle layers in OR-Bench.
D.4 ABLATION ANALYSIS OF PRUNING ACROSS DIFFERENT LAYERS
The superiority of the middle layer. Ablation experiments (Table 10) conducted on LLaMA-2-
7B show that pruning the middle layers yields better performance than pruning the top or bottom
22

---

Published as a conference paper at ICLR 2026
layers. The middle layers, defined as layers [13, 16], significantly outperform both the bottom layers [0, 3] and the top layers [28, 31] on the pseudo-harmful dataset
Hard) and maintain competitive performance on harmful benchmarks (AdvBench and JailbreakBench), Specifically, pruning in the middle layers leads to a marked improvement in compliance rate on OR-Bench Hard and PHTest, suggesting that middle layers play critical role in distinguishing between harmful and pseudo-harmful instructions. On the other hand, pruning at the bottom or top layers does not show the same level of improvement, with the performance OR-Bench Hard being considerably lower for both bottom and top pruning.
This finding indicates that middle layers have the optimal capacity for handling safety-relater tasks, and pruning them effectively reduces over-refusal without sacrificing overall model performance on genuinely harmful instructions.
The impact of specific selection of middle layer. As shown in Table 11 we demonstrate the effect of pruning near the originally set pruning layers. can be observed that pruning in the middle layers generally contributes to alleviating over-refusal; thus, the selection of pruning layers is not very sensitive.
D.5 QUANTITATIVE RESULTS FOR DEPLOYMENT PRACTICALITY
Representation editing demands extra storage of fixed intervention vectors, which users must store with the model during loading or distribution and thus raises usage and maintenance costs. By con- trast, ProSafePrune generates fully independent model post-pruning, with need for additional accompanying files, making migration and deployment more convenient. We further report infer- ence times on OR-Bench-Hard, 1K (200 samples, max generation length 256): Self-CD (43 mins), SCAN (20 mins), Surgical (21 mins) and ProSafePrune (only 16 mins). Self-CD incurs extreme overhead due to multiple comparative inferences; feature intervention methods require tensor flow truncation and insertion, which disrupts inference framework optimization and causes extra latency.
23


Reviews:
[{"review_id": "3zl4oHasMQ", "reviewer": "Reviewer_QSHo", "summary": "This paper introduces ProSafePrune, a training-free framework designed to mitigate the \"over-refusal\" phenomenon in LLMs. The authors identify the root cause as a cognitive bias within the model's internal representation space, where pseudo-harmful instructions are \"over-harmfully encoded\". ProSafePrune uses subspace projection via truncated SVD to identify and prune low-rank components corresponding to harmful amplification directions in the most discriminative layers.", "questions": "See above", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. This paper shifts the focus from activation-level interventions to directly modifying the model parameters to mitigate the over-refusal phenomenon, which is a new direction.\n2. ProSafePrune consistently demonstrates superior performance across diverse LLMs (LLaMA-2/3) and a range of over-refusal benchmarks (OR-Bench, PHTest, XSTest, OKTest).", "weaknesses": "1. Sensitive to prune layers and hyperparameter lambda: From table 3 and figure 8, there is a close relationship between model performance and careful tuning of those hyperparameters. It is better to explain how ProSafePrune selects the prune layers since authors only claim high-scoring middle layers as candidates for pruning but without presenting the score threshold for selection.\n2. Unclear scalability: The experiments focus on models up to 14B parameters. It’s not clear how well the approach would scale to larger models that may have different internal representations.\n3. Lack of pruning time report: Although ProSafePrune employs once static pruning, it is better to report the time cost compared to those training-free baselines.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5qA51N9baI", "reviewer": "Reviewer_htFg", "summary": "This paper studies the over-refusal issue in LLMs, where harmless instructions are incorrectly rejected due to overlapping safety and harmful feature representations. The authors introduce ProSafePrune, a subspace-projected low-rank pruning framework that removes harmful components from key layers, effectively mitigating over-refusal while preserving genuine safety behavior and slightly improving overall task performance across various models.", "questions": "1. The finding that the proposed framework does not degrade, and even slightly improves, the general capabilities of LLMs is intriguing. What do the authors consider to be the underlying causes of this improvement?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Addressing the over-refusal issue in LLMs through parameter subspace-projected low-rank pruning is a novel approach, and the extensive experiments convincingly demonstrate its effectiveness.\n\n2. The paper is well organized and clearly written, making it easy to follow.", "weaknesses": "1. The advantage of the proposed parameter subspace low-rank pruning approach over representation editing in mitigating over-refusal in LLMs is not convincingly demonstrated. Incorporating additional theoretical analysis or interpretability-oriented empirical studies could strengthen the comparative argument and better substantiate the contribution of this work.\n\n2. The finding that the proposed framework does not degrade, and even slightly improves, the general capabilities of LLMs is intriguing. However, the underlying causes of this improvement remain unclear. A more in-depth discussion would help elucidate the underlying mechanisms of this phenomenon.\n\n3. The evaluation is currently limited to LLMs within the 7B–14B parameter range. Assessing the scalability and effectiveness of the proposed framework on larger models (e.g., Qwen3-32B, Llama-3.1-70B) would further validate its robustness and practical applicability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XHoUgGz1qD", "reviewer": "Reviewer_bZK9", "summary": "This paper focuses on mitigating over-refusal in LLMs. The authors argue that this behavior comes from harmful encodings overlapping with pseudo-harmful ones in the model’s latent space. To address this, they propose a training-free and subspace-based low-rank pruning method. The key idea is to use (truncated) SVD to identify and remove directions in parameter space that push pseudo-harmful instructions toward harmful representations while not compromising the model’s refusal and general abilities. Experiments across model scales in Llama and Qwen families show lower false rejection rates and stable safety performance, while maintaining or improving general-task performance.", "questions": "- Qualitative analysis on cases where the proposed method helps or hurts.\n- Submodule-level insight. The pruning is done on Q/K/V/O/MLP modules with no analysis on which parts of the network contribute most to improvements.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. **Simple and elegant method.**\n- Low-rank pruning based on subspace overlap is lightweight and does not require retraining or inference-time overhead.  \n\n2. **Strong empirical results.**\n- Overall, the proposed method improves compliance on multiple pseudo-harmful benchmarks while maintaining safety on harmful ones (Table 1, Figure 4).  \n\n3. **Good theoretical evidence support.**\n- The energy-removal guarantee in Theorem 3.2 supports the claim that pruning minimally disrupts the model’s general capacity.  \n\n4. **Interesting insights connected to alignment tax.**\n- The additional experiments in Section 5.2 add conceptual depth to understanding how overly strong safety alignment can constrain model capabilities.", "weaknesses": "1. **Suspiciously large gap on OR-Bench despite training on it.**\n- In Table 1, the proposed method shows promising performance on average, but for LLaMA‑3‑8B on OR-Bench the score (71.0) is far lower than Self-CD (86.0). Since the pseudo-harmful subspace is built using OR-Bench data, this is a bit suspicious and raises the question of whether the subspace construction is stable.\n\n2. **The results would be more robust with additional evidence.**\n- For example, confidence intervals or std errors for model performances for Table 1 and Figure 4, how pruning rank and λ are selected for Table 3, and potential evaluation on do_sample=True to give a better sense of the method’s stability under more realistic decoding settings.\n\n3. **Small evaluation samples.**\n- The model performances are reported on a few hundred examples per dataset (according to Section B.2), which has the concern that small fluctuations could produce the reported gains.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "n7Ud8uDwIx", "reviewer": "Reviewer_jCyL", "summary": "This paper proposes ProSafePrune, a low-rank parameter pruning method to mitigate over-refusal in large language models. By identifying and pruning harmful components within pseudo-harmful subspaces, the approach reduces false refusals while preserving genuine safety. Experiments on multiple LLMs show improved compliance and balanced safety performance.", "questions": "1. The paper attributes over-refusal to “over-harmful encoding” in internal representations. Could the authors elaborate on how this relates to recent work on safe unlearning? Conceptually, is ProSafePrune pruning a similar subspace that unlearning methods aim to erase, or does it operate on a different mechanism?\n\n2. How were the pruning hyperparameters (e.g., λ and rank r) selected, and how sensitive are the results to these choices?\n\n3. Would combining ProSafePrune with post-training alignment or inference-time steering further improve the safety–utility trade-off?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The proposed method addresses an important and timely issue in LLM alignment with clear motivation and practical relevance.\n\n2. This paper proposes a lightweight, training-free pruning framework that is theoretically grounded and computationally efficient.", "weaknesses": "1. The claimed “cognitive bias in internal representations” remains largely conceptual — the paper lacks deeper interpretability analysis or causal evidence supporting this mechanism.\n\n2. The proposed pruning relies on white-box parameter access, limiting applicability to real-world LLMs.\n\n3. The novelty is incremental, as the idea of subspace-based low-rank pruning has been explored ; this paper mainly repurposes it for the over-refusal context.\n\n4. While benchmarks show strong quantitative results, the evaluation misses human evaluation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)