FALCON: Fine-grained Activation Manipulation by Contrastive Orthogonal Unalignment for Large Language Model
Authors: Jinwei Hu
Zhenglin Huang
Xiangyu Yin
Wenjie Ruan
Guangliang Cheng
Yi Dong
Xiaowei Huang
Venue: NeurIPS 2025 poster ()
Keywords: LLM Unlearning;Contrastive Learning;Responsible AI
Primary area: social_and_economic_aspects_of_machine_learning

Abstract: Large language models have been widely applied, but can inadvertently encode sensitive or harmful information, raising significant safety concerns. Machine unlearning has emerged to alleviate this concern; however, existing training-time unlearning approaches, relying on coarse-grained loss combinations, have limitations in precisely separating knowledge and balancing removal effectiveness with model utility.  In contrast, we propose $\textbf{F}$ine-grained $\textbf{A}$ctivation manipu$\textbf{L}$ation by $\textbf{C}$ontrastive $\textbf{O}$rthogonal u$\textbf{N}$alignment (FALCON), a novel representation-guided unlearning approach that leverages information-theoretic guidance for efficient parameter selection, employs contrastive mechanisms to enhance representation separation, and projects conflict gradients onto orthogonal subspaces to resolve conflicts between forgetting and retention objectives. Extensive experiments demonstrate that FALCON achieves superior unlearning effectiveness while maintaining model utility, exhibiting robust resistance against knowledge recovery attempts.

FALCON: Fine-grained Activation Manipulation by Contrastive Orthogonal Unalignment for Large L Language Model
Jinwei Hu, Zhenglin Huang, Xiangyu Yin, Wenjie Ruan,
Guangliang Cheng, Yi Dong', Xiaowei Huang!
School of Computer Science and Informatics, University of Liverpool, UK
Abstract
Large language models have been widely applied, but can inadvertently encode sensitive or harmful information, raising significant safety concerns. Machine unlearning has emerged to alleviate this concern; however, existing training-time unlearning approaches, relying on coarse -ranised loss combinations, have limita- tions in precisely separating knowledge and balancing removal effectiveness with model utility. In contrast, we propose Fine- tneeggraied Activation manipul Lation by Contrastive Orthogonal uNalignment (FALCON), novel representation-guee -uuided unlearning approach that leverages coormationnthhoreee guidance for efficient pa- rameter selection, employs contrastive mechanisms enhance representation sepa- ration, and projects conflict gradients onto orthogonal subspaces to resolve conflicts between forgetting and retention objectives. Extensive experiments demonstrate that FALCON achieves superior unlearning effectiveness while maintaining model utility, exhibiting robust resistance against knowledge recovery attempts. Our im- plementation is available at: https: / thub. com/ Sonm/Charles.w22//AAL.... tree/main
Introduction
Recent advancements in generative AI [7],. powered by Parameter-Efficient Fine-Tuning (PEFT) techniques, have enabled LLMs to internalize linguistic knowledge and excel across diverse tasks
29 While these models gain their capabilities from massive datasets, this reliance on large-scale corpora creates significant risks: harmful, biased, or sensitive information can become encoded and amplified, resulting in ethical violations, regulatory noncompliance, and potential misuse 28
[13] or training models with expertly curated datasets refuse harmful queries (60), computationally expensive and often inadequate against adversarial attacks [85). In contrast, while retraining an entire model on a cleaned dataset to eliminate harmful impacts is theoretically feasible, it prohibitively resource-intensive for modern LLMs (44]. Additionally, adversaries can exploit PEFT to reintroduce such unwanted information. highlighting the urgent need for more effective and scalable solutions for publicly accessed LLMs [63].
Existing mitigation strategies, such as LLM guardraits
To solve harmful or sensitive information in machine learning models, Machine Unlcarning (MU) has emerged promising solution, supported by growing regulations such the "right to be forgotten" under the GDPR [67]. It commonly developed in the Lon-LLMs domain and has proven effective at
Corresponding authors: (yi. dong, xiaouei. huang) uc.uk
39th Conference on Neural Information Processing Systems (NeurIPS 2025).

---

When transferred to maintain responsible LLMs, MU offers significant advantages, being far more computationally efficient than full retraining. Unlearned models also exhibit greater inherent safety, they lack the undesired knowledge necessary for malicious behaviors [27 50].
removing specific data influences while preserving model performance [53, 82
Despite its potential, LLM unlearning still faces several fundamental issues: (I1) existing approaches typically rely on empirical methods like grid search to identify intervention parameters, lacking efficient and interpretable guidance within deeper L.LM architectures, rely on yoarse-grainet manipulation (using simplistic loss combinations that induce random represen- tation dispersion with uncontrolled gradient dynamics, struggling to balance knowledge removal and utility preservation) rather than fine- fine-grained representation manipulation (achieving more effective knowledge separation through targeted representation modification and regulated gradient dynamics for reducing damage to model utility), and (I3) knowledge recovery methods such as jailbreaking attack can recover the undesired information from the unlearned model [70]
(I2) current methods normally
FALCON
Updeted
Frozen
Step 2.2: Orthogonalizing Gradient for Contilat Resolution
CyALCON
Redain Activation (R) IE
Vx
Retain LOSS
Yes
Step 1: Ideniity Low- Entanglement Paramaters via Information- Theoretic Guidance
Step 2:1: Contrastive
Representation Untearning
COS/
((((F)
Forgettable
V PALCCN
Unwanted
No
Representaaiion
4 Cxx
Mutual Information
v Cvr
Forget Dataset
Stop Model Unioarning *
Ropresomiation Principal Offict Vector
Updated
Forget Activation
Figure 1: Schematic overview of FALCON. The pipeline comprises three stages: parameter selec-
tion based on mutual information (Step 1); contrastive orthogonal unalignment, which consists of contrastive mechanism both forgetting and retention datasets (Step 2.1) and orthogonal gradient conflict resolution (Step 2.2); and model unlearning guided by these components (Step 3).
To address the aforementioned issues of selective knowledge unlearning in LLMs, we propose Fine-grained Activation manipuLation by Contrastive Orthogonal uNaligmment (FALCON), a representation-guidee framework for targeted knowledge removal with minimal impact on gen- eral capabilities. For I1, FALCON uses mutual information (MI) as an auxiliary signal to assess dependencies between forget and retain data, based on which it introduces two core mechanisms for fine-grained disentanglement and unlearning (Step 1). To tackle 12, FALCON utilizes singular value decomposition (SVD) to identify principal directions in activation space to steer representa- tions along axes misaligned with forgettable knowledge, enabling more thorough removal (Step 2.1). Meanwhile, FALCON uses gradient orthogonal projection strategy, which constrains updates away from irention-sensitive directions, reducing interference with preserved content (Step 2.2). These mechanisms enable precise unlearning with limited data access and remain effective even under single-layer interventions. Afterwards, the projected gradients are used to update the model parameters (Step 3). For I3, we provide comprehensive empirical evidence and analysis in Section 5.3 and Appendix to support our claims. Our contributions are as follows:
We propose FALCON, representation-guiinn guidee framework that combines contrastive mech- anisms and gradient projection to achieve fhnieveffne-greee fine-grained representation unalignment in LLMs.
We introduce information-thcoretic metrics for quantifying knowledge entanglement, enabling principled parameter selection and providing empirical insights into knowledge distribution across model architectures.
We demonstrate the scalability, effectiveness, and resistance to knowledge recovery FALCON through extensive experiments, highlighting its ability to unlearn selective knowledge while preserving utility across various LLMss.
2

---

2
Related work
Our paper focuses on LLM unlearning for undesired knowledge, information-theoos metrics, and contrastive learning. We highlight the developments and limitations of LLM unlearning in this section, while related advancements in information- thheorrttct metrics, contrastive learning, and gradient projection detailed in the Appendix A and
LLM Unlearning LLM unlearning refers the selective removal of specific knowledge from large language models while preserving their overall functionality [87]. Current approaches can be broadly categorized into training-time methods and inference-time methods [3] Among training-time
approaches, which represent the mainstream methodology, two primary directions have emerged.
The first direction focuses on gradient optimization [84 38 91 20] which suppresses harmful
knowledge through loss-driven techniques but often causes catastrophic forgetting and instability when distributions are highly similar or lack fine-grained knowledge localization. The second adaptation, targeting intermediate hidden representations for modification (50) 195 68], but relying on empirical layer selection and lacking targeted separation mechanisms. While these aforementioned training-tima methods achieve permanent unlearning by targeting specific layers and parameters, they currently rely heavily on coarse- gaanse-graineddd loss combinations that struggle to disentangle deeply embedded knowledge representations flexibly [40].
direction emphasizes representation-guidee
tiference-time methods offer alternative approaches like task vectors and model editing. Task vector approaches address efficiency concerns through arithmetic operations on sarameter-efficient modules, enabling lightweight unlearning under resource constraints 36 [ but oversimplify knowledge structure through linear assumptions that fail to capture complex knowledge entanglement. In contrast, model editing usually modifies intermediate hidden states logits to alter model behavior S 39
35 such as contrastive decoding methods that prevent inappropriate responses [94]. Moreover, ECO [5]] has also demonstrated promising performance. though functions more as a guardrail's definition for filtering sensitive content [14 33 rather than directly serving as an unlearning algorithm [55]. However, these methods' dependence on modular arithmetic operations fundamentally limits their granularity in knowledge separation and constrains generalizability across diverse scenarios. Additionally, in-context unlearning has emerged as another inference-tine -ime approach, leveraging tailored prompts to dynamically suppress undesired outputs [93] 62). While flexible, this method's 's effect remains inherently temporary as the undesired knowledge persists in the model's 's representation space [54]]
Despite these advancements, existing training- tiiming--imeeeee methods fall short in achieving precise knowledge disentanglement between information to be forgotten and retained. To address these limitations, we propose FALCON, targeted representation unalignment approach that achieves more precise separation through contrastive learning, gradient projection, and information-theoretie guidance. Through its contrastive mechanism and gradient projection, our approach enables fine-grained knowledge separation and resolves optimization conflicts between forgetting and retention objectives, while enhanced resistance compared current state-of-the-ar training-ting time methods.
3 Problem Formulation
3.1 Problem Setup
The task of LLM unlearning involves selectively removing specific knowledge (forget set) from the model while retaining critical information (retain set). However, this process is complicated by the issue of knowledge entanglement, where representations of the forget and retain sets overlap significantly within the model's parameters [89]. This entanglement arises due to the distributed nature of knowledge across multiple layers and features, making it difficult to isolate knowledge for removal without affecting retained information. To formalize the unlearning process, adopt the general formulation proposed by Liu et al. (54):
min BLaeyeE [C
AB [C()=;;)]]
zz;
where C(y) lz; 9) measures the discrepancy between the model's prediction and the target response y for given input x under the model's parameters 6. Here, D, and Dvx denote the forget set and retain
Further discussion on ECO is shown in Appendix.
P3
3

---

set, respectively. The variable yy specifies the intended output for the forget set after unlearning. while the hyperparameter 2 controls the trade-off between forgetting and retention objectives. For simplicity, we will refer to this objective ming ( (0) in subsequent sections.
Despite the generality of above formulation, it does not explicitly quantify the representations of forgotten and retained knowledge. This lack of quantification poses challenges in precisely guiding the unlearning process [66]. To address this, a principled metric is needed to evaluate and minimize knowledge entanglement, ensuring that unlearning primarily affects the forget set while minimizing interference with the retain set. Consequently, we introduce information- measures, specifically continuous entropy and mutual information, to quantify the dependency between the activations of the forget and retain sets. Let F and R. represent the activations of the forget and retain sets at a specific layer of the model, respectively. The degree of knowledge entanglement between representations can be formulated as the MI (F;;
IFF; HFF) + (R) (F,
where HFF) and (R) are the continuous entropies of the activations F and R, and HJF, R) denotes their joint entropy. These measures provide systematic approach to identify parameters with minimal entanglement and guide the LLM unlearning process. The details of these metrics are
shown in Appendix C
3.2
LLM unlearning with MI Guidance
To quantify knowledge entanglement during machine unlearning, pendency between the activations of the forget set F()) and the retain set R(() at each layer 1. The MI ((F(R;;R() serves as an indicator to guide the unlearning process by minimizing entangle- ment between FiI) and (4), To minimize the entanglement between the forget and retain sets' representations, we formulate the parameter selection for specific LLM layers as:
MI measure the de-
1" argmin (FF)RR())
(3)
Given the selected layer 1", the LLM unlearning problem guided by MI can be reformulated as:
min (0) subject to Eqs. (3)
(4)
0
This formulation ensures that the unlearning process is conducted on the parameters with min- imal knowledge entanglement, effectively suppressing the undesired knowledge while reducing interference with the retained knowledge.
4 Methodology
To address the challenges of more thorough selective multi-domain knowledge unlcarning and enhanced robustness against knowledge recovery in LLMs, propose FALCON shown in Figure 1 and Appendix. [.1]] a framework that advances both precision and effectiveness in knowledge manipulation. Unlike prior approaches that rely on -aarsegrraieed loss combinations, FALCON introduces three key mechanisms: (1) mutual information-based guidance to identify parameters where knowledge representations are least entangled, enabling interpretable parameter selection; (2) contrastive mechanism with enhanced representation separation to achieve fine-grained knowledge manipulation while ensuring robust resistance against knowledge recovery attempts; and (3) gradient orthogonal projection to resolve optimization conflicts and ensure training stability. This holistic design enables precise, interpretable, and robust knowledge unlearning LLMs, transcending traditional loss-combination methods.
4.1 Information- Theoore-ii Guidance for Unlearning
In this paper, utilize principled approach to selective multi-domain knowledge unlearning in LLMs through mutual information. MI provides a natural measure of representational entanglement between the forget and retain datasets across model layers. By identifying parameters that minimize MI, we can target unlearning interventions where forget and retain representations exhibit minimal overlap, thus preserving desired knowledge while selectively removing unwanted information.
We extend this measure to the multi-domain scenario where the forget set F consists of multiple sub-domains F2,.2..... F...
Our approach quantifies two critical relationships: (1) the interaction

---

between each sub-domain and the retain set R, measured by T( ;R()) at layer I, where lower values indicate reduced entanglement and thus more selective unlearning; and (2) the inter-domain
dependencies captured by ; F
for sub-domains F, and F, (i * j), which characterizes potential conflicts or redundancies that may impact unlearning effectiveness.
To quantify the overall representational conflicts between the forget and retain datasets, I ( R()), and the interdependence among forgettable sub-domains, T(F("; F at layer i, we define the
aggregate MI 7()
10 Σ I(F())
E 1(7 (0)
where m denotes the number sub-domains in the forget set F, and is a balancing coefficient that controls the relative importance of inter-domain dependencies. For each layer 1, since the activations are high-dimensional and continuous, direct entropy calculation is infeasible (75). Instead, we utilize Kernel Density Estimation (KDE) to approximate the underlying global data distribution, estimating continuous entropy in activation space as defined in Appendix C [79]. Specifically, we use a multivariate Gaussian kernel, which offers smooth and flexible density estimation well-suited to high- high-dimensional data. The estimated probability density function for activations A is given by:
N
p(a)
Σ n=l
Osp K
(6)
Nh
h
where E Rd represents a single sample from the activations A, including F and R, with d denoting the feature dimensionality of the activations, .N the number of samples, (-) represents the kernel function and h as the adaptive bandwidth calculated using Scott's rule [69]. defined as / - oN के., which is particularly suitable for nigh-dimensional data due its dimensionality- based adjustment. Here, a is the standard deviation of the data. This adaptive bandwidth selection effectively balances bias and variance, ensuring robust density estimation for diverse activation distributions (6] To mitigate the curse of dimensionality, we apply Principal Component Analysis (PCA), which has been widely adopted across various domains in prior work [47 65 71 to reduce activation dimensions before performing KDE (2) retaining at least 95% of variance to ensure minimal information loss while significantly lowering computational complexity.
Using the KDE-based entropy estimations, we approximate the overall mutual information i at each layer based Eq. (5). The optimal layer 1* for unlearning is then determined by minimizing I:
1" arg min 7(0)
By identifying the layer with the lowest MI, we locate the model region where the forger and retain datasets are least entangled, minimizing the overlap between the two types of knowledge. Concurrently, this layer exhibits higher entanglement among sub-domains within the forget set, enabling efficient updates shared representations across forgettable sub-domains. This dual property makes the layer an optimal target for unlearning, where parameters with minimal mutual interference are prioritized remove undesired knowledge while more easily preserving essential and generalizable knowledge for downstream tasks.
4.2 Contrastive Orthogonal Unalignment
To achieve selective knowledge unlearning in LLMs, we first apply MI-guided parameter selection to identify layers with minimal knowledge entanglement, which remains fixed throughout unlearning. We then devise Contrastive Orthogonal Unalignment through contrastive mechanisms and gradient projection, employing alternating strategy between forget and retain datasets to iteratively refine representations while balancing knowledge removal and retention objectives.
4.2.1 Contrastive Representation Unlearning
The core task of LLM unlcarning is to selectively separate knowledge representations to be forgotten from those to be retained. Contrastive learning provides an effective mechanism for this task by
Discussion MI-guided parameter selection is shown in Appendix. F2
5

---

learning discriminative representations through comparing similar and dissimilar samples. In our context, we leverage contrastive learning to maximize the distance between representations that should be forgotten while maintaining the coherence of retained knowledge.
To facilitate thorough unlearning, we construct Principal Offset Vectors (POVs) that steer model activations away from undesired knowledge by redirecting updated forgettable representations into subspaces intentionally misaligned with the principal directions of frozen counterparts, as identified via SVD, thereby achieving representational decoupling within the model.
Mathematically, given an activation matrix H. R(B-L)xD where B is the batch size, L the sequence length, and D the hidden dimension, we perform SVD to obtain the dominant principal directions U1...., UK corresponding to the top-K K singular values. The POV' 7 defined as:
Htt f {(- (1 Ei= (aea .c)
(8)
(r. moe .c)
Here, E IDD is randomly initialized vector, w controls the influence of principal directions, and I E RDxDi is the identity matrix. The term introduces optional perturbations while f(-) is flexible transformation operator, potentially including non-linear mappings (e.g., tanh), adaptive projections, or adversarially -inspired perturbations, enhancing disentanglement and recovery resis- tance. This design ensures 7 is directed away from dominant principal subspaces, combining deterministic guidance and transformations to improve robustness. Unlike generic random vectors, POVs deliberately target dominant features to improve adversarial robustness and unlearning efficacy.
For each input sample, we define three types of representations: the anchor representation Ha from the updated model for the forget set, the positive representation H given by the POV defined in Eq. @). and the negative representations H from the frozen model. To ensure consistent scaling, representations are normalized, and their similarity scores are measured using cosine similarity:
S
Hald Ht|, S
=
Hald H5
(9)
where Z is the number of negative samples. Building on these similarity scores, we define the forget loss Ly using the InfoNCE objective:
[8]
exp(St /T)
C5
log
(10)
exp /T) + E exp(SS
where is temperature scaling parameter. This loss encourages the updated model's representations to align with the POVs while diverging from the frozen model's representations undesired knowl- edge. By leveraging both directional guidance through POVs and contrastive learning, approach achieves more precise and efficient representation unalignment in activation space.
In addition to unlearning undesired representations, preserving critical knowledge for downstream tasks is essential. We define retain loss in Eq. (II) to measure alignment between the updated model's activations (?(") and frozen model's 's activations (HL for the retain set. This retention alignment loss, functioning as a self-super variant contrastive loss, maximizes consistency between updated and frozen activations to ensure effective knowledge preservation during unlearning.
|
Ed Hy [d] H.[[d]
LR =
(11)
ER (H
v VE (H1)1))
This loss ensures alignment between the updated and frozen model activations for the retain set, preserving critical knowledge while complementing the unlearning objective. Combined with the forget loss Cr, this approach achieves an effective balance between unlcarning and retention.
4.2.2
Orthogonalizing Gradient for Conflict Resolution
After computing the forget loss L, and retain loss LR, we address optimization direction mis- alignment between unlearning and retaining by employing a gradient projection mechanism that orthogonalizes conflicting gradients onto subspaces, minimizing interference and promoting bal- anced optimization. Given the gradients of the forget and retain losses, denoted as VLF and V VLR, respectively, the conflict can be quantified using the cosine similarity:

---

vCr VLR
cos (VLr, VCx)
(12)
|V [V
where cos(-) < 0 indicates opposing directions, signifying a conflict between the two objectives. To mitigate this conflict, we adjust the gradients by projecting one onto the orthogonal complement of the other. Specifically, if cos(.) < 0, we project VLr onto the subspace orthogonal to V LR::
VLF VVR
payj = VL
VL
(13)
This adjustment ensures that VC orthogonal to VR., climinating interference from the retain objective during the update for the forget objective. Once the gradients are adjusted, the final update direction of the FALCON determined by combined gradients:
VLFACCNN = aV CF BVC
(14)
where a and / are hyperparameters balancing the contributions of the forget and retain objectives.
This mechanism mitigates gradient conflicts, enabling joint optimization while minimizing inter- ference. By enforcing orthogonality between adjusted gradients, approximates a Pareto-optimal solution. The model then updates its weights using the conflict-reduced gradient, allowing for more flexible adaptation. To further enhance efficiency and stability, we leverage the second-order opti- mizer Sophia (52), as suggested in [25] 41]. for refined weight updates, ensuring more effective and stable optimization process for selective knowledge unlearning.
5 Experiments
To validate FALCON's effectiveness, we conduct extensive experiments to answer the following research questions: RQ1: Does FALCON with MI guidance, establish quantifiable measure for principled parameter selection while achieving superior performance in harmful knowledge unlearning tasks? (Section RQ2: Does FALCON maintain strong generalizability across diverse unlearning tasks including enfity unlearning and copyrighted content unlearning? (Section 5.2 RQ3: Beyond efficient parameter space reduction through MI guidance, does FALCON's algorithmic design offer competitive computational efficiency? (Appendix E.3) RQ4: Can FALCON effectively resist recovery attempts of unlearned knowledge? (Section
More complete experiments and
ablation study are shown in Appendix
5.1 Harmful Knowledge Unlearning
To validate RQ1, we use the WMDP [50 benchmark for harmful knowledge unlearning assessment, WikiText [59] for measuring perplexity, and MMLU [26] for evaluating model utility. We test
FALCON on three pre-trained LLMs: Zephyr-7B-Beta [76], Yi-6B-Chat [86), and Mistral-7B.
Instruct-v0.3 42), comparing against all baselines from [50] with details in Appendix
5.1.1 Mutual Information for Parameter Selection
Hestmap of Matual Information Across Different Layers
Ba-Wii
68
CskenWWak
86
X0
Mitral7B-ttirrd--333
C14
Figure 2: Heatmaps of MI across LLM layers show that lower MI values indicate layers better suited for unlearning, with early layers being more domain-specific and deeper layers more entangled.
Visualization of MI for LLMSs Figure 2 presents MI heatmaps illustrating knowledge entanglement between forget sets (WMDP-Bio, W WMDP-Cyber) and the retain set (WikiTeext-2raww-v) across LLM layers. This metric provides an interpretable measure for identifying layers with minimal entangle- ment for targeted unlearning. All models show lower MI values in earlier layers, indicating more domain-specific and disentangled representations, which aligns with both intuition and experimental observations [50]. Yi-6B-Chat demonstrates particularly complex entanglement patterns between
7

---

domains, presenting greater difficulty for unlearning multi-domain knowledge and making it an ideal candidate for our effectiveness analysis experiments in Soctioo[3.1.2 Beyond identifying optimal intervention parameters, MI-guided selection improves efficiency by narrowing the parameter search space compared to exhaustive methods like grid search, scaling effectively with model complexity.
Gradient Conflicts Analysis We empirically
validate the underlying principle of MI guidance by analyzing gradient conflicts be- tween forget and retain objectives across lay- ers. As shown in Figure 3 layers with low MI values exhibit significantly reduced con- flicts, with cosine similarities near zero, in- dicating minimal interference between objec- tives. Conversely, high-MI layers show pro- nounced, fluctuating conflicts, highlighting the issues of entangled representations. These results confirm that mutual information reliable auxiliary signal for guiding param- eter selection, as low-MI parameters reduce interference, support stable updates, and help mitigate conflicts between unlearning and re- tention goals.
Gradient Conflicts Across Different Layers
0.4
Lages
Low
0.2
Cosige Simalarty
0.2
-0.4
-0.6
Iterations
Figure 3: Gradient conflicts across layers with mini- mum (blue) and maximum (orange) MI values com- puted during parameter selection Mistral-7B.
5.1.2 Unlearning Effectiveness and Utility Analysis
We evaluate FALCON against all baseline methods across three LLM architectures shown in Table | and Appendix E..] with our evaluation focusing on three key met- rics: WMDP scores for measuring unlearning effectiveness, MMLU scores for assessing general knowledge retention, and perplexity (PPL) for model stability. Our primary ob-
Table Unlearning effectiveness and utility across models methods. Metries with (t) indicate preferable increases; (L) indicate preferable decreases.
Method
WMDP (L)
MMLU (1)
PPI. (L)
Bio
Cyber
Zephyr-7B
63.7
43.8
58.1
1.5
LLMO
36.3
20.3
50.3
438
jective is minimize WMDP scores while
+ SCRUB
38.7
35.4
50.0
16.5
maintaining MMLU and PPL values close
+ SSD
53.1
43.2
52.8
1.6
to the base model's performance (MMLU
RMU
34.5
28.9
57.4
1.5
and PPI., this indicates successful knowl- FALCON
26.7
25.3
57.4
1.5
edge removal without compromising general
Yi-6B-
65.4
42.6
61.8
1.5
capabilities. To ensure quantifiable com-
+ LLMO
56.2
39.9
373
5.4
parison, we prioritize maintaining general model utility and report each method's best
+ SCRUB
38.7
35.5
50.0
16.4
+ SSD
55.1
43.7
53.8
1.6
unlearning performance under this setting. Results demonstrate FALCON's superior per- formance compared to baselines that struggle with effectivennss--tiile utility balance and show increased uncertainty in their perplexity. On Zephyr-7B, FALCON achieve lower forgetting scores while preserving general capabilitics. This advantage more clear Yi-6B-Chat with its complex knowledge entanglement: RMU show significant biological domain degradation when constrained to maintain MMLU above 60%, while FALCON maintain consistent effectiveness with superior general performance. These findings validate our fine- mme-grained representation guided mechanisms for targeted unlearning with preserved utility, even in scenarios with complex knowledge entanglement.
RMU
50.8
33.5
59.6
1.6
FALCON
27.7
25.3
60.3
1.5
5.2 Cross-Domain Generalizability Assessment
To address RQ2, we conduct additional experiments on copyrighted content and entity unlearning
using the MUSE 772 and TOFU (56) benchmarks with additional baselines [66. For RQ3, we compare computational efficiency across methods in Appendix E.3 All aforementioned experiments
optimizers for fair comparison, with complete implementation details in Appendix

---

5.2.1 Copyrighted Content Unlearning
For copyrighted content unlearning, utilize the MUSE benchmark and Llama-2-7b to assess FALCON's effectiveness in removing protected news articles while preserving general capabilities. As shown in Table 2 FALCON achieved the lowest forget metrics scores (0.02 and 0.03) while maintaining competitive retention (0.54), Unlike baselines, FALCON consistently balanced copyright removal with knowledge preservation, demonstrating broader applicability beyond harmful content removal.
Table 2: Evaluation on MUSE News over 10 epochs.
Method
frrget_knwwemn [,
barge__vrbmemm_ ve
retain _anen__ ROUGET
Finetuned
0.64
0.58
0.55
Retain
0.33
0.21
0,56
GradAscent
0.00
0.00
GradDiff
0.41
8.92c-3
0.37
NPO
0,56
0..35
0.51
SimNPO
0,54
0).36
0.51
RMU
0.48
0.05
0.51
FALCON
0.02
0.03
0.54
5.2.2 Entity Unlearning
Table 3: TOFU evaluation across varying sizes over 10 epochs.
We evaluate FALCON's ability to remove knowledge about fictitious entities using TOFU with varying forget data sizes (1/5/10%). Our method maintain strong forget qual- ity (FQ1) and model utility (MU1) across different splits on Llama-3.2- IB-ntttucct. Even with only un- learning epochs, FALCON consis-
Forget01
Forget05
Forget10
Method
FQ
MU
FQ
MU
FQ
MU
Finetuned
0,01
0.60
2.96e 13
0.60
8.08e-22
0.6
Retain
1.0
0.60
1.0
0.60
1.0
0.59
GradAscent
0.27
0.33
1.94e-119
1.06e-239
0
GradDiff
0.77
0.43
2.04e-110
0.22
1.06e-239
0.49
ldkDPO
0.01
0.51
4.02e-06
0.04
4.26e-10
0.08
tently outperform baselines in bal- ancing knowledge removal with pre- served utility. Notably, while other
NPO
0.92
0.56
0.32
0.42
0.02
0.46
RMU
0.16
0.55
1.46c-7
0.57
1.4e-20
0.59
FALCON
0.99
0.55
0.92
0.59
0.52
0.60
methods like Graenscent suffers significant utility degradation with larger forget sets, FALCON remains effective, demonstrating our method's generalizability to entity unlearning tasks.
5.3 Resistance Against Knowledge Recovery Attempts
We conduct experiments on Yi-6B- Chat evaluate FALCON's resis- tance against knowledge recovery at- tempts [55] for RQ4. Logit Lens [1]]. which projects intermediate ac- tivations onto the model's vocabu- lary space, serves as powerful tech- nique for probing the model's internal knowledge representations and poten- tial recovery of unlearned information. Figure 4: Logit lens probing results on different components. As shown in Figure [ the logit lens analysis across different architectural components such as MLP and attention layers demonstrates that the unlearned knowledge remains consistently inaccessible, with performance staying close to the unlearned bascline and far below the original model's 's performance.
Origiaa
WMDD--Bo Boo Store
0.38
0.30
0.25
30
Layers
Additionally, as shown in Table 4 FALCON exhibits strong resilience against enhanced GCG in QA setting, an advanced vreii-optimizztiion based jailbreaking attack that compromises other baselines such as RMU 73 Even with increasing attack iterations, the recovered WMDP scores remain close to the unlearned baseline, demonstrating robust unlearning through funda- mental changes to the model's internal representations rather than superficial knowledge mask-

---

ing. Further evaluation using conversational templates for jailbreaking attacks (detailed in Ap- pendix [Ε,6) further validates our method's robustness against knowledge recovery attempts.
These results across both
Table 4: Knowledge recovery results via enhanced GCG attack.
probing techniques validate FALCON's s effectiveness in creating a more permanent and recovery-resistan form of knowledge removal.
Dataset
Original Score
Unlearning Score
Recovery Score via Enhanced GCG GCG-1000
GCG-500
ECG-150X
GCG-2000
WMDD-Bio WMDP-Cyber
65.4
27.6
28.4
42.6
25.3
26,3
26.4
25.8
24.7
Practical Implications of LLM Unlearning for Responsible AI
The problem setting addressed by FALCON stems from the growing challenge directly employing LLMs or deploying them as autonomous agents in safety-critical environments [32] As these models become increasingly embedded in diverse real-world applications, selectively removing undesired or harmful knowledge after deployment remains difficult (54). Unlike conventional machine learning models where unwanted data can simply be excluded future training cycles, LLMs encode information across billions of parameters, making precise removal extremely challenging. This limitation creates a critical gap between learning capabilities and responsible deployment. The issue is further amplified by regulatory demands such the GDPR's "right to be forgotten" [67], and by empirical evidence that even state-of-the L.LSs and their agentic variants can inadvertently reproduce sensitive or hazardous content when prompted, raising urgent concerns about information safety and controllability.
FALCON provides fine-gram the-grained unlearning mechanism that identifies harmful knowledge and decouples from beneficial reasoning. This targeted process enables models to forget unsafe information while retaining legitimate competence, supporting the emerging need for responsible LLM deployment [T). As LLMs operate in dynamic, real-world contexts, the capacity for precise and interpretable knowledge modification becomes essential for responsible AI. We advocate viewing unlearning not as an academic objective as core practical infrastructure for transparent, compliant, and responsible Al systems.
7 Conclusion
This paper presents FALCON, a fine-grained representation-guue guided framework for LLM unlearning. Leveraging mutual information guidance and contrastive orthogonal unalignment, it enables precise and efficient unlearning through principal component-base based representation separation and gradient conffict resolution. Extensive experiments demonstrate its superior performance in effectively removing undesired knowledge while preserving essential information across diverse tasks, along with resistance against knowledge recovery and efficient optimization guidance. However, this work currently limited to text-based LLM unlearning, with experiments conducted on relatively smaller models due computational constraints. Future directions include extending unlearning multi- modal LLMs and refining strategies to disentangle intertwined knowledge in deeper architectures.
Acknowledgment
This work is partially funded by the European Union (under grant agreement ID 101212818). Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or European Health and Digital Executive Agency (HADEA). Neither the European Union nor the granting authority can be held responsible for them. This work is partially supported by Innovate UK through AI-PASSPORT under Grant 10126404. Yi's contribution is partially supported through the Royal Society international exchanges programme and in part by the Engineering and Physical Sciences Research Council [grant number EP/Y009800/1], through funding from RAi UK.
Additioonl discussions provided in Appendix F
10

---

References
[1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et Gpt-4 technical report. arXiv preprint arXhv:2303.0877 2023.
[2 Naomi Altman and Martin Krzywinski. The curse (s) of dimensionality. Nat Methods, 15(6):399- 400, 2018.
[3] Shuang Ao, Yi Dong, Jinwei Hu, and Sarvapali Ramchurn. Safe pruning lora: Robust distance- guided pruning for safety alignment adaptation 1lms. arXiv preprint (tt.:2550..193311 2025.
[4] Giuseppe Attanasio, Debora Nozza, Dirk Hovy, Elena Baralis, et al. Entropy- based attention regularization frees unintended bias mitigation from lists. In Findings of the Association for Computational Linguisties: ACL 2022, pages 1105-1119, Association for Computational Linguistics, 2022.
[5] Fazl Barez, Tingchen Fu, Ameya Prabhu, Stephen Casper, Amartya Sanyal, Adel Bibi, Aidan O' O'Gara, Robert Kirk, Ben Bucknall, Tim Fist, Luke Ong, Philip Torr, Kwok- Yan Lam, Robert Trager, David Krueger, Sören Mindermann, José Hernandez-Orallo, Mor Geva, and Yarin Gal. Open problems in machine unlearning for ai safety, 2025.
[6] Elsidieg Belhaj. A modified rulc-of-thumb method for kernel density estimation. 2024.
[7] Junbum Cha, Kyungjae Lee, Sungrae Park, and Sanghyuk Chun. Domain generalization by nuttualinformmtioo regularization with pre-trained models. In European conference on computer vision, pages 440-457. Springer, 2022.
[8] Sungmin Cha, Sungjun Cho, Dasol Hwang, Honglak Lee, Taesup Moon, and Moontae Lee. Learning to unlearn: Instance-wist unlearning for pre-trained classifiers. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pages 11186-11194 2024.
[9] Cheng Chen, Ji Zhang, Jingkuan Song. and Lianli Gao. Class gradient projection for continual learning. In Proceedings of the 30th ACM International Conference on Multimedia, pages 5575-5583, 2022.
[10] Xin Chen, Hanxian Huang, Yanjun Gao, Yi Wang, Jishen Zhao, and Ke Ding. Learning to maximize mutual information for chain-of-thought distillation. arXiv preprint 24X.0.2403.03338, 2024.
[11] Wenying Deng, Beau Coker, Rajarshi Mukherjee, Jeremiah Liu, and Brent Coull. Towards unified framework for uncertainty-aware nonlinear variable selection with theoretical guarantees. Advances in Neural Information Processing Systems, 35:27636-276511 2022.
[12] Ann-Kathrin Dombrowski and Guillaume Corlouer. An information-theoretie study of lying in LLMs. In ICML 2024 Workshop on LLMs and Cognition, 2024.
[13] Yi DONG, Ronghui Mu, Gaojie Jin, Yi Qi, Jinwei Hu, Xingyu Zhao, Jie Meng. Wenjie Ruan, and Xiaowei Huang. Position: Building guardrails for large language models requires systematic design. In Forty-first International Conference Machine Learning, 2024.
[14] Yi Dong, Ronghui Mu, Yanghao Zhang, Siqi Sun, Tianle Zhang, Changshun Wu, Gaojie Jin, Yi Qi, Jinwei Hu, Jie Meng, et al. Safeguarding large language models: A survey, Artificial Intelligence Review, 58(12):382, 2025.
[15] Yijiang River Dong, Hongzhou Lin, Mikhail Belkin, Ramon Huerta, and Ivan Vulić. Undial: deff-distillation with adjusted logits for robust unlearning in large language models. arXiv preprint arXiv:2402.10052. 2024.
[16] Vineeth Doma, Anmol Mekala, Wenlong Zhao, Andrew McCallum, Zico Kolter, and Pratyush Maini. OpenUnleamning unified framework for Ilm unlearning benchmarks. hhttps: // github.com/locus.ll htw.ab/.coo/occssla/ggeeee..c watee-unleaannnggn 2025. Accessed: February 27, 2025.

---

[17] Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang. Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 2024.
[18] Ronen Eldan and Mark Russinovich. Who's harry potter? approximate unlearning in Ilms. arXiv preprint arX0o:3310.02238, 2023.
[19] Linus Ericsson, Henry Gouk, Chen Change Loy, and Timothy M Hospedales. Self-supervised representation learning: Introduction, advances, and challenges. IEEE Signal Processing Magazine, 39(3):42-61 2022.
[20] Chongyu Fan, Jiancheng Liu, Licong Lin, Jinghan Jia, Ruiqi Zhang. Song Mei, and Sijia Liu. Simplicity prevails: Rethinking negative preference optimization for Ilm unlearning. arXiv preprint arXiv:2410.07163, 2024.
[21] Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal. Detecting hallucinations in large language models using semantic entropy. Nature, 630(8017):625-636 2024.
[22] Jack Foster, Stefan Schoepf, and Alexandra Brintrup. Fast machine unlearning without retraining through selective synaptic dampening. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 12043-12051, 2024.
[23] Marylou Gabrié, Andre Manoel, Clément Luneau, Nicolas Macris, Florent Krzakala, Lenka Zdeborová, al. Entropy and mutual information in models of deep neural networks. Advances in neural information processing systems, 2018.
[24] Piotr Garbaczewski. Differential entropy and dynamics of uncertainty. Journal of Statistical Physics, 123:315-22 1-555,--55,20 2006.
[25] Kang Gu, Md Rafi Ur Rashid, Najrin Sultana, and Shagufta Mehnaz. Second-order infor- mation matters: Revisiting machine unlearning for large language models. arXiv preprint 1055572 2024.
[26] Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In International Conference Learning Representations, 2021.
[27] Dan Hendrycks, Nicholas Carlini, John Schulman, and Jacob Steinhardt. Unsolved problems in ml safety. arXiv preprint arXiv:2109. 139/6, 2021.
[28] Chia- Yi Hsu, Yu-Lin Tsai, Chih-Hsun Lin, Pin- Pin-Yu Chen, Chia-Mu Yu, and Chun- Chun-Ying Huang. Safe lora: the silver lining of reducing safety risks when fine-tuning large language models. arXiv preprint arXiv:2405.1683. 2024.
[29] Edward Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022.
[30] Haigen Hu, Xiaoyuan Wang, Yan Zhang, Qi Chen, and Qiu Guan. A comprehensive survey on contrastive learning. Neurocompuling, page 128645, 2024.
[31] Jinwei Hu, Yi Dong, Shuang Ao, Zhuoyun Li, Boxuan Wang, Lokesh Singh, Guangliang Cheng, Sarvapali D Ramchur,, and Xiaowei Huang. Position: Towards a responsible lIm-empowered multi-agent systems. arXiv preprint arXiv:2502.01714 2025.
[32] Jinwei HU, Yi DONG, Zhengtao DING, and Xiaowei HUANG. Enhancing robustness of IIm-
driven multi-agent systems through randomized smoothing. Chinese Journal of Aeronautics, page 103779, 2025.
[33] Jinwei Hu, Dong, and Xiaowei Huang. Trust-oriented adaptive guardrails for large language models. arXiv preprint arXiv:2408.0m 08959, 2024.
[34] Jinwei Hu, Yi Dong, Youcheng Sun, and Xiaowei Huang. Tapas are free! training-free adaptation of programmatic agents via Ilm-guided program synthesis in dynamic environments. arXiv preprint arXiv:2508. 2025.
12

---

[35] James Y Huang, Wenxuan Zhou, Fei Wang. Fred Morstatter, Sheng Zhang. Hoifung Poon, and Muhao Chen. Offiset unlearning for large language models. arXiv preprint arXiv:2404. 10045, 2024.
[36] Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannaneh Ha- jishirzi, and Ali Farhadi. Editing models with task arithmetic. In The Eleventh International Conference Learning Representations, 2023.
[37] Shadi Iskander, Kira Radinsky, and Yonatan Belinkov. Shielded representations: Protecting sensitive attributes through iterative gradient-based projection. In Findings of the Association for Computational Linguistics: ACL 2023, pages 5961-5977. July 2023.
[38] Joel Jang, Dongkeun Yoon, Sohee Yang, Sungmin Cha, Moontae Lee, Lajanugen Logeswaran, and Minjoon Seo. Knowledge unlearning for mitigating privacy risks in language models. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki, editors, Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), July 2023.
[39] Jiabao Ji, Yujian Liu, Yang Zhang, Gaowen Liu, Ramana Kompella, Sijia Liu, and Shiyu Chang. Reversing the forget-retain objectives: An efficient Ilm unlearning framework from logit difference. Advances in Neural Information Processing Systems, 37:12581-1261 2024.
[40] Jinghan Jia, Jiancheng Liu, Yihua Zhang. Parikshit Ram, Nathalie Baracaldo, and Sijia Liu. Wagle: Strategic weight attribution for effective and modular unlearning large language models. arXiv preprini arXiv:2410. 0002400..7509. 17599, 2024.
[41] Jinghan Jia, Yihua Zhang, Yimeng Zhang, Jiancheng Liu, Bharat Runwal, James Diffenderfer, Bhavya Kailkhura, and Sijia Liu. Soul: Unlocking the power of second-order optimization for Ilm unlearning. arXiv preprint arXiv:2404. 18239, 2024.
[42] Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023.
[43] Junfeng Jiao, Saleh Afroogh, Yiming Xu, and Connor Phillips. Navigating lIm ethics: Advance- ments, challenges, and future directions. arXiv preprint arXiv:2406..888 2024.
[44] Hongpeng Jin, Wenqi Wei, Xuyu Wang, Wenbin Zhang, and Yanzhao Wu. Rethinking learning rate tuning in the era of large language models. In 2023 IEEE 5th International Conference on Cognitive Machine Intelligence, pages 112-121, 2023.
[45] Jonathan Kahana and Yedid Hoshen. A contrastive objective for learning disentangled represen- tations. In European Conference on Computer Vision. pages 579-595. Springer, 2022.
[46] Junyaup Kim and Simon S Woo, Efficient two-stage model retraining for machine unlearning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 4361 16611,369,12 2022.
[47] Matthaus Kleindessner, Michele Donini, Chris Russell, and Muhammad Bilal Zafar. Efficient fair pca for fair representation learning. In International Conference Artificial Intelligence and Statistics, pages 1250-5270. PMLR, 2023.
[48] Meghdad Kurmanji, Peter Triantafillou, Jamie Hayes, and Eleni Triantafillou. Towards un- bounded machine unlearning. Advances in neural information processing systems, 36, 2024.
[49] Linyang Li, Ruotian Ma, Qipeng Guo, Xiangyang Xue, and Xipeng Qiu. BERT-ATTACK: Adversarial attack against BERT using BERT. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu, editors, Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing (EMNLP), pages 1193-6202. Association for Computational Linguistics, November 2020.
13

---

[50] Nathanicl Li, Alexander Pan, Anjali Gopal, Summer Yue, Daniel Berrios, Alice Gatti, Justin D. Li, Ann-Kathrin Dombrowski, Shashwat Goel, Gabriel Mukobi, Nathan Helm-Burgee Rassin Lababidi, Lennart Justen, Andrew Bo Liu, Michael Chen, Isabelle Barrass, Oliver Zhang, Xiaoyuan Zhu, Rishub Tamirisa, Bhrugu Bharathi, Ariel Herbert- Voss, Cort B Breuer, Andy Zou, Mantas Mazeika, Zifan Wang. Palash Oswal, Weiran Lin, Adam Alfred Hunt, Justin Tienken- Tienken-Harder, Kevin Y. Shih, Kemper Talley, John Guan, Ian Steneker, David Campbell, Brad Jokubaitis, Steven Basart, Stephen Fitz, Ponnurangam Kumaraguru, Kallol Krishna Karmakar, Uday Tupakula, Vijay Varadharajan, Yan Shoshitaishvili, Jimmy Ba, Kevin M. Esvelt, Alexandr Wang, and Dan Hendryeks. The WMDP benchmark: Measuring and reducing malicious use with unlearning. In Forty-first International Conference on Machine Learning, 2024.
[51] Chris Liu, Yaxuan Wang, Jeffrey Flanigan, and Yang Liu. Large language model unlearning via umbedding-corrupeed prompts. Advances in Neural Information Processing Systems, 17:11 11 118266, 2024.
[52] Hong Liu, Zhiyuan Li, David Leo Wright Hall, Percy Liang, and Tengyu Ma. Sophia: A scalable stochastic second-order optimizer for language model pre-training. The Twelfth International Conference on Learning Representations, 2024.
[53] Junxu Liu, Mingsheng Xue, Jian Lou, Xiaoyu Zhang, Li Xiong, and Zhan Qin. Muter: Machine unlearning on adversarially trained models. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 4892-4902, 2023.
[54] Sijia Liu, Yuanshun Yao, Jinghan Jia, Stephen Casper, Nathalie Baracaldo, Peter Hase, Yuguang Yao, Chris Yuhao Liu, Xiaojun Xu, Hang Li, et al. Rethinking machine unlearning for large language models. Nature Machine Intelligence, pages 1-14, 2025.
[55] Jakub Lucki, Boyi Wei, Yangsibo Huang, Peter Henderson, Florian Tramèr, and Javier Rando. An adversarial perspective on machine unlcarning for AI safety. In Transactions on Machine Learning Research, 2024.
[56) Pratyush Maini, Zhili Feng, Avi Schwarzschild, Zachary Chase Lipton, and J Zico Kolter. TOFU: A task of fictitious unlearning for 1lms. In First Conference on Language Modeling, 2024.
[57] Andrey Malinin and Mark Gales. Predictive uncertainty estimation via prior networks. Advances in neural information processing systems, 31, 2018.
[58] Kim Martineau. Why we're teaching Ilms to forget things. htttp:://eeeaarda iim. com/ hotg/11mmunlle laaninn October 2024.
[59] Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. arXiv preprint arXiv: (00.66990777433 2016.
[60] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishknn, Chong Zhang. Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 27730 277444. Curran Associates, Inc., 2022.
[61] Vaidehi Patil, Peter Hase, and Mohit Bansal. Can sensitive information be deleted from LLMs? objectives for defending against extraction attacks. In The Twelfth International Conference on Learning Representations, 2024.
[62] Martin Pawelczyk, Seth Neel, and Himabindu Lakkaraju. In-context unlearning: Language models as few-shot unlearners. In Forty-first International Conference on Machine Learning, 2024.
[63] Xiangyu Qi Yi Zeng, Tinghao Xie, Pin- Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. Fine-tuning aligned language models compromises safety, even when users do not intend to! In The Twelfth International Conference on Leurning Representations, 2024.
14

---

[64] Xin Qiu and Risto Mikkuulaienn. Semantic density: Uncertainty quantification for large language models through confidence measurement in semantic space. In The Thirty-eighth Anmual Conference on Neural Information Processing Systems, 2024.
[65] Yifu Qiu, Zheng Zhao, Yftah Ziser, Anna Korhonen, Edoardo Maria Ponti, and Shay Co- hen. Spectral editing of activations for large language model alignment. Advances in Neural Information Processing Systems, 07756658--69887 2024.
[66] Youyang Qu, Ming Ding, Nan Sun, Kanchana Thilakarathna, Tianqing Zhu, and Dusit Niyato. The frontier of data erasure: Machine unlearning for large language models. arXiv preprint arXiv:2403.15779 2024.
[67] Protection Regulation. Regulation (eu) 2016/679 of the european parliament and of the council. Regulation (eu), 679:2016, 2016.
[68] Domenic Rosati, Jan Wehner, Kai Williams, Lukasz Bartoszeze, Robie Gonzales, Subhabrata Majumdar, Hassan Sajjad, Frank Rudziez, et al. Representation noising: A defence mechanism against harmful finetuning. Advances in Neural Information Processing Systems, 37:1 16666 12676, 2024.
[69] David W Scott. Multivariate density estimation; theory, practice, and visualization. John Wiley & Sons, 2015.
[70] Thanveer Shaik, Xiaohui Tao, Haoran Xie, Lin Li, Xiaofeng Zhu, and Qing Li. Exploring the landscape of machine unleaming: A comprehensive survey and taxonomy. IEEE Transactions on Neural Networks and Learning Systems. pages 1-21, 2024.
[71] Shun Shao, Yftah Ziser, and Shay B. Cohen. Gold doesn't 't always glitter: Spectral removal of linear and nonlinear guarded attribute information. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 1611-1622, Dubrovnik, Croatia, May 2023. Association for Computational Linguistics.
[72] Weijia Shi, Jaechan Lee, Yangsibo Huang, Sadhika Malladi, Jieyu Zhao, Ari Holtzman, Daogao Liu, Luke Zettlemoyer, Noah A. Smith, and Chiyuan Zhang. Muse: Machine unlearning six-way evaluation for language models. 2024.
[73] T Ben Thompson and Michael Sklar. Firt: Fluent student-teaches redteaming, arXiv preprint arXiv:2407.17447, 2024.
[74] Michael Tschannen, Josip Djolonga, Paul Rubenstein, Sylvain Gelly, and Mario Lucic. On mutual information maximization for representation learning. In International Conference on Learning Representations, 2020.
[75] Dor Tsur, Ziv Goldfeld, and Kristjan Greenewald. Max-sliced mutual information. Advances in Neural Information Processing Systems, 36, 2024.
[76] Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro von Werra, Clémentine Fourrier, Nathan Habib, et al. Zephyr: Direct distillation of Im alignment. arXiv preprint arXiv:23110. 16944, 2023.
[77] Aleksandra Urman and Mykola Makhortykh. The silence of the llms: Cross-lingual analysis of political bias and false information prevalence in chatgpt, google bard, and bing chat. 2023.
[78] Joost Van Amersfoort, Lewis Smith, Yee Whye Teh, and Yarin Gal. Uncertainty estimation using a single deep deterministic neural network. In International conference on machine learning, pages 9690-9700. PMLR, 2020.
[79] Janett Walters-Williams and Yan Li. Estimation of mutual information: A survey. In Rough Sets and Knowledge Technology: 4th International Conference, RSKT 2009, Gold Coast, Australia, July 14-16, 2009. Proceedings 4, pages 389-396. Springer, 2009.
[80] Jiaan Wang, Jianfeng Qu, Kexin Wang, Zhixu Li, Wen Hua, Ximing Li, and An Liu. Improving the robustness of knowledge-groundee dialogue via contrastive learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 19135-19143, 2024.
15

---

[81] Junda Wu, Tong Yu, Rui Wang, Zhao Song, Ruiyi Zhang, Handong Zhao, Chaochao Lu, Shuai Li, and Ricardo Henao. Infoprompt: Information theoretic soft prompt tuning for natural language understanding. Advances in Neural Information Processing Systems, 36, 2024.
[82] Yongliang Wu, Shiji Zhou, Mingzhuo Yang. Lianzhe Wang, Heng Chang. Wenbo Zhu, Xinting Hu, Xiao Zhou, and Xu Yang. Unlearning concepts in diffusion model via concept domain correction and concept preserving gradient. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 8496-8504, 2025.
[83] Tzu-Hsuan Yang and Cheng-Te Te Li. When contrastive learning meets graph unlcarning: Graph contrastive unlearning for link prediction. In 2023 IEEE International Conference on Big Data, pages
IEEE, 2023.
[84] Yuanshun Yao, Xiaojun Xu, and Yang Liu. Large language model unlearning. In Socially Responsible Language Modelling Research, 2023.
[85] Ziyi Yin, Muchao Ye, Tianrong Zhang, Jiaqi Wang. Han Liu, Jinghui Chen, Ting Wang, and Fenglong Ma. Vqattack: Transferable adversarial attacks on visual question answering via pre-trained models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38,
pages 6755-6763, 2024.
[86] Alex Young, Bei Chen, Chao Li, Chengen Huang, Ge Zhang. Guanwei Zhang, Heng Li, Jiangcheng Zhu, Jianqun Chen, Jing Chang, et al. Yr: Open foundation models by 01. ai. arXiv preprint arXiv:2403. 046522, 2024.
[87] Dawen Zhang, Pamela Finckenberg- Bromman. Thong Hoang, Shidong Pan, Zhenchang Xing, Mark Staples, and Xiwei Xu. Right to be forgotten in the era of large language models: Implications, challenges, and solutions. Al and Ethics, pages 1-10, 2024.
[88] Jinghan Zhang, Junteng Liu, Junxian He, et al. Composing parameter-efficient modules with arithmetic operation. Advances in Neural Information Processing Systems, 12610, 2023.
[89] Ningyu Zhang, Yunzhi Yao, Bozhong Tian, Peng Wang, Shumin Deng, Mengru Wang, Zekun Xi, Shengyu Mao, Jintian Zhang, Yuansheng Ni, al. A comprehensive study of knowledge editing for large language models. arXiv preprint ar arXiv:2401.01286, 2024.
[90] Qiuchen Zhang, Carl Yang, Jian Lou, Li Xiong, et al. Contrastive unlearning: A contrastive approach to machine unlearming. arXiv preprint arXi:2401.11455, 2024.
[91] Ruiqi Zhang, Licong Lin, Yu Bai, and Song Mei. Negative preference optimization: From catastrophic collapse to effective unlearning. arXiv preprint urXiv:2404.058868 2024.
[92] Zuyu Zhang, Yan Li, and Byung-Seok Shin. Embracing domain gradient conflicts: Domain generalization using domain gradient equilibrium. In Proceedings of the 32nd ACM International Conference Multimedia, pages 5594-5603, 2024.
[93] Ce Zheng, Lei Li, Qingxiu Dong, Yuxuan Fan, Zhiyong Wu, Jingjing Xu, and Baobao Chang. Can we edit factual knowledge by in-context learning? In The 2023 Conference on Empirical Methods in Natural Language Processing, 2023.
[94] Qihuang Zhong, Liang Ding, Juhua Liu, Bo Du, and Dacheng Tao. Rose doesn't do that: Boosting the safety of instruction-tunee large language models with reverse prompt contrastive decoding. arXiv preprint arXiv:2402.11882 2024.
[95] Andy Zou, Long Phan, Justin Wang, Derek Duenas, Maxwell Lin, Maksym Andriushchenko, J Zico Kolter, Matt Fredrikson, and Dan Hendrycks. Improving aligmment and robustness with circuit breakers. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
16

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No]" ", is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). ) In general, answering "[No] " or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendíx. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: The abstract and introduction clearly summarize the main contributions of FALCON, including its practicality and effectiveness.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]

---

Justification:
The paper discusses current limitations in conclusion.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or specch-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification: This work present many experimental evidences and it is primarily empirical in nature.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed reproduce the main perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: The paper provides detailed descriptions of datasets, model configurations, and evaluation metrics to facilitate reproducibility.
Guidelines:
18

---

The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (c.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
also be provided via detailed
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g.. with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: An anonymized GitHub repository will be released upon acceptance, contain- ing implementation and experiment code
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
ssssnnoiiccc)) for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https)
/ //nips.cc/publi: http://ccppbbcigiiduesss///////e
for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.
19

---

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Training setups, hyperparameters, and baseline configurations are thoroughly documented in the main paper and appendix.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [NA]
Justification: Due to the intrinsic non-deterministic behavior of LLM, traditional significance testing not directly applicable.
Guidelines:
The answer NA means that the paper does include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: The paper reports the use of specific models, sizes and prove the unlearning efficiency in appendix.
20

---

Guidelines:
The answer NA that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: picttps:/neurrppee ct/ppblic//ccccccccccccccccccclt http:///tehiisiieelliiiieeeeee
Answer: [Yes]
Justification: The research complies with NeurIPS ethical standards and focuses enhane- ing model safety through principled unlearning.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations in their jurisdiction..
Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative socictal impacts of the work performed?
Answer: [Yes]
Justification: The paper discusses both the benefits of preventing misuse through knowledge removal.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g.. gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time. improving the efficiency and accessibility of ML).
11. Safeguards
21

---

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (c.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: This work does not involve the release of high-risk models or datasets that require specific safety measures.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All datasets, models, and code used from prior work are properly cited and gain consent.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, wwppppaitthhaaeeeo com/ dataaasssess has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question:
Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The code will be released upon acceptance due double-blind review con- straints.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions structured templates. This includes details about training, license, limitations, etc.
22

---

The paper should discuss whether and how consent obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects. does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
This work does not involve human subjects or crowdsourced data.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This work does not involve human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recogmize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it an important, original, or non-standard component of the core methods in this research? Note that the LLM used only for writing, editing, formatting purposes and does not impact the methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non non-standard components.
Please refer to our LLM policy (https:/nneurips. ctt.rrrpppppeeccooeeeeeesess/e/e 2025/ LLMM for what should should be described.
23


Reviews:
[{"review_id": "7G71OurQyP", "reviewer": "Reviewer_GXzf", "summary": "The paper introduces FALCON (Fine-grained Activation Manipulation by Contrastive Orthogonal Unalignment), a novel framework for selective knowledge unlearning in large language models (LLMs). FALCON addresses key challenges in machine unlearning, such as precise knowledge separation, preservation of model utility, and resistance to knowledge recovery attempts.", "questions": "1. Please refer to the Weakness section.\n2. What are the advantages and disadvantages of using task vectors [1] or negative preference optimization [2] to solve toxicity/unlearning problems compared to the methods mentioned in the paper.\n\n[1] Editing Models with Task Arithmetic. ICLR 2023.\n\n[2] Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning. Arxiv 2024.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n1. The paper presents a well-designed framework (FALCON) with a clear theoretical foundation (mutual information, contrastive learning, gradient projection). The use of SVD for principal directions and gradient orthogonalization is technically sound.\n2. FALCON could be used in cross-domain generalizability assessment beyond harmful content, which is valuable.\n\nWeaknesses\n1. Experiments are conducted on relatively small LLMs (≤7B parameters). The scalability to larger models remains unverified.\n2. The granularity of parameter selection is not very clear. The gradient should be calculated on each parameter matrix or neuron, but it seems that parameter selection is expressed as the granularity of layers.\n3. The use of two basic models and baseline systems in Section 5.1 and 5.2 is confusing, and there should at least be some common baselines for comparison.", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's feedback and happy to address each concern one by one below.\n\n$\\textbf{For Weakness 1 and 3:}$\n\nThanks for pointing out your concern about experiments on models and baseline methods. Our experimental design is carefully aligned with established practices in the machine unlearning literature and represents current state-of-the-art evaluation standards. \n\nThe model selection strictly follows best practices established by leading works in this field. For example, RMU [1], the current state-of-the-art representation-guided adaptation method, conducted their core experiments on Zephyr-7B. We not only replicated their experimental setup but comprehensively evaluated all baseline methods they employed, including LLMU [2], SCRUB [3], and SSD [4], ensuring fair and meaningful comparisons across WMDP benchmark with RMU.  In WMDP benchmark experiments, we employed both Yi-series and Mistral-series models (Appendix E.1) to demonstrate our method's generalization for multiple-architecture evaluation, which are also the primary architectures evaluated in the RMU and WMDP paper.\n\nFor entity unlearning and copyrighted content unlearning tasks, we adopted the latest OpenUnlearning framework [5], which represents the community's recognition on unified evaluation standards. It is also worth noting that most existing unlearning methods in the literature, such as LLMU [2] and SimNPO [6], are also evaluated on less than 7B-scale models due to the hardware and resource constraints of LLM unlearning. The models we employed (e.g., LLaMA-3.2, LLaMA-2-7B) are the standard evaluation models recommended by this OpenUnlearning framework and community, ensuring reproducibility and generalization of our method. We also provided extensive evaluations with widely recognized baselines like NPO [7], covering entity unlearning and copyrighted content unlearning tasks comprehensively. FALCON's core innovations are theoretically model-agnostic. Our approach targets the intrinsic representational structures of LLMs rather than specific architectural details, providing a foundation for scaling to larger models. Given hardware limitations, maintaining experimental consistency with SOTA methods and utilizing community-recognized unified evaluation frameworks provides sufficient evidence for FALCON's efficacy.\n\n$\\textbf{For Weakness 2:}$\n\nWe clarify that our parameter selection is guided by mutual information rather than gradients. We compute $I(F_l; R_l)$ where $F_l$ and $R_l$ represent the activations (intermediate hidden representations) at layer $l$ for forget and retain datasets, respectively. Since activations are naturally layer-level outputs in transformer architectures, layer-level granularity for MI-based selection is theoretically appropriate. The gradient conflicts shown in our experiments serve as intuitive empirical evidence that layers with lower MI indeed facilitate better knowledge disentanglement. When MI is minimal between forget and retain representations, the gradient conflicts between forgetting and retention objectives are also reduced, enabling more effective decoupling of knowledge to be removed versus preserved. Subsequently, we perform parameter updates within the selected optimal layer l indeed targeting specific parameter matrices, normally focusing on MLP matrices like other leading works [1]. This choice is motivated by the fact that MLP matrices, particularly the output projection weights, control the final knowledge output of each layer and directly influence information propagation to subsequent layers, while being relatively safe to modify without overly disrupting the model's fundamental representational capabilities.\n\n$\\textbf{For Question 2:}$\n\nTask vectors [8] offer computational efficiency through arithmetic operations on model weights, enabling lightweight unlearning without requiring extensive retraining. Their modular approach allows for easy composition and reversal of edits. However, task vectors suffer from significant limitations: they oversimplify knowledge structures through linear assumptions that fail to capture complex knowledge entanglement in LLMs, lack fine-grained control over specific knowledge removal, and may inadequately address deeply embedded representations that require non-linear transformations.\n\nNegative Preference Optimization [7] reformulates unlearning as preference learning, leveraging established preference optimization techniques. NPO can handle diverse unlearning scenarios through flexible preference specification. Nevertheless, NPO is prone to catastrophic collapse when forget and retain distributions exhibit high similarity, lacks fine-grained knowledge localization control, and may struggle with precise separation of entangled knowledge representations.\n\nTo better mitigate your concern between our method and existing mainstream baselines, we also provide a more comprehensive discussion that will be integrated into the Related Work section and Appendix in the camera-ready version to address your concerns as follows:\n\nExisting LLM unlearning approaches operate through distinct methodological paradigms with inherent trade-offs. Methods like Task Vectors [8] use arithmetic operations on model weights for computational efficiency but oversimplify knowledge structure through linear assumptions that fail to capture complex knowledge entanglement. Gradient-based approaches including LLMU [2] and GradAscent [11] apply gradient ascent to forgotten knowledge but frequently induce catastrophic forgetting and optimization instability. Preference optimization methods such as NPO and its variants reformulate unlearning as preference learning, yet are prone to catastrophic collapse when distributions exhibit high similarity and lack fine-grained knowledge localization control [6,7]. Representation manipulation methods like RMU [1] modify intermediate activations using MSE loss and random unit vectors for steering, offering more targeted intervention than gradient methods but relying on empirical layer selection and lacking targeted separation mechanisms.\n\nSimilar to RMU, FALCON belongs to the representation-based paradigm, optimizing only a subset of intermediate layers and parameters for effective LLM unlearning. However, FALCON advances through three key innovations: information-theoretic parameter guidance using mutual information to identify optimal intervention layers, targeted representation disentanglement via contrastive learning with Principal Offset Vectors for precise knowledge separation, and gradient orthogonal projection for conflict resolution between objectives. Compared to RMU's MSE-based random steering with heuristic selection, our contrastive mechanism provides more principled and targeted representation manipulation, achieving superior unlearning effectiveness while preserving model utility.\n\n$\\textbf{Reference List for Reviewer 1 and Reviewer 2:}$\n\n[1] Li et al., \"The WMDP benchmark: Measuring and reducing malicious use\nwith unlearning\". ICML 2024\n\n[2] Yao et al., \"Large Language Model Unlearning\". Neurips 2024\n\n[3] Kurmanji et al., \"Towards unbounded machine unlearning\". Neurips 2024\n\n[4] Foster et al., \"Fast machine unlearning without retraining through selective synaptic dampening\", AAAI 2024\n\n[5] Dorna et al., \"OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metrics\".\n\n[6] Fan et al., \"Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning\", Neurips SafeGenAi 2024\n\n[7] Zhang et al., \"Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning\", COLM 2024\n\n[8] llharco et al., \"Editing Models with Task Arithmetic\". ICLR 2023.\n\n[9] Maini et al., \"TOFU: A task of fictitious unlearning for LLMs\". COLM 2024\n\n[10] Łucki et al., \"An Adversarial Perspective on Machine Unlearning for AI Safety\". TMLR 2024.\n\n[11] Jang et al., \"Knowledge Unlearning for Mitigating Privacy Risks in Language Models\". ACL 2023."}, {"review_id": "TapDJbdd7i", "reviewer": "Reviewer_aqr3", "summary": "Existing training-time unlearning methods rely on coarse-grained loss combinations. This work proposes a representation-guided framework for targeted knowledge removal. It utilizes information-theoretic guidance to identify layers with minimal entanglement and employs contrastive mechanisms to enhance the separation of representations.", "questions": "1. **Robustness to Knowledge Recovery:** The reason why the proposed method is robust against knowledge recovery techniques needs more discussion. It would be helpful to include a more detailed and intuitive explanation or theoretical analysis. Alternatively, comparing with other strong unlearning baselines under these attack methods could better demonstrate the effectiveness of the proposed defense. \n2. **Evaluation Metrics in Table 2:** In Table 2, FALCON seems to achieve the second-lowest forget metric score. Have you defined a new criterion for determining which method performs best?\n3. The first issues for current research is that “existing approaches typically rely on empirical methods like grid search to identify intervention parameters”, could you give several related work?\n4. Minor part: In Table 3, for forget01, NPO seems to achieve the best MU.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "# Strengths\n\n- Introduces a new fine-grained framework for LLM unlearning.\n- The use of mutual information-based guidance and gradient orthogonal projection enhances the interpretability and robustness of the unlearning process.\n- Provides extensive experiments on three commonly used benchmarks and multiple LLMs.\n\n# Weaknesses\n\n- **Validation of Layer Selection Metric on TOFU:** To validate the metric used for identifying layers with minimal entanglement, visualizations and analysis of gradient conflicts on the TOFU dataset should be provided. Since the forget and retain sets in this dataset are relatively similar, demonstrating that mutual information can disentangle knowledge related to similar QA pairs would strengthen the argument.\n- **Baselines:** For the TOFU dataset, since retained data is used for FALCON, the retained version of DPO should be included as a baseline, not just IdkDPO. Additionally, it is unclear whether the version of NPO used is the original or one of the extended variants (NPO-RT or NPO-KL).\n- **Limited Experiments:**\n    - The MUSE benchmark appears to only report results on the “book” subset. Results on the “news” subset should also be included.\n    - The TOFU dataset experiments are limited to only one model (LLaMA-3.2-1B-Instruct), whereas the original paper provides results for finetuned and retained models based on LLaMA-2-7B. The same problem for WMDP. These limitations make it unclear whether the proposed method can generalize to other model architectures or larger models.", "was_revised": true, "final_justification": "The concerns have been addressed. I would like give 4 to this paper.", "rebuttal": "Thanks for your comments and we fully understand your concerns. To provide clear responses to your feedback, we use W to denote weakness and Q to denote question to address each concern below. $\\textbf{The reference list please follow Reviewer 1 due to text limit.}$\n\n$\\textbf{W1: }$\n\nTo mitigate your concern, we are pleased to provide additional experimental analysis on TOFU benchmark [9] to demonstrate the efficiency and practicality of our MI-guided method. We evaluated the computational overhead of MI estimation across different sample sizes while maintaining PCA dimensionality reduction at 95% variance retention to ensure computational feasibility. The results below show computational efficiency across varying sample proportions ranging from 10% to the full dataset and exhibit consistent results.\n\nMI-guided Method Cost Analysis (TOFU + Llama-3.2):\n\nSample Size | Time(s) | Optimal Parameter Layer\n------------|---------|------------------------\n10%         | 67      | 3\n30%         | 79      | 3\n50%         | 110     | 3\n70%         | 165     | 3\n100%        | 260     | 3\n\nFurthermore, we provide the normalized MI results on the full sample size below (lower is better) to give an intuitive understanding of our method. Due to rebuttal requirements that prevent figure inclusion, we present these results in tabular form here and will provide visualizations in the camera-ready version.\n\nLayer         | 0    | 1    | 2    | 3    | 4    | 5    | 6    | 7    | 8    | 9    | 10   | 11   | 12   | 13   | 14   | 15\n--------------|------|------|------|------|------|------|------|------|------|------|------|------|------|------|------|------\nNormalized MI | 0.78 | 0.43 | 0.19 | 0.00 | 0.18 | 0.07 | 0.05 | 0.14 | 0.23 | 0.41 | 0.51 | 0.65 | 0.65 | 0.68 | 0.80 | 1.00\n\nThe aforementioned results demonstrate: (1) MI computation scales reasonably with sample size, maintaining manageable computational overhead; (2) layer selection remains consistent across different sample sizes, indicating stability of the MI-guided approach; (3) the method successfully identifies optimal intervention layers, with Layer 3 indeed being the parameter selection in accordance with our best performance in our experimental results on TOFU, even forget and retain sets have high similarity. \n\nThese results confirm that MI-guided parameter selection is both computationally feasible and scalable, providing a principled foundation for efficient unlearning. Combined with our original computational efficiency analysis in Appendix E.3, both the MI-guided parameter selection and our method's inherent efficiency demonstrate the scalability of our approach.\n\n$\\textbf{W2:}$\n\nRegarding baselines, our experimental setup strictly follows two widely community-recognized frameworks: the TOFU benchmark [9] and the OpenUnlearning framework [5].\n\nDPO Clarification: Our implementation includes IdkDPO (often abbreviated as DPO in some papers), which utilizes standard DPO with specialized preference pair construction for unlearning context. IdkDPO sets preferred responses to \"I don't know\" for forget queries while treating original answers as rejected responses. This expression was initially introduced in TOFU paper and adopted as a core baseline in OpenUnlearning.\n\nNPO Clarification: Our experiments utilized the NPO implementation from the OpenUnlearning framework as specified by Zhang et al. [7], which employs the negative feedback term for forget data combined with a standard retain loss term.\n\nBoth frameworks represent community consensus on standardized evaluation protocols. TOFU established IdkDPO as a basic baseline for entity unlearning, while OpenUnlearning provides unified implementations of core methods. Our adherence ensures reproducibility and fair comparison with established benchmarks in the literature.\n\n$\\textbf{W3:}$\n\nTo clarify, the results presented in our paper is MUSE News. However, for results across different MUSE subsets, we are happy to provide additional results on the MUSE Books below to address this concern and it will be integrated into the final version for a comprehensive evaluation.\n\nMethod     |F_knowmem|F_verbmem|R_knowmem\n-----------|--------------|--------------|---------------\nFinetuned  |0.47          |1.0           |0.69           \nRetain     |0.3           |0.14          |0.69           \nGradAscent |0             |0             |0              \nGradDiff   |0.18          |0.16          |0.3            \nNPO        |0.32          |0.84          |0.55           \nSimNPO     |0.32          |0.84          |0.54           \nRMU        |0.29          |0.79          |0.48           \nFALCON     |0.06          |0.05          |0.68\n\nFor model selection on TOFU, we follow established unified benchmark OpenUnlearning while demonstrating generalization across different architectures. For MUSE we still used LLaMA-2-7B series following standard configurations, while for TOFU we deliberately employed LLaMA-3.2 series as provided by the OpenUnlearning to demonstrate the reviewer's concerned generalizability across diverse model architectures from established to latest versions. For WMDP [1], we conducted comprehensive experiments on Zephyr-7B, the primary model used in the WMDP paper, where we reproduced all baselines for fair comparison. While hardware limitations prevent us from testing on the larger models shown in their supplementary experiments, we selected same architectures including Yi and Mistral series used in WMDP papers to demonstrate our method's generalizability across different model families. This cross-architectural validation validates FALCON's broad applicability, and it is worth noting that most established unlearning works [2, 6] are also primarily evaluated on models of similar scale (7B and below), reflecting the community's standard practice for unlearning evaluation.\n\n$\\textbf{Q1:}$\n\nFALCON's robustness against knowledge recovery stems from creating fine-grained representational changes rather than random directions. Our contrastive mechanism with Principal Offset Vectors systematically steers activations away from the dominant subspaces associated with unwanted knowledge, while orthogonal gradient projection ensures updates move away from retention-sensitive directions. This approach intuitively creates targeted geometric separation in the model's representational space, making forgotten knowledge structurally inaccessible rather than relying on random perturbations. Our comparative evaluation against Enhanced GCG attacks demonstrates that while existing baselines like RMU [1] show vulnerability to increasing attack iterations, FALCON maintains consistent resistance across attack intensities [10]. Furthermore, our logit lens analysis in Section 5.3 also empirically reveals that unlearned knowledge remains inaccessible across different architectural components, validating that our method achieves persistent internal representational changes. Moreover, we  provide additional intuitive comparative results with other strong baselines below to address your concerns. Under identical jailbreaking attacks, RMU's knowledge can be substantially recovered while FALCON maintains resistance.\n\nMethod        |Original|Unlearned|Recovery\n--------------|--------|---------|---------------\nRMU (Bio)     |65.4    |50.8     |58.5 ± 3.2\nFALCON (Bio)  |65.4    |27.7     |28.1 ± 0.5\nRMU (Cyber)   |42.6    |33.5     |41.8 ± 2.1\nFALCON (Cyber)|42.6    |25.3     |25.5 ± 0.8\n\n$\\textbf{Q2 and 4:}$\n\nWe understand your confusion regarding evaluation criteria and would like to clarify the unlearning evaluation. Unlearning evaluation requires holistic assessment $\\textbf{beyond forget metrics alone, as effective unlearning must balance knowledge removal with utility preservation.}$ While FALCON achieves very low forget scores (0.02 and 0.03), our key strength lies in superior retain performance (0.54). This balance is crucial because methods like GradAscent achieve perfect forget scores (0.00) but completely fail to preserve useful knowledge (retain score: 0.00), rendering them impractical for deployment. Comprehensive metric explanations are in Appendix D.3.\n\nRegarding NPO's slightly higher Model Utility (0.56 vs 0.55) in Forget01, FALCON demonstrates superior Forget Quality (0.99 vs 0.92), indicating more effective knowledge removal. As forget dataset size increases to Forget05 and Forget10, FALCON consistently outperforms baselines with optimal balance, achieving the highest MU (0.60) in Forget10 while maintaining effective forgetting. These results demonstrate FALCON's scalability across varying dataset complexities and architectures. We will update this clarification in the camera-ready version.\n\n$\\textbf{Q3:}$\n\nWe will include the citations and discussion in the camera-ready version based on your valuable suggestions. Early methods like GradAscent [11] typically perform full-parameter optimization, which can be computationally expensive and lead to catastrophic forgetting. Selective parameter modification approaches, including RMU [1] and task vector methods [8], have demonstrated that targeted parameter updates can effectively address the limitations of full-parameter optimization and have contributed this insight to subsequent unlearning methods. However, these methods still rely on empirical techniques such as extensive grid search across model layers to obtain their observations regarding optimal parameter selection [1]. Despite demonstrating the effectiveness of selective parameter modification, the fundamental limitation remains that these methods lack principled theoretical guidance for parameter selection, as model architectures, knowledge entanglement patterns, and task-specific conditions vary significantly across different scenarios. This highlights the necessity for our MI-guided approach that provides interpretable, theory-driven parameter identification."}, {"review_id": "iuHrGv8gOM", "reviewer": "Reviewer_RCfm", "summary": "This paper focuses on large language model (LLM) unlearning with activation manipulation. Specifically, this work aims to improve the existing unlearning methods, regarding the limitations in relying on coarse-grained loss combinations. To this end, this work proposes a Fine-grained activation manipulation by contrastive orthogonal unalignment, termed FALCON, that incorporating information-theoretic guidance for efficient parameter selection, and project conflict gradients onto orthogonal subspaces to solve the onflicts between forgetting and retention objectives. Various experiments are conducted to demonstrate the performance effectiveness.", "questions": "1. Could the authors further elaborate what is “fine-grained” compared with “coarse-grained” in the context of LLM unlearning considered in this work?\n2. Could the authors point out specific results about the “theoretical insights” claimed in line 59?\n3. Could the authors abstract several unique insights on ”fine-grained” unlearning in the context of LLM?\n4. Could the authors also discuss the computation-time cost of the existing framework compared with other methods?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. This work consider the information-theoretic guidance for effectively measure the dependence on retain and forget data, and using SVD to identify principle directions in activation spaces are new (in the reviewer views) in unlearning.\n2. The overall framework of FALCON is sounded and easy-to-understand, with intuitive illustrations.\n3. The proposed method are generally demonstrated to be effective on various unlearning tasks and different LLMs.\n\nWeaknesses:\nOverall I think the current presentation is good for a new algorithmic proposal, while it could be better if the authors revise some part of clarification to make it clearer.\n1. It is not very clear about the unlearning definition/target of the existing scenario, although the authors provide eq.1 as an general formulation which pursue two objectives like forgetting and retaining. The formal definition of unlearning in the context of LLM should be further elaborated, either on erasing specific knowledge within the LLM or just care about the specific output behavior.\n2. It is a little bit hard to understand the fine-grained in the context of llm unlearning compared with the coarse-grained loss objective design in previous literature. To some extent, those mutual information-based reguarlization, SVD, and gradient conflicts analysis are also existing in previous literature of representation learning or exactly llm unlearning. It would be better if the authors highlight some new insights on the core intuition behind those technical combination.\n3. Although the experimental results are promising on various unlearning tasks, could the authors also consider the efficiency of the existing methods compared with other baselines?\n4. It is not easy to understand whether the proposed framework indeed approximate retrained model or realize the two pursuit in a special way. It could be better if the authors can further discuss this point.", "was_revised": true, "final_justification": "Most of my concerns have been addressed, while I'm not fully convinced by the clarification about \"theoretical insights\" as there is no formal theoretical results, and most claims are supported with empirical evidence, while it doesn't affect my appreication on the work merits. In addition, it would also be better if the authors could discuss the practicality of the problem setting considered in this unlearning works and the real-world scenarios in the final version.", "rebuttal": "We appreciate the reviewer's feedback and support. \n\nFor $\\textbf{Question 1}$, we provide a detailed explanation below for your convenience:\n\nCoarse-grained unlearning approaches refer to existing methods that rely on simplistic loss combinations and empirical parameter selection strategies. These methods typically: (1) use grid search or empirical method to identify intervention parameters without principled guidance, (2) apply uniform modifications across broad parameter spaces without considering the specific knowledge distribution, and (3) employ random representation dispersion with uncontrolled gradient dynamics that struggle to precisely separate knowledge to be forgotten from knowledge to be retained. Such approaches often result in significant interference between forgetting and retention objectives, leading to degraded model utility or incomplete knowledge removal.\n\nIn contrast, fine-grained unlearning as implemented in FALCON achieves more precise knowledge manipulation through: (1) Information-theoretic guidance that provides principled parameter selection by identifying layers with minimal knowledge entanglement using mutual information, (2) Targeted representation modification through our contrastive mechanism and Principal Offset Vectors that specifically steer activations away from unwanted knowledge directions identified via SVD, and (3) Regulated gradient dynamics via orthogonal projection that resolves conflicts between forgetting and retention objectives, minimizing damage to model utility. This fine-grained approach enables surgical knowledge removal with controlled interference, achieving superior balance between unlearning effectiveness and utility preservation compared to existing coarse-grained methods.\n\nWe will provide a more detailed clarification and explanation of this distinction in the camera-ready version to ensure clarity for readers.\n\nFor $\\textbf{Question 2}:$\nRegarding the clarification on the specific theoretical insights claimed in line 59. Our approach provides the first principled, theoretically grounded method for parameter selection in LLM unlearning, moving beyond empirical grid search to a systematic framework based on knowledge entanglement quantification. The intuitive insights based on information-theoretic metrices are demonstrated through several key findings in our work:\n\n1. Knowledge Distribution Patterns Across Model Architectures (Section 5.1.1, Figure 2): Our mutual information analysis reveals consistent patterns across different LLM architectures, showing that earlier layers exhibit lower MI values and more domain-specific representations, while deeper layers show higher entanglement. This provides intuitive understanding of how knowledge is distributed and organized within LLM architectures.\n\n2. Gradient Conflict Correlation with Knowledge Entanglement (Figure 3): We demonstrate a strong relationship between mutual information and optimization conflicts - layers with low MI values exhibit significantly reduced gradient conflicts (cosine similarities near zero), while high-MI layers show pronounced, fluctuating conflicts. This validates MI as a reliable theoretical indicator for identifying optimal intervention parameters.\n\nThese insights advance our intuitive understanding of knowledge representation and distribution in LLMs, providing a theoretical methodology for more principled unlearning approaches.\n\nFor $\\textbf{Question 3}:$\nWe identify several critical insights that necessitate fine-grained approaches for LLM unlearning:\n\n1. $\\textbf{Knowledge Entanglement Complexity:}$ LLMs exhibit intricate knowledge entanglement patterns where forget and retain information are deeply intertwined across multiple layers and features. Unlike traditional machine learning models, LLMs store knowledge in distributed representations that cannot be effectively addressed through coarse-grained loss combinations or global parameter modifications.\n\n2. $\\textbf{Precision Requirements:}$ LLM unlearning demands precisely removing specific knowledge while preserving semantically related but distinct information. For example, forgetting a particular entity's biographical details while maintaining general knowledge about similar entities requires fine-grained manipulation of specific representational directions rather than broad parameter updates. However, the massive parameter space and complex transformer architectures of LLMs make naive approaches like full-parameter gradient ascent computationally prohibitive and prone to catastrophic forgetting. Fine-grained methods enable targeted interventions that maintain model utility while achieving effective knowledge removal.\n\n3. $\\textbf{Optimization Conflicts:}$ LLM unlearning inherently involves conflicting objectives between forgetting and retention that manifest at the gradient level. Fine-grained approaches like our gradient orthogonal projection can resolve these conflicts in specific parameter subspaces, while coarse-grained methods struggle with such optimization dynamics.\n\nThese insights underscore why LLM unlearning requires sophisticated, fine-grained approaches that can navigate the complex knowledge structures and optimization landscapes inherent to large-scale language models.\n\nFor $\\textbf{Question 4}$:\nWe have provided $\\textbf{computational cost comparison between our method and other baselines in Appendix E.3 (line 821)}$, which demonstrates FALCON's competitive efficiency compared to other methods. Our results show that FALCON achieves 28.69 samples per second and completes 0.72 optimization steps per second, maintaining competitive performance with other unlearning methods while providing superior effectiveness. Additionally, to address your concern and validate the efficiency of our MI-guided approach, we have conducted specific analysis on the computational cost of MI-based parameter selection using the TOFU dataset. As shown in our MI cost analysis table, the parameter selection process scales reasonably with sample size, requiring only 67 seconds for 1\\% of data up to 260 seconds for the full dataset. Importantly, this one-time parameter selection cost represents a small fraction of the total unlearning time, and the selected layers remain consistent across different sample sizes, demonstrating both the efficiency and stability of our MI-guided approach. Detailed discussion are shown at Review2, Question1 and new results will be integrated into camera ready version.\n\nMI Computational Cost Analysis (TOFU + Llama-3.2):\n\nSample Size | Time(s)\n------------|--------\n10%         | 67\n30%         | 79\n50%         | 110\n70%         | 165\n100%        | 260"}, {"review_id": "LWL5HFvf6C", "reviewer": "Reviewer_ASj8", "summary": "This paper introduces FALCON, a novel framework for accurate and robust knowledge de-learning in LLMs. Unlike previous works that rely on retraining, parameter overwriting, or on-the-fly tuning, FALCON is a representation-guided framework that combines contrastive mechanisms and gradient projection to achieve fine-grained representation de-learning in LLMs.\nExperiments on multiple de-learning tasks on multiple open source LLMs show its good performance.", "questions": "See above.", "limitations": "yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1.\tThis paper focus on knowledge unlearning, which is a high-impact problem, particularly in light of privacy and safety regulations. FALCON leverages mutual information guidance and contrastive orthogonal unalignment, it enables precise and efficient unlearning.\n\n2.\tThis method explicitly tackles the common trade-off between forgetting and preserving useful knowledge, which is often ignored or vaguely addressed in prior work.\n3.\tThe paper demonstrates FALCON’s effectiveness in various settings, including robustness to jailbreak-style recovery attacks, and generalization across tasks and model sizes.\n\nWeaknesses:\n\n1.\tWhile MI-guided layer selection is compelling, the paper does not quantify its cost in practice, especially for large models or datasets. It’s unclear whether this process is scalable.\n2.\tSince FALCON relies on internal activation and gradient access, it is not applicable to closed-source models like GPT-4 or Claude. It is good to include discussion or experiments on how FALCON could be adapted or approximated for black-box models or low-resource settings.\n3.\tThe method hinges on identifying principal directions of harmful representations, but it is not evaluated how stable this direction is under perturbations or data sampling variance.", "was_revised": true, "final_justification": "I have responsed to author. This is my final Justification", "rebuttal": "We thank you for your feedback and support of our work. We are very happy to provide additional experiments and discussion to address your remaining concerns.\n\n$\\textbf{For Weakness 1:}$\nFor the computational cost and scalability of MI-guided parameter selection, we additionally provide a detailed cost analysis below to demonstrate the practical feasibility of our design.\n\nMI Computational Cost Analysis (TOFU + Llama-3.2):\n\nSample Size | Time(s)\n------------|--------\n10%         | 67\n30%         | 79\n50%         | 110\n70%         | 165\n100%        | 260\n\nThe results show reasonable results of only 4.3 minutes for the complete TOFU dataset [1]. This represents manageable overhead as a one-time preprocessing step. Importantly, our MI-guided approach eliminates the need for costly empirical grid search across all model layers that traditional methods require, making our approach both theoretically principled and computationally practical for larger models and datasets. Additional discussion can be found in Review2, Q1. Furthermore, our Appendix E.3 provides computational cost comparison of our method, which demonstrates FALCON's competitive efficiency compared to baseline methods, indicating the scalability of our approach.\n\n$\\textbf{For Weakness 2:}$\nWe understand your interest in exploring black-box LLM unlearning methodology. Actually, FALCON is a white-box algorithm, which requires access to internal activations and also gradients. However, FALCON's core design principles could be possibly adapted for black-box scenarios through contrastive prompt engineering that mirrors our contrastive orthogonal unalignment mechanism. Building upon recent advances in in-context unlearning [2,3], this adaptation could leverage surrogate models, smaller accessible models trained to approximate the behavior of target closed-source systems [4], to identify principal directions of unwanted knowledge representations through our SVD-based analysis, then systematically design prompts that incorporate counter-examples and directional guidance that implement our Principal Offset Vectors (POVs) concept at the prompt level. The information-theoretic principles underlying our mutual information calculations could provide crucial guidance for optimizing such unlearning prompts by quantifying the entanglement between different knowledge domains within the prompt structure itself, enabling systematic optimization of prompt templates that maximize separation between forget and retain domains within API-only constraints.\n\n$\\textbf{For Weakness 3:}$\nWe understand your concern about principal direction design. To address this concern, we provide comparative experiments between random perturbation-based baselines and our method under identical knowledge recovery attempts.\n\nThis observation actually highlights a key advantage of our approach. While RMU [5] relies on random perturbations which are inherently unstable and susceptible to variance, our method uses Principal Component Analysis to identify the dominant subspaces of harmful representations through SVD decomposition. This principled approach provides significantly more stable directions compared to random perturbations. Our empirical results under identical jailbreaking attempts demonstrate this stability advantage through comparative evaluation showing that our method maintains consistent performance with minimal variance, while RMU's knowledge can be substantially recovered under identical conditions. Additionally, our logit lens results (Section 5.3, Figure 4) also demonstrate that our method is more stable compared to the logit lens visualization results of other baselines in [4].\n\nMethod        |Original|Unlearned|Recovery\n--------------|--------|---------|---------------\nRMU (Bio)     |65.4    |50.8     |58.5 ± 3.2\nFALCON (Bio)  |65.4    |27.7     |28.1 ± 0.5\nRMU (Cyber)   |42.6    |33.5     |41.8 ± 2.1\nFALCON (Cyber)|42.6    |25.3     |25.5 ± 0.8\n\nThese results demonstrate that our SVD-based principal directions are inherently more stable than random vector approaches. This stability stems from the mathematical properties of principal components, which capture the most significant variance directions in the data rather than arbitrary perturbations.\n\n$\\textbf{Reference List:}$\n\n[1] Maini et al., \"TOFU: A task of fictitious unlearning for LLMs\". COLM 2024\n\n[2] Pawelczyk et al., 2024 - \"In-context unlearning: Language models as few-shot unlearners\". ICML2024\n\n[3] Zheng et al., 2023 - \"Can we edit factual knowledge by in-context learning?\". EMNLP2023\n\n[4] Pan et al., 2024. \"Differential privacy in deep learning: A literature survey\". Neurocomputing\n\n[5] Li et al., \"The WMDP benchmark: Measuring and reducing malicious use with unlearning\". ICML 2024"}, {"review_id": "3RTENa4TWH", "reviewer": "Reviewer_amhk", "summary": "This paper introduces Falcon, a novel machine unlearning method designed to remove specific information from large language models (LLMs) while preserving the utility of their representations.\nFalcon consists of two key components:\na) Layer selection — identifying layers where the target information to retain and the information to forget are least entangled, based on mutual information analysis;\nb) Representation disentanglement — applying contrastive orthogonal unalignment followed by orthogonal gradient conflict resolution to guide the model in effectively unlearning undesired information while preserving useful features.", "questions": "See weaknesses", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\nThe paper is generally well-written (aside from a few sections noted below) and easy to follow.\n\nThe authors present a comprehensive experimental setup, effectively demonstrating the potential of their method.\n\nThe method is shown to be highly resilient against knowledge recovery attempts, which is an important and timely contribution.\n\nWeaknesses and Suggestions:\n\nThe paper is not always fully self-contained. For example, the baselines are only briefly discussed in the Appendix, while the main text repeats the MI-based idea in Sections 3.2 and 4.1. It would be more helpful to consolidate the explanation of the proposed method and dedicate more space in the main paper to clearly describe the baselines and how they differ from the approach proposed here.\n\nSeveral key ideas in the paper—such as removing unwanted information using SVD or PCA, or preserving desired information while discarding irrelevant components (as in Equation 9)—have been explored in prior work that is not cited. Specifically, related approaches appear in the following references:\n\n[1] https://arxiv.org/pdf/2203.07893\n\n[2] https://proceedings.mlr.press/v206/kleindessner23a/kleindessner23a.pdf\n\n[3] https://proceedings.neurips.cc/paper_files/paper/2024/file/684c59d614fe6ae74a3be8c3ef07e061-Paper-Conference.pdf\n\nI’m not suggesting that these works necessarily undermine the novelty of your method, but a more thorough discussion of the similarities and differences would help clarify your contribution and position it more clearly within the existing literature.", "was_revised": true, "final_justification": "This is a solid paper with reasonable contributions. The authors acknowledged the literature gap I pointed out, so I have maintained my fairly positive score.", "rebuttal": "We sincerely appreciate your feedback and support for our work. Regarding your remaining concerns and suggestions, we are pleased to adopt them.\n\n$\\textbf{For Weakness and Suggestions 1: }$\n\nRegarding the repetition between Sections 3.2 and 4.1, we clarify our organizational rationale: Section 3.2 establishes the formal problem formulation with mutual information as the theoretical foundation for quantifying knowledge entanglement, while Section 4.1 provides concrete implementation details including KDE-based estimation and multi-domain extension. Though intended to separate theoretical formulation from practical implementation, we acknowledge this may created redundancy and will consolidate similar equations and descriptions in the camera-ready version.\n\nRegarding baseline comparisons, we provide a detailed discussion below that will be integrated into the Related Work section and Appendix in the final version.\n\nExisting LLM unlearning approaches operate through distinct methodological paradigms with inherent trade-offs. Parameter-space methods like Task Vector [1] use arithmetic operations on model weights for computational efficiency but oversimplify knowledge structure through linear assumptions that fail to capture complex knowledge entanglement. Gradient-based approaches including LLMU [2] and GradAscent [3] apply gradient ascent to forgotten knowledge but frequently induce catastrophic forgetting and optimization instability. Preference optimization methods such as NPO and its variants reformulate unlearning as preference learning, yet are prone to catastrophic collapse when distributions exhibit high similarity and lack fine-grained knowledge localization control [4,5] . Representation manipulation methods like RMU [6] modify intermediate activations using MSE loss and random unit vectors for steering, offering more targeted intervention than gradient methods but relying on empirical layer selection and lack targeted separation. Similar to RMU, FALCON belongs to the representation-based paradigm, optimizing only a subset of intermediate layers and parameters for effective LLM unlearning. However, FALCON advances through three innovations: information-theoretic parameter guidance using mutual information to identify optimal intervention layers, targeted representation disentanglement via contrastive learning with Principal Offset Vectors for precise knowledge separation, and gradient orthogonal projection for conflict resolution between objectives. Compared to RMU's MSE-based random steering with heuristic selection, our contrastive mechanism provides more principled and targeted representation manipulation, achieving superior unlearning effectiveness while preserving model utility.\n\n$\\textbf{For Weakness and Suggestions 2: }$\n\nThese suggested works indeed present important insights that share fundamental concepts with our approach. SEA [7] demonstrates how SVD on cross-covariance matrices can project LLM activations toward directions with maximal covariance with positive demonstrations while minimizing covariance with negative ones for inference-time behavior modification. Efficient Fair PCA [8] provides an elegant solution for fair dimensionality reduction by ensuring projected data's group-conditional means coincide while maintaining computational efficiency. SAL [9] introduces a clever approach applying SVD to cross-covariance matrices between representations and protected attributes, pruning high-covariance directions to remove sensitive information with superior computational efficiency. While all these works share the fundamental insight of using SVD and PCA to manipulate information in neural representations efficiently, each addresses distinct challenges with novel technical contributions.  We will certainly include proper citations to these works in our methodology section of the camera-ready version to acknowledge how similar techniques have been developed and applied across the literature.\n\n$\\textbf{Reference List:}$\n\n[1] llharco et al., \"Editing Models with Task Arithmetic\". ICLR 2023.\n\n[2] Yao et al., \"Large Language Model Unlearning\". Neurips 2024\n\n[3] Jang et al., \"Knowledge Unlearning for Mitigating Privacy Risks in Language Models\". ACL 2023.\n\n[4] Zhang et al., \"Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning\", COLM 2024\n\n[5] Fan et al., \"Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning\", Neurips SafeGenAi 2024\n\n[6] Li et al., \"The WMDP benchmark: Measuring and reducing malicious use with unlearning\". ICML 2024\n\n[7] Qiu et al., \"Spectral Editing of Activations for Large Language Model Alignment\". NeurIPS 2024\n\n[8] Kleindessner et al., \"Efficient fair PCA for fair representation learning\". AISTATS 2023\n\n[9] Shao et al., \"Gold Doesn’t Always Glitter: Spectral Removal of Linear and Nonlinear Guarded Attribute Information\". EACL 2023"}]

Author rebuttal:

Meta-review: This paper focuses on LLM unlearning through activation manipulation. It aims to improve existing unlearning methods by addressing the limitations of relying on coarse-grained loss combinations, and instead proposes a representation-guided framework for targeted knowledge removal. The approach leverages information-theoretic guidance to identify layers with minimal entanglement and employs contrastive mechanisms to enhance representation separation.

All five reviewers gave this paper positive scores, highlighting its effective framework with solid theoretical foundation and extensive experiments. One main initial concern raised by several reviewers was whether the baselines were set up correctly and whether the selected baselines were sufficient. During the discussion stage, most of these concerns from the reviewers were addressed.
Decision: Accept (poster)