Unified Parameter-Efficient Unlearning for LLMs
Authors: Chenlu Ding
Jiancan Wu
Yancheng Yuan
Jinda Lu
Kai Zhang
Alex Su
Xiang Wang
Xiangnan He
Venue: ICLR 2025 Poster (2025)
Keywords: Large Language Model Unlearning; Machine Unlearning; Influence Function
Primary area: interpretability and explainable AI

Abstract: The advent of Large Language Models (LLMs) has revolutionized natural language processing, enabling advanced understanding and reasoning capabilities across a variety of tasks. Fine-tuning these models for specific domains, particularly through Parameter-Efficient Fine-Tuning (PEFT) strategies like LoRA, has become a prevalent practice due to its efficiency. However, this raises significant privacy and security concerns, as models may inadvertently retain and disseminate sensitive or undesirable information. To address these issues, we introduce a novel instance-wise unlearning framework, LLMEraser, which systematically categorizes unlearning tasks and applies precise parameter adjustments using influence functions. Unlike traditional unlearning techniques that are often limited in scope and require extensive retraining, LLMEraser is designed to handle a broad spectrum of unlearning tasks without compromising model performance. Extensive experiments on benchmark datasets demonstrate that LLMEraser excels in efficiently managing various unlearning scenarios while maintaining the overall integrity and efficacy of the models.

Published as a conference paper at ICLR 2025
UNIFIED PARAMETER-EFRICHES UNLEARNING FOR LLMS
Chenlu Ding
Jiancan Wu
Yancheng Yuan
Jinda Lu
Kai Zhang!
Alex Su
Xiang Wang!
Xiangnan Hel
University of Science and Technology China
2 Hong Kong Polytechnic University
MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, USTC
ABSTRACT
The advent of Large Language Models (LLMs) has revolutionized natural lan- guage processing, enabling advanced understanding and reasoning capabilities across a variety tasks. Fine-tuning these models for specific domains, par- ticularly through Parameter-Efficien Fine-Tuning (PEFT) strategies like LoRA, has become a prevalent practice due to its efficiency. However, this raises sig- nificant privacy and security concerns, models may inadvertently retain and disseminate sensitive or undesirable information. To address these issues, we introduce novel instance- wise unlearning framework, LLMEEaser, which sys- tematically categorizes unlearning tasks and applies precise parameter adjust- ments using influence functions. Unlike traditional unlearning techniques that are often limited in scope and require extensive retraining, LLMEraser is de- signed to handle broad spectrum of unlearning tasks without compromising model performance. Extensive experiments on benchmark datasets demonstrate that Mraser excels in efficiently managing various unlearning scenarios while maintaining the overall integrity and efficacy of the models. Our code is available at ht / /i hubb com/ ocw.aceannnnnc////a/oo raMer.
INTRODUCTION
Large language models (LLMs) demonstrate remarkable capabilities in knowledge understanding and complex reasoning (Li et al., 2023; Zhang et al., 2024b; Li, 2024; Li et al., 2024; Lee et al., 2024), having sparked increasing interest in adapting LLMs to specific domains through fine-tuning techniques (Li & Liang, 2021; Dettmers et al., 2023; Zhang et al., 2023; Zaken et al., 2022). Among them, Parameter-Efficien Fine-Tuning (PEFT) (Li & Liang, 2021; Liu et al., 2021), such as LoRA (Hu et al., 2022), has emerged as the mainstream paradigm, offering significant reductions in re- source costs by fine-tuning only a small subset of parameters. While highly effective, the reliance on domain-speciin data for fine-tuning raises concerns regarding data leakage and privacy (Lu et al., 2024; Blanco-Justicia al., 2024), such as potentially memorizing or propagating sensitive, biased, copyrighted, or harmful information (Liu et al., 2024c; Qu et al., 2024). In this light, researchers have introduced unlearning techniques (Jang et al., 2023; Kurmanji et al., 2023; Kumar et al., 2023) into LLMs, to "forget" specific data without requiring the time-consuming and resource-intensive process of retraining.
Prior efforts in exploring unlearning in LLMs primarily focus on removing specific concepts (Kassem et al., 2023; Jang et al., 2023). typical example is the erasure LLM's ability to recall information related to the Harry Potter series (Eldan & Russinovich, 2023). While these ef- forts yield valuable insights, they risk inadvertently affecting related concepts, such as other novels with similar titles. In this work, broaden the scope by investigating aistance-wise unlearning
Work done at The Hong Kong Polytechnic University.
1 Equul contribution. {dingchenlu201103, wujean } gmail.com :Corresoondence to Jiancan Wu, Yancheng Yuan, and Xiangnan He. yancheng pol.eeoo.yyaannnuo..c.o.ome xiangnanhe gmail.com)
(wujcan gmail.commm

---

Published as a conference paper at ICLR 2025
Table I: A summary of existing LLM unlearning methods and their application scenarios. 'E' and 'A' are abbreviations for Exact unlearning and Approximate unlearning respectively.
Preserve Model
Free from Retrain/Pretrain
Reluted Work
Mode
Method
IR
QM
RC
Architecture
Retrain
Retrain
SISA (Bourtoule 2021)
Retrain Sub-model
FairSISA (Kadhe et al., 2023)
Retrain Sub-model
APA (Ha al., 2024c)
Retrain Sub-model
1
Grudient Ascent
A
Fine tuming
X
X
RUL RUhen & Yang, 2023)
A
Fine tuming
X
x
EZURee (Wang et al., 2024)
A
Fine-tuning
X
X
X
LLMEraser (Ours)
A
Parameter Editing
V
tasks, which allow to target more nuanced aspects of model behavior. To this end, first present various instance-wise unlearning tasks for LLMs, as illustrated in Figure 1. More case studies can be found in Appendix G. Specifically, consider training instance (2r,y) in supervised fine- tuning dataset, where x represents the query and is the response. We can categorize the LLMs unlearning tasks the instance level as follows:
Instance Removal (IR). It removes the sample : = (z, y) from the training set.
Query Modification (QM). It adjusts the input tokens in query æ, such as removing specific noisy tokens or correcting certain erroneous tokens.
Response Correction (RC). It corrects the model's response y. including updating outdated an- swers or rectifying incorrect classification results.
In this work, we focus on unlearning the domain-specific data used solely in PEFT, which re- quires updating the PEFT adapters (e.g., LoRA). Technically, recent LLM-unlcarning efforts can be roughly grouped into two categories. Exact unlearning approaches divide data into disjoint shards and retrain adapters (Bourtoule et al., 2021; Hu et al., 2024c). Despite effectiveness, these methods have inherent limitations inevitably destroying the model's original structure and neces- sitating the retraining cost. Approximate unlearning methods, on the other hand, aim to replicate the performance of the retrained model, often aligning the output of the target data elosely with ran- domness through KI Ill-divergence-base PEFT (Liu et al., 2024a; Qu et al., 2024). Nonetheless, this paradigm primarily focuses on data removal (e.g., g. IR) and hardly corrects biased or inaccurate data (e.g., QM, RC), as it falls short in guiding the output of the target data towards accurate information, rather than mere randomness. See Table for the summary of current L.LMs unlearning methods, with detailed descriptions available in Appendix A. Overall, both approaches struggle to efficiently handle these instance-wir wise LLM unlearning tasks and are not specifically designed for unlearning within the PEFT framework. It calls for general LLM unlearning method capable of addressing these various tasks.
In pursuit of parameter-efficient unlearning, we identify the influence function (Koh & Liang, 2017) as a promising tool. At its core is to formulate the parameter changes caused by perturbations in the form of the inverse Hessian- vector-product (Agarwal et al., 2016), where Hessian matrix represents the curvature of the loss function w.r.f. model parameters. However, the direct application of the influence function LLMs presents two significant challenges: the expensive cost of calculating the inverse dressian-vector-proddde for vast model parameters and the cumulative errors introduced by approximation strategies (e.g,, stochastic estimation (Agarwal al., 2016)). Consequently, the use of influence functions for LLM unlearning remains largely underexplored. To fill this research gap, we propose a unified parameter-efficieet unlearning framework, LLMEraser, for various instance- wise unlearning tasks. Specifically, for each type of unlearning task, LLMEraser leverages influence functions to directly calculate the parameter changes in the PEFT adapters and then efficiently up- date the adapter parameters, thus bypassing the need for time- tuning. Furthermore, we reformulate the calculation of the inverse dessiann-vectorproddee finite-sum quadratic programming problem (Nesterov, 2013; Beck & Teboulle, 2009), significantly reducing computational complexity while mitigating the approximation errors from stochastic es- timation. LLMEraser has several advantages: model-agnostic, applicable to various instance- wise unlearning tasks, and ensuring fast model updates. We conduct experiments on both LLMs and Multimodal Large Language Models (MLLMs), specifically focusing LLMSs for Recommenda-
model retraining or fine- into a
2

---

Published as a conference paper at ICLR 2025
Ins t Removal
Query Modification
Exact LLM Unlearning
the largest one from sot of numbers. 1001, 22, 500, -3999, 1o6, B5, 2266
Imput: Find
Inpol: Solve the following
equation sy stem. Give me
Output
Adaptert
Shard1
the final answer. 3x 4y = =200
Output: 1e6
Oulpul: =3
Aggregated Adapter
Adapter2
Shard2
Training Set
LLM
Jmpult: Fiad out the largest berbers. 1001, 22, 500, -2939,1e6,,,8, 26, Output: 1e6
Imputt: Solve the following
Adapterk
Shardk
equation system. Give the final answer. 3x 1, 2x by 12
Input
Adapterk'
Unlearning
Request
Output: x =
Response Correction
Approximate LLM Unlearning
Output
Imput: Solect the oldest. person from the list. Georg Washingtan, Confucius, Michael Jordan, Iichelangelo. Oulput: Geone Washinglon)
*
KL-divergence-- based Fine-tuning
Training Set +
LLM
Adapter
Inguat: Select the al dest person from George Washington, Confucius, Michael Jordan, Michelangelo. Outpat: Confucius
Unlearning Request
(b) Overview of exact/approximate LLM Unlearning. Figure 1: la: A brief description of the different types LLM unlearning tasks. l The framework of exact LLM unlearning method, approximate unlearning method.
(a) Taxonomy LLM unlearning tasks.
tion (LLM4Recc as well as MI MLLLM relation mining tasks to validate the effectiveness of LLMEraser. Our extensive evaluations across these diverse scenarios demonstrate that L.LMEraser consistently outperforms the state-of- sate-of-theear the-art art unlearning methods.
2 PRELIMINARY
This section introduces key concepts underpinning our methodology. We cover instruction tuning enhance LLMs' understanding of human instructions, followed by PEFT, highlighting LoRA for efficient updates. Lastly, we discuss the influence function, which analyzes parameter changes from data perturbations. These foundations set the stage for the techniques discussed later.
2.1 INSTRUCTION TUNING
Instruction tuning is key technique that leverages carefully curated datasets of instructions and corresponding responses to enhance LLMs' capacity to comprehend and respond to human instructions (Wei et al., 2022; Liu al., 2023b; Sanh et al., 2022). Given downstream task dataset Z = {=|= = (rr 3)} containing n instances, where æ represents description of the human instruction and y is the corresponding response, L.MMs fine-tuned using the following autoregressive (Brown et al., 2020; Touvron et al., 2023a) objective:
max Σ
log (P (1h /
[, b)),
(x.N) t=1
where & is LLMs' parameters, V4 the t-th token of y, and Yer represents tokens preceding ye-
2.2 CHAMMETER-EFFIIEEN FINE-TUNING
LLMS typically consist of billions of parameters, making full fine-tuning computationally expen- sive. Parameter-Efffcicnt Fine-Tuning (PEFT) addresses this challenge by updating only small number of the parameters while still achieving satisfactory performance. Among them, LoRA (Hu et al., 2022) stands out as particularly effective, which freezes the original pretrained parameters while introducing pairs of www-rank-decompen Formally, the optimization objective for LoRA is expressed as follows:
weight matrices to simulate parameter updates.
lvl
max
log (P (1( Nets 1 + A
(2)
(5.g)E2
1=1
3

---

Published as a conference paper at ICLR 2025
where θ is the trainable parameters that is significantly smaller in size compared 4.
2.3 INFLUENCE FUNCTION
The influence function was first applied in machine learning by Koh & Liang (2017) to analyze the outputs of black-box models. For the dataset Z, focus the following empirical risk minimiza- tion
& Ben-David, 2014; Vapnik, 1998; Bartlett & Mendelson, 2002) problem:
}
θ E arg min R(2:0)]R(2)0)
Σ 2((x,)):0)
(3)
n
(a,9)EZ
where θ is the trainable model parameter and is the minimizer of Equation 3. L (=; is the loss function, and for Equation 2, is defined as:
141 L ((x,,)): = - log (P x. Veee A)(()))
(4)
When a training example (a, is upweighted by an infinitesimal amount e, perturbed loss for Onw (c) can be expressed as:
+ new (c) E arg min {2, (C.
(5)
When A O, the parameter change A0(() new (c) θ can be approximately calculated by ap- plying a Taylor expansion Equation 3. Please refer to (Koh Liang, 2017) for detailed derivation. Specifically, AO(e) can be written as:
AO(e) ~ -elf VeL (Ix v): 6) VVe
(6)
where Hg ((Z;A) is the Hessian matrix, VeL((x, y); Q) represents the gradient of L. w.r.f.
parameters θ, evaluated at é.
METHOD
In this work, we propose LLMEraser, framework that updates the PEFT adapter parameters to handle various instance-wise unlearning tasks. As shown in Figure 2, approach leverages the influence function to directly estimate the parameter changes for various unlcarning tasks, circum- venting the resource-consuming fine-tuning retraining procedures. Moreover, present novel algorithm to accelerate the computation the inverse Hessian- cessiin-vector-producc in the influence func- tion, enabling its efficient implementations in LLMs. Finally, we summarize how LLMEraser works.
3.1 TAXONOMY OF LLM UNLEARNING TASKS
We focus on instance-wise unlearning tasks for LLMs, specifically for PEFT that uses domain- specific data. For an instance 2 (a., y). where æ represents the query and y the response, we propose taxonomy of unlearning tasks based the operation applied to the target instance.
Instance Removal (IR). When a specific instance = = (z, y) is either restricted from use or contains harmful content, it necessitates complete elimination from the training set, along with its associated influence on the model.
Query Modification (QM). This category involves modifying the query æ, transforming into = (a', .). could not only delete outdated or incorrect tokens the query æ, such as noisy interactions from user's history, but also update erroneous outdated tokens with correct ones.
(x, y)
Response Correction (RC). Here, the focus is on rectifying the output component y of the instance That is, replacing 2 (z. 3) with z' (x, y²). For binary classification tasks, such as answering "Yes" or "No", it corrects mislabeled outputs by flipping the labels. For other tasks, such as multi- class classification question answering, it is applied to rectify inaccurate responses.
4

---

Published as a conference paper at ICLR 2025
Instruction Tuning
Parameter Updating
Old Adapter
Training Set
PEFT
Unlearning Request
LLME raser
New
A
Adapter
Influence & Function
Parameter Changes
Updated <
Training Set
LLM Unlearning
The framework LLMEEaser. The old adapter is obtained through PEFT on domain- specific data. When an unlearning request arrives (e.g., deleting or correcting certain data from the training set), LLMEraser utilizes influence functions to compute the parameter changes caused by such request. These estimated parameter modifications are added the old adapter's 's weights, resulting in the new adapter parameters essentially the unlearned model parameters.
Figure 2:
Our proposed taxonomy expands the concept of LLM unlearning beyond the removal of entire in- stances. It introduces more fine-grained categorization defined at the token level within both queries and responses, allowing for nuanced control of model behavior.
3.2 LLMERASER
The key strength of LLMEraser lies in its capacity to directly estimate the adapter's parameter changes caused by various unlearning tasks. For the sake of clarity and without sacrificing general- ity, we employ the loss function in LoRA (ef. Equation 4) as example, while other alternatives would yield similar formulations.
To develop unified approach for solving all unlearning tasks in taxonomy, begin by sidering a general case where perturbations are applied to both the query (æ) and response (y) com- ponents of instance This generalized framework allows task as a special case this perturbation scenario. Formally, we define the perturbation & applied 2ş (rr + 6n, y + oy), where o, and 8, represent perturbations the query and response, respectively. We now formulate the perturbed empirical risk minimization problem as:
model each specific unlearning
es(c) E arg min (R(2:0) + cL ((x 8z, 6)); cf cf v): O)),
6
where @s(() is the minimizer of the optimization problem after applying perturbation ó of magni- tude to the sample z. Following the derivation in (Koh & Liang, 2017). when the sample size (i.e., z 0), we can safely estimate the parameter change A0;
sufficiently large, by taking as follows:
1 n AOs (V)R(2)6)) (G(x. G(x + on + 8y))
where G( y) is an abbreviations for VeL (z, corresponding parameter changes for different unlearning tasks.
y): 6). Next, we present the perturbations and
Instance Removal. The deletion of data corresponds to the perturbation function in Equation 5. By setting _ like Equation it is equivalent removing instance 2. The set of deleted instances is denoted SIR- By aggregating the gradients of all deleted instances, the parameter change A@I can be expressed as follows:
A R (vEn(Z;0))
Σ
g(x,y).
(Z.))
Query Modification. Modifying certain tokens in the query is equivalent perturbing with 8,. where 8, represents deleting noisy tokens or correcting inaccurate tokens, while keeping the re- sponse unchanged (i.e., o, 0). Hence, the perturbed instance is represented 2ạ (x++5,, y),
5

---

Published as a conference paper at ICLR 2025
with the set of instances requiring the removal or modification of specific tokens represented by SoM- By aggregating the gradients of all instances in SQM. the parameter change A induced by query modification can be shown as follows:
A
(VAR(Z)))) 1 n
Σ (z.M)
(z
Σ
Veg(x V (10)
Response Correction. Correcting the response solely corresponds to ó₂ = 0 while perturbing the response y with 8y. Here 5y represents updates to outdated answers adjustments erroneous classification results. With 25 = (2, y + 6y), the set of instances with rectified labels is Sgc. The parameter change A@Re is as follows:
AORC = n (VER(2:6))
Σ (N.
g(r
E
G + 88)
(11)
G,)
However, computing inverse Hessian vector-produuct results presents significant challenges. Al- though CG (Hestenes et al., 1952; Fletcher, 2000; Shewchuk et al., 1994) shows some promise, requires full-bateh gradient computation (Koh & Liang, 2017), making it impractical for large-scale datasets. Stochastic estimation (Agarwal al., 2016) expands (V) T/Z θ))-1 into truncated power series and iteratively estimates parameter changes, but it suffers from cumulative approxima- tion errors (Blanco-Justicia et al., 2024; Basu al., 2021). Next, we elaborate new efficient and scalable algorithm for computing T for different unlearning tasks.
3.3 A NEW ALGORITHM FOR COMPUTING PARAMETER CHANGES
Inspired by the previous studies (Ding et al., 2025), LLMEraser reformulates the calculation of parameter changes as solving an equivalent optimization problem expressed in summation form, enabling efficient resolution using mini-batch algorithms. Specifically, we focus on the following optimization problem regarding A:
min F(A)
2 A A V (b,
(12)
where (.) represents the inner product of vectors, and b defined as:
(a G(x,y).
if Task IR if Task IM if Task RC
b =
(ua)) 9( y) 1 E E(tto..)esm G(x + da, v),
(13)
(x.)) G(x.v) v) _ (x G(x, y dy),
Since Ô is the minimizer of Equation it satisfies the second-order necessary optimality condi- tion (Nocedal & Wright, 1999; Luenberger et al., 1984; Bertsekas, 1997), resulting in the matrix V²,((Z;)) being symmetric and positive semidefinite. Thus, Equation 12 is essentially convex quadratic problem, with gradient of V R(Z;)))
Given that A0Tusk can be interpreted as the solution the linear system V2, b, ad- dressing T is effectively equivalent to optimizing Equation 12. Due to the summation form of Vi Z; 6), Equation 12 can be reformulated as the following finite-sum formation:
F(A)
Σ f ((z, A),
(14)
(x,9)E2
where f((x, v), A) is defined as:
1 f((2, v) A) 1 VOLL (1 (I, A (b,
(15)
By employing scalable algorithms (e.g., SGD) to optimize problem 12, we can obtain the solu- tion for A It is worth noting that both the function value and the gradient can be efficiently

---

Published as a conference paper at ICLR 2025
computed using the Hessian- vectorr-eeerr-ppe protuuct (HVP)', reducing the complexity from O(p²) O(p) (Pearlmutter, 1994). where p is the number of trainable parameters. The pseudocode for computing parameter changes can be found in Appendix B. Error analysis for proposed algorithm can be found in Appendix D.
3.4 THE WORKFLOW OF LLMERASER
LLMEraser focuses on unlcarning domain-specifir data and updating the parameters of the PEFT adapters. Overall, the workflow of LLMEraser is follows:
Leverage domain-specific data and apply PEFT techniques to train and obtain the initial adapter, which captures the model's performance on the original dataset.
When certain data becomes unavailable, process and validate the unlearning request ensure compliance with regulations or organizational policies before initiating the unlearning procedure.
Utilize LLMEraser, which employs influence functions to efficiently calculate the necessary changes in the model parameters resulting from the specified unlearning request. This step en- sures that the impact of the unavailable data is removed from the model.
Apply the computed parameter adjustments to the parameters of the previously trained adapter, effectively updating to reflect the removal of the unavailable data. This yields the final unlearned model parameters while preserving efficiency and minimizing retraining overhead.
4 EXPERIMENT
In this section, carry out extensive experiments assess the performance and efficiency of L..MEraser. The experiments are designed to explore the following key research questions: RQ1: How does I.LMEraser perform across various unlearning tasks? RQ2: How does LLMEraase per- form at different unlearning ratios? RQ3: How does the efficiency of LLMeraser compared to other unlearning methods?
4.1 EXPERIMENTAL SETUPS
We conduct experiments both LLMs and Multimodal Large Language Models (MLLMs), fo- cusing specifically on LLMs for Recommendation (LLM4Rec) (Bao et al., 2023; Liao et al., 2024) and MILM relation mining tasks (Wu et al., 2024c; Ye al., 2024), validate the effectiveness of our proposed LLMEraser. We choose LLaMA2-7B (Touvron et al., 2023b) as our backbone LLM and LLaVA 1.5-7B (Liu et al., 2023a) for the MLLM experiments. Comprehensive details on task, datasets, baselines, and evaluation metrics for our proposed LLMEraser can be found in Appendix C.
4.2 RESULTS ANALYSIS FOR VARIOUS UNLEARNING TASKS (RQ1)
We design a variety of comprehensive experiments to thoroughly validate the effectiveness of LLMEraser across the three unlearning tasks we have proposed.
4.2.1 RESULTS ANALYSIS ON INSTANCE REMOVAL
For instance removal, we directly delete a proportion of training instances and subsequently evaluate the performance of each unlearning method. The experimental results on LLM4Rec are shown in Table We can find that: (1) LLMEraser closely mirrors the performance of Retrain. The perfor- mance gap between LLMEraser and Retrain is merely 0.0038, constituting only 0.6% of Retrain's 's performance. This can be attributed to our method's 's direct estimation the parameter changes be- tween the retrained model and the original model, allowing for a highly accurate calculation of these changes. (2) Other unlearning methods exhibit notable declines in model performance. Specifically, Gradient Ascent and E2URee show average decreases of 2.7% and 2.4%, respectively, they do not explicitly aim to approximate the Retrain model during the fine-tuning process.
!HVP has corresponding implementation in PyTorch; refer to https://pytont ottp///rro.ooo/oo/os////o/////c. stable/autograd html for details.

---

Published as a conference paper at ICLR 2025
4.2.2 RESULTS ANALYSIS ON QUERY MODIFICATION & RESPONSE CORRECTION
Adversarial attack experiments are widely employed to assess the efficacy of data modification for unlearnin techniques (Wu et al., 2023; Moon et al., 2024; Cha et al., 2024). The core idea is first randomly introducing corrupted instances into the dataset, which inevitably leads to a decline in model performance, and then leveraging unlearning techniques correct these noisy data on the model. Following this setting, we evaluate the performance of LLMEraser in both query modifica- tion and response correction tasks.
For query modification, we conduct experiments on the LLM4Rec task by adding adversarial noise to the user interaction sequences, i.e., randomly deleting some items from the sequences (Inter- action Removal) or replacing them with corrupt ones (Interaction Replacement), and then using LLMEraser to rectify the data. Table 3 presents the experimental results. We can observe that: (1) LLMEEaser brings a substantial urility gain to the model compared to the corrupted baseline, signif- icantly reducing the negative impact of noisy data. Specifically, achieves an average improvement of 5.1% compared to the corrupted model in both settings, with a peak increase of 5.5% in inter- action removal setting. Moreover, its performance is closest to that of Retrain, demonstrating its effectiveness in correcting inaccurate input information. (2) SISA and RecEraser fail to improve performance. Their average results in both settings decreased by 7.0% and 31.3% compared to the corrupted baseline. The reasons may lie in their dataset partitioning and submodel retraining strat- egy. potentially leading to a loss of crucial contextual information and introducing inconsistencies in learned representations. (3) RecEraser underperforms SISA in most cases. Designed on tradi- tional recommendation models, RecEraser relies on users' collaborative signals to optimize shard partitioning: however, this strategy fails effectively adapt to LLM4Rec.
For response correction, we introduce noise into the training data of the MLLMs task by randomly assigning incorrect labels to a portion of the samples. In the spurious biases task for MLLMs, we reverse 40% the original "yes/no" labels. For the hard hallucination mining task in MLLMs, we assign random labels to 40% of the samples. We leverage LLM unlearning to mitigate the neg- ative impact of such noisy data, aiming to approximate the performance of retraining with clean data. The experimental results of response correction unlearning task on spurious biases task and hard hallucination mining task are presented in Table 4 and 5, respectively. We can draw the fol- lowing observations: (1) LLMEraser effectively performs response correction, achieving average improvements of 14.2% and 18.9% on the spurious biases task and hard hallucination mining task, respectively, compared to the corrupted baseline. Compared to other methods, LLMEEraser shows the smallest performance gap relative Retrain. On the spurious biases task and hard hallucination mining task, the average differences with Retrain are 0.024 and 0.048, which account for 2.9% and 7.5% of Retrain's 's performance, respectively. Whether addressing label reversal in binary classifica- tion or correcting labels in multi-class scenarios, LLMEraser can eliminate the negative impact of noisy labels and restore them to their clean, original state. (2) The improvement brought by SISA is not significant. Although SISA ensures that dirty data is replaced with clean data during retraining, its data segmentation strategy can inevitably hurt model performance.
4.3 RESULTS ANALYSIS FOR DIFFERENT UNLEARNING RATIOS (RQ2)
To assess the sensitivity of various unlearning methods to different scales of unlearning data, we con- duct experiments using different unlearning ratios in instance removal and query modification tasks. For the instance removal, we employ TallRec as the M.MM framework, where 5% and 10% of in- stances are removed. Meanwhile, for query modification, I.LARA is utilized as the backbone, where 5% and 10% of user interactions are deleted. The experimental results are shown in Figure 3. From these results, we can find that: (I) In the instance removal task, LLMEraser consistently performs closest to Retrain across different unlearning ratio settings, with an average performance decline of only 1.18%. This indicates that LLMEraser can effectively delete data while minimizing the neg-
Table 2: Experimental results on the instance removal task with 5% of training data removed, using TALLRec the LLM4Rec model on the BookCrossing dataset.
Original
Retrain
Gradient Ascent
E2URec
LLMEraser (Ours)
AUC
0.6400
0.6357
0.6187
0.6205
0.6319
8

---

Published as a conference paper at ICLR 2025
Table Experimental results on the QM task, using LLaRA the LLM4Rec model on the Movic- Lens and LastFM datasets. "10% Interaction Removal" refers to 10% of users have items removed from their interaction sequences, "5% Interaction Replacement" refers 5% of users have items replaced with noisy interactions. Corrupted refers to the model trained with the noisy data.
Movielens
LastFM
Method
HitRatio@1
ValidRatio
HitRatio @
ValidRatio
Retrain
0.4565
0.9684
0.4508
1.0000
Corrupted
0,4222
0.9375
0.4344
1.0000
10% Interaction Removal
SISA
0.4130
0.9684
0,4132
0.9918
RecEraser
0.2717
0.9684
0.4298
0.9918
LLMEraser (Ours)
0.4456
0.9684
0.4463
0.9918
Retrain
0.4565
0.9684
0.4508
1.0000
Corrupted
0.4316
0.9684
0.4344
0.9918
5% Interaction Replacement
SISA
0,3804
0.9684
0,4050
0.9918
RecEraser
0.3152
0.9684
0.3689
1,0000
LLMEraser (Ours)
0,4516
0.9789
0.4426
1.0000
Table Experimental results on the MM-SPUBENCH for RC tasks, where Corrupted denotes we assign wrong labels for 40% of the training samples.
SPUBENC
Method
Average
All
BG
TN
CO
RS
Col.
Ori.
LS
PA
Sha.
Retrain
0.88
0.80
0.83
1.00
0.78
0.86
0.86
0.66
0.70
0.82
0.84
Corrupted
0.76
0.62
0.67
0.80
0.67
0.76
0.65
0.68
0.67
0.70
0.71
SISA
0.84
0.65
0.79
1.00
0.64
0.79
0.86
0.73
0.57
0.76
0.77
LLMEraser
0.86
0.70
0.80
1.00
0.78
0.85
0.84
0.76
0.67
0.81
0.81
ative impact on model performance. (2) In the query modification task, LLMEraser consistently achieves the best performance across various unlearning ratios, with an average improvement of 4.9% compared to corrupted method. Notably, at unlearning ratio of 10%, the relative improve- ment reaches 5.1%. The average difference between LLMEraser and Retrain is only 0.0079. In comparison to SISA and RecEraser, LLMEraser demonstrates superior ability to maintain model utility. This highlights the effectiveness of LLMEraser, demonstrating its robust performance across varying unlearning demands. (3) We observe interesting phenomenon in query modification task under adversarial attack settings, with a sufficiently high unlcarning ratio (in this case, 5% and 10%), both SISA and Receraser require retraining all shards with the same clean data, resulting in equiva- lent outcomes. Despite the direct use of clean data for retraining, they still struggle to obtain optimal model performance.
4.4 RESULTS ANALYSIS FOR UNLFARNING EFFICIENCY (RQ3)
Efficiency is key metric in evaluating unlearning techniques, par- ticularly for LLMs. We here conduct experiments, comparing our proposed LLMEraser against existing techniques. For a fair com-
Table 6: Execution time in the QM task.
parison, we report the execution time in the QM task, where 5% of users have items replaced with noisy interactions. All methods are run on a single Nvidia A A100 GPU. Table 6 presents the results. We can observe that: (1) Due to the parallel training of sub-models, the retraining time of both SISA and RecEraser can be reduced to some extent. However, RecEraser requires data partitioning based on similarity, which introduces additional computational overhead. Moreover, both methods remain highly inefficient as unlearning re- quests necessitate retraining of the adapters. (2) In contrast, our proposed LLMEraser exhibits remarkable efficiency handling unlearning tasks. By directly mod- ifying model parameters, LLMEraser achieves speedup of approximately 31.25 times compared to retraining, requiring only about 1.4 x seconds to update the parameters. This reduction in execution time demonstrates the effectiveness of approach in accelerating the computation of
Method
Time (s)
Retrain
5.4 x 10a
SISA
1.8 x 104
RecEraser
2.0 x 104
LLMEraser
1.4 x 103

---

Published as a conference paper at ICLR 2025
Table Experimental results on the R-BENCH for RC tasks, where Corrupted denotes we assign wrong labels for 40% of training samples.
Method
Recall
F1-Score
Precision
Accuracy
Yes
Retrain
0.70
0.66
0.63
0.65
0.55
Corrupted
0.47
0,50
0.53
0.54
0,44
SISA
0.47
0.49
0.52
0.52
0.45
LLMEraser (Ours)
0.68
0.63
0.58
0.56
0.50
Unlearning Ratio: 0.05
Unlearning Ratio: 0.05
066
0.6357
0.6319
0.4
0.4565
InaRatio@]
0.6187
042
0.3152
Original
Compped
Unlearning Ratio: 0.10
Unlearning Ratio: 0.10
0,6000
0,4457
0,6298
0,6277
IitRaaiio@!l
0,4239
0,3804
0,6114
0.3152
Orightal
Retrin Grodient Acxent E2UR
Mi
Comppted
Retmin
SISA
RecEmer L.Me
(a) Impact unlearning ratio IR. (b) Impact unlearning ratio QM.
Figure 3: 3a: Experimental results of the instance removal task using TallRec as the LLM4Rec
model on the BookCrossing dataset, where 5% and 10% of the training data were randomly deleted. 3b: Experimental results of the query modification task using LLaRA as the LLM4Rec model on the MovieLens dataset, where interactions were randomly removed from 5% and 10% of users.
parameter changes. Additional experimental results and related analyses on the memory usage and exccution time of LLMEraser can be found in Appendix E.
5 LIMITATIONS
LLMEraser offers efficient parameter updates without the need for retraining, making it versatile across different unlearning tasks while also reducing computational overhead. Despite the improve- ments brought by LLMEraser, its potential shortcomings should not be overlooked. Calculating parameter changes for different unlearning tasks requires accessing the gradient information of the target data and assumes the availability of the training set. Furthermore, the influence function's liance on the first-order Taylor expansion of the optimization objective leads to inevitable estimation errors, representing an inherent limitation of such approach.
CONCLUSION AND FUTURE WORK
This paper introduces LLMEraser, a unified parameter-effiiient unlearning framework. By sys- tematically categorizing and addressing various unlearning tasks, L.MEErase leverages influence functions for parameter adjustments, circumventing the cumbersome retraining processes common traditional methods. Extensive experiments on benchmark datasets show that LLMEraser excels in efficiently handling various unlearning tasks while preserving the overall integrity and efficacy of the models. Additionally, LLMEraser opens new avenues for future research, encouraging the exploration of enhanced unlearning techniques and their implications in diverse applications, such as data privacy and ethical AL. Future studies could explore the broader applicability of LLMEraser and potential optimizations for its computational efficiency and accuracy.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENTS
This research is supported by the National Science and Technology Major Project (203320122102)), National Natural Science Foundation of China (92270114, 62302321, U24B20180, 62121002). The work of Yancheng Yuan is supported by the Research Center for Intelligent Operations Research and The Hong Kong Polytechnic University under grant P0045485. This research is also supported by the advanced computing resources provided by the Supercomput- ing Center of the USTC.
REFERENCES
Naman Agarwal, Brian Bullins, and Elad Hazan. Second order stochastic optimization in linear time. CoRR, abs/ tbss/100..0394,, 2016.
lan Arawjo, Chelse Swoopes, Priyan Vaithilingamm, Martin Wattenberg, and Elena L. Glassman. Chainforge: A visual toolkit for prompt engineering and LLM hypothesis testing. In CHI, pp- 304:1-304:18. ACM, 2024.
Keqin Bao, Jizhi Zhang, Yang Zhang, Wenjie Wang, Fuli Feng, and Xiangnan He. Tallrec: An effective and efficient tuning framework align large language model with recommendation. In RecSys, pp. 1007-1014. ACM, 2023.
Peter L. Bartlett and Shahar Mendelson. Rademacher and gaussian complexities: Risk bounds and structural results. J. Much. Learn. Res., 3:463-482, 2002.
Samyadeep Basu, Phillip Pope, and Soheil Feizi. Influence functions in deep learning are fragile. In ICLR. OpenReview.net, 2021.
Amir Beck and Mare Teboulle. fast iterative shrinkage-thressholdnng algorithm for linear inverse problems. SIAM J. Imaging Sci., 2(1):183 202, 2009.
Dimiur P Bertsekas. Nonlinear programming. Journal of the Operational Research Society, 48(3):
334-334, 1997.
Alberto Blanco-Justicho Najeeb Jebreel, Benet Manzanares- David Sánchez, Josep Domingo- Ferrer, Guillem Collell, and Kuan Feik Tan. Digital forgetting large language models: A survey of unlearning methods. CoRR, abs/2404.02062, 2024.
Lucas Bourtoule, Varun Chandrasekaran, Christopher A. Choquette-Choo Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. Machine unlearning. In SP, pp. 141- 159. IEFE, 2021.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voor Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In NeurIPS, 2020.
Robin D. Burke. Hybrid recommender systems: Survey and experiments. User Model. User Adapt. Interact., 12(4))331 17000 2002.
William Cain. Prompting change: exploring prompt engineering in large language model ai and its potential to transform education. TechTrends, 68(1):47-57,20 2024.
Iván Cantador, Peter Brusilovsky, and Tsvi Kuflik (eds.). Proceedings the 2nd International Work- shop on Information Heterogeneity and Fusion in Recommender Systems, HetRec '11, Chicago, Illinois, USA, October 27, 201 2011 ACM.
Sungmin Cha, Sungjun Cho, Dasol Hwang, Honglak Lee, Taesup Moon, and Moontae Lee. Learn- ing to unlearn: Instance-wise unlearning for pre-trained classifiers. In AAAI, pp. 11186-11194. AAAI Press, 2024.
11

---

Published as a conference paper at ICLR 2025
Chong Chen, Fei Sun, Min Zhang, and Bolin Ding. Recommendation unleamning. In WwW, pp- 7768-2777.20 ACM, 2022.
Jiaao Chen and Diyi Yang. Unlearn what you want to forget: Efficient unlearning for llms. In EMNLP, pp. 12041-12052. Association for Computational Linguistics, 2023.
Yuxin Chen, Junfei Tan, An Zhang, Zhengyi Yang, Leheng Sheng, Enzhi Zhang, Xiang Wang, and Tat-Seng Chua. On softmax direct preference optimization for recommendation. In NeurIPS, 2024.
Zhikai Chen, Haitao Mao, Hang Li, Wei Jin, Hongzhi Wen, Xiaochi Wei, Shuaiqiang Wang, Dawei Yin, Wenqi Fan, Hui Liu, and Jiliang Tang. Exploring the potential of large language models (Ilms)in learning on graphs. SIGKDD Explor, 25(2):42-61, 2023.
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton. Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay. Noam Shazeer, Vinodkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengcheng Yin, Toju Duke, Anselm Lev- skaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier Garcia, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito. David Luan, Hyeontack Lim, Barret Zoph, Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivani Agrawal, Mark Omernick, Andrew M. Dai, Thanumalayan Sankaranarayana Pillai, Marie Pellat, Aitor Lewkowyez, Erica Moreira, Rewon Child, Oleksandr Polozov, Katherine Lee, Zongwei Zhou, Xuezhi Wang, Bren- nan Saeta, Mark Diaz, Orhan Firat, Michele Catasta, Jason Wei, Kathy Meier- Hellteme,,sstem, Douglas Hck, Jeff Dean, Slav Petrov, and Noah Fiedel. Palm: Scaling language modeling with pathways. J. Mach. Learn. Res., 24:240:1-2400 11.. 2023.
Nasrin Dehbozorgi, Mourya Teja Kunuku, and Seyedamin Pouriyeh. Personalized pedagogy through Ilm-based recommender system. In AIED Companion (2), volume 2151 of Commu- nications in Computer and Information Science, pp. 63-70. Springer, 2024.
Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient finetuning of quantized llss. In NeurlPS, 2023.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT (I), pp. 4171-4186. As- sociation for Computational Linguistics, 2019.
Chenlu Ding, Jiancan Wu, Yancheng Yuan, Junfeng Fang, Cunchun Li, Xiang Wang, and Xiangnan He. Addressing delayed feedback in conversion rate prediction via influence functions. arXiv preprint arXiv:2502.01669 2025.
Jesse Dodge, Gabriel Ilharco, Roy Schwartz, Ali Farhadi, Hannaneh Hajishirzi, and Noah Smith. Fine-tuning pretrained language models: Weight initializations, data orders, and early stopping. arXiv preprint ar arXiv:2002.06:33 06330, 2020.
Ronen Eldan and Mark Russinovich. Who's harry potter? approximate unlearning in llms. CoRR, abs/2310.02238,, 2023.
Junfeng Fang. Houcheng Jiang, Kun Wang, Yunshan Ma, Xiang Wang, Xiangnan He, and Tat- Seng Chua. Alphaedit: Null-space constrained knowledge editing for language models. CoRR, abs/2410.02355, 2024a.
Junfeng Fang. Shuai Zhang, Chang Wu, Zhengyi Yang, Zhiyuan Liu, Sihang Li. Kun Wang, Wenjie Du, and Xiang Wang. Molte: Towards molecular relational modeling in language models. In ACL (Findings), pp. 1943-1958. Association for Computational Linguistics, 2024b.
Roger Fletcher. Practical methods of optimization. John Wiley & Sons, 2000.
F. Maxwell Harper and Joseph A. Konstan. The movielens datasets: History and context. ACM Trans. Interact. Intell. Syst. 111):19 2016.
12

---

Published as a conference paper at ICLR 2025
Magnus Rudolph Hestenes, Eduard Stiefel, et al. Methods of conjugate gradients for solving linear systems, volume 49. NBS Washington, DC, 1952.
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, An- drea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for NLP. In ICML, volume 97 of Proceedings of Machine Learning Research, pp. 2790-2799, PMLR, 2019.
Cunchen Hu, Heyang Huang, Liangliang Xu, Xusheng Chen, Jiang Xu, Shuang Chen, Hao Feng, Chenxi Wang, Sa Wang. Yungang Bao, Ninghui Sun, and Yizhou Shan. Inference without inter- ference: Disaggregate LLM inference for mixed downstream workloads. CoRR, abs/2401.11181 2024a.
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. The Tenth Inter- national Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022.
OpenReview.net, 2022. URL https: httpttpp://oennevieewwe owwnreview. net/
Jun Hu, Wenwen Xia, Xiaolu Zhang, Chilin Fu, Weichang Wu, Zhaoxin Huan, Ang Li, Zuoli Tang, and Jun Zhou. Enhancing sequential recommendation via IIm-based semantic embedding learn- ing. In WWW (Companion Volume), pp. 103-111, ACM, 2024b.
Zhiyu Hu, Yang Zhang, Minghao Xiao, Wenjie Wang, Fuli Feng, and Xiangnan He. Exact and efficient unlearning for large language model-based recommendation. CoRR, abs/2404.10327, 2024c.
Joel Jang, Dongkeun Yoon, Sohee Yang. Sungmin Cha, Moontae Lee, Lajanugen Logeswaran, and Minjoon Seo. Knowledge unlearning for mitigating privacy risks in language models. In ACL pp. 14389-14408. Association for Computational Linguistics, 2023.
Hongye Jin, Xiaotian Han, Jingfeng Yang, Zhimeng Jiang, Zirui Liu, Chia- Yuan Chang, Huiyuan Chen, and Xia Hu. LLM maybe longlm: Self-extend LLM context window without tuning. CoRR, abs/2401.01325,, 2024.
Swanand Ravindra Kadhe, Anisa Halimi, Ambrish Rawat, and Nathalie Baracaldo. Fairsisa: En- semble post-processing to improve fairness unlearning in 1mms. CoRR, abs/2312.007420 2023.
Aly M. Kassem, Omar Mahmoud, and Sherif Saad. Preserving privacy through dememorization: An unlearning technique for mitigating memorization risks in language models. In EMNLP, pp. 4360-4379. Association for Computational Linguistics, 2023.
Pang Wei Koh and Percy Liang Understanding black-box predictions via influence functions. In ICML, volume 70 of Proceedings of Machine Learning Research, pp. 1885-1894. PMLR, 2017.
Xiaoyu Kong, Jiancan Wu, An Zhang, Leheng Sheng, Hui Lin, Xiang Wang, and Xiangnan He. Cus- tomizing language models with instance-wise lora for sequential recommendation. In NeurIPS, 2024.
Vinayshekhar Bannihatti Kumar, Rashmi Gangadharaiah, and Dan Roth. Privacy adhering machine un-learning in NLP. IJCNPP (Findings), pp. 268-277. Association for Computational Linguis- tics, 2023.
Meghdad Kurmanji, Peter Triantafillou, Jamie Hayes, and Eleni Triantafillou. Towards unbounded machine unlearning. In NeurIPS, 2023.
Chanhee Kwak, Junyeong Lee, Kyuhong Park, and Heeseok Lee. Let machines unlearn machine unlearning and the right to be forgotten. AMCIS. Association for Information Systems, 2017.
Byung- Kwan Lee, Beomchan Park, Chae Won Kim, and Yong Man Ro. Collavo: Crayon large language and vision model. In ACL (Findings), pp. 1121-1138. Association for Computational Linguisties, 2024.
13

---

Published as a conference paper at ICLR 2025
Likun Li, Haoqi Zeng, Changpeng Yang, Haozhe Jia, and Di Xu. Block-wise lora: Revisiting fine-grained lora for effective personalization and stylization in text-to-image generation. CoRR, abs/2403.07500 2024.
Shaoxu Li. Diffstyler: Diffusion- based localized image style transfer. CoRR, abspaaaa. 2024.
Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. In ACL/JJCNLP (1), pp.
Association for Computational Linguistics, 2021.
Zongxi Li, Xianming Li, Yuzhang Liu, Haoran Xie, Jing Li, Fu Lee Wang, Qing Li, and Xiaoqin Zhong. Label supervised llama finetuning. CoRR, abs/2310.012088 2023.
Jiayi Liao, Sihang Li, Zhengyi Yang, Jiancan Wu, Yancheng Yuan, Xiang Wang, and Xiangnan He.
Llara: Large bangugge-ocommmnndttio assistant. In SIGIR, pp. 1785-1795. ACM, 2024.
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. CoRR, abs/2310.03744, 2023a.
Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre- train, prompt, and predict: A systematic survey of prompting methods in natural language pro- cessing. ACM Comput. Surv., 55(9):195:1 2023b.
Sijia Liu, Yuanshun Yao, Jinghan Jia, Stephen Casper, Nathalic Baracaldo, Peter Hase, Xiaojun Xu, Yuguang Yao, Hang Li, Kush R. Varshney, Mohit Bansal, Sanmi Koyejo, and Yang Liu. Rethinking machine unlearning for large language models. CoRR, abs/2402.088777, 2024a.
Xiao Liu, Kaixuan Ji, Yicheng Fu, Zhengxiao Du, Zhilin Yang, and Jie Tang. P-uning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks. CoRR, abs/2110.07640 2021.
Yiheng Liu, Hao He, Tianle Han, Xu Zhang, Mengyuan Liu, Jiaming Tian, Yutong Zhang, Jiaqi Wang, Xiaohui Gao. Tianyang Zhong, Yi Pan, Shaochen Xu, Zihao Wu, Zhengliang Liu, Xin Zhang, Shu Zhang, Xintao Hu, Tuo Zhang, Ning Qiang, Tianming Liu, and Bao Ge. Under- standing Ilms: A comprehensive overview from training to inference. CoRR, abs/2401.02038, 2024b.
Zheyuan Liu, Guangyao Dou, Zhaoxuan Tan, Yijun Tian, and Meng Jiang. Towards safer large language models through machine unlearning. In ACL (Findings), pp. 1817-1829. Association for Computational Linguistics, 2024c.
Weikai Lu, Ziqian Zeng, Jianwei Wang, Zhengdong Lu, Zelin Chen, Huiping Zhuang, and Cen Chen. Eraser: Jailbreaking defense in large language models via unlearning harmful knowledge. CoRR, abs/2404.0588/) 2024.
Ximing Lu, Sean Welleck, Jack Hessel, Liwei Jiang, Lianhui Qin, Peter West, Prithviraj Am- manabrolu, and Yejin Choi. QUARK: controllable text generation with reinforced unlearning. In NeurlPS, 2022.
David G Luenberger, Yinyu Ye, et al. Linear and nonlinear programming, volume 2. Springer, 1984.
Hanjia Lyu, Song Jiang, Hanqing Zeng, Yinglong Xia, Qifan Wang, Si Zhang, Ren Chen, Christo- pher Leung, Jiajie Tang, and Jiebo Luo. LIm-rec: Personalized recommendation via prompting large language models. In NAACI-HIT (Findings), pp. 583-612. Association for Computational Linguistics, 2024.
Wenyu Mao, Jiancan Wu, Weijian Chen, Chongming Gao, Xiang Wang, and Xiangnan He. Reinforced prompt personalization for recommendation with large language models. CoRR, 1bb/2407..7115, 2024.
Joanna Misztal- Radecka and Bipin Indurkhya. Getting to know your neighbors (KYN), explaining item similarity in nearest neighbors collaborative filtering recommendations, UMAP (Adjunct Publication), pp. 59-64. ACM, 2020.
14

---

Published as a conference paper at ICLR 2025
Saemi Moon, Seunghyuk Cho, and Dongwoo Kim. Feature unlearning for pre-trained gans and vaes. In AAAI, pp. 11420-21428. AAAI Press, 2024.
Daye Nam, Andrew Macvean, Vincent J. Hellendoorn, Bogdan Vasilescu, and Brad A. Myers. Using an LLM to help with code understanding. In ICSE, pp. 97:1-97:13, ACM, 2024.
Yurii E. Nesterov. Gradient methods for minimizing composite functions. Math. Program., 140(1): 125-161, 2013.
Jorge Nocedal and Stephen J Wright. Numerical optimization. Springer, 1999.
Long Ouyang, Jeffrey Wu, Xu Jiang. Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback. In NeurlPS, 2022.
Martin Pawelezyk, Seth Neel, and Himabindu Lakkaraju. In-context unlearning: Language models as few shot unlearners. arXiv preprint arXiv::311..7559, 2023.
Michael J. Pazzani and Daniel Billsus. Content-basee recommendation systems. In The Adaptive Web, volume 4321 of Lecture Notes in Computer Science, pp. 325-341. Springer, 2007.
Barak A. Pearlmutter. Fast exact multiplication by the hessian. Neural Comput., 6(1):147-160, 1994.
Youyang Qu, Ming Ding, Nan Sun, Kanchana Thilakarathna, Tianqing Zhu, and Dusit Niyato. The frontier of data erasure: Machine unlearning for large language models. CoRR, abs/2403.15779, 2024.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Arun Raja. Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal V. Nayak, Debajyoti Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han Wang, Matteo Manica, Sheng Shen, Zheng Xin Yong. Harshit Pandey, Rachel Bawden, Thomas Wang, Trishala Neeraj, Jos Rozen, Abheesht Sharma, Andrea Santilli, Thibault Févry, Jason Alan Frics, Ryan Techan, Teven Le Scao, Stella Biderman, Leo Gao. Thomas Wolf, and Alexander M. Rush. Multitask prompted training enables zero-shot task generalization. In ICLR. OpenReview.net, 2022.
Ayush Sekhari, Jayadev Acharya, Gautam Kamath, and Ananda Theertha Suresh. Remember what you want forget: Algorithms for machine unlearning. In NeurlPS, pp. 18075-18086, 2021.
Shai Shalev-Shwartz and Shai Ben-David. Understanding Machine Learning From Theory to Algorithms. Cambridge University Press, 2014.
Wenbo Shang and Xin Huang. survey of large language models on generative graph analyties: Query, learning, and applications. CoRR, abs/2404. 14809, 2024.
Leheng Sheng, An Zhang, Yi Zhang, Yuxin Chen, Xiang Wang, and Tat-Seng Chua. Language models encode collaborative signals in recommendation, arXiv preprint arXiv.2407.0544/ 2024.
Jonathan Richard Shewchuk et al. An introduction the conjugate gradient method without the agonizing pain. 1994.
Jiabin Tang. Yuhao Yang, Wei Wei, Lei Shi, Lixin Su, Suqi Cheng, Dawei Yin, and Chao Huang. Graphgpt: Graph instruction tuning for large language models. In SIGIR, pp. 491-500. ACM, 2024.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurélien Rodriguez, Ar- mand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language models. CoRR, abs/2302, 13971, 2023a.
15

---

Published as a conference paper at ICLR 2025
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton-Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurélien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and fine-tuned chat models. CoRR, Abs/2200.09228, 2023b.
Vladimir Vapnik. Statistical learning theory. John Wiley & Sons google schola, 2:831-842, 1998.
Hangyu Wang, Jianghao Lin, Bo Chen, Yang Yang, Ruiming Tang, Weinan Zhang, and Yong Yu. Towards efficient and effective unlearning of large language models for recommendation. CoRR, abs/2403.03536, 2024.
Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoe V. Le. Finetuned language models are zero-shot learners. In ICLR. OpenReview.net, 2022.
Jiancan Wu, Xiangnan He, Xiang Wang, Qifan Wang, Weijian Chen, Jianxun Lian, and Xing Xie. Graph convolution machine for context-aware recommender system. Frontiers Comput. Sci., 16 (6):166614, 2022.
Jiancan Wu, Yi Yang, Yuchun Qian, Yongduo Sui, Xiang Wang, and Xiangnan He. GIF: A general graph unlearning strategy via influence function. In WWW, pp. 651-661. ACM, 2023.
Jiancan Wu, Xiang Wang, Xingyu Gao, Jiawei Chen, Hongcheng Fu, and Tianyu Qiu. On the effectiveness of sampled softmax loss for item recommendation. ACM Trans. Inf. Syst., 42(4): 98:1-98:26, 2024a.
Junkang Wu, Yuexiang Xie, Zhengyi Yang, Jiancan Wu, Jinyang Gao, Bolin Ding, Xiang Wang, and Xiangnan He. 8-dpo: Direct preference optimization with dynamic B. In NeurlPS, 2024b.
Mingrui Wu, Jiayi Ji, Oucheng Huang, Jiale Li, Yuhang Wu, Xiaoshuai Sun, and Rongrong Ji. Evaluating and analyzing relationship hallucinations in large vision-language models. In ICMI.. OpenReview.net, 2024c.
Xin Xin, Tiago Pimentel, Alexandros Karatzoglou, Pengjie Ren, Konstantina Christakopoulou, and Zhaochun Ren. Rethinking reinforcement learning for recommendation: A prompt perspective. In SIGIR, pp. 1347-1357. ACM, 2022.
Junjie Xu, Zongyu Wu, Minhua Lin, Xiang Zhang, and Suhang Wang. LLM and GNN are comple- mentary: Distilling LLM for multimodal graph learning. CoRR, abs/2406.01033. 2024a.
Xuhai Xu, Bingsheng Yao, Yuanzhe Dong, Saadia Gabriel, Hong Yu, James A. Hendler, Marzyeh Ghassemi, Anind K. Dey, and Dakuo Wang. Mental-llm: Leveraging large language models for mental health prediction via online text data. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 6(1):31:1331:33 2024b.
Wenqian Ye, Guangtao Zheng. Yunsheng Ma, Xu Cao, Bolin Lai, James M. Rehg, and Aidong Zhang. Mm-spubench: Towards better understanding of spurious biases in multimodal llms. CoRR, abs/2406. 17126, 2024.
Charles Yu, Sullam Jeoung, Anish Kasi, Pengfei Yu, and Heng Ji. Unlearning bias in language mod- els by partitioning gradients. In ACL (Findings), pp. 0032-6048. Association for Computational Linguistics, 2023.
16

---

Published as a conference paper at ICLR 2025
Elad Ben Zaken, Yoav Goldberg, and Shauli Ravfogel. Bitfit: Simple parameter-efficient fine- tuning for transformer-basee masked trgguage-mdeele ACL (2), pp. 1-9. Association for Computational Linguistics, 2022.
Biao Zhang, Zhongtao Liu, Colin Cherry, and Orhan Firat. When scaling meets LLM finetuning:
The effect of data, model and finetuning method. In ICLR. OpenReview.ner, 2024a.
Qingru Zhang, Minshuo Chen, Alexander Bukharin, Pengcheng He, Yu Cheng, Weizhu Chen, and Tuo Zhao. Adaptive budget allocation for parameter-efficiet fine-tuning. In ICLR. OpenRe- view.net, 2023.
You Zhang, Jin Wang. Liang-Chih Yu, Dan Xu, and Xuejie Zhang. Personalized lora for human- centered text understanding. AAAI, pp. 19588-19596. AAAI Press, 2024b.
Yushun Zhang, Congliang Chen, Naichen Shi, Ruoyu Sun, and Zhi-Quan Luo. Adam can converge without any modification update rules. In NeurlPS, 2022.
Yuyue Zhao, Jiancan Wu, Xiang Wang. Wei Tang, Dingxian Wang, and Maarten de Rijke. Let me do it for you: Towards LLM empowered recommendation via tool learning. In SIGIR, pp- 1796-1806. ACM, 2024.
Cai-Nicolas Ziegler, Sean M. McNee, Joseph A. Konstan, and Georg Lausen. Improving recom- mendation lists through topic diversification. In WWW, pp. 22-32. ACM, 2005.
A OVERVIEW OF EXISTING LLM UNLEARNING METHODS
SISA (Bourtoule et al., 2021): It works by dividing the training dataset into partitions, allowing for targeted unlearning of specific instances. The methodology typically involves the following steps: data partitioning, retraining, and aggregation. However, notable limitation of SISA is that it does not preserve the model urchitecture and requires retraining of sub-models, which can lead increased computational costs.
FairSISA (Kadhe et al., 2023): FairSISA improves upon SISA by incorporating fairness hancements. It still relies on the paradigm of retraining sub-models to handle unlearning requests. This approach inherently alters the model architecture and necessitates the retraining of the sub- models, which can limit the flexibility and efficiency of the unlearning process.
APA (Hu et al., 2024c): This study introduces the first exact unlearning approach for large lan- guage model-based recommendation (LLMRec), focusing the removal of personal data to com- ply with privacy regulations. The Adapter Partition and Aggregation (APA) method is proposed, which combines data partitioning with parameter aggregation to reduce inference latency while maintaining performance. This approach enables efficient unlearning without incurring the ex- tra costs typically associated with traditional methods. However,it can affect the integrity of the adapter structure and necessitates retraining of sub-models.
Gradient Ascent: It utilizes the gradient of the target instance to fine-tune the adapter by moving in the direction of the negative gradient of the deleted data. However, this approach is not effective for input modification and output correction tasks, as gradient ascent of target instances cannot adequately handle these scenarios.
EUL (Chen & Yang, 2023): This work introduces lightweight approach for LLMS to efficiently forget specific information without complete retraining. It incorporates unlcarning layers into transformer architectures, utilizing selective teacher-student formulation, and employs fusion mechanism to combine multiple unlearning layers into a unified layer. This enables LLMs to dynamically handle sequence of deletion requests while maintaining model performance. The introduction of adapters alters the model's structure, and the KL divergence- based methods are only effective for instance removal tasks, obtaining a model trained on clean data is not feasible.
E2URec (Chen & Yang, 2023): This method uses lightweight LoRA modules and teacher- student framework to forget specific data while maintaining performance. However, the extra LoRA module changes the original model architecture. and the teacher-student framework quires pretraining on both retained and forgotten data, which is intricate and cannot perform well other tasks like editing.
17

---

Published as a conference paper at ICLR 2025
B ALGORITHM FOR CALCULATING PARAMETER CHANGES
The algorithm for calculating the parameter changes Task can be found in Algorithm This algorithm accelerates the computation of parameter changes resulting from unlearning requests and applicable large-scale data scenarios.
Algorithm Calculate Parameter Changes A
1: Input: target data, train data loader, old adapter, loss fun, n, Task, Dinit Air
2: Output: Parameter changes TO
3: if Task IR then
4: - 1/- > (aay)) (ar, y)
5: else if Task = RC then
6:
else if Task Sm. (xx v) n ( G( 8z, v)
7:
IM then
8:
b +
E Sac ((r - Ee G(z, 0y)
9: end if
A +- initialize( (nnit)
11: oplimizer <- Adam [A], Atr)
12: while not converge do
13: data get.batch((rrinnnddaa...ooo
14:
batch - loss fun data.y)
15:
batch
((batch loss, old adapter parameters())
16:
hop + V(bbtch.ggrat old oldooddddddd adapter
o.t..adattee.pprrameeeeee parameters()) output
pplianizer.zerogrradl.
18:
funve - (hop, p) (b.p)
funv_vala wmn.vvalue.backuue wuno.ualue.backuuadd backward((
20:
optimizer.ster stee((
end while
22:
Return Parameter changes A =
C EXPERIMENTAL DETAILS
In this section, we briefly introduce the tasks used to validate LLMEraser on the unlearning tasks for IR, QM, and RC, as discussed in Section 4. These tasks are designed to assess LLMEraser's 's effee- tiveness handling unlearning scenarios, where specific instances or data are removed or corrected when certain unlearning request arrives.
For LLM4Rec unlearning tasks, our implementation is based on two representative PEFT meth- ods: TallRec (Bao et al., 2023) for item rating, and LLaRA (Liao et al., 2024) for item ranking. Specifically, we frame the rating tasks (TallRec) as binary classification problem, predicting whether or not the user prefers a target item. We employ AUC as the evaluation metric. For the ranking tasks (LLaRA), which recommend items to users from a candidate set, we utilize HitRa- tio@1 and ValidRatio evaluate the relevance of recommended items among all candidates and the proportion of effective responses separately.
terms of MILMs unlearning tasks, we focus on hard hallucination mining, egg. understanding of relation (Wu al., 2024c) and spurious biases (Ye al., 2024). We structure the evaluation as binary or multi-choice classification problems, which aim to select the ground-truth from the noisy labels. Specifically, for relation understanding, we follow (Wu al., 2024c) present the Recall, FI-Score, Precision, Classification accuracy, Yes ratio as the evaluation metrics. For spurious biases, we follow (Wu et al., 2024c) to show the classification accuracy for 9 types of spurious correlations, which is Background (BG), Texture and Noise (TN). Co-occum Co-occurring Objects (CO), Relative Size (RS), Colorization (Col.), Orientation (Ori.), Lighting and Shadows (LS), Perspective and Angle (PA), and Shape (Sha.).
Datasets: Our experimental datasets for LLM4Rec unlearning tasks include three commonly used recommendation datasets: BookCrossing (Ziegler et al., 2005), Movielenn (Harper & Konstan,
18

---

Published as a conference paper at ICLR 2025
2016), and LastFM (Cantador et al., 2011). We follow the data preprocessing and dataset partitioning as described in (Bao et al., 2023) and (Liao et al., 2024). For MLLMs unlearning tasks, we utilize MMSpuBench (Ye et al., 2024), and R-Bench (Wu et al., 2024c) with the representative masked instances for evaluation, partitioning the data into training (60%), validation (20%), and testing (20%) set.
Baselines: We carefully select the following methods for comparison. Original: The original model without unlearning modifications. Retrain: It retrains the adapters using the dataset after correc- tion or removal. SISA (Sekhari et al., 2021): It divides the training data into disjoint shards and subsequently retrains sub-models (adapters) associated with the shards containing unlearning data. RecEraser (Chen et al., 2022): An enhancement of SISA, refining the aggregation strategy and taking into account collaborative signals during data partitioning. Gradient Ascent: finetunes adapters using the reverse gradients of the deleted data. E2URec (Wang et al., 2024): An approach implement instance removal based on KI. divergence within teacher-student framework.
D ESTIMATION ERRORS ANALYSIS OF LLMERASER
The approximation emrors in LLMEraser consist of two primary components: first, the errors in- troduced by the Taylor expansion approximation in the derivation of the influence function, where high-order terms neglected; and second, the errors arising from the new algorithm proposed in Section 3.3 for solving the inverse Hessian-vecto-produue. We will conduct the error analysis in two parts accordingly.
D.1 ERRORS ANALYSIS FOR TAYLOR EXPANSION APPROXIMATION
Without loss of generality, we consider approximation error in Equation 6. other words, we will I)
analyze the error ()(+))
The derivation below follows from (Zhang al., 2022), where we assume that H& is invertiable. As we discussed in our paper, this can be guaranteed if the second-order sufficient condition holds at é.
Since (eww is an optimal solution to the perturbed loss function defined in Equation 5, we have
V RZz; (c)) + eVVL((z, y); Onew (e)) = 0.
Since θ now (c) θ when is sufficiently small, it follows from the Taylor expansion that
= (2:(Z + cVol((x, y): θ) + [g + <V ()]AO(c) + o(I)))(c)))
Since Ô is optimal solution the loss function defined in Equation 3, we have VeR(2;)) 0. Therefore,
AO -Hg +
Since θ is optimal solution to the loss function defined in Equation 3, H3 is positive semidefinite. Therefore, the assumption that Hg invertiable implies that Hg is positive definte. Therefore, we know that
A0(c) - - (H- +(4)) + o( (0())))
Therefore, as c c 0,
+
611 = o(lc) + o (C()I) - 0.
In applications, know that O1/nn) where is the number of training samples. There- fore, e should be very small and our approximation to AO(e) by the influence function should be accurate for applications with very large training datasets.
19

---

Published as a conference paper at ICLR 2025
Table 7: Memory usage (measured in megabytes, MB) for different LoRA ranks (8, 16, 32) on the QM task, using LLaRA as the LLM4Rec model on the LastFM dataset, where 10% of users have items replaced with noisy interactions.
Method
LoRA r
LoRA 16
LoRA r= 32
Retrain
33040 MB
33868 MB
34128 MB
SISA
33040 MB
33868 MB
34128 MB
LLMEraser (Ours)
30760 MB
31386 MB
31834 MB
D.2 ERRORS ANALYSIS FOR OUR PROPOSED ALGORITHM
For our proposed Algorithm, the estimation errors analysis is as follows. For a given (approximate) solution Á to the Equation 12, the error is defined as
err(A) ( = (() bI = IVF(A)].
where the function F-) defined in Equation 14. Therefore, the theoretical analysis of err( is equivalent to the error analysis of |V((A)) for the sequence {A, hr2 generated by the optimiza- tion algorithm for solving the problem Equation 9, Equation 10, and Equation 11.
Since we use ADAM as default optimizer for solving Equation 9, Equation 10, and Equation 1 11, we analyze the error F for the sequence (ár generated by ADAM. follows from (Zhang et al., 2022) that ADAM can converge without modifications if the hyper-parametery are appropriately chosen (say the default choice 31 = 0.9, B2 = 0.999).
Moreover, under reasonable assumptions (see (Zhang et al., 2022) for more details), it holds that
SIST min B VF(A))) ([LloT//Ii) = Ö(1/VT).
Since for sufficiently large T, log < T for any q > 0, we know we can achieve
min E VF(A))) c SIST
for small e > 0 in Ō 3) ~ () iterations. This proof also ensures the convergence of the algorithm proposed Section 3.3.
E DISCUSSION ABOUT THE EFFICIENCY OF LLMERASER
Our proposed algorithm in Section 3.3 for computing the parameter changes not only accelerates the calculation of parameter changes but also significantly reduces GPU memory consumption. As highlighted in our paper, while Conjugate Gradients (CG) is an effective method for computing parameter changes, it requires full-batch computation (Agarwal et al., 2016), which is infeasible for LLMs. Our new algorithm overcomes this limitation, making practical compute adapter's parameter changes in the context of LLMs.
Specifically, LLMEraser formulates the parameter updates as an inverse vussiaa-veccto--roddue (Equation 9, Equation 10, and Equation 11). Importantly, although the inverse Hessian appears in the formulation, does not require explicit computation or inversion of the Hessian matrix. Di- -eccoo--roouuuu
rectly calculating the inverse Hessian-vector product has time complexity of O(p²) and space complexity of Op²), as the Hessian matrix needs to be stored-making it highly memory-intensive.
Our method transforms the computation of the inverse solving for the ceessiin-veeto--rrooeee The Hessian-vector perurpuc if computed directly via the full Hessian matrix multiplication, would have a time and space complexity of O(²²). However, using HVP (Hessian- free methods), we avoid the explicit computation and storage of the Hessian matrix, reducing both time and space complex- ity to (p) (Pearlmutter, 1994). By further leveraging mini-bated optimization for Equation 12, LLMEraser achieves space complexity of O(p), ensuring its scalability.
coriian-veeto--rrdduee into the problem of enabling efficient resolution through mini-batch algorithms.
The results for the LastEM dataset using the LLaRA backbone with LoRA ranks of 8, 16, and 32 shown in the Table 8.
20

---

Published as a conference paper at ICLR 2025
Table 8: Experimental results on the QM task for different LoRA ranks (8, 16, 32), using LLaRA as the LLM4Rec model on the LastFM dataset, where 10% of users have items replaced with noisy interactions. "Corrupted" refers the model trained with the noisy data.
LoRA =
LoRA 16
LoRA r 32
Method
HitRatio@)
ValidRatio HitRatio@@1
ValidRatio HitRatio@ ValidRatio
Retrain 0,4508
1.0000
0.4417
0.9836
0.4215
0.9918
Corrupted
0.4344
0,9918
0,4098
1.0000
0.4016
1.0000
LLMEraser 0.4426
1.0000
0.4344
1.0000
0.4180
1.0000
Table 9: Execution time (measured seconds) for different LoRA ranks (8, 16, 32) on the QM task, using LLaRA as the LLM4Rec model on the LastFM dataset, where 10% of users have items replaced with noisy interactions.
Method
LoRA r 8
LoRA r= 16
LoRA r= 32
Retrain
1.68 x 10*
1.69 x 10*
1.69 x 104
LLMEraser (Ours)
1.50 X 103
1.53 x 103
1.56 x 103
We can observe that LLMEraser effectively reduces the negative impact of noisy data and brings significant utility gain. The HitRatio @ improves by an average of 4.9%, and the performance is comparable that of Retrain. This demonstrates that L.MEErasr can effectively forget and correct the adverse effects caused by noisy data.
Regarding GPU memory usage, we measure the GPU utilization of the LLaRA backbone with LoRA rank sets to 8, 16, and 32. The statistical information and the experimental results (with memory usage measured in megabytes (MB) are shown Table 7.
The GPU utilization of SISA is identical to that of Retrain because SISA (Kwak et al., 2017) ef- fectively requires retraining all parameters (We report the memory usage required to train a single shard). Similarly, fine-tuning- based methods such as gradient descent also necessitates updating all parameters. The backbone of the LLM we used LLaMA2-7B (Touvron et al., 2023b).
The runtime results for LoRA with ranks 8, 16, and 32 on the LastFM dataset are shown in Table 9. The evaluation is measured in seconds.
In summary, the time and space complexity of LLMEraser both Op)). where p represents the number parameters. This indicates that LLMEraser is highly efficient in terms of both time and space, as its performance scales linearly with the number of parameters. This efficiency makes LLMEraser a suitable choice for real-world applications where computational resources and time critical considerations.
F RELATED WORK
F.1 LARGE LANGUAGE MODELS
Recent advancements in natural language processing (NLP) (Nam et al., 2024; Jin et al., 2024) have been significantly driven by the development of pretrained language models and Large Language Models. The introduction of models like BERT (Devlin et al., 2019) and GPT-2 (Radford et al., 2019) marked pivotal shift in leveraging large-scale unsupervised pretraining, enabling superior performance across various NLP tasks through fine-tuning. The scaling of language models led to the emergence of I.L.ss such as GPT-3 (Brown et al., 2020) and PaI. (Chowdhery et al., 2023), which have pushed the boundaries of language understanding and generation. These models, with billions of parameters, are capable of performing complex reasoning and handling diverse tasks with minimal instruction.
Recent research has explored parameter-efficient fine-tuning techniques, which adapt large mod- els specific applications without requiring extensive computational resources. Techniques like Adapter modules (Houlsby et al., 2019) and Low-Rank Adaptation (LoRA) (Hu et al., 2022) have gained popularity for their efficiency and effectiveness in maintaining performance while reducing
21

---

Published as a conference paper at ICLR 2025
the number of trainable parameters. Furthermore, instruction tuning (Liu et al., 2023a: Tang al., 2024) using domain-speciff data has emerged as a key strategy to enhance model performance in specialized contexts. Works by Ouyang et al. (2022) and Dodge et al. (2020) illustrate how tai- loring models to specific tasks through targeted instruction can significantly improve their utility, particularly in complex domains, demonstrating the importance of context and relevance in model training.
LLMs have found extensive applications in various downstream tasks (Fang et al., 2024b; Hu al., 2024a; Wu et al., 2024b), demonstrating their versatility across domains such as natural language processing, information retrieval, and knowledge graph augmentation (Zhang et al., 2024a; Xu al., 2024b; Fang et al., 2024a; Sheng et al., 2024). For instance, LLMs are employed to enhance the accuracy of query-based systems by leveraging their ability understand and generate contextually relevant responses, improving user experience in search applications (Liu et al., 2024b; Shang & Huang, 2024). Additionally, they are utilized in graph analyties, enabling complex reasoning tasks and facilitating the extraction of insights from structured data (Chen et al., 2023; Xu et al., 2024a). The adaptability LLMs through prompt engineering further supports their deployment specific cases, allowing for tailored outputs that meet diverse requirements (Arawjo et al., 2024; Cain, 2024).
In similar vein, LLMs are increasingly being integrated into recommendation systems, building on their capabilities in natural language processing and understanding user preferences. Traditional recommendation systems often rely collaborative filtering (Misztal-Radecka & Indurkhya, 2020; Wu et al., 2024a), content-based approaches (Pazzani & Billsus, 2007; Wu et al., 2022), or hybrid models (Burke, 2002). Recent advances, including Reinforced Prompt Personalization (Mao al., 2024; Xin et al., 2022), and the incorporation of LLMs into recommendation systems via tool learn- data
ing (Zhao al., 2024; Dehbozorgi et al., 2024) or fine-tuning with recommendation-speetnn (Kong et al., 2024: Chen et al., 2024), have significantly improved personalization. These methods enable I.IMS better capture user preferences and context (Lyu et al., 2024; Hu et al., 2024b), ultimately enhancing the accuracy and relevance of recommendations.
F.2 LARGE LANGUAGE MODELS UNLEARNING
The concept of unlearning in Large Language Models has garnered considerable attention as con- cerns over data privacy and model integrity have intensified. In-context unlearning, proposed by Pawelczyk et al. (2023). allows the selective removal of data points by supplying flipped labels during inference, effectively maintaining performance while unlearning specific information. Ad- ditionally, Quark by Lu et al. (2022) employs reinforcement learning framework to control and reduce undesirable behaviors, enhancing text generation without extensive retraining.
Chen & Yang (2023) introduce lightweight unlearning method that integrates unlearning layers into transformer architectures, facilitating efficient data removal. Knowledge Unlearning by Jang et al. (2023) demonstrates that targeted gradient ascent can effectively forget sensitive informa- tion, surpassing traditional methods in performance retention. The technique proposed by Eldan & Russinovich (2023) facilitates the removal of specific facts related to the Harry Potter series while preserving the model's 's overall performance.
Other approaches, such as the Partitioned Gradient Update (PGU) method by Yu et al. (2023), aim to reduce social biases effectively. Collectively, these studies underline the significance unlearning in LLMs, paving the way for safer, more responsible AI applications.
G MORE EXAMPLES OF VARIOUS UNLEARNING TASKS
22

---

Published as a conference paper at ICLR 2025
Instance Removal ase Study
Given user's historical cesteractions-llllnee seeerraddiinss-lleeeeedeeeiiii detemiiee whether the user will enjoy the -oo mov-oe- answering "Yes" or "Ne". User's liked items: GedFather: User's disllikddd items Star Wars,
Query
Target movieeie Iron-Man
Response No.
Query Modification Case Study
This user has watched:
The Rich Man's Wife [emb], Air Force One [emb], Murder 1600 [emb], Absolute Power in the previous [emb]. Please predict the next movie this user will watch. Choose the answer from the following 10 movie titles: Face/Off [emb], Primal Fear [emb], Ransom [emb], Men in Black [emb], Twelve Monkeys [emb], Lone Star [embl, Mr. Holland's Opus [emb], Jackie Chan's First Strike [emb], Waiting for Guffman [emb], The Long Kiss Goodnight [emb].
Query
Answer:
Response
Face/Off
After Query Modification
This user has watched: Rheh-Rich-ane Man's Wife [emb)), Air Force One [emb], Murder at 1600 [emb], Absolute Power in the previous [emb]. Please predict the next movie this user will watch. Choose the answer from the following 10 movie titles: Face/Off [emb], Primal Fear [emb], Ransom [emb], Men in Black [emb], Twelve Monkeys [emb), Lone Star [emb), Mr. Holland's Opus [emb), Jackie Chan's First Strike [emb), Waiting for Guffman [emb], The Long Kiss Goodnight (emb).
Query
Answer:
Figure Instance Removal Case Study & Query Modification Case Study.
23

---

Published as a conference paper at ICLR 2025
Response Correction Case Study
Query
the elephant in red mask standing next tree in green mask?
Response Yes
After Response Correction
Response No
Response Correction Case Study
You are helpful assistant that can answer questions for an image. will provide you 4 options. 'n Rspoonse FormatinChoice: A single character from A, B, C, D. DAnWhhic feature best indicates the identity of the object that has a floral pattern and placed on chair?'nChoices:A The object's soft textureinB. The indoor setting'nC. The wooden chairinD. The background clutter
Query
Response
After Response Correction
Response
Figure 5: Response Correction Case Study.
24


Reviews:
[{"review_id": "fvU2msbyHj", "reviewer": "Reviewer_W27P", "summary": "This paper suggest a unified and feasible way to address multiple unlearning scenarios under a single framework. It first generalizes three unlearning objectives. Then, proposes to use influence-functions to approximate the influence of datapoints requested to be unlearned. It then proposed to use PEFT to avoid full finetuning of the model, by training each adapter separately.", "questions": "In the limitation section is mentioned a limited due to \"first-order Taylor expansion\", but I thought influence functions rely on second order expansion, which one is it?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 2, "strengths": "The paper unifies three unlearning objective under a single simple framework.\nThe paper directly addresses the complexity concerns the framework seems to have in two simple ways: using fast Hessian-vector product, and using mini-batches. It then provides a convincing argument for why such problem formulation would converge using SGD.", "weaknesses": "The paper is a bit hard to read. I would suggest putting most equations in the appendix and focus on high-level explanation in the body of the paper. It is unclear exactly what exactly how adapters are used in Algorithm 1. The evaluation is a bit limited, consider comparing this method performance to other unlearning approaches. Evaluating efficiency in seconds can be a nice addition but the main evaluation should compare an asymptotic running-time complexity.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4NUxo0lsbF", "reviewer": "Reviewer_8gYQ", "summary": "This paper introduce a novel instance-wise unlearning method that utilizes influence functions to adjusting parameters for unlearning tasks. Specifically, the method can remove specific sample from the training dataset, adjust input tokens in user queries, as well as correct model response. Experimental results show that the method can perform unlearning tasks with high accuracy while ensuring efficiency.", "questions": "see weakness. Also, how is the performance of the method on text-generation tasks? Could you evaluate your approach on some QA datasets?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The problem is important and interesting. The three unlearning tasks are practical and meaningful in real-world settings.\n\n2. The unlearning approach edits model parameters based on influence functions, which avoids finetuning or re-training the models\n\n3. The evaluations are comprehensive. \n\n4. The writing is clear and easy to follow.", "weaknesses": "My concern is mainly about evaluation datasets. \n\n1. Although the paper claims to address privacy concerns in LLMs, the authors primarily evaluated the method using tabular datasets for recommendation tasks and a multimodal dataset for a classification task. While these tasks are representative, they are relatively simple and straightforward. Although the authors applied prompt engineering to those data, the training data are still very similar. \n\n2. In real AI applications, LLMs are used for text generation more often, and the training data consists of documents from various sources and may contain sensitive information, such as bank accounts, addresses, and SSNs. Thus, this paper lacks evaluations in more practical and meaningful scenarios that would better demonstrate its effectiveness in addressing privacy concerns in real-world settings.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PHMcks1npD", "reviewer": "Reviewer_8XJc", "summary": "This paper studies machine unlearning for parameter-efficient fine-tuning (PEFT) of large language models (LLM). The authors first propose a framework called LLMEraser for three different unlearning tasks: instance removal, query modification, and response correction. The core of LLMEraser is based on the influence function, based on which the algorithm can minimize the effect of the removed or modified samples. Some experiment results are presented to show the effectiveness of the proposed algorithm.", "questions": "1) How effectively can the proposed algorithm let the model forget the samples?\n2) How effectively can the proposed algorithm maintain the model performance on generative tasks?\n3) If a sample is also used in the pre-training, can it still be unlearned? \n4) How can we tell whether the unlearning (or failure to unlearn) is on the PEFT module but not on the pre-trained model?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1) The paper shows the motivation and derivation of the proposed algorithm/framework.\n2) The proposed algorithm demonstrates that it can effectively maintain good performance on different tasks, including recommendation and relation mining tasks.\n3) The proposed algorithm is also shown to have high efficiency compared to the existing algorithms.", "weaknesses": "1) All the experiments show how well the models can perform after unlearning a certain amount of training samples. However, it seems the paper does not present or compare how effectively the algorithms can let the model \"forget\" those training samples, which is the original goal of the unlearning algorithm.\n2) As PEFT is fine-tuning the model, it is unclear how to distinguish the influence of a sample when it is in the fine-tuning training set or the pre-training training set. When the sample was also used in the pre-training set, it is also unclear whether it is reasonable to require the PEFT to eliminate those \n3) The experiment settings in the paper do not include the generative tasks of LLMs.\n4) Some minor typos, such as \"handling handle\" near the end of page 9.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HQFCFQNbtE", "reviewer": "Reviewer_nWJj", "summary": "This paper introduces LLMEraser, a unified framework for parameter-efficient unlearning in large language models (LLMs), specifically tailored to address privacy and security concerns in domain-specific fine-tuning. The framework utilizes influence functions to perform instance-wise unlearning tasks such as instance removal, query modification, and response correction. LLMEraser allows unlearning without requiring full model retraining, and has demonstrated efficacy in preserving model performance across various unlearning tasks.", "questions": "1. Could the authors provide a detailed analysis of estimation errors to demonstrate their impact on performance? Given the importance of precision in unlearning tasks, such an analysis would help establish whether the errors introduced by the method are indeed within an acceptable range and do not significantly detract from the efficacy of LLMEraser in high-precision unlearning scenarios.\n\n2. The LLM4Rec unlearning tasks are different from those unlearning tasks used in the baseline paper. Could the authors provide performance results on additional unlearning tasks more closely aligned with those proposed in other baselines? Additionally, how do the LLM4Rec tasks reflect the generalizability of LLMEraser across diverse unlearning scenarios? \n\n3. While LoRA was used in the experiments, it would be helpful to know the specific experimental setup, including the number of trainable parameters. Since computational complexity is directly related to the number of trainable parameters and rank values in LoRA, could the authors clarify these settings and present results across different ranks? Additionally, could they specify the sample sizes used in unlearning experiments, as sample size likely affects the overall complexity?\n\n4. While the method demonstrates computational efficiency, I am concerned about memory efficiency, particularly for broader applications. Could the authors provide an analysis of the memory cost and specify whether LLMEraser could run on a lower-memory GPU than an Nvidia A100? If the method is limited to high-memory GPUs, this could hinder its practical application. A comparison of memory usage with baseline methods is also important to clarify this point.\n\nI would be happy to engage with the authors to help improve the presentation of the method and evaluation, but my concerns are not insignificant. Clarifications would need to resolve my questions in order for my score to improve.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is written clearly and is easy to follow, with clearly presented formulations.\n2. The taxonomy of unlearning tasks is well-defined, providing clarity on the types of unlearning scenarios addressed. \n3. Extensive experiments are conducted across diverse unlearning tasks with both LLMs and MLLMs, covering recommendation and multimodal relation mining applications. \n4. LLMEraser accelerates the computation of the inverse Hessian-vector-product in the influence function, enabling efficient implementations. This improvement is valuable for LLM applications, where computational efficiency is a growing concern.", "weaknesses": "1. While LLMEraser reduces the need for retraining, the memory-intensive computation of inverse Hessian-vector products remains demanding. This requirement may limit scalability, particularly for very large models or environments with limited GPU memory.\n2. The method’s reliance on first-order Taylor expansion in influence functions can introduce estimation errors. A lack of detailed error analysis makes it difficult to assess the impact of these errors, especially in tasks requiring high unlearning precision.\n3. Limited generalizability is a concern, as the LLM experiments primarily focus on LLMs for recommendation (LLM4Rec). Broader evaluation across more diverse LLM unlearning tasks would strengthen the claims.\n4. Key experimental details, such as PEFT configurations and the number of trainable parameters, are not fully provided. These details are essential for evaluating the method's complexity and efficacy across different settings.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)