Offset Unlearning for Large Language Models
Authors: James Y. Huang
Wenxuan Zhou
Fei Wang
Fred Morstatter
Sheng Zhang
Hoifung Poon
Muhao Chen
Venue: Accepted by TMLR ()

Abstract: Despite the strong capabilities of Large Language Models (LLMs) to acquire knowledge from their training corpora, the memorization of sensitive information in the corpora such as copyrighted, biased, and private content has led to ethical and legal concerns. In response to these challenges, unlearning has emerged as a potential remedy for LLMs affected by problematic training data. However, previous unlearning techniques are either not applicable to black-box LLMs due to required access to model internal weights, or violate data protection principles by retaining sensitive data for inference-time correction. We propose $\delta$-unlearning, an offset unlearning framework for black-box LLMs. Instead of tuning the black-box LLM itself, $\delta$-unlearning learns the logit offset needed for unlearning by contrasting the logits from a pair of smaller models. Experiments demonstrate that $\delta$-unlearning can effectively unlearn target data while maintaining similar or even stronger performance on general out-of-forget-scope tasks. $\delta$-unlearning also effectively incorporates different unlearning algorithms, making our approach a versatile solution to adapting various existing unlearning algorithms to black-box LLMs.

Published in Transactions on Machine Learning Research (05/2025)
Offset Unlearning for Large Language Models
James Y. Huang
Muanggjammasse edu
University of Southern California
Wenxuan Zhou University of Southern California
showwenz@usc.edd edu
Fci Wang
Juang398@usc..co
University of Southern California
Fred Morstatter University of Southern California
fred edu
Sheng Zhang Microsoft Research
shashan@@microoofff.comm
Hoifung Poon Microsoft Research
hoijung@miicrooff..com com
Muhao Chen
muuhchen@@uaaavis.ccc eduu
University of California, Davis
Reviewed on OpenReview: https: openreview. net/ forum? id= A4RLPHPXCU
Abstract
Despite the strong capabilities of Large Language Models (LLMs) to acquire knowledge from their training corpora, the memorization of sensitive information in the corpora such as copyrighted, biased, and private content has led to ethical and logal concerns. Tn re- sponse to these challenges, unlcarning has emerged as a potential remedy for LLMs affected by problematic training data. However, previous unlearning techniques are either not appli- cable to black-box LLMs due required access to model internal weights, or violate data protection principles by retaining scnsitive data for inferenco-time correction. We propose 8- UNLLARNNNG, offset unlearning framework for black-box LLMs. Instead of tuning the black-box LLM itself, 5-UNLEARNING learns the logit offset needed for unlearning by contrasting the logits from pair of smaller models. Experiments demonstrate that &- UNLEARNING can effectively unlearn target data while maintaining similar or even stronger performance on general nut-of-forget-scopp tasks. 8- J-UNLeARNING also effectively incorpo- rates different unlearning algorithms, making our approach versatile solution adapting various existing unlearning algorithms to black-box LLMss
1 Introduction
Large Language Models (LLMs) are capable of memorizing a large amount of information derived from their training corpus. While LLMs are empowered by the abundance of knowledge they acquire during training, their training data may contain sensitive information that should not. be memorized by LLMs. Previous studies have shown LLMs can reproduce copyrighted materials Chang et al. (2023); Eldan & Russinovich (2023): Karamolegkou al.] (2023), generate harmful and biased content Shaikh et al. (2023), and reveal private information Staab et al. (2024), raising both ethical and legal concerns. The introduction of data
protection regulations such the right to be forgolten Hoofnagle et. al. (2019); Zhang et. al. (0023 Min

---

Published in Transactions on Machine Learning Research (05/2025)
Unlearning Method
Black-box
Privacy
Gradient Ascent
X
Data Relabeling
X
/
In-context Unlearning
/
X
8-UNLEARNING
/
Table 1: Comparison with existing unlearning methods. Previous techniques either require access to LLM's internal weights, retain sensitive information for inference.
et al. 2024g also highlights the need for erasing the influence of problematic data when deploying LLMs in real- world applications.
One potential solution to this challenge is unlearning, where the goal is to "forget" a set of training data without hurting the model's performance on out-of-forget-scope tasks. An exact unlearning approach would roquire retraining the model from scratch with forget set data removed Bannihatti Kumar al.) (2023) However, given the enormous amount of resources required retrain LLMs, it is generally more practical to employ approzimate unlearning techniques that modify the behavior of trained model in post hoc manner. However, most previous LLM unlearning techniques require access to model internal weights Jang
let al.
2023
Eldan & Russinovich (2023); Yao et al.] (2023); Chen & Yang (2023 Meng et al. (0223 Wu
et al. algorithms, Gradient Ascent maximize the likelihood
2023 making them infeasible for black-box LLMs. For example, as two widely used unlearning
forget set data, while Data Relabeling minimizes
the likelihood of relabeled forget set data. Both of these methods require fine- fine-tuning the LLMs. Black-box LLM unlearning useful since this opens up the possibility of modular, customizable unlearning without the need to update the base LLM itself. Alternatively, in-context unlearning Pawelczyk et al. (0223 prompts LLMs with counterfactual forget set instances to steer model behavior at inference time. However, this approach comes with two major limitations. First, model developers still maintain an explicit list of sensitive information to be used during inference. Such practice is not only in violation of privacy regulations but also susceptible to malicious attacks such as prompting leaking Perez & Ribeiro (022).. 1. Second, in-context unlearning cannot effectively deal with an ever-growing set of knowledge be unlearned given the challenges of processing long context with LLMs Li et al. (2024). Tab. summarizes the strengths and weaknesses of
existing unlcarning algorithms.
In this work, propose 5-UNLEARNING, an offset unlearning framework for arbitrary black-box LLM without updating its internal weights. Instead of tuning the black-box LLM itself, 6-UNLEARNING learns the logit offset needed for unlcarning by contrasting the logits from a pair of smaller, whitc-box models. During unlearning, we first compute the logit offset by taking the difference in logits from the two smaller models. Then, we add the logit offset between the two smaller models to the logits of the larger model. The intuition behind this is that we can learn the offset term that approximates how a larger model should modify its prediction in the face of sensitive queries from the behavior adaptation of smaller model. 5- UNLEARNING does not require access to the larger model's internal weights, nor retains any sensitive data for inference after unlearning. Our method also enables flexible version control and customization, since for different unlearning requests we only need to maintain a pool of smaller models, which can be combined with the same base LLM in a plug-and-play manner. This allows us to efficiently curate the pool of knowledge available to different applications using specialized unlearning modules, which largely in line with previous efforts to modularize knowledge access for LLMs but from a different, complementary perspective Feng et al.) h024h.
We evaluate the effectiveness of 5-UNLEARNING on TOFU Maini et al. (2024), a widely used LLM unlearning benchmark containing knowledge about fictitious authors. Experiments show that when targeting the same forget performance, S-UNLEARNING maintains similar or even stronger performance on for-of-forget-ccope data compared to directly fine-tuned larger models while requiring no parameter updates to the larger model.
Our contribution three-fold. First, we propose 8- UNLEARNING, an unlearning framework for arbitrary black-box LIM without modifying its parameters by only fine-tuning smaller model to update the logits
2

---

Published in Transactions on Machine Learning Research (05/2025)
of larger one. Second, 8- B-UNLEARNING can achieve the same level of unlearning as directly fine-tuning the larger model while still matching or even outperforming direct fine-tuming baselines on general tasks outside the unlearning scope. Third, 8- d-UNLEARNING can be integrated into different unlearning algorithms, demonstrating the versatility of our approach.
2 Related Work
In this section, we sunmarize two lines of research that are highly related to our work.
Machine Unlearning for LLM. Prior works have explored machine unlearning as a way to mitigate the influence of undesirable training data on LLMs. Given the vast cost incurred by retraining LLMs from scratch Bannihatti Kumar et al. (2023). most unlearning methods apply post hoc finetuning or adaptation to steer the behavior on the forget set Jang et al. (2023): Eldan & Russinovich (2023); Yao et al. (2023)
Chen & Yang (2023). Gradient ascent based methods fine-tune models by minimizing the likelihood of forget
set data Jang et al. (0023; ; Chen & Yang (2023); Maini et al. (0024). Alternatively, several works proposed to maximize the likelihood of relabelled target data, where the original answer is replaced with a generic, insensitive response Eldan & Russinovich (2023): Patil et al. (2024). Auxiliary training objectives can also be
introduced to maintain model performance on out-of-forgete fot-off-orret-scope data Yao et al. (2023): : Wang et al. (2023) Another related line of research is model editing, where the goal is to identify and alter knowledge captured by local compouents within models Meng et al.) (2023); WW et al. (2023). While both model editing and unlearning attempt to modify the behavior of trained LMs, unlcarning focuses climinating the effect of specific set of training data without necessarily creating new answer mappings Liu et al. (2024c). It is worth noting that all of the aforementioned approaches require access to the model's internal weights. In-context unlearning Pawelczyk et al. (2023), while being applicable to black-box LLMs, still requires storing sensitive information for inference and therefore fails to address data privacy concerns. In this work, we propose an unlearning framework that does not require access to LLM weights, nor storage of sensitive information for inference.
Logit Ensemble. The potential of combining logits from different models has been studied in various context. One line of research focuses on control and improving LLM generation quality by contrasting
SShi the logits from different models or layers at decoding-tine Liu et al. 2022 :: Shi et al. (2023): Li al. (2023);
Chuang et al. 20024.. Logit ensemble has also been shown an effective way of adapting LLMs to various
downstream lasks. Ormazabal et al. (2023) propose to adapt LLMs to different domains through a learned
combination with smaller domain experts. Mitchell et al. (2024) leverage an ensemble of difference-sizes
models to study the effect of pretraining and finetuning at different scales. Concurrently, Liu et al. (2024a) propose Proxy-Tuning that combines the logits from smaller tuned models with larger LLMs to enhance instruction following capabilities. [Lu al. (2024b) ensemble the logits of a main LLM with a paraphrase model that leads to monotonie prompt paraphraser for rewriting prompts with enhanced generalizaion effects. Zhao et al.] (2024) use the logits from unsafe LLMs to guide the jailbreaking of safer LLMs during decoding. In this work, propose to utilize smaller LLMs to capture the logit offset needed for unlearning sensitive data from black-box LLMs while maintaining general performance on out-of-forget-ssooe tasks.
3 Method
In this section, we formulate the unlearning problem ($3.1 discuss the technical details of our d- UNLEARNING framework (.2), and highlight the strength of 5- UNLLERNNNNGG compared to existing methods
$3.3
3.1 Problem Definition
Given a target forget set Sy taken from the training data S of an LLM M, the goal of unlearning is to obtain a new model M' that resembles a model trained without Sy. This implics M' should "forget" all information from the forget set without hurting the performance on out-ot--orgee-scoo data. Ideally, unlearning can be accomplished by retraining M on SY,S,, i.e. the training set with forget set data removed. However, given
3

---

Published in Transactions on Machine Learning Research (05/2025)
Answer with sensitive information
Sensitive Query
5-Unlearning
Unlearning
Answer without sensitive information
Offsee
Black-box LLM
Figure 1: Overview of 8- UNLEARNING. In order to adapt the behavior of a black-box LLM without updating its parameters, combine it with pair of smaller, white-box models (which call offset models). For unlearning, compule the logit offiset of these two models and add it lo the logits of the black-box LLM given the same query. Both of the two offset models are initialized from the same checkpoint, making the logit offset zero initially. The goal of 6-UNLEARNING is to fine-tune one of them such that their logit offset, after being added to the logits of the black-box LLM, can steer its prediction away from generating sensitive information.
the prohibitive cost of retraining the LLM from scratch, it is generally more practical to approximate M' by directly updating M. The unlcarning problem can also optionally include retain set S, on which the model after unlearning should not forget any information and maintain performance.
3.2 Offset Unlearning
d-UNLEARNING is based on the idea a product-ofexperls Hinton (2002) and its subsequent applications to ensemble of language models Liu et al. (2021): Meng et al. (2022) Li et al. (023) Fig. provides an overview U-UNLEAGNIGG.
Suppose want. to unlearn forget set S from LLM M. Instead of directly updating the parameters of M, we introduce pair of smaller, offset models M. and M2 We define their logit offset the difference between the logits of two offsct models M2 and M, given the same query. For unlcarning, we add the logit offset to the logits of M given the same query, essentially forming a logit ensemble of M, M and Mg- Both M, and My are initialized from the same checkpoint, making the logit offset zero for all data initially. During unlearning, we only update the parameters of M2 while keeping M and M, frozen, and use the logit ensemble to generate the final output. In this way, we encourage M2 to deviate from its initialization M. given sensitive query and learn the correct logit offset that applies to the logits of M, steering its prediction away from generating sensitive information. Formally, the logits of the ensemble le are computed as follows:
le (ye 14, }-() = l (Velg. + <<[ar;; (velg. y<+)) - L. (yr 44, yc()),
where las lug, and la, are the logits from their respective models, is the query, and is factor controlling the strength of applying the offsee term to M. Since the logits are in the log space, the additive combination of them can also be interpreted the following product--ffexxerts::
Pas; (vell HcL)
Pe Pe yee) x Pas(Vel, yee)
PMr. (velg. Mee))
Essentially, the probability of each token predicted by A is scaled by the probability ratio between Mý and Mo. which reflects how M2 changes its token distribution relative to its initialization M, after unlcarning. Specifically, when querying non-sensitive, out-of-forgel-scopp information, the probability ratio between M' and M, should close to one, making the token distribution of the ensemble similar to that of the original LLM M. When querying sensitive information that the model should forget, the token distribution of M; differs from that of M, to adjust the probability ratio, thus steering the token distribution of the ensemble away from that of M.
4

---

Published in Transactions on Machine Learning Research (05/2025)
During training, optimize any unlearning objective the prediction of the ensemble instead of on the
original model M. For example, to unlearn the model using Gradient Ascent Jang et al. (2023 Chen & Yang (203)) where the objective is to minimize the likelihood of forget set data, we maximize the following loss function for instance output length 1:
C log Pe (yelg,
3.3 Merits of d-Unlearning
The design of 6-UNLEARNING leads the following key merits.
Applicability to Black-box LLMs. contrast to most previous unlearning methods, UNLEARNING is applicable to not just open-soureed models, but also black-box LLMs without access internal weights. Instead of updating M, d-UNLEARNING only obtains the logits from M, and learns the logit offset needed to adjust its prediction using smaller white-box models.
Privacy Protection. Prior work has proposed in-context unlearning Pawelczyk et al. (2023) make un- learning possible for black-box LLMs. However, a key drawback of this approach is that the model developer still maintains explicit, ever-growing list of sensitive information used to construct queries for unlearning during inference, which defeats the purpose of privacy protection. For comparison, 5- B-UNLAARNIN does not require storage of any explicit sensitive information after unlearning is completed.
Version Control and Customization. 6- B-NNLEARNING also facilitates flexible version control and user customization, as instead of storing multiple versions of the larger model, we only need to keep track of a pool of smaller models. These models can be combined with the same base LLM plug-and-play manner. By using specialized unlearning modules, we can efficiently curate the pool of knowledge available to different applications.
4 Experiment
this section, we provide a description of the evaluation setting ($4.1) a summary baseline unlearning algorithms on which we apply our framework as well as other implementation details 4.2 and the main results 4.3
4.1 Evaluation Setting
We conduct our experiments on TOFU Maini al. (2024), ; a widely used unlearning benchmark designed for evaluating LLMs. The benchmark defines an unlearning task that targets information derived from a collection of fictitious author profiles that do not exist in real world. This creates controlled unlearning setting with well-defined unlearning scope and source of knowledge. TOFU designates Forget Set which contains knowledge about a small subset of fictitious authors that we aim to unlearn. TOFU also includes three other datasets with knowledge about the retained fictitious authors (Retain Set), real world authors (Real Author), and general world facts (World Fact) respectively. Ideally, the model should retain all knowledge had about these three retain datasets before and after unlearning.
The Forget Set evaluates forgel performance, i.e., how well the model removes target information from its memory, while the latter three sets focus retain performance, indicator of how well the model maintains its performance on out-of-forget-scopp data. The latter three sets also represent series of oor-of--orget--scope data with decreasing levels of relevance to the forget set. Generally speaking, it is more challenging for a model to remember out-of-forget-ssoope data that are more relevant to the forget set, a phenomenon known as knowledge entanglement Maini et al. [2024).
We follow the settings outlined in TOFU and report the following metrics for forget performance. ROUGE measures how well the generated output from the LLM matches the correct answer. Specifically,
5

---

Published in Transactions on Machine Learning Research (05/2025)
the ROUGE-L recall score Lin 2004
Probability computes the conditional probability of the correct answer given the prompt. Truth Ratio measures how likely the correct answer is compared to a collection of wrong answers perturbed from the correct answer. Since the model is fine-tuned on one specific phrasing of the correct answer, thus potentially having inflated probability compared to other phrasing with similar meanings, Truth Ratio is computed using a paraphrased version of the original correct answer on the forget set and retain set. Following the original evaluation pipeline, we normalize Truth Ratio so that a. higher truth ratio indicates better unlearning performance. Forget Quality measures the difference between distributions of Truth Ratios based on Kolmogorov-Smirnot test. For retain performance, we report the aggregated Model Utility, which is computed by taking the harmonic mean of ROUGE-L, Probability, and Truth Ratio on all three retain datasets.
As we will demonstrate in $5.1 there is generally a trade-off between forget performance and retain per- formance. For example, a model can have a near-zero ROUGE score on the forget set but completely unusable if the model always outputs gibberish given any prompt. Hence, we need to determine a target for- gel performance as a stopping criterion to facilitate direct comparison between different unlearning methods. In experiments, we use the ROUGE score of the retraining baseline on the forget set the stopping criterion, since retraining corresponds to an ideal sccnario where the model has never been exposed to the forget set
+ Following Yao et al. 20244), we match all models to the target ROUGE score by adjusting the learning rate.
In addition to TOFU, we assess the unlearned model preserves general utilities on well-established bench- marks, including ARC Clark et al. (2018), HellaSwag Zellers et al. (2019). WinoGrande [Sakaguchi et al. [021) and OpenBookQA Mihaaylov et al. (2018). Since solving these general tasks does not involve knowl- edge about the data we aim to unlearn, the model after unlearning should maintain as much performance as possible. We follow the default evaluation setting from
Gao et al. (0023) and report accuracy on all four
tasks under the zero-shot setting.
4.2 Model Configuration
Unlearning Algorithms. 8- 8-UNLEARNING is a general unlearning framework compatible with different ex- isting unlearning algorithms. We compare 5-UNLEARNING with its corresponding direct fine-tuning baseline when incorporated with each of the following commonly used unlearning algorithms. Gradient Ascent Jang et al.
2023); Chen & Yang (2023) minimizes the likelihood of the forget set. Gradient Difference Liu et al. | Yao et al.
(023) minimize forget set likelihood while maximize retain set likelihood. KT. Minimiza- tion Maini et al. [024] penalizes the distributional distance between models before and after unlearming. Dala Relabeling Eldan & Russinovich (2023) trains the model on forget set questions paired with an alter- native answer that abstains from answering the question such as "I don't have that information." We also include the Retraining baseline which fine-tunes the initial model with the forget set excluded, which serves as the upper bound in terms of balancing forget and retain performance.
(2022
Implementation. We run our experiments on the widely used Llama2 model family Touvron et al. (2023) Specifically, we use Llama2-133-a 13b-chat-hf as the larger model and Llama2- 7b-chat-hf as the smaller offset model. Note that while Llama2 models All models are trained using NVIDIA A100 GPUs for 5 epochs with batch size of 32. We set to 1 for our experiments.
4.3
Main Results
Our experimental results on TOFU are shown in [Tb. 2
The model before unlearning cxhibits strong memorization over both the forget set and retain set, indicated by high ROUGE and probability scores. This is as expected since the model is explicitly trained on the full dataset of fictitious authors to simulate model's exposure to private information. Retraining significantly reduces the model's knowledge the forget set while maintaining similar model utility as it is before unlearning. Although retraining would not
By definition the retraining baseline has forget quality of 1.0, representing an upper hound for this metric. Hence, is infeasible to use forget quality as the stopping criterion for forget performance.
6

---

Published in Transactions on Machine Learning Research (05/2025)
TOFU Forget
TOFU Retain
ARC
HS
WG
OBQA
Method
RL (4)
P (4)
TR. (t)
FQ (t)
MU (t)
Acc
Acc
Acc
Acc
Before Unlearning
95.6
98.3
49.5
1.3c-13
62.1
44.0
58.1
67.9
36.2
Retraining
38.9
15.2
65.6
1.0
62.9
45.0
58.5
68.0
34.6
Gradient Ascent
Direct Fine-tuning
38.8
3.4
53.3
2.6e-7
32.7
39.9
56.4
65.2
34.4
8- UNLEARNING
38.6
15.2
57.9
4.0e-6
48.6
42.2
56.3
65.7
32.8
Gradient Difference
Direct Fine-tuning
38.9 38.1
2.1
51.9
1.4e-6
51.4
40.4
56.3
64.9
32.6
8- UNLEARNING
6.2
52.5
6.7e-6
50.5
40.9
55.7
65.2
35.4
KL Minimization
Direct Fine-tuning
39.8
3.1
53.4
1.4e-6
33.5
39.2
56.5
65.0
34.0
A-UNLEARNING
39.6
14.1
57.5
1.8e-5
50.4
43.7
57.2
66.9
34.4
Data Relabeling
Direct Fine-tuning
38.1
92.5
53.3
6.6c-12
56.1
43.5
57.9
68.9
34.6
6-UNLEARNING
36.3
91.5
50.8
3.0e-13
58.6
44.2
58.0
68.0
34.8
Table 2: Results on TOFU and general benchmarks. We report ROUGE-L recall (RL), Probability (P). Truth Ratio (TR) and Forget Quality (FQ) on the Forget Set and Model Uuility (MT) on retain data from the TOFU benchmark. We report accuracy on general benchmarks. Higher scores are better except ROUGE and probability on the Forget Set. Better scores are underlined for each of the four unlcarning strategics.
Retain Set
Real Author
World Fact
Method
RL
P
TR
RL
P
TR
RL
P
TR
Before Unlearnéng
96.3
97.9
51.2
85.2
44.5
55.7
87.7
42.5
56.3
Retraening
95.8
97.7
50.4
89.5
45.8
58.5
85.5
43.0
57.4
Gradient Ascent
Direct Fine-tuning
51.2
8.0
51.6
52.3
43.9
58.3
80.2
44.6
60.6
S-UNEEARNING
41.0
26.1
48.9
75.0
45.3
57.4
82.1
47.0
63.7
Gradient Difference
Direct Fine-tuning
56.8
58.9
55.1
61.4
35.0
47.9
80.4
38.9
53.7
S-UNEEARNING
53.4
47.8
51.9
60.6
36.1
45.9
83.2
41.3
59.1
KL Minimization
Direct Fine-tuning
53.0
51.0
55.8
42.2
56.4
83.3
43.3
58.8
U-NLEAANNNG
46.1
27.9
50.9
80.4
45.1
57.5
84.9
46.3
64.0
Data Relabeling
Direct Fine-tuning
85.0
95.3
48.0
82.5
38.0
46.3
87_7
39.2
49.2
6-UNLEARNING
72.4
95.1
49.6
78.7
41.5
52.6
86.9
42.3
55.5
Table Detailed results on TOFU retain data, namely the Retain Set, Real Author and World Fact. Higher numbers are better for all metrics.
be feasible in real world scenarios, its performance gives us a better understanding of the gap between exact unlearning and post hoc approximate unlearning methods.
We first examine the forget quality of different post-hoc unlearning methods on the TOFU Forget Set. As shown in Tab. 2 both direct. fine-tuning and 6-UNLFARNING can reach level of unlearning similar to

---

Published in Transactions on Machine Learning Research (05/2025)
90
50
80
80
70
ROVISE
70
60-
60
70
Retraining
90
70
Spochs
Forgen
Figure 2: Unlearning trajectory of Gradient Ascent using direct fine-tuning (left), &- S-UNLIARNING (middle), and the tradeoff curve between forget and retain performance (right) over the course of unlearning. For training trajectories we report ROUGE score on all four TOFU datasets. For the tradeoff curve we report Forget Set ROUGE versus Non-forget Set ROUGE score.
retraining in terms of ROUGE score the generated response. Although direct fine-tuning tends to assign lower probabilitics to the correct answer on out of the mothods we investigate, 8-UNLEARNING produces higher truth ratio and forget quality in all three cases. A higher truth ratio is desirable since indicates the presence of other highly likely alternatives, making the correct answer less distinguishable from other wrong answers.
Interestingly, data relabeling retains a very high probability score and very low forget quality despite having a similarly low ROUGE score as other algorithms on the forget set. This is likely due to relabeling being the only method that does not explicitly minimize the likelihood of the original forget set answers.
We then investigate how well the unlearned model maintains its performance on data outside the unlearning scope. On TOFU retain data, 5-UNLEARNING preserves more model utility than direct finc-tuning on out of the methods we compare. In particular, 8- 6-UNLEARNING demonstrates superior retain performance when applying to gradient ascent and KL minimization, beating direct fine-tuning by more than 15 points. Taking a closer look at how these models perform on individual TOFU retain data, we observe in (abb. 3 that direct fino-tuning tends to perform better on the Retain Set, while J-UNLEARNING outperforms direct fino-tuning on the Real Author and World Fact. This indicates slight. divergence between direct. fine-tuning and UNLEARNING in terms of how balance performance across different types of knowledge during unlearning. This is likely result of changing training dynamics with the introduction of offsct models, which we will study in more detail in 1.1
In addition to TOFU, we also evaluate the utility of the unlearned model on general task benchmarks. Performance on these tasks also an important indicator of retain performance, reflecting whether general capabilities of LLMs are preserved after unlearning. As shown in Tab. 2 8- B-UNLEARNING achieves competitive performance on most metrics when compared direct fine-tuning basclines, and closes the performance gap between before and after unlearning. 8- d-UNLEARNING consistently outperforms direct fine-tuning with KL minimization, and bring improvement on out of tasks with gradient difference and data relabeling.
Overall, our experiments demonstrate that J-UNLEARNING is strong alternative to direct fine-tuning, with matching or even superior performance in terms of both forget and retain performance. 6-UNLEARNING is also effective across different unlearning algorithms, showing the versatility of our approach.
5 Analysis
In this section, we provide analyses on the training trajectory of the unlearning process ($ and the effect of varying offset strength ($5.2).
8

---

Published in Transactions on Machine Learning Research (05/2025)
80
60
ROUGE
40
Forget Set
Retain Set
20
Real Author
World Fact
0.2
0.5
Offset Strength (a)
Figure 3: Effect of varying offset strength on the model after 6-UNLEARNING with Gradient Ascent
5.1 Unlearning Trajectory
To better understand how forget and retain performance change over the course of unlearning, we study the training trajectory of both direct fine-tuning and 8 d-UNLEARNING. As shown in Fig. 2] when targeting the same ROUGE score on the forget set, 8- G-UNLNANN exhibits a steeper decline on the forget set imitially compared to direct fine-tuning, which is also coupled with a steeper decline on the Retain Set. As unlcarning progresses, direct fine-tuning starts to lose performance on the Real Author set that the model should not forget, while 8- UNLEARNING still maintains relatively stable performance.
When comparing the training trajectory different TOFU datascts, we clearly observe the varying degrees of knowledge entanglement with the Forget Set. Being the most similar to the Forget Set, Retain Set performance starts to degrade at early stages, followed by the Real Author set. Performance on the World Fact, which is the least relevant to the Forget Set, only declines slightly towards the end of unlearning. This highlights the importance of finding a good balance betweon forget and retain performance for an unlcarning method. We also study this trade-off from a more direct perspective by plotting the curve forget set ROUGE score versus non-forget set ROUGE score in Fig. 2 (right). A desired model should lie the upper right corner, which represents strong forget and retain performance. While Direct fine-tuning maintains more performance retain data initially, 5- UNLLARNNIN achieves better balance at higher unlearning levels direct fine-tuning starts to lose more performance non-forget sets.
5.2 Effect of Offset Strength
As mentioned in $3.2 can adjust the value of to control the strength of the logit offset being added to the larger model's logits. We experiment with using difference a values during inference and study its effect on forget and retain performance. As shown in Fig. 3 A low offset strength makes the effect of logit offset negligible, and the prediction of the ensemble is essentially dominated by the larger model M without unlearning. As we gradually increase the offset strength, the unlearning effect becomes more prominent and forget set ROUGE score decreases significantly. Similar to what we obscrve in Fig. 2 Retain Set performance largely follows the trajectory of Forget. Set, while Real Author and World Fact performance are less influenced by the increase of unlearning offset strength. When we surpass the level of offset strength used in training, we observe continued performance degradation on all four datasets. The ROUGE score on forget set drops below 10 when increases to 5, a score much lower than the retraining baseline (which has a ROUGE score of 38.9). However, at this offset strength the model becomes unusable, indicated by poor performance across all three non-forget set.
We present an example from the Forget Set in Tab. 4 to provide a better understanding of model behavior at different offset strength levels. At a=0.2, the model can perfectly reproduce the answer that is supposed
9

---

Published in Transactions on Machine Learning Research (05/2025)
Example
Sensitive Query
In which geure does Hina Ameen primarily write?
Ground Truth
Hina Ameen primarily contributes to the geology genre.
0.2
Hina Ameen primarily contributes to the geology genre.
0.5
Hina Ameen primarily contributes to the genre of geology. Her extensive knowledge of
a = 1.0
Hina Ameen works primarily in the genre mythology. Her literature has a deep connection...
a 2.0
As the book primarily consist narrations revolved historical Daker period
5.0
As writers focus deep introsvosity embits poert calurity reveiased world literature reflect
Table 4: Example response by 8- N-UNEAANNING on the Forget Set with varying oflset strength during inference.
TOFU Forget
TOFU Retain
Model
RL ())
P (L)
TR (t)
FQ (1)
MU (t)
Retraining
37.2
11.6
57.7
1.0
58.8
7B Direct Fine-tuning
37.4
4.1
50.1
1.2e-4
37.8
7B + 3B Offset
37.3
37.0
46.7
1.5e-7
52.2
7B + 1.5B Offset
36.9
17.5
49.9
6.7e-6
49.1
7B + 0.5B Offset
37.0
14.7
43.8
4.9c-10
48.8
Table 5: Results on different offset model size.
to be forgotten, showing that unlcarning is not taking effect at low offset strength. At a=0.5, the model is still capable of recalling the correct answer, despite in slightly different phrasing, At a=1, we obtain a fluent response with the sensitive information from the ground truth removed, demonstrating success of unlearning. If we further increase offset strength, the model starts to generate gibberish and eventually becomes unusable. In conclusion, using the same offset strength as in training leads to the best results overall.
5.3 Choice Offset Model
To study how the choice of offset model affects unlearning performance, we run a series of experiments using Qwen family models as they offer a wide range of model sizes. Specifically, we apply offset unlearning with gradient ascent to target Qwve2.55-7biinstee
model using 3B, 1.5B and 0.5B offset models. As shown in Tab. 5 all models can reach level of unlearning similar to retraining as measured by ROUGE score of the generated responses, and using a stronger offset model generally leads to better performance on retain sets. Distributional metrics such as truth ratio and forget quality are more sensitive to the choice of offset model especially when the offset models are weak, as weak offset models tend to behave very differently from the target model during unlcarning, and thus making the final logit ensemble deviate more from direct finc- luning, Similar to what we observe in Tab. using different ofiset models affects how the model balances performance across different types of knowledge during unlearning.
6 Conclusion
In this work, propose 8- UNLEARNING, offset unlearning framework applicable to black-box LLM that does not require access to model's internal weights. Instead of modifying model parameters, 5- J-UNLEARNING learns the logit oflset needed to steer model behavior on the larget forget set data. Experiments show that 6-UNLEARNING is on par with and sometimes even stronger than direct fine-tuning in terms of both forget quality and model utility. We also demonstrate that 8-UNLAANNING compatible and effective when combined with various unlearning algorithms, thus providing a versatile solution adapting existing algorithms to black-box LLMss.
10

---

Published in Transactions on Machine Learning Research (05/2025)
Acknowledgment
We appreciate the reviewers for their insightful comments and suggestions. James Y. Huang was supported by gift fund from the USC Center Secure & Trusted ML. Fei Wang supported by Amazon ML PhD Fellowship. Muhao Chen was supported by the DARPA FoundSci Grant HR00112490370, the NSF of the United States Grant ITE 2333736.
References
Vinayshekhar Bannihatti Kumar, Rashmi Gangadharaiah, and Dan Roth. Privacy adhering machine un- Icarning in NLP. In Jong C. Park, Yuki Arase, Baotian Hu, Wei Lu, Derry Wijaya, Ayu Purwarianti, and Adila Alfa Krisnadhi (eds.), Pindings of the Association for Compulational Linguistics: IJCNLP-AACL 2023 (Findings), pp. 268-277, Nusa Dua, Bali, November 2023, Association for Computational Linguistics. doi: 10.18653/ v1 weeee/.ininns--eeee 1.. URL https::/aaclanthologyy hww.aaatttooooooooo/22./.2.... wtt. fidding---jennlp 25.
Kent Chang, Mackenzie Cramer, Sandeep Soni, and David Bamman. Speak, memory: An archaeology of books known to ChatGGT//PTT4 In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Longuage Processing, pp. 7312 7327, Singapore, December 2023. Association for Computational Linguistics. doi: 00.1865//11/0233.0 www.m..0.....mmmm...... 453.. URL https://acllanthol..g www.ogg.......g..oo.. www. emnlp-main. 453
Jiaao Chen and Diyi Yang. Unlearn what you want to forget: Efficient unlearning for LLMs. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 12041-12052, Singapore, December 2023. Association for Compu- wttp:tthhhoooooooo/22233333
tational Linguistics. doi: 2.116533/1122233.cm wwtp://0....mmmmmmammmmm mannlppmmain...... URL emnlp-main. 738
Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James R. Glass, and Pengcheng He. Dola: Decoding by contrasting layers improves factuality in large language models. In The Twelfth International Conference on Learning Representations, 2024. URL
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. orXiv preprint arXiv:1803. 15X5:1880..0557, 2018.
Ronen Eldan and Mark Russinovich. Who's harry potter? approximate unlearning in llms, orXiv preprint arXiv:2310.0200 2023.
Shangbin Feng, Weijia Shi, Yuyang Bai, Vidhisha Balachandran, Tianxing He, and Yulia Tsvetkov. Knowl- edge card: Filling LLMs' knowledge gaps with plug-in specialized language models. In The Twelfth International Conference on Learning Representations, 2024. URL WbWtDYI2IK
Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Lau- rence Golding, Jeffrey Hsu, Alain Le Noachh, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, Eric Tang, Anish Thite, Ben Wang, Kevin Wang. and Andy Zou. A framework for few-shot language model evaluation, 12 2023. URL
Geoffrey E Hinton. Training products of experts by minimizing contrastive divergence. Neural compulation, 14(8):1771-1800, 2002.
Chris Jay Hoofnagle, Bart Van Der Sloot, and Frederik Zuiderveen Borgesius. The european union general data protection regulation: what is and what means. Information & Communications Technology Law, 28(1):65-996 98, 2019.
11

---

Published in Transactions on Machine Learning Research (05/2025)
Joel Jang, Dongkeun Yoon, Sohee Yang, Sungmin Cha, Moontae Lee, Lajanugen Logeswaran, and Min- joon Seo. Knowledge unlearning for mitigating privacy risks in language models. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 61st Annual Meeting of the As- sociation for Computational Linguistics (Volume 1: Long Papers), pp. 14389-14408, Toronto, Canada, URL https:
July 2023. Association for Computational ional Linguistics. doi: 10. 10.18653/v1 /2022.2002-cooog. //aclanthology.org wwe.:22....1..1.. 1ool-long.800
Antonia Karamolegkou, Jiaang Li, Li Zhou, and Anders Sogaard. Copyright violations and large language models. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 7403-7412, Singapore, December 2023. Association for Computational Linguistics. doi: 10. 10.1855/v1120223 emnll-main.dee main.458. URL org/2023 emnlp-mane mai..ai........ 458
Tianle Li, Ge Zhang, Quy Duc Do, Xiang Yue, and Wenhu Chen. Long-context Ilms struggle with long in-context learning, arXiv preprint arxiv:2404.. 02060, 2024.
Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori Hashimoto, Luke Zettle- moyer, and Mike Lewis. Contrastive decoding: Open-ended text generation optimization. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 12286-12312, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10.18653/vvl /1116653/112223.aa// acc-oo... 687. URL https://aclanthone
oro.org///0 2rgg/023...o acl-llnc..lo..... 687
Chin-Yes Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pp. 74 81, 2004.
Alisa. Liu, Maarten Sap, Ximing Lu, Swabha Swayamdipta, Chandra Bhagavatula, Noah A. Smith, and Yejin Choi. DExperts:
Decoding- controlled text generation with experts and anti-experts. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceedings of the 59th Annual Meeting of the Asso- ciation for Computational Linguistics and the 1th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 6691-6706, Online, August 2021. Association for Computational Linguistics. doi: 10.18653/v1, /1233/11222211/ wac--lln.lllonneee 522. URL http:tttss////aaatoolooggg hwtp://hooooyyy./22221.1.1 acc-///ooo...coooooo.ooo. 5222
Alisa Liu, Xiaochuang Han, Yizhong Wang, Yulia Tsvetkov, Yejin Choi, and Noah Smith. Tuning language models by proxy. Xixio preprint arXiv:2401.0855 088665, 2024a.
Bo Liu, Qiang Liu, and Peter Stone. Continual learning and private unlearning. In Sarath Chandar, Razvan Pascanu, and Doina Precup (eds.), Proceedings of The 1st Conference on Lifelong Learning Agents, volume 199 of Proceedings of Machine Learning Research, pp. 244. PMLR, 22-24 Aug 2022. URL https: https://proceedings mlr
html
Qin Liu, Fei Wang, Nan Xu, Tianyi Yan, Tao Meng, and Muhao Chen. Monotonic paraphrasing improves generalization of language model prompting. arXiv preprint arxiv:2403. 16038, 2024b. URL https:
//arxiv. oww.vvo.rg//dtf... wttp://///p////t//////p//// /203. 16038. pff
Sijia Liu, Yuanshun Yao, Jinghan Jia, Stephen Casper, Nathalic Baracaldo, Peter Hasc, Xiaojun Xu, Yuguang Yao, Hang Li, Kush R Varshney, et al. Rethinking machine unlearning for large language models. ar preprint 440.020777.877 2024c.
Pratyush Maini, Zhili Feng, Avi Schwarzschild, Zachary C Lipton, and Zico Kolter. Tofu: task of fictitious unlearning for lmms. arXiv preprint 2rXiv:2101.0111, 2024.
Kevin Meng, Arnab Sen Sharma, Alex J Andonian, Yonatan Belinkov, and David Bau. Mass-editing memory in a transformer. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview ottp:/perrevieeeeeeffrruurr/uaep
net/forum? wiwtoruuniiiakkbAHHTHH.. gyS
Tao Meng, Sidi Lu, Nanyun Peng, and Kai-Wei Chang. Controllable text generation with neurally- decomposed oracle. Advances in Neural Information Processing Systems, 35:28125- 25-28125-28119,, 2022.
12

---

Published in Transactions on Machine Learning Research (05/2025)
Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can suit of armor conduct electricity? a new dataset for open book question answering. In Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun'ich Mun'ichi Tsujüi (eds.), Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 2381-2391, Brussels, Belgium, October-November 2018. Association for Computational Linguistics. doi: 10. 1 1260 URL httpss:/aclaantiologg.or///////
Sewon Min, Suchin Gururangan, Eric Wallace, Weijia Shi, Hannaneh Hajishirzi, Noah A. Smith, and Luke Zettlemoyer. SILO language models: Isolating legal risk in a nonparametric datastore. In The Twelfth International Conference on Learning Representations, 2024. URL https:// new.:/runfffmiid rukOnyQPec
Eric Mitchell, Rafael Ralailov, Archit Sharma, Chelsea Finn, and Christopher D Manning. An emulator for fine-tuning large language models using small language models. In The Twelfth International Conference on Learning Representations, 2024. URL https://openrevveew http://wwww...oooooooo//oo//ooa.
Aitor Ormazabal, Mikel Artetxe, and Eneko Agirre. CombLM: Adapting black- language models through small fine-tuned models. In Houda Bouamor, Juan Pino, and Kalika Bali (ods.). Proceedings of the 2023
Conference on Empirical Methods in Nalural Language Processing, pp. 2961 2974, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023 eeme..emmlllmman 180. URL https:
www.aatthooooy..rr22222222 www.::2...mmmmmmmmmmmmmmmmmmmlmm commmp-main... 180.
Vaidehi Patil, Peter Hase, and Mohit Bansal.
Can sensitive information be deleted from LLMs? objectives for defending against extraction attacks. In The Twelfth International Conference on Learning Represen- lations, 2024. URL. http:///ppppaeeeeee..aee///rrtar
Martin Pawelczyk, Seth Neel, and Himabindu Lakkaraju. In-context unlearming: Language models as few shot unlearners. arXiv preprint xxXi:2310.07579, 2023.
Fábio Perez and Ian Ribeiro. Tgnore previous prompt: Attack techniques for language models. arXiv preprint urXiv:2211.09521 2022.
Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge al scale. COMMUNICATIONS OF THE ACM, 64(9), 2021.
Omar Shaikh, Hongxin Zhang, William Held, Michael Bernstein, and Diyi Yang. On second thought, let's not think step by step! bias and toxicity in zero-shot reasoning. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 4454-4470, Toronto, Canada, July 2023. Association for Com- putational Linguistics. doi: 10.18653/v1 / a20 244. URL hutps://aclanthoolggy htrpo/looyyrgg/2222200 o02//023.
acl ac1-long.e 244
Weijia Shi, Xiaochuang Han, Mike Lewis, Yulia Tsvetkov, Luke Zettlemoyer, and Scott Wen-tau Yih. Trusting your evidence: Hallucinate less with context-aware decoding. arXiv preprint arXio:2305.14733 2023.
Robin Staab, Mark Vero, Mislav Balunovic, and Martin Vechev. Beyond memorization: Violating privacy via inference with large language models. In The Twelfth International Conference Learning Represen- tations, 2024. URL htttp::/ wwtp://oommdddddnnoooooooeeee
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bash- lykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arxie preprint wrXiv:2307.09288 2023.
Lingzhi Wang. Tong Chen, Wei Yuan, Xingshan Zeng, Kam-Fai Wong, and Hongzhi Yin. KGA: A general machine unlearning framework based on knowledge gap alignment. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 13264-13276, Toronto, Canada, July 2023. Association for Computational Linguisties. doi: 10.18653/v1. 1/t////112222//// 22222222cccllcc acl- long. 740. URL httpap:://cllanttooogg...o www.oogg...g//33///// acl-aaclloon
acl ael-long.11 770
13

---

Published in Transactions on Machine Learning Research (05/2025)
Xinwei Wu, Junzhuo Li, Minghui Xu, Weilong Dong, Shuangzhi Wu, Chao Bian, and Deyi Xiong. DEPN: Detecting and editing privacy neurons in pretrained language models. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 2875-2886, Singapore, December 2023. Association for Computational Linguistics. doi: (0.18653/v1/2000 /t////22...mlpmmaii...
URL https:://aclaatthooogg...oo org/ conlp-mming 174
Jin Yao, Eli Chien, Minxin Du, Xinyao Niu, Tianhao Wang, Zezhou Cheng, and Xiang Yue. Machine unlearning of pre-trained large language models. arXiv preprint w:Xiv2240.111159 2024.
Yuanshun Yao, Xiaojun Xu, and Yang Liu. Large language model unlearning. ar preprinl 0rXi:2330010068: 2023.
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. HellaSwag: Can a machine really finish your sentence? In Anna Korhonen, David Traum, and Lluís Mårquez (eds.), Proceedings of the 57th Arnual Meeling of the Associalion for Compulational Långuisties, pp. 4791 4800, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10. 10.18653/11/119-144 1472. URL https: www.aaathoooooooog//111---1-
Dawen Zhang, Pamela Finckenberg- Bromann Thong Hoang, Shidong Pan, Zhenchang Xing, Mark Staples, and Xiwei Xu. Right to be forgotten in the era of large language models: Implications, challenges, and solutions. arXiv preprint arXh::237.003941 2023.
Xuandong Zhao, Xianjun Yang, Tianyu Pang, Chao Du, Lei Li, Yu-Xiang Wang, and William Yang Wang. Weak-lo-strong jailbreaking on large language models. arxiv preprint 2im:22440 17256, 2024.
14


Reviews:
[{"review_id": "6ymK2hLyf9", "reviewer": "Reviewer_d6iE", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "#### **Strengths**  \n\n1. **Novel Unlearning Framework for Black-Box LLMs**  \n   - The introduction of **δ-Unlearning** offers a **practical, parameter-free** solution for unlearning in **black-box LLMs**.  \n   - Unlike existing methods, it does not require **fine-tuning the LLM itself**, making it **cost-effective** and **applicable to proprietary models**.\n\n2. **Versatility Across Unlearning Algorithms**  \n   - δ-Unlearning is designed to be **modular** and **integrates well** with different unlearning techniques (**Gradient Ascent, Gradient Difference, KL Minimization, and Data Relabeling**).  \n   - This flexibility increases the method's **applicability across different use cases**.\n\n3. **Stronger Privacy Compliance**  \n   - The method does not **store sensitive data** or **require access to model weights**, addressing concerns raised by **data protection laws** like the GDPR.  \n   - Avoids **in-context unlearning limitations**, where storing sensitive information for inference can **violate privacy**.\n\n4. **Improved Retain Performance Compared to Direct Fine-Tuning**  \n   - **Maintains or even enhances performance** on out-of-forget-scope tasks compared to standard fine-tuning approaches.  \n   - Empirical results on **TOFU and general NLP benchmarks** (ARC, HellaSwag, WinoGrande, OpenBookQA) **demonstrate its ability to preserve general knowledge**.\n\n5. **Scalability and Version Control**  \n   - The approach **enables efficient version control** by maintaining **a pool of smaller models** instead of retraining the entire LLM.  \n   - Supports **customizable and modular unlearning**, allowing **domain-specific** knowledge updates without affecting the base LLM.\n\n6. **Thorough Experimental Validation**  \n   - The **TOFU benchmark** is well-suited for evaluating unlearning, providing a **controlled setting** with fictitious author knowledge.  \n   - Evaluation metrics (ROUGE, Probability, Truth Ratio, Forget Quality) give a **clear assessment of both forgetting and retention trade-offs**.\n\n---\n\n#### **Weaknesses / Areas for Improvement**  \n\n1. **Dependency on Smaller White-Box Models**  \n   - **δ-Unlearning relies on training smaller white-box models** to compute logit offsets, which may **limit its applicability** in environments where **such models are unavailable** or where creating high-quality smaller models is challenging.  \n   - The effectiveness of the approach may **depend on the quality of the smaller models**, which is not fully analyzed in the paper.\n\n2. **Lack of Real-World Evaluation Beyond TOFU**  \n   - While TOFU is a **strong benchmark**, the paper does not test δ-Unlearning on **real-world unlearning tasks**, such as removing **sensitive, copyrighted, or biased content** from an LLM.  \n   - Evaluating on datasets with **legal or ethical constraints** (e.g., **removing personally identifiable information**) would further validate the framework.\n\n3. **Limited Analysis of Trade-Offs in Forget and Retain Performance**  \n   - Although the paper discusses the **trade-offs between forgetting and retention**, further **fine-grained analysis** of when **forgetting causes unintended knowledge loss** (knowledge entanglement) would strengthen the claims.  \n   - A **more detailed breakdown** of how δ-Unlearning affects different types of knowledge (e.g., factual vs. contextual) would be useful.\n\n4. **Hyperparameter Sensitivity of Logit Offsets (α)**  \n   - The offset strength parameter **α** is crucial for effective unlearning, yet **there is limited discussion** on how to **optimally select α**.  \n   - Further experiments on **adaptive or automated tuning** of α based on model behavior could enhance applicability.\n\n5. **Computational Overhead**  \n   - While δ-Unlearning **avoids full model retraining**, training **additional offset models** adds some **computational cost**.  \n   - A direct **comparison of computational efficiency** with traditional unlearning methods (in terms of FLOPs or training time) would be beneficial.\n\n6. **Clarity in Presentation and Explanation of Some Concepts**  \n   - The **mathematical formulation** of logit offsets and their influence on LLM predictions could be more **intuitively explained** for a broader audience.  \n   - A **visualization of how logits shift** before and after unlearning would provide better clarity.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces **δ-Unlearning**, a novel offset unlearning framework for **black-box Large Language Models (LLMs)** that allows unlearning specific training data without modifying the model’s internal weights. The key contributions are:\n\n1. **Introduction of δ-Unlearning**: The proposed method enables unlearning in black-box LLMs by leveraging a **logit offset** technique. Instead of fine-tuning the LLM itself, **δ-Unlearning computes and applies a logit offset using two smaller, white-box models** that contrast the logits of sensitive versus general knowledge.\n\n2. **Effective Unlearning While Preserving General Performance**: Experiments show that δ-Unlearning achieves **comparable or superior performance** to traditional fine-tuning methods in removing targeted knowledge while maintaining accuracy on tasks outside the forget scope.\n\n3. **Versatility Across Unlearning Methods**: δ-Unlearning is **compatible with various unlearning algorithms** such as **Gradient Ascent, Gradient Difference, KL Minimization, and Data Relabeling**, making it a **flexible** and **modular** approach.\n\n4. **Black-Box Applicability and Privacy Compliance**: Unlike previous methods that require access to LLM **weights** or store sensitive data at inference, δ-Unlearning can **unlearn data from black-box models without violating data protection regulations**.\n\n5. **Efficient Version Control and Adaptability**: The approach **modularizes unlearning**, enabling users to **curate knowledge** by maintaining a pool of smaller models that can be combined in a **plug-and-play** manner with the base LLM.\n\n6. **Extensive Empirical Validation**: The method is evaluated on the **TOFU benchmark**, showing strong unlearning capabilities while maintaining general model utility. It also demonstrates robust performance on general NLP benchmarks, proving its adaptability beyond targeted unlearning tasks.\n\n### **New Knowledge Introduced**  \n- **Offset-based unlearning as an alternative to model retraining**: δ-Unlearning shows that modifying **only the logits** (rather than model weights) can achieve effective unlearning.\n- **Modular and interpretable unlearning for black-box LLMs**: The approach allows for **targeted and customizable** knowledge removal without violating **privacy constraints**.\n- **Trade-offs between forgetting and retaining knowledge**: The paper provides **insightful analysis** on how different **unlearning strengths** impact knowledge retention and general model performance.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "### **Critical Changes (Necessary for Acceptance)**  \n\n1. **Expand Real-World Evaluation Beyond TOFU**  \n   - While TOFU is a useful benchmark, the paper would significantly **benefit from experiments on real-world unlearning scenarios**, such as:  \n     - Removing **personally identifiable information (PII)** from LLM outputs.  \n     - Forgetting **copyrighted or proprietary content**.  \n     - Addressing **biased or harmful responses**.  \n   - This would strengthen the claim that **δ-Unlearning is practical for real-world applications**.  \n\n2. **Analyze the Impact of Offset Model Quality**  \n   - δ-Unlearning relies on **two smaller white-box models** to generate logit offsets. The paper should **investigate how the choice of these models affects unlearning performance**.  \n   - Key questions to explore:  \n     - Does the size of the **offset models** influence the effectiveness of unlearning?  \n     - How do different training strategies for these smaller models affect the **final logit offsets**?  \n     - Are there cases where δ-Unlearning fails due to poor offset model choices?  \n   - If these factors significantly impact the method's effectiveness, **guidelines should be provided** for selecting appropriate offset models.  \n\n3. **Provide a More Detailed Analysis of Forget vs. Retain Trade-offs**  \n   - The paper currently discusses the **trade-off between unlearning and retention**, but further **quantitative insights** would be useful.  \n   - Suggested improvements:  \n     - Show how different **types of knowledge (e.g., factual, contextual, reasoning-based) are affected** by δ-Unlearning.  \n     - Provide **more granular analysis** on how performance degrades across different datasets in TOFU.  \n     - Discuss whether **δ-Unlearning disproportionately affects certain types of information**, leading to unintended consequences.  \n\n4. **Clarify Computational Efficiency and Overhead**  \n   - While δ-Unlearning avoids full model retraining, it **introduces additional computational steps** through offset model training.  \n   - The paper should include:  \n     - A direct **comparison of computational cost** (e.g., FLOPs, memory usage, or wall-clock time) between δ-Unlearning and traditional fine-tuning-based unlearning.  \n     - Discussion of whether δ-Unlearning scales effectively for **larger LLMs** (e.g., 30B+ parameters).  \n\n### **Recommended Changes (Would Strengthen the Paper)**  \n\n5. **Improve the Explanation of Logit Offsets and Their Impact**  \n   - The paper presents the mathematical formulation of **logit offsets**, but some parts could be more **intuitively explained**.  \n   - Suggested improvements:  \n     - **Add a visualization** showing how **logits shift before and after unlearning**.  \n     - Provide **concrete examples** where δ-Unlearning successfully **removes unwanted knowledge**.  \n     - Offer a **more detailed interpretation** of how the logit offset technique works in practice.  \n\n6. **Discuss Adaptive or Automated Offset Strength Selection (α)**  \n   - The paper currently sets **α = 1**, but there is **no discussion on how to tune α optimally**.  \n   - Suggested additions:  \n     - Investigate **different ways to select α dynamically** instead of fixing it.  \n     - Explore whether an **adaptive α** can improve unlearning effectiveness while minimizing unintended knowledge loss.  \n\n7. **Broaden the Related Work Section**  \n   - The discussion on **machine unlearning** is well-grounded, but adding references to **recent advances in continual learning, domain adaptation, and knowledge editing** could provide **additional context**.  \n\n8. **Enhance Paper Readability and Structure**  \n   - Some explanations, particularly in **Section 3 (Method) and Section 5 (Analysis)**, could be **streamlined** for better readability.  \n   - Specific improvements:  \n     - Reword complex mathematical passages for clarity.  \n     - Make the **figures and tables more self-explanatory** by adding **more detailed captions**.  \n     - Reduce **redundant phrasing** in discussions of unlearning benchmarks.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "PfYIqgMX1l", "reviewer": "Reviewer_2dTb", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "1. The approach is conceptually innovative by leveraging logit offsets to modify a black-box model’s outputs. However, the experiments rely solely on closely related llama2 variants. This raises concerns about whether the mechanism holds when the offset model’s architecture significantly differs from that of the large model. Moreover, even when using similar architectures, factors like different initialization and training recipes might yield different logits, so the claim of general “black-box unlearning” appears a bit overextended.\n\n2. The framework’s ability to integrate with multiple unlearning algorithms is a strength, as is its potential for preserving privacy by not storing sensitive information at inference time. On the downside, maintaining a pool of smaller offset models introduces system complexity and extra overhead, and it is unclear how mismatches in architecture or size between the large model and offset models might affect overall performance. This additional layer might complicate real-world deployment and scalability.\n\n3. Empirical evaluations on the TOFU benchmark and general tasks show promising trade-offs between forgetting and retaining knowledge. However, the results seem to be sensitive to the offset strength parameter $\\alpha$. Without a robust strategy for tuning $\\alpha$, practical deployment might be challenging, especially if the optimal value is difficult to determine and maintain across different scenarios.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces a novel framework for unlearning sensitive or problematic information from large language models without accessing their internal weights. Instead of fine-tuning the large, black-box model directly, the method learns a logit offset using a pair of smaller, white-box offset models. This approach is designed to modify the output distribution of the target model while preserving its overall performance on non-sensitive tasks. The framework is versatile, as it can be combined with various existing unlearning algorithms, and it demonstrates empirical effectiveness on a synthetic benchmark (TOFU) as well as on several general task benchmarks.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. It's important to include empirical studies that examine the effect of using offset models with different architectures from the target LLM. In addition, it would strongly support the claim of effective black-box unlearning by demonstrating robustness when the offset model diverges in structure, size, or training protocol from the large model.\n\n2. The authors should discuss and quantify the additional computational overhead introduced by maintaining an ensemble of models. It would be informative to also have a trade-off analysis showing how the size of the offset models relates to unlearning performance (and whether smaller models might suffice compared to larger ones).\n\n3. Since the results are sensitive to the offset strength $\\alpha$, it would be helpful to discuss strategies for selecting the optimal $\\alpha$ or an adaptive mechanism for setting $\\alpha$. \n\n4. I suggest including the ROUGE score of the retraining baseline in Fig. 2, as this would help understand how δ-Unlearning performs when unlearning levels are near those achieved by full retraining, rather than only at higher unlearning levels.", "broader_impact_concerns": "The proposed framework, if misused, could potentially erase information deliberately for unethical purposes. Outlining safeguards, transparency measures, or audit mechanisms for the unlearning process could mitigate the risk of misuse and promote responsible deployment.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "DIaXKXuEss", "reviewer": "Reviewer_wcJf", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1. The problem of unlearning in large language models (LLMs) is critical, and the introduction effectively presents the issue with clarity.\n\n2. This paper highlights common drawbacks of existing LLM unlearning methods and proposes a systematic approach that can be integrated with other unlearning techniques.\n\n3. The experimental results demonstrate the efficiency of the proposed method across various unlearning tasks.\n\nWeaknesses:\n\n1. Some parts of the methodology are unclear—for instance, the process for selecting the smaller model $M_o$ is not well-defined.\n\n2. Although the paper claims that previous unlearning methods require an explicit list of sensitive information, the model offset stage still depends on forgetting data for unlearning small models. Additionally, it does not address the growing size of the forgetting data, an issue that is omitted from both the methodology and experimental sections.\n\n3. The experimental results are not entirely convincing, as direct fine-tuning outperforms the proposed method on several evaluations.\n\n4. The experiments only compare the proposed method with direct fine-tuning without providing a detailed explanation of the fine-tuning process, nor do they explore a broader range of small models.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "1. This paper summarizes the drawbacks of current unlearning methods for large language models (LLMs) and proposes a logit offset method to reduce reliance on model weights, addressing the challenges posed by the ever-growing forgetting set.\n\n2. Experimental results demonstrate that the proposed method achieves performance comparable to direct fine-tuning approaches on LLMs.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. This paper can provide more details about how to select the smaller models and whether different smaller models can have effects on the larger model.\n\n2. This paper can provide more explanation about how the proposed method can solve the two drawbacks of current LLM unlearning methods\n\n3. Then, regarding the experiment results, this paper can provide a more comprehensive result to prove the proposed method can be superior to other methods", "broader_impact_concerns": "No further concerns", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper introduces a novel framework for unlearning sensitive or problematic information from large language models without accessing their internal weights. Instead of fine-tuning the large, black-box model directly, the method learns a logit offset using a pair of smaller, white-box offset models. This approach is designed to modify the output distribution of the target model while preserving its overall performance on non-sensitive tasks. The framework is versatile, as it can be combined with various existing unlearning algorithms, and it demonstrates empirical effectiveness on a synthetic benchmark (TOFU) as well as on several general task benchmarks.

For the strength, the paper presents a novel and timely contribution to understanding LLMs (how they learn), addressing an important challenge with practical implications. The methodology is well motivated and clearly described, and the experimental results are generally sound, demonstrating consistent improvements over relevant baselines. While there are some areas that could benefit from further clarification or expanded analysis, they do not detract significantly from the overall quality and relevance of the work. The paper meets the TMLR acceptance criteria in terms of originality, soundness, clarity, and significance, and I believe it would be a valuable addition to the community. However, there are several points to be improved. While the paper presents some interesting contributions, its applicability appears to be confined to a narrow set of scenarios. The primary claim that comparable unlearning performance can be achieved without accessing the target model's weights relies on several caveats. Specifically, it requires a smaller model from the same family, access to the weights of this smaller model, and careful hyperparameter tuning for the new parameters. The authors address reviewers' concerns well in their rebuttal. Thus, I would like to recommend accept as is.
Decision: Accept (poster)