MOUCHI: Mitigating Over-forgetting in Unlearning Copyrighted Information

Authors: Irfan Akbar, Dongmin Park, Patara Trirat, Jae-Gil Lee
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: machine unlearning, LLM unlearning, derivative knowledge

Abstract:
Large language models are trained on massive internet datasets, which may inadvertently memorize illegal copyrighted content, making its inclusion unavoidable. Unlearning is a potential solution to remove such content. However, existing unlearning methods often suffer from **over-forgetting**, where the process unintentionally erases knowledge similar to the copyrighted content that falls under fair use and should be preserved. To address this issue, we propose **MOUCHI**, a novel unlearning framework that introduces the concept of **derivative knowledge**, a subset of information derived from copyrighted content that must be retained during unlearning. MOUCHI first generates derivative knowledge and then incorporates a derivative loss function into the unlearning process to mitigate over-forgetting in unlearning copyrighted content. Due to its plug-and-play nature, MOUCHI can be effortlessly integrated into existing unlearning methods. Experimental results show that MOUCHI reduces unintended knowledge loss, improving performance by **up to 145%** compared to baseline methods when evaluated on the derivative set.

Decision: reject
Reviewer scores: 6, 5, 5, 3

Reviews and discussion:
<thread>
creation date:
1730224871874

thread_type:
official_review

<thread_object>
creator: reviewer_gjpq

<content>
summary:
Copyright concerns in large language models (LLMs) have become especially prominent due to the widespread use of these models, making them more vulnerable to misuse. Unlearning is a potential solution for removing such content. However, existing unlearning methods often suffer from over-forgetting, where the process unintentionally erases knowledge that is similar to copyrighted content falling under fair use and should be preserved. There is a trade-off between unlearning and over-forgetting.

This paper analyzes and identifies the over-forgetting problem in current LLM unlearning methods, then introduces the concept of derivative knowledge, and applies it to the MOUCHI framework by proposing the $L_{drv}$ loss. Experimental results show that this method achieves better performance than previous unlearning methods.

strengths:
This paper investigates the issue of over-forgetting in unlearning methods. Its strengths include:

1. Analyzing and identifying the over-forgetting problem in current LLM unlearning methods.
2. Proposing the concept of derivative knowledge, which can be viewed as a buffer zone between the knowledge to be forgotten and the knowledge that needs to be retained.
3. Carefully designing a derivative generation module, with experimental results surpassing those of previous unlearning methods.

weaknesses:
1. There are inconsistencies in the paper's wording. For example, in line 17, it states, "...the concept of derivative knowledge, a subset of information derived from copyrighted content that must be retained during unlearning." However, in line 76, it says, "the concept of derivative knowledge—a subset derived from the target copyrighted information that needs to be removed."

2. More experiments are needed to validate the effectiveness of the method. Currently, the paper is based on only one language model and a relatively small dataset (TOFU), where each author has only 20 QA questions. Such results may have certain limitations. There are many existing, more comprehensive unlearning datasets available, and broader testing is required to assess the method's generalizability.

[1] Muse: Machine unlearning six-way evaluation for language models. arXiv preprint arXiv:2407.06460, 2024.

[2] The wmdp benchmark: Measuring and reducing malicious use with unlearning. arXiv preprint arXiv:2403.03218, 2024.

[3] RWKU: Benchmarking Real-World Knowledge Unlearning for Large Language Models. arXiv preprint arXiv:2406.10890, 2024.

questions:
1. Since the TOFU dataset is relatively small, this paper divides it into four subsets (by augmenting part of the data). Could you provide a detailed distribution of these subsets?

2. Regarding the evaluation metric "Derivative," if MOUCHI is not included, will the model not be trained on $D_{drv}$ ? Or is $D_{drv}$ integrated into other subsets for training?

3. Given the diversity of ChatGPT's generated content, has the determination of δmin and δmax undergone multiple validations to ensure its accuracy?

4. In the example on the right side of Figure 6, is the given "A" the knowledge that the model needs to retain? If so, what specifically needs to be forgotten? For clarity, it would be helpful to include related Q&A from both the Forget set and the Derivative set for the same author in the presentation, illustrating what should be retained versus what should be forgotten.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730454475159

thread_type:
official_review

<thread_object>
creator: reviewer_w5ru

<content>
summary:
To address the over-forgetting in LLM unlearning, this paper proposes MOUCHI, a new unlearning framework that generates a set of derivative knowledge to enrich the retain set. The derivative knowledge lies between the forget set and the retain set, which helps to better specify the boundaries of the knowledge to be removed. Experimental results demonstrate that MOUCHI can provide better control over over-forgetting.

strengths:
1.This paper is well motivated as it focuses on the over-forgetting issue in unlearning.

2.This paper proposes a simple method to expand the retain set in the form of data synthesis.

weaknesses:
1.The proposed method lacks novelty as it simply prompts the model to generate derivative knowledge and uses KL divergence for filtering. And there is no improvement in the loss function for the derivative set, which can be regarded as an expansion of the retain set. Is there a generation-free way to induce derivative knowledge?

2.This paper only conducts experiments on TOFU and needs to verify the effectiveness on more unlearning datasets, such as scenarios where the range of unlearning knowledge is particularly extensive.

3.Some implementation details are lacking. For example, the forget set in TOFU consists of 200 fictional authors and has no connection with real-world knowledge. Then what is the derivative knowledge of these fictional authors? Some generated results need to be provided.

questions:
Please see the weakness.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730668838525

thread_type:
official_review

<thread_object>
creator: reviewer_c6sn

<content>
summary:
The paper titled "MOUCHI: Mitigating Over-Forgetting in Unlearning Copyrighted Information" addresses the critical issue of copyright infringement within large language models (LLMs). Due to the extensive and often indiscriminate data used in their training, LLMs can inadvertently memorize and reproduce copyrighted content, posing a significant legal and ethical challenge. The proposed MOUCHI framework introduces "derivative knowledge" as a subset of information derived from copyrighted content but intended for retention during unlearning to maintain fair-use knowledge. MOUCHI integrates a derivative loss function to differentiate and preserve derivative knowledge, allowing the model to continue answering general questions related to copyrighted material while removing infringing content. The framework is designed to work seamlessly with existing unlearning methods, providing a plug-and-play solution. Experimental results show that MOUCHI successfully reduces unintended knowledge loss by up to 145% over baseline methods.

strengths:
The paper is well-written, with a clear explanation of the problem and the proposed solution. 

The concept of over-forgetting and derivative knowledge is effectively introduced and explained, making the paper accessible to a broad audience.

Copyright concerns in LLMs are increasingly pressing, particularly as these models become more widely deployed. The focus on balancing copyright compliance with preserving relevant knowledge is timely and important. MOUCHI attempts to address the complex need to avoid copyright infringement while retaining content that may fall under fair use.

weaknesses:
The MOUCHI framework, while straightforward, maybe too simplistic to fully address the nuanced requirements of copyright compliance. 

MOUCHI may still risk indirect copyright violations by retaining derivative knowledge that could closely resemble the copyrighted content "CopyBench: Measuring Literal and Non-Literal Reproduction of Copyright-Protected Text in Language Model Generation"

Utility evaluation is limited, which weakens the claim of solving overforgetting problems. The authors may need to run common benchmarks such as MMLU to ensure the utility of LLM is not sabotaged. 

The paper does not compare MOUCHI with some recent and relevant unlearning baselines, such as Goldfish Loss "Be like a goldfish, don’t memorize!" and "Avoiding Copyright Infringement via Machine Unlearning"

Code is not given.

Although the authors reference several key studies, the paper could be strengthened by discussing additional recent works that are pertinent to the topic of copyright compliance in LLMs. Listed below:

- Foundation Models and Fair Use
- Rethinking LLM Memorization through the Lens of Adversarial Compression
- Llms and memorization: On quality and specificity of copyright compliance
- SHIELD: Evaluation and Defense Strategies for Copyright Compliance in LLM Text Generation
- Digger: Detecting copyright content misusage in large language model training
- Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-4

questions:
KL Divergence value is constrained by ChatGPT? Is this possible without an iterative approach?

How to know whether the generated dataset is correct and does not include hallucinations?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730739005091

thread_type:
official_review

<thread_object>
creator: reviewer_bvjx

<content>
summary:
This paper tackles the problem of over-forgetting during the unlearning process of LLMs, i.e., unintentionally removing similar content which should be preserved. The authors propose a concept of derivative knowledge, which is a subset of information derived from the copyrighted content and should be retained during unlearning. In particular, 1) the set derivative knowledge will be generated and 2) a derivative loss function is included. Different unlearning approaches can incorporate their proposed method and obtain various performance improvements in terms of model utility and forget quality.

strengths:
* Propose an interesting and effective approach to tackle the over-forgetting issue of LLMs' unlearning process
* The approach can be integrated into existing unlearning methods to achieve various performance improvements
* Detailed analysis of the experiment results

weaknesses:
* Lack of manual analysis of the derivative knowledge: the authors also mention that experts/lawmakers need to be involved to check the boundary, otherwise it is difficult to judge how good (enough) the KL-based semantic similarity is in this context. I would suggest the authors to incorporate human judgements in this process.
* Experiments are restricted to one dataset, also one scenario/domain. Maybe the authors could comment on how easy/difficult for the approach to be adapted to other types of copyrighted information.

questions:
See weaknesses.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732118808167

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Additional Experiment Results (2/2)

comment:
**Table A2.** MOUCHI's Performance on the MUSE-News Dataset
| \\( \mathcal{L}_{fgt} \\) | \\( \mathcal{L}_{rt} \\) | MOUCHI | Forget ↓        | Derivative ↑     | Retain ↑        |
|:-------------------------:|:------------------------:|:--------:|:-----------------:|:------------------:|:-----------------:|
|  GA                   | none                   | ✗      | 0.000*±0.000    | 0.000*±0.000     | 0.000*±0.000    |
|                         |                        | ✓      | 0.000*±0.000    | **0.444±0.103**      | **0.312±0.093**     |
|                       | GD                     | ✗      | 0.043*±0.004    | 0.410±0.115      | 0.184±0.061     |
|                         |                        | ✓      | 0.047*±0.005    | **0.820±0.109**      | **0.561±0.092**     |
|                         | KL                     | ✗      | 0.000*±0.000    | 0.000*±0.000     | 0.000*±0.000    |
|                         |                        | ✓      | 0.080*±0.025    | **0.556±0.063**      | **0.396±0.108**     |
|  DPO                    | none                   | ✗      | 0.002*±0.021    | 0.017*±0.107     | 0.001*±0.002    |
|                         |                        | ✓      | **0.261±0.057**     | **0.970±0.031**      | **0.782±0.079**     |
|                         | GD                     | ✗      | **0.151±0.021**    | 0.831±0.044      | 0.654±0.104     |
|                         |                        | ✓      | 0.192±0.039     | **0.978±0.043**      | **0.765±0.081**     |
|                         | KL                     | ✗      | 0.010*±0.010    | 0.040*±0.035     | 0.021*±0.023    |
|                         |                        | ✓      | **0.278±0.060**     |**0.733±0.066**      | **0.639±0.057**     |
|  NPO                    | none                   | ✗      | 0.056±0.038     | 0.524±0.065      | 0.353±0.101     |
|                         |                        | ✓      | **0.138±0.027**     | **0.881±0.042**      | **0.576±0.086**    |
|                         | GD                     | ✗      | **0.152±0.040**     | 0.558±0.073      | 0.357±0.125     |
|                         |                        | ✓      | 0.214±0.029     | **0.893±0.091**      | **0.869±0.113**     |
|                         | KL                     | ✗      | 0.014*±0.002    | 0.061*±0.028     | 0.057*±0.036    |
|                         |                        | ✓      | **0.155±0.042**     | **0.874±0.056**      | **0.751±0.066**     |
| TV                      |                        | ✗      | 0.741±0.025     | 0.619±0.042      | 0.736±0.102     |
|                         |                        | ✓      | **0.721±0.016**     | **0.815±0.072**      | **0.741±0.009**     |
|                         |    |   **Best Improvement**     | **28.9%**       | **60%**          | **143.4%**      |

**Table B1.** TV on TOFU
| MOUCHI | Forget ↓     | Derivative ↑  | Retain ↑     | Normal ↑     |
|:--------:|:--------------:|:---------------:|:--------------:|:--------------:|
| ✗      | **0.762±0.011**  | 0.752±0.071   | 0.735±0.054  | **0.491±0.031**  |
| ✓      | 0.771±0.082  | **0.831±0.033**   | **0.761±0.078** | 0.482±0.082  |

**Table B2.** TV on MUSE-Books
| MOUCHI | Forget ↓     | Derivative ↑  | Retain ↑     |
|:--------:|:--------------:|:---------------:|:--------------:|
| ✗      | **0.831±0.072**  | 0.852±0.033   | 0.811±0.041  |
| ✓      | 0.850±0.017  | **0.903±0.031**   | **0.866±0.053**  |

**Table B3.** TV on MUSE-News
| MOUCHI | Forget ↓     | Derivative ↑  | Retain ↑     |
|:--------:|:--------------:|:---------------:|:--------------:|
| ✗      | 0.741±0.025  | 0.619±0.042   | 0.736±0.102  |
| ✓      | **0.721±0.016**  | **0.815±0.072**   | **0.741±0.009**  |
</content>
</thread_object>

</thread>
<thread>
creation date:
1732118839954

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Additional Experiment Results (1/2)

comment:
**Table A1.** MOUCHI's Performance on the MUSE-Books Dataset
| \\( \mathcal{L}_{fgt} \\) | \\( \mathcal{L}_{rt} \\) | MOUCHI | Forget ↓       | Derivative ↑    | Retain ↑        |
|:-------------------------:|:------------------------:|:--------:|:----------------:|:-----------------:|:-----------------:|
|  GA                  | none                       | ✗      | 0.468±0.154    | 0.096*±0.075    | 0.034*±0.013    |
|                         |                        | ✓      | **0.383±0.074**| **0.723±0.199** | **0.844±0.107** |
|                       | GD                     | ✗      | 0.648±0.133    | 0.913±0.034*    | 0.924±0.066     |
|                         |                        | ✓      | **0.542±0.056**| **0.970±0.012** | 0.924±0.031     |
|                         | KL                     | ✗      | 0.623±0.089    | 0.952±0.021     | 0.942±0.070     |
|                         |                        | ✓      | **0.601±0.045**| **0.996±0.006** | **0.988±0.014** |
|  DPO                    | none                   | ✗      | 0.012*±0.011   | 0.035*±0.034    | 0.332±0.112     |
|                         |                        | ✓      | **0.196±0.087**| **0.990±0.017** | 0.429±0.118     |
|                         | GD                     | ✗      | 0.213±0.088    | 0.853±0.040     | 0.488±0.073     |
|                         |                        | ✓      | **0.182±0.038**| **0.970±0.072** | 0.225±0.092     |
|                         | KL                     | ✗      | 0.032*±0.128   | 0.902±0.052     | 0.905±0.064     |
|                         |                        | ✓      | **0.511±0.081**| **0.991±0.010** | **0.938±0.026** |
|  NPO                    | none                   | ✗      | 0.444±0.170    | 0.718±0.192     | 0.835±0.100     |
|                         |                        | ✓      | **0.375±0.153**| **0.993±0.008** | **0.981±0.023** |
|                         | GD                     | ✗      | 0.480±0.163    | 0.814±0.108     | 0.864±0.093     |
|                         |                        | ✓      | **0.496±0.159**| **0.993±0.008** | **0.947±0.046** |
|                         | KL                     | ✗      | 0.472±0.081    | 0.512±0.084     | 0.594±0.013     |
|                         |                        | ✓      | **0.419±0.115**| **0.997±0.004** | **0.997±0.004** |
| TV                      |                        | ✗      | 0.831±0.072    | 0.852±0.033     | 0.811±0.041     |
|                         |                        | ✓      | **0.850±0.017**| **0.903±0.031** | **0.866±0.053** |
|                         |    |   **Best Improvement**     | **14.5%**      | **94.7%**       | **66.3%**       |
</content>
</thread_object>

</thread>
<thread>
creation date:
1732118914263

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Response Summary to All Reviewers

comment:
We profoundly appreciate the reviewers' valuable feedback and constructive comments, which have greatly contributed to improving our paper. Overall, the reviewers have recognized the novelty of analyzing and identifying the over-forgetting problem (**all reviewers**), the clarity of the paper (**all reviewers**), and the usability of MOUCHI with existing approaches (Reviewers **[bVJx](https://openreview.net/forum?id=N2wPtFVK6o&noteId=FIWnvNrZnY)** and **[w5RU](https://openreview.net/forum?id=N2wPtFVK6o&noteId=9b9zKLG1IT)**). However, the reviewers have suggested that additional comparisons with recent datasets and baselines are necessary to further support the claim of MOUCHI's effectiveness in mitigating over-forgetting. We believe that the following responses address all the issues and questions raised by the reviewers. 

### **Additional Experiments**
- According to the reviewers' suggestions, we have included a more recent dataset, MUSE [1], incorporated the MMLU benchmark [2] for general utility evaluation, and added a recent baseline, Task Vector (TV) [3,4].

### **Further Clarifications of MOUCHI's Design Choices**
- The main reason for initially focusing on a single dataset was the absence of datasets specifically designed for removing copyrighted information; existing ones target general unlearning tasks. We selected TOFU [5] as it was the closest match to our research objectives, though it still required some modifications. Based on the reviewers' suggestions, we expanded our experiments to include the MUSE dataset, which we also found to be suitable for our study.
- The robustness of ChatGPT as a substitute for experts in obtaining boundary KL divergence values was tested by generating the boundary QnA multiple time and averaging the results.


Once again, we sincerely thank the reviewers for their time and thoughtful suggestions to improve our paper. We are happy to address any additional questions during the discussion period. With these clarifications, we hope the reviewers will now feel confident in supporting this paper for publication by reconsidering their scores.

---

**References**

[1] Weijia Shi *et al*. Muse: Machine unlearning six-way evaluation for language models. *arXiv preprint arXiv:2407.06460, 2024*.

[2] Dan Hendrycks *et al*. In *Proceedings of the 9th International Conference on Learning Representations (ICLR)*, 2021.

[3] Gabriel Ilharco *et al*. Editing models with task arithmetic. In *Proceedings of the 11th International Conference on Learning 
Representations (ICLR)*, 2023

[4] Guangyao Dou *et al*. Avoiding copyright infringement via machine unlearning. *arXiv preprint arXiv:2406.10952*, 2024.

[5] Pratyush Maini *et al*. TOFU: A task of fictitious unlearning for LLMs. *In Proceedings of the Conference on Language Modeling (COLM)*, 2024.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732118939328

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Revision Note

comment:
Based on the reviewers' feedback, we have highlighted the revised sections of the paper in **red**. The updates include:
- Improving the consistency of the derivative knowledge's definition (Reviewer **[gJPq](https://openreview.net/forum?id=N2wPtFVK6o&noteId=IjJoHmgQ8q)**)
- Adding additional related works (Reviewer **[c6sN](https://openreview.net/forum?id=N2wPtFVK6o&noteId=k2029RKIGj)**)
- Including the MMLU benchmark for broader utility evaluation (Reviewer **[c6sN](https://openreview.net/forum?id=N2wPtFVK6o&noteId=k2029RKIGj)**)
- Adding more recent baselines and datasets (**all reviewers**)
- Modifying the experiment section to reflect the addition of new baselines and datasets
- Updating the qualitative result of MOUCHI for better clarity (Reviewers **[w5RU](https://openreview.net/forum?id=N2wPtFVK6o&noteId=9b9zKLG1IT)** and **[gJPq](https://openreview.net/forum?id=N2wPtFVK6o&noteId=IjJoHmgQ8q)**)
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper addresses the issue of over-forgetting during the unlearning process of Large Language Models (LLMs), where similar content that should be preserved is unintentionally removed. The authors introduce the concept of derivative knowledge, a subset of information derived from copyrighted content that must be retained during unlearning. Specifically, 1) the set of derivative knowledge is generated, and 2) a derivative loss function is proposed. The suggested method can be incorporated into various unlearning approaches, leading to improvements in both model utility and the quality of forgetting. However, most reviewers argue that the proposed method lacks technical contribution, as it simply prompts the model to generate derivative knowledge and uses KL divergence for filtering. Additionally, there is no improvement in the loss function for the derivative set, which seems to be an extension of the retain set. Besides, there are missing details regarding the implementation.