How Far Are We from True Unlearnability?

Authors: Kai Ye, Liangcai Su, Chenxiong Qian
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: shortcut learning, unlearnable dataset, loss landscape

Abstract:
High-quality data plays an indispensable role in the era of large models, but the use of unauthorized data for model training greatly damages the interests of data owners. To overcome this threat, several unlearnable methods have been proposed, which generate unlearnable examples (UEs) by compromising the training availability of data. Clearly, due to unknown training purposes and the powerful representation learning capabilities of existing models, these data are expected to be unlearnable for models across multiple tasks, i.e., they will not help improve the model's performance. However, unexpectedly, we find that on the multi-task dataset Taskonomy, UEs still perform well in tasks such as semantic segmentation, failing to exhibit $\textit{cross-task unlearnability}$. This phenomenon leads us to question: $\textit{How far are we from attaining truly unlearnable examples?}$ We attempt to answer this question from the perspective of model optimization. To this end, we observe the difference in the convergence process between clean and poisoned models using a simple model architecture. Subsequently, from the loss landscape we find that only a part of the critical parameter optimization paths show significant differences, implying a close relationship between the loss landscape and unlearnability. Consequently, we employ the loss landscape to explain the underlying reasons for UEs and propose Sharpness-Aware Learnability (SAL) to quantify the unlearnability of parameters based on this explanation. Furthermore, we propose an Unlearnable Distance (UD) to measure the unlearnability of data based on the SAL distribution of parameters in clean and poisoned models. Finally, we conduct benchmark tests on mainstream unlearnable methods using the proposed UD, aiming to promote community awareness of the capability boundaries of existing unlearnable methods.

Decision: accept (poster)
Reviewer scores: 6, 6, 6

Reviews and discussion:
<thread>
creation date:
1730165764264

thread_type:
official_review

<thread_object>
creator: reviewer_ydjs

<content>
summary:
This paper addresses the concept of “unlearnable examples” (UEs), which are data samples designed to prevent unauthorized model training by impairing their usability. While current methods produce UEs effective in single-task settings, the paper reveals their limitations in multi-task scenarios. The authors explore unlearnability through the lens of model optimization and introduce Sharpness-Aware Learnability (SAL) as a metric to measure parameter unlearnability. Additionally, they propose “Unlearnable Distance” (UD) to benchmark existing UE methods across tasks.

strengths:
1. The paper identifies a crucial gap in current unlearnable example (UE) methods by showing their limited effectiveness in multi-task scenarios.
2. The authors present SAL as a metric that evaluates the unlearnability of parameters, providing a fresh approach that goes beyond traditional test accuracy to measure how model parameters respond to UEs during training.

weaknesses:
While the paper highlights the limitations of existing unlearnable example (UE) methods in multi-task scenarios, it does not provide solutions or adaptations to improve UE robustness across tasks.

questions:
Given the identified limitations of existing unlearnable examples (UEs) in multi-task scenarios, could SAL and UD metrics be used to improve UE robustness across diverse tasks?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730447601247

thread_type:
official_review

<thread_object>
creator: reviewer_h1lc

<content>
summary:
This paper addresses the challenges of unlearnable examples (UEs) in the context of model training, particularly highlighting their unexpected performance in multi-task Taskonomy. The authors reveal that existing unlearnable methods fail to achieve true unlearnability, prompting an exploration of the convergence differences between clean and poisoned models through loss landscape analysis. The authors introduce two key concepts: Sharpness-Aware Learnability (SAL) for quantifying parameter unlearnability, and Unlearnable Distance (UD) for measuring data unlearnability. The paper benchmarks existing methods using UD, aiming to illuminate the limitations of current approaches and guide future research towards more effective unlearnable examples.

strengths:
- The paper is  well written and easy to follow.
- The motivation and background is explained clearly. Arguments and claims are supported by  experimental results.
- The paper provides a new perspective  to  evaluate the data unlearnability through the introduction of SAL and UD. These metrics provide valuable insights into the effectiveness of unlearnable methods and address a gap in existing evaluation methods that rely solely on post-training performance metrics

weaknesses:
Limited Scope of Experiments: While the paper investigates unlearnability in multi-task settings, the experiments primarily focus on specific datasets, such as CIFAR-10 and CIFAR-100. It would be valuable to see results on larger and more complex datasets, like ImageNet.

questions:
- Revisiting Figure 1, it highlights that the Error Minimization (EM) method struggles to perform effectively in multi-task scenarios. I'm curious if any recent methods have also faced similar limitations.
- I'm curious about the  generalizability of the evaluation metrics.  The experiments predominantly emphasize classification tasks; including evaluations on other downstream tasks, such as semantic segmentation, object detection, and even applications in large language models, would enhance the study's relevance and generalizability. This narrow focus may limit the applicability of the findings to other domains or unlearnable methods not considered in this research.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730563171753

thread_type:
official_review

<thread_object>
creator: reviewer_6uf4

<content>
summary:
The authors investigate the limitations of current unlearnable examples (UEs), which aim to protect data ownership by making data unusable for model training. They discover that UEs generated for one task (e.g., image classification) can still contribute to performance improvement in other tasks (e.g., semantic segmentation) within multi-task learning settings. To address this, they introduce the concept of Sharpness-Aware Learnability (SAL) as a new metric to measure the unlearnability of model parameters. They further propose Unlearnable Distance (UD), a metric built on SAL, to quantify the unlearnability of data. Using UD, they benchmark existing unlearnable methods, revealing the gap between current research and the goal of creating truly unlearnable examples.

strengths:
1. Novel perspective on unlearnability: The authors explore the concept of cross-task unlearnability. They highlight the limitations of existing unlearnable methods that primarily focus on single-task scenarios, revealing their ineffectiveness in multi-task settings. 

2. Focus on the training process: Instead of relying solely on test accuracy, the authors analyze the model optimization process during training to understand how UEs impact learning. This may provide a more in-depth understanding of the unlearnability mechanism compared to previous studies that focused on post-training model performance.

3. Introduction of new metrics: The authors propose two novel metrics:

    a. Sharpness-Aware Learnability (SAL): This metric quantifies the unlearnability of parameters by considering the impact of parameter updates on the loss landscape during training. 

    b. Unlearnable Distance (UD): This metric, built on SAL, measures the overall unlearnability of data by analyzing the proportion of learnable parameters in models trained on UEs compared to models trained on clean data.

weaknesses:
1. Limited explanation of multi-task unlearnability failure: While the authors identify the failure of existing UEs in multi-task scenarios, they provide a limited explanation for this phenomenon. They point towards the inherent difficulties of multi-task learning, specifically conflicts between different tasks, as a potential reason. However, a more detailed analysis of how these conflicts affect unlearnability is lacking. For instance, do UEs designed for one task interfere with the learning of other tasks? Or do the shared representations learned in multi-task models somehow overcome the unlearnability effect of UEs? The authors do not fully address these questions.

2. Limited scope of multi-task experiments: The multi-task experiments are primarily conducted on the Taskonomy dataset using only 4 tasks, which might not be representative of all multi-task learning scenarios. Exploring the effectiveness of UEs on other multi-task datasets with more diverse tasks and complexities would strengthen the claim of cross-task unlearnability failure.

3. Lack of discussion on practical implications: The authors briefly mention the need for better data protection strategies, but they do not discuss the practical implications of their findings in detail. How can the insights from this research be translated into real-world solutions for protecting data ownership? What are the potential challenges in implementing these solutions? 

4. Poor visualization of figures: Some figures are not well-visualized, especially the color, linewidth and font size are not well-chosen. For example, figures 1, 2, and 4 are hard to read in the printed version.

questions:
Please refer to the weakness.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732380985122

thread_type:
official_comment

<thread_object>
creator: authors

<content>
comment:
We thank all reviewers for the constructive and insightful efforts in evaluating this work. We have uploaded revised files, with several modifications:
1. two new image about performance of existing UEs in multi-task scenarios (one in Introduction, one in Appendix);
2. more experimental results in larger dataset (Imagenet-100) and other multi-task UEs (on taskonomy dataset) with some new findings;
3. a clearer and more complete description of the contributions to our findings and proposed indicator UD.

The above points are marked in blue (in both main paper and supplementary materials).

Besides these points, we have also revised typos, references, figure captions, and all other minor points mentioned in the reviewing process.

Thanks again for the constructive efforts in the comments and reviews.

Author
</content>
</thread_object>

</thread>
<thread>
creation date:
1733073964383

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Rebuttal and Revision Summary

comment:
**We sincerely thank you for your insightful feedback and constructive suggestions, which have significantly improved our manuscript. Revisions are highlighted in blue in the manuscript. Below, we summarize the key revisions and rebuttal made to address the reviewers' comments and ensure alignment with the conference's standards**: 
___
### **1. Additional Experiments on Multi/Cross-Task UEs**
- **Multi-Task UEs**: We add new experiments with three popular UE methods (EM, OPS and AR) and random noise for comparison on 12 different tasks, as shown in Fig. 1 which shows that existing UEs fail in multi-task scenarios.
- **Cross-Task UEs**: Ablation experiments on 4 common CV tasks with EM perturbations (see Appendix A.2). Results show UEs from single tasks struggle to transfer to other tasks and are even ineffective within their own tasks(in multi-task models).

### **2. Additional Experiments on ImageNet-100 Dataset**
We have added results on ImageNet-100 of our proposed UD and SAL metrics. The majority of unlearnable methods perform similarly on the ImageNet-100 dataset as they do on the CIFAR dataset.

### **3. More Explanation and Analysis of Failure of Existing UEs**
- It is not surprising that existing UEs fail in multi/cross-task scenarios, not only because many of them (AR, OPS, LSP) are based on **class-related linearly separable noise** (which obviously fails in non-classification tasks), but also because the multi-task learning training process (dedicated to **resolving conflicts between tasks**) can further **undermine the unlearnability** of existing UEs. 
- Essentially, based on our research findings, practical UEs need to influence the update of model parameters (specifically, **the proportion of learnable parameters**), and the changes in model parameters caused by UEs **have inconsistent trends in their impact on the loss of different tasks**.
- Given the unknown training objectives, data privacy protection based on UEs must overcome the challenges posed by multi-task scenarios to be effective in practice. We have provided two insights for future research aimed at addressing multi-task UEs, as detailed below:
   - The construction of UEs should not be decoupled from the model itself, as the manifestation of sample unlearnability stems from abnormal updates during the model parameter optimization process, with samples predating model selection in the training process. Therefore, the construction of UEs should consider the underlying training objectives and the model;
   - The construction of multi-task UEs should focus more on the feature extraction from the model backbone; only if the feature extraction of multi-task learning is affected can the performance of the model's multi-task heads be further impacted, thereby demonstrating unlearnability.

### **4. More Discussion on Practical Implications of Our Study**
We provide and summarize the practical implications of our study from two aspects:
- **For Current Work**: Unmasking the Critical Gap in UE Methods. Our study highlights the limitations of existing UE methods and underscores the importance of cross/multi-task unlearnability for data protection. Essentially, current UEs are ineffective when deployed, and we seek to alert data protectors to these inherent limitations. Additionally, we aim to provoke significant attention, calling for a comprehensive reevaluation of how UEs handle cross/multi-task unlearnability.
- **For Future Work**: Providing an Effective Tool for Evaluating UE Methods. We provide a powerful tool to assess the practical unlearnability of UE methods, introducing two metrics (i.e., SAL and UD) during training. These metrics help data protectors and researchers predict UE effectiveness, minimizing resource waste. Moreover, SAL and UD could guide the development of more effective UEs by highlighting the essential characteristics they should possess.

### **5. Presentation Improvement**
- Modified Figure 1,2,4 to enhance their readability and clarity. 
- Modified the Introduction and Section 3 (Sharpness-Aware Unlearnablity Explanation) with more detailed descriptions and more straightforward and insightful explanations.
- Added more details in the Appendix.

___

Best regards,

*The Authors*
</content>
</thread_object>

</thread>


Metareview:
metareview:
In this work, the authors attempt to answer the question of "How Far Are We from True Unlearnability" from the perspective of model optimization. The authors observe the difference of convergence process between clean models and poisoned models on a simple model using the loss landscape and find that only a part of the critical parameter optimization paths show significant differences, implying a close relationship between the loss landscape and unlearnability. Consequently, the authors employ the loss landscape to explain the underlying reasons for UEs and propose Sharpness-Aware Learnability (SAL) for quantifying the unlearnability of parameters based on this explanation. Based on this, the authors propose an Unlearnable Distance (UD) metric to measure the unlearnability of data based on the SAL distribution of parameters in clean and poisoned models. 

The novel aspect of this paper is that the authors aim to re-evaluate the true unlearnability of data in this paper and tackling this issue from the perspective of model optimization. Specifically, they introduced a metric called Sharpness-Aware Learnability (SAL) to quantify the unlearnability of parameters.

The finding of this paper is interesting, and the paper is also clearly written. The empircal analyses firmly validate the authors' finding.