Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a strategy to address a shortcoming of prior 'unlearnable' strategies which aim to prevent training on unauthorized, namely the lack of transferability across training settings and datasets. Typical strategies add perturbations generated for a specific training setting and target dataset, and so typically do not transfer. This paper proposes a strategy called Transferable Unlearnable Examples (TUE) based on Classwise Separability Discriminant (CSD) to enhance linear separability of perturbation classes to improve the transferability of unlearnable examples. This method is an extension of prior contrastive learning algorithms to also balance linear separability, with interpolation for coverage of missing classes. Experiments show its improvements over other recent methods in most cases, particularly transferability. 
strength_and_weaknesses: Strengths:
- Experiments on three datasets support that the proposed method generally improves both unlearnability and improves unlearnability transfer compared to recent methods (by a large margin in some cases, but certainly more consistently across different settings). 

Weaknesses: 
- The motivation is somewhat unclear at points. Paper clarity and organization could be improved. 

clarity,_quality,_novelty_and_reproducibility: - The proposed method is described relatively clearly, but the organization of the paper could be improved. 
    - The text is occasionally redundant, and some points are mentioned multiple times before sufficiently defined or described. Adding forward references in the introduction or related work to sections later may help improve the overall organization. 
    - It can be difficult to find some model and training details and would be difficult to reproduce some experiments with the details provided (Section 3.2.1 or the compared methods, for example). 
- The motivation is not completely clear to me; the experiments seem to aim preventing learnability in general, but the stated application is preventing training on protected user data, which would rarely be the full set of available training data. It is not clear to me that the proposed method would be effective in such a case where the data is only partly obscured. The unsupervised case is also stated as left to future work on page 2 but is given significant focus. 
- It is not clear to me how the interpolations for new classes are validated in Section 5.3 beyond overall transfer performance.
- Why does UCL have no protection in supervised settings? While the other two baselines are discussed in Section 3, this is not; it would be nice to add a short description. 
- Minor presentation notes: the space has been overly compressed around tables and figures (particularly captions); the labels in Figure 1 are too small to read; there are occasional typos (Unlearnbale, unsuperivsed, unlearnbility); \citet and \citep are often flipped. 
summary_of_the_review: This paper presents an effective strategy to promoting transferable unlearnability in image classification settings. While the performance improvements from the method are encouraging, particularly for transfer, the writing of the paper itself could be improved.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposed an unlearnable strategy, TUE. It aims to simultaneously enjoy training-wise transferability and data-wise transferability by enhancing linear separability. Experimental results show that the proposed method shows advantages in both transferability settings.

strength_and_weaknesses: Strength:
1. This paper proposed a simple but effective unlearnable strategy that can preserve both training-wise transferability and data-wise transferability.
2. The paper is easy to understand, the formula is concise, and the algorithm description is clear. 
3. The performance gain seems to be significant.
4. The visualization of unlearnable examples helps understand the effectiveness of linear separability.

Weaknesses:
1. In section 3, the authors mentioned that this paper focuses on supervised unlearnable examples, which seems to contradict the training-wise transferability of the proposed method.
2. The first term of equation 5 combines unsupervised loss explicitly, while the second term L_S combines supervised loss implicitly. What if we consider both unsupervised and supervised loss explicitly?
3. It is not clear how much interpolation affects data-wise transferability.
4. What are the backbones of EMN and SN in Table 5 and Table 6?
5. No ablation experiments on the two proposed losses.

clarity,_quality,_novelty_and_reproducibility: Clarity
The algorithm of this paper is clarity and easy to understand.
However, something is puzzling in experimental results.

Quality
This paper provides an effective strategy and some meaningful conclusions.

Novelty
The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.


summary_of_the_review: Overall, this work presents a simple, yet seemingly effective unlearnable strategy, and the paper is well written and easy to understand. But there are some deficiencies in the experiment, so I would give score 6 for now.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper presents an unlearnable strategy to prevent third parties from training on the data without authorization. The unlearnable strategy is based on class-wise separability discriminant that seeks to transfer better to other training settings and datasets, in contrast to existing methods that only work on a target training setting and a target dataset. Experiments on digit datasets suggest the feasibility of generating unlearnable examples across training settings and datasets.
strength_and_weaknesses: - The paper systematically establishes the limitations of existing supervised unlearnable methods, and then proposes a simple strategy to make the unlearnable examples transfer to unsupervised and other datasets. 

- The proposed strategy is simple, and adds a class-wise separability discriminant loss to the contrastive loss for learning a transferable representation.

- Interpolation data-wise transferability strategy is not convincing. Just linearly interpolating class samples or creating new class categories is not going to be representative of the real-world situations. Further experimentation is required to establish the claims to this end.

- Comparison with supervised contrastive learning that makes use of samples within the class categories as positives, and other class categories as negatives would be useful, as supervised contrastive learning yields representations that are similar to the ones desired from class separability discriminant. 

- It is also not clear to me the rationale of the paper. If the goal is to avoid unauthorized use of the data, the data could be encrypted, or the security authentication protocol can be defined accordingly. Adding a noise to make it linearly separable seems difficult for challenging datasets.

clarity,_quality,_novelty_and_reproducibility: The paper is well-written. It identifies the limitation of supervised learnable examples and combines it with an unsupervised representation approach to make it data-wise and training-wise transferable. Novelty of the paper is very incremental, and the intended use case is somewhat clear. Experimental set-up is sound and results are reproducible.
summary_of_the_review: The paper presents an application of representation learning to dataset protection and security. It identifies the limitation of supervised unlearnable examples in transferring to a new situation, and presents a remedy to make unlearnable examples transferable. More experiments on use cases and challenging datasets will improve the quality of the paper.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes Classwise Separability Discriminant (CSD), an unlearnable strategy to transfer the unlearnable effects (perturbations to avoid unauthorized data usage) to other training settings and datasets by enhancing the linear separability. Experiments showcase transferability of the proposed unlearnable examples across training settings and datasets
strength_and_weaknesses: Strengths
-the paper proposes training-wise and data-wise transferability for unlearnable effects using contrastive learning
- the proposed methods uses linear separability in addition to similarity between augmentations 
-experiments are extensive and detailed and show the improvement in data-wise and training-wise transferability 


clarity,_quality,_novelty_and_reproducibility: The paper is well written and novel. The experimental settings are detailed for reproducibility


summary_of_the_review: The paper addresses an important problem of improving data-wise and training-wise transferability of unlearnable examples. The proposed approach is novel and detailed results show it outperforms its counterparts.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The study considers the setting of "unlearnable examples", the introduction of appropriately crafted samples that, when combined with real data, distort learning from the (perturbed) dataset. The authors study how to transfer such examples to other datasets, and propose a methodology for doing so.

Strengths of the paper include an interesting setting, a clearly motivated method through Clustering Separability Discriminant analysis, and a very thorough/extensive experimentation method. Reviewers raised several concerns, too many to list here; nevertheless the authors eventually addressed them via the rebuttal process, adding a significant amount of new experiments to improve their paper. Overall, they should be encouraged to add these to the supplement in the final submission. On the negative side, the paper could benefit from polishing, which was found by several reviewers to need improvement.
justification_for_why_not_higher_score: Results are convincing, especially after the extensive added experiments, but presentation/organization could be better.
justification_for_why_not_lower_score: Results are convincing and experiments are quite thorough, especially after the rebuttal.
Decision: Accept: poster