Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposes ConfounderGAN, a GAN whose generator can be used to create a noise to an image to make it *unlearnable*, by creating a spurious correlation between the image and the label. The proposed approach has been evaluated on several image classification tasks and the results show it can help reduce the accuracy of a model trained on noisy data in the *non-adaptive* setting.
strengths_and_weaknesses: 
**Strengths** 

The paper tackles an important issue. The results seem promising. However, several issues need to be addressed to make the paper more convincing.

**Weaknesses** 

- The proposed approach seems a lot like a data poisoning attack. However, discussions on data poisoning and its relation to this solution are missing. Instead, terms like encryption are used, which can be misleading. 
- The paper did not discuss the asymmetry between users and attackers as discussed in recent literature (e.g., [a]), which may give a false sense of security to users as these types of countermeasures have been proven to be ineffective.

[a]- Data Poisoning Won't Save You From Facial Recognition. (Radiya-Dixit et al., 2021)
questions: 
- Please consider discussing the relation between this solution and data poisoning in more detail. 
- I suggest, using "perturbed" images instead of "encrypted" images
- The current discussion on how attackers can bypass the defence is a bit limited. What happens when some users have both their perturbed and un-perturbed data online? Since ConfounderGAN can be obtained by any user, the attacker can also use it to train a denoiser. Overall, more discussion on the adaptive setting could help. 
limitations: Nothing to report.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposed using GAN to produce confounder noise for unlearnable examples. The proposed method address an important issue that personal data is being used for unauthorized machine learning training. Unlike existing methods that require bi-level optimizations with multiple backward passes, the proposed method can generate confounder noise in a forward pass after training, making it very practical in a real-world application. Empirically, the proposed method outperforms existing methods. 

strengths_and_weaknesses: Strengths
- Well-motivated method for efficiently generating unlearnable examples, and the context of unauthorized machine learning training is well explained. 
- The proposed method is efficient and technically sound. Existing works rely on optimizations that may not be practical for a user to generate unlearnable examples on the fly. Using GAN, the unlearnable version of the image can be generated in a forward pass, which improves usability in a practical setting. 
- Comprehensive empirical evaluations of different datasets and models are appreciated. Results demonstrated the proposed method consistently outperforms existing methods.

---
Weaknesses/Limitations:
- For experiments on the different proportions of unlearnable examples, it does not make sense to compare Polyline 5 with others. Polyline 5 is still regarded as the 100% unlearnable case. It would be interesting to see $ D_{out,en} $ with $D_{nat}$. 
- Legends on Fig 5 (c-d) color is not clear for EMN v.s. the proposed method for only $ D_{in,en} $
- Line 215, "When the model trainer downloads these images..." I believe the goal of unlearnable examples is to make the model unable to predict the protected classes/users rather than high-performing models. 
- Line 237 "we first" -> "We first"
- Once the data is released, the defender may not modifies the data anymore, and the model trainer can retroactively apply new models/methods [1]. An adaptive case should be carefully examined.
- Comparison with DeepConfuse [2], which also able to generate unleranable samples for $ D_{out,en} $

[1] Data Poisoning Won’t Save You From Facial Recognition, ICML 2021 Workshop AML
[2] Learning to Confuse: Generating Training Time Adversarial Data with Auto-Encoder. NeurIPS 2019


---
After the author's response, I increased my rating score to 7. 

A detailed analysis of the adaptive method to reverse the original image is well explained and discusses the potential limitation of the proposed method. Based on the author's response, in practice, the owner should keep the parameters of ConfounderGAN private to prevent the model trainer reverse the unlearnable data. 
questions: Please address the questions in the Strengths And Weaknesses section.


limitations: Please address the potential limitations in the Strengths And Weaknesses section.
review_title: null

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposes a GAN that makes personal image data unlearnable by DL methods for data protection. The authors utilize the confounder property present in the noise produced by the generator. This property builds spurious correlations between images and labels, disallowing the model to learn the correct mappings. The discriminator is used to ensure that this generated noise is undetectable. The authors conduct experiments using six image classification datasets, 3 of which are natural object datasets and 3 are medical datasets. Specifically, a confounder based framework has been proposed for image data encryption. 
strengths_and_weaknesses: The paper has been very well written, introduction to concepts have been well laid out and the structure of the paper is clear. The authors have done a thorough study of the past works in this particular domain. There are, however, some typos throughout the paper. For example: Line 12 -> "thereby, remaining the normal utility", Line 14 -> "The experiments are conducted in six image classification datasets, including three natural object datasets and three medical datasets" (this reads slightly ambiguous so a better choice of word for including might be consisting/comprising), Line 237 -> sentence capitalization.
questions: Section 2, Related work reviews data privacy, data poisoning and causal confounder in DL. However, only data privacy and confounder in deep learning have been discussed. Has data poisoning been combined with data privacy? There was no clear demarcation in the discussion of the two concepts under Data Privacy.
Section 5.2.1 could be better structured for ease of reading. 
limitations: There are very few possible negative societal impacts, that are not straightforward in nature. The authors have, however, discussed the positive impact, which is data privacy in DL.
review_title: null

Meta-review: metareview: All the reviewers were excited by the idea and a efficient method to solve very critical problem with rigorous experimental support. They all agreed that the paper is above bar for publications. We hope the authors will further improve the paper for camera ready submission. 
confidence: Certain
Decision: Accept