Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposed to utilize intermediate checkpoints from a single training process to protect data from unauthorized use. It also proposed a novel feature alignment (FA) technique that improves the accuracy of its proposed self-ensemble protection (SEP) method. FA uses an existing theory called neural collapse to align the last layer feature of a sample for each checkpoint to the feature mean of the target class samples. It has minor improvement in performance. But the method is simple which will make it useful. 
strength_and_weaknesses: **Strength: **

- The objective is straightforward and the problem of data protection is exciting and vital. 

- Improving performance with intermediate checkpoints from a single training is an excellent idea to save time and resources. 

**Weaknesses:**

- The feature alignment section could be explained a little better. Especially the neural collapse theory should be explained a little more since it is the primary tool used to develop FA.

- The experiments are limited. More results on different datasets comparing different models should have been shown. Only one model and dataset are tested while the proposed method is compared with the previous methods in Table 1.

- It is unclear what model and dataset are used to produce Table 3.

clarity,_quality,_novelty_and_reproducibility: Sec 3 is not well explained, it could be rewritten for better understanding. The quality is acceptable. They have done minimal experiments to support their claims. It needs more experiments. Their self-ensemble idea is interesting, but it is mainly an extension of previous work.
summary_of_the_review: The paper attempts to solve a compelling problem and proposes a simple solution. However, it could not support empirically enough. As mentioned in the weakness section, it needs a little more work and rewriting.

***
The authors have clarified the raised doubts, updating the ratings accordingly. 
***
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper is about perturbing images such that it is not possible to use them for training a model which has good performance. The authors propose to build upon existing works which use adversarial perturbations to corrupt the training set. As contributions, this work proposes to ensemble the perturbation gradients coming different model snapshots of a same training deep network. The perturbation gradients come from a feature alignment loss such that perturbed samples when passed to the DNN result in feature maps which are close to the mean feature of other classes.
strength_and_weaknesses: Strength:
1) The paper is clear and very well written.
2) Lots of experiments to support the paper claims with interesting ablation studies.

Weakness and suggestions:
1) Adversarial training easily circumvents the data corruption. The proposed method suffers from the same drawback of other existing methods.
2) L2 perturbation(not only L-inf) could complete the study. Maybe mixed perturbations could be helpful against adversarial training.
3) What about exponential moving average as a comparison to the ensemble of snapshots? It would be a way to avoid storing all these snapshots.
4) Typos in the abstract and other places for the perturbation bounds: with l_inf=8 instead of L_inf=8/255 for examples.
clarity,_quality,_novelty_and_reproducibility: Novelty lies in using a feature alignment loss and an ensemble of snapshots instead of a single DNN to attack the images. The paper is well written and clear.
summary_of_the_review: The paper is enjoyable to read but it only marginally improves over existing methods both technically and experimentally.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a simple new ensemble method for protecting data from someone training on it.  The work shows the empirical success of their method on multiple datasets and compared to strong baselines like adversarial poisons.
strength_and_weaknesses: The main strength of the paper is in the empirical success compared to strong baselines and defenses.

In Figure 3, the peak test accuracy of the proposed method is just as high as for adversarial poisons.  Is there a way to reduce this peak?  Otherwise, early stopping might be a highly effective defense.

It might be worth considering the AR poisons of “Autoregressive Perturbations for Data Poisoning” as they are a recent stronger baseline than adversarial poisons.

How did you tune the attack hyperparameters (e.g. step size) of adversarial poisons since you are using a smaller perturbation radius than they used in the experiments in their paper?

In general, the writing is a bit messy and hard to follow, but the overall structure and content is good, and I was still able to understand the writing nonetheless.

clarity,_quality,_novelty_and_reproducibility: The paper is clear enough, although the writing is hard to follow at times.  I did not notice some of the hyperparameters that are needed to re-produce the experiments, but maybe they were already there and I just did not see them.  Crucially, the hyperparameters for the competitor are important since this work does not use the same constraint space as the paper for adversarial poisons.
summary_of_the_review: In general, while the writing could use some work and there is an additional baseline that would be nice to include, I currently lean towards acceptance.  The paper provides significant empirical contribution and extensive testing.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: **Summary:** This paper takes on the problem of poisoning examples so that they are hard to train on. It is a defense against attackers who  seek to collect data in an unauthorized way (e.g., via web scraping) from an entity that wishes to share data publicly (e.g., a website storing art). The defense takes advantage of two properties of neural networks: (1) neuron collapse and (2) that gradients are very different for different checkpoints throughout training. The method involves applying gradients from different checkpoints in training to each example. Doing so seems to productively prevent new networks from learning on these modified examples.

**Strengths:** This method is quite simple in practice, and it is very efficient insofar as it doesn't require an ensemble of separately trained models to produce the results. It seems like a nice advance over previous work in this area.

**Weaknesses:** The paper is a bit hard to make sense of, and the details of the algorithm aren't that clearly described in plain text. I urge the authors to take another pass over the writing to make the paper more accessible. In addition, it's unclear (at least to me) how this method would work in non-classification settings. How would this apply to something like stable diffusion or even a self-supervised image pre-training task? Would the same protection work? Is neuron collapse at play here?

**Overall:** All of the reviewers were in favor of acceptance. This is a nice incremental result, and I agree with their assessment. It is a bit more on the incremental side, so I wouldn't be heartbroken if the paper were rejected.
justification_for_why_not_higher_score: The result is quite incremental. It's a neat result that will move the field forward and appears deserving of publication, but it isn't earthshattering in a way that requires the attention of the broader community. If it did get a spotlight, it would be because this was a new problem to me (and perhaps to many others) that deserves to be highlighted. It's a cool problem that is increasingly applicable as models are trained from giant web scrapes.
justification_for_why_not_lower_score: The reviewers had a consensus that the paper deserved to be accepted. They were all pleased with the novelty of the idea and the sufficiency of the results. I think this paper is generally pretty incremental, so I wouldn't be upset if this paper were to be rejected. The technical results are solid, but the novelty/distinction from previous work is the biggest point of concern.
Decision: Accept: poster