Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: Paper considers a setting of `unlearnable examples' where a given dataset is perturbed in a way to make it hard to learn the true task. In essence, data here gets perturbed with correlated noise such that when learning is attempted, models learn to focus on the noise rather than on the true features useful for generalisation. While prior work focused on generating class-wise poisons, in this work the noise is generated per sample using a Markov process, producing linearly separatable noise. Paper thoroughly evaluates the setting and demonstrates that the approach effectively stops generalisation when the whole dataset is poisoned and struggles in a similar way in presence of adversrail training or dilution with clean data. 
strengths_and_weaknesses: 
Strengths:
+ Interesting setting
+ Idea of hardness of learning is rather fascinating

Weaknesses:
+ Unclear how much of the evaluation is an artifact of chosen optimisation hyperparameters
+ Unclear how one compares performance of different correlated noises
questions: Thank you very much for the paper, it is a very interesting read! I only have a handful of questions:

1. Are tables 4 and 5 computed over a number of models? Given how close the numbers are, it would be great to know if the differences are observed on distributional level, not just per model
2. Given that we can produce arbitrary correlated noise of different flavors, how should one think about it? What is the fundamental difference between the noises in the related literature and the one produced in the paper? This naturally leads to my final question.
3. Given the argument of easier learnability of different noises, it turns the question to how much observed behaviour is an artifact of the optimisation procedure itself? Did you try running the experiments with different lr/optimiser options? 

Minor:
* Punctuation missing around eqs in some places
limitations: N/a
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposes a new data poisoning attack to prevent data scraping. The proposed method adds class conditional autoregressive (AR) noise to training data to prevent people from using the data for training, and the method is data and model independent, which means that the same noise can be used to poison different datasets and models of different architectures.

The intuition behind the idea is that easy to learn noise is more effective at data poisoning, and AR noise generated in the proposed way is easy for neural network to learn. The authors show that a manually specified 3-layer CNN with AR filter can easily learn class information from the AR noise. Experiments on four benchmark datasets (CIFAR10, STL10, SVHN, CIFAR100) show that the proposed method performs better than other four baselines (Error-min, Error-max, Regions, Random noise).
strengths_and_weaknesses: Strengths:

- The proposed method is novel as autoregressive process hasn't been used before to do data poisoning. The method is easy to implement and the same AR coefficients can be used for different datasets and architectures as long as the numbers of classes are the same. Though code is not available, pseudo code (algorithms) and implementation details are provided. It is better that if actual code can be provided for reproduction of the results.

- The paper is well-written and easy to follow. Empirical results on four different datasets show that the method performs better than other baselines, both under normal setting and defense settings.

Weakness:

- Though the proposed method performs better than other baselines compared in the paper, when tested against adversarial training, the performance is not satisfactory. It performs similarly to other baselines under this setting and the poisoning effect is not good, especially when the radii is large.

- As pointed out in the paper, assuming that all the data can be poisoned is not realistic. In section 4.3.3, the poisoning methods are evaluated using a mix of poisoned and clean data. Under this setting, the performance of the proposed method is not good and similar to those of other baselines.


questions: About the process of AR noise generation:

- It is clear how to generate AR noise at the beginning inside the sliding window. How about the subsequent steps? Take the example in Figure 2 as an example, if the sliding window slides one step to the right, there are three values to be generated. Are $x_{t-7}$ up to $x_t$ used to generate the next one ($x_{t+1}$)? Then $x_{t-6}$ up to $x_{t+1}$ are used to generate $x_{t+2}$, and so on. 
limitations: The author points out that the method does not perform well against adversarial training and experiments show that when evaluated using a mix of poisoned and clean data, the performance is also not good.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposed autoregressive poisoning techniques to protect data from being exploited by unauthorized machine learning models. The proposed method does not rely on optimizations while generic towards different model architectures and different datasets. This paper also provides insight into why the proposed method is effective. 

strengths_and_weaknesses: Strengths:
- The proposed method is efficient and technically sound. Existing works rely on optimizations which is the bottleneck. The proposed method does not rely on optimizations, and the parameters for AR are easy to find.  
- The existing works are also shown that do not transfer well between model architectures or datasets. Experiment results show that one set of AR is generic across different architectures or datasets. 
- The efficient and generic can be very practical considering real-world applications.
- Experiment results also demonstrated AR generated unlearnable examples are more robust towards augmentations.

---
Limitations:
- Once the data is released, the defender may not modifies the data anymore, and the model trainer can retroactively apply new models/methods [1]. The adaptive case should be carefully examined. Consider that if the parameters for AR are leaked, can it be used to recover the original image? Or if a portion of the clean images are leaked, using pair of clean and unlearnable versions, is it possible to reverse the AR process? 
- In section 3.3, the assertion that the noises are easy to learn is more effective for poisoning, this could also mean they are easy to detect. Such as calculating sample-specific loss at the end of each training epoch. Although only detecting such samples does not make them "learnable," but adaptive method (if there are any) can be applied to these samples. Or the model trainer could wait for future advancement for the recovery method as mentioned in [1]. 

[1] Data Poisoning Won’t Save You From Facial Recognition, ICML 2021 Workshop AML

---
After the author's response, I increased my rating score to 7. My main concerns over possible reverse operation if parameters are leaked have been well addressed. 

questions: - In experiments section line210:  "We say that poisoning effectiveness drops from setup A to setup B if the network from poison-trained on setup B has higher test set accuracy than the network poison-trained on setup A. " I find this is confusing.
- For experiments in Table 4, for clean only, is it the same subset of data as in mixing poisons/clean? 
limitations: Please address the potential limitations in the Strengths And Weaknesses section.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper proposes to use autoregressive processes to generate perturbations for data poisoning. The generated perturbations, despite looking complex, are actually very simple. One advantage of the proposed method is that its generated perturbations are dataset and architecture independent. The paper evaluates the proposed method on multiple datasets and networks, showing the effectiveness of the perturbations when the poison rate is high.
strengths_and_weaknesses: The strengths of this paper include
1. The proposed attack method is interesting.
2. The proposed method has good transferability.

But I still have the following concerns:

1. Is the proposed method only applicable to $\ell_2$ norm? The paper uses the sentence "We measure AR perturbations
in $\ell_2$ because measuring in $\ell_\infty$ would underestimate the extent to which these perturbations are less perceptible than purely  $\ell_\infty$ random noise" to explain why it uses $\ell_2$ norm. But this sentence is hard to follow, and this short explanation is not convincing. The paper should provide more clear and convincing explanation about why it *only* uses $\ell_2$ norm.

2. The proposed attack requires high poison rate to be effective? In most experiments, the paper uses poison rate 1. In Table, the lowest poison rate is 0.6. The assumption of high poison rate is very strong. In practice, if the data is collected from multiple sources, then the attack is not effective? In the case that the data is collected from one source (the adversary), the entity who trains the model would be more cautious about the quality of data due to the high risk when the data only comes from one source.

3. Section 3.3 is not easy to follow, and the logic is not very clear. I think Section 3.3 is one of the most important parts in the paper since it explains why the proposed method works. After reading Section 3.3, I am still very confused. The relation between Lemma 3.1 and the effectiveness of the proposed method in *poisoning attacks* is not obvious. 

questions: 1. Is the proposed method only applicable to $\ell_2$ norm?
2. The proposed attack requires high poison rate to be effective?
3. Is the proposed method only applicable to computer vision tasks?
limitations: 1. The paper only studies $\ell_2$ norm.
2. The poison rate is high. The lowest poison rate studied in the paper is 0.6.
3. The theoretical analysis is not sufficient. The relation between Lemma 3.1 and the effectiveness of the proposed method in poisoning attacks is not obvious.
review_title: null

Meta-review: metareview: The paper proposed a novel auto-regressive perturbation method to make the data unlearning. The method is independent to models and data, making it more easy to be used. Reviewers found the idea is novel and intuitively reasonable. The authors responded to reviewers' detailed questions about the method and experiments. The rebuttal succeeded to remove the confusions and convince us about the empirical significance. We suggest the authors improve the paper according to the review comments in the next version. 
confidence: Certain
Decision: Accept