Autoregressive Perturbations for Data Poisoning
Authors: Pedro Sandoval-Segura
Vasu Singla
Jonas Geiping
Micah Goldblum
Tom Goldstein
David W. Jacobs
Venue: NeurIPS 2022 Accept (2022)
Keywords: autoregressive processes
poisons
data poisoning
data protection
imperceptible perturbations
adversarial machine learning

Abstract: The prevalence of data scraping from social media as a means to obtain datasets has led to growing concerns regarding unauthorized use of data. Data poisoning attacks have been proposed as a bulwark against scraping, as they make data ``unlearnable'' by adding small, imperceptible perturbations. Unfortunately, existing methods require knowledge of both the target architecture and the complete dataset so that a surrogate network can be trained, the parameters of which are used to generate the attack. In this work, we introduce autoregressive (AR) poisoning, a method that can generate poisoned data without access to the broader dataset. The proposed AR perturbations are generic, can be applied across different datasets, and can poison different architectures. Compared to existing unlearnable methods, our AR poisons are more resistant against common defenses such as adversarial training and strong data augmentations. Our analysis further provides insight into what makes an effective data poison. 

But Autoregressive Perturbations for Data Poisoning
Pedro Sandoval-Segura" Vasu Singla !+
Jonas Geiping' David W. Jacobs'
Micah Goldblum
Tom Goldstein
Unniversity of Maryland
NNew York University
{psando, vsingla, jgeiping, tomg,
dwj} [wj]eumd.com edu goldblumenyu. edu
Abstract
The prevalence of data scraping from social media as a means to obtain datasets has led to growing concerns regarding unauthorized use of data. Data poisoning attacks have been proposed as a bulwark against scraping, as they make data "unlearnable" by adding small, imperceptible perturbations. Unfortunately, existing methods require knowledge of both the target architecture and the complete dataset so that a surrogate network can trained, the parameters of which are used generate the attack. In this work, we introduce autoregressive (AR) poisoning, a method that can generate poisoned data without access to the broader dataset. The proposed AR perturbations are generic, can be applied across different datasets, and can poison different architectures. Compared existing unlearnable methods, our AR poisons are more resistant against common defenses such as adversarial training and strong data augmentations. Our analysis further provides insight into what makes effective data poison.
Introduction
Increasingly large datasets are being used to train state-of-the-art neural networks 24 26 25
collecting enormous datasets through web scraping makes it intractable for a human to review samples in meaningful way obtain consent from relevant parties B. In fact, companies have already trained commercial facial recognition systems using personal data collected from media platforms 15.. To prevent the further exploitation of online data for unauthorized or illegal purposes, imperceptible, adversarial modifications to images can be crafted to cause erroneous output for a neural network trained the modified data [12 This crafting of malicious perturbations for the purpose of interfering with model training is known as data poisoning.
In this work, we focus poisoning data to induce poor performance for network trained on perturbed data. This kind of indiscriminate poisoning. which seeks to damage average model performance, is often referred to as an availability attack [I] 2 40 [8]
[9] [C Because we assume
the data is hosted on a central server controlled by the poisoner, the poisoner allowed to perturb large portion of it. Throughout this work. unless stated otherwise, poisoning
the entire dataset, refers to the perturbing of every image in the training dataset. This makes the creation unlcarnable data different from other poisoning methods, such backdoor
[ and targeted poisoning
attacks 28
We introduce autoregressive (AR) data poisoning for degrading overall performance of neural networks on clean data. The perturbations that we additively apply to clean data are generated by AR processes that are data and arehitecture-iddeeedded where each new element is a linear combination of p previous ones, plus noise. This means AR perturbations are cheap to generate, not requiring any optimization backpropagation through network parameters. AR perturbations are generic; the same set AR processes can be re-used to
An AR(p) process is Markov chain,
Authors contributed equally.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

96x96
AR Poison
11.65% Accuracy
STL-10
Clean
83.65% Accuracy
AR Process
AR Process K
32x32
AR Poison
11.75% Accuracy
CIFAR-10
Clean 93.62% Accuracy
Figure 1: The same set of AR processes can generate perturbations for different kinds of data without access to surrogate network or other images in a dataset. A ResNet-18 18 trained on these AR poisons is unable to generalize to clean test data, even with the help of strong data augmentations.
generate diverse perturbations for different image sizes and new datasets, unlike other poisoning methods which need to train surrogate network the target dataset before crafting perturbations.
Our method also provides new insight into why data poisoning works. We work on top of the result that effective poisons are typically easy to learn [27] and construct AR perturbations which are separable by a manually-specified CNN. Working under the intuition that highly separable perturbations should be easily learned, we use the manual specification of parameters as a way of demonstrating that our AR perturbations are easily separable. Our manually-speccffed CNN makes use of what we call AR filters, which are attuned to detect noise from specific AR process. AR poisoning's 's effectiveness is competitive or better than error-maximizing maximmaxiiizing, error-minimizing and random noise poisoning across a range of architectures, datasets, and common defenses. AR poisoning represents paradigm shift for what a successful indiscriminate poisoning attack looks like, and raises the question of whether strong indiscriminate poisons need to be generated by surrogate networks for a given dataset.
Background & Related Work
Error-minimizing and merrim-maimizzing Noise. To conduct poisoning attacks on neural networks, recent works have modified data to explicitly cause gradient vanishing 31 to minimize the loss with respect to the input image 188. Images perturbed with error- minimiziiiiig noises are a surprisingly good data poisoning attack. A ResNet-18 18 (RN-18) trained on a CIFAR-10 20 sample- minple-wiseeee error- minimizing poison achieves 19.9% final test accuracy, while the class-wise variant achieves 16.4% final test accuracy after 60 epochs of training [8] More recently, strong adversarial attacks, which perturb clean data by maximizing the loss with respect to the input image, have been shown to be the most successful approach thus far [TO] An maximiziig poison can poison a network to achieve 6.25% test accuracy on CIFAR-10 10. But both error- groormiinmizing and error- paroommxxmizzin poisons require surrogate network, from which perturbations are optimized. The optimization can be expensive. For example, crafting the main CIFAR-10 poison from [00] takes roughly 6 hours on 4 GPUs. In contrast, our AR perturbations do not require access to network parameters and can be generated quickly, without the need for backpropagation or GPU. We provide technical overview of error- prror-minimizing and error- mroo-maximizing perturbations in Section 5
Random Noise. Given their simplicity, random noises for data poisoning have been explored as necessary baselines for indiscriminate poisoning. If random noise, constrained by an foo norm, is applied sample-wise to every image in CIFAR-10, 10, a RN-18 18 trained on this poison can still generalize to But if the noise applied class-wise, where every image of class is modified with an identical additive perturbation, then a RN-18 trained on this CIFAR- 10 poison will achieve around chance accuracy; i.e. -10% (39 18 27 The random perturbations of 39 consist of fixed number of uniform patch regions, and are nearly identical to the class- wise poison, called "Regions- 16." from (27] All the random noises that we consider are class-wis wisse and we confirm they work well in standard training setup using RN-18, but their performance varies across architectures and they are rendered ineffective against strong data augmentations like Cutout 77. CutMix 4II, and Mixup 42 Conversely, our AR poisons degrade test performance more than error-maximizing error- minimizing, and random poisons on almost every architecture. We show that AR perturbations are effective against strong data augmentations and can even mitigate some effects of adversarial training.
the test set, with -90% accuracy [C [8]

---

Understanding Poisoning, A few works have explored properties that make for effective poisons. For example. [27] find that poisons which are learned quickly have a more harmful effect on the poison-trained network, suggesting that the more quickly perturbations help minimize the training loss, the more effective the poison is. 3 perform a related experiment where they use a single linear layer, train on perturbations from variety of poisoning methods, and demonstrate that they can discriminate whether a perturbation is saror-minimizing or aroo-mmximizing with high accuracy. We make use of ideas from both papers, designing AR perturbations that are provably separable and Markovian in local regions.
Other Related Work. Several works have also focused on variants of "unlearnable" poisoning attacks. p propose to employ gradient alignment [[]] to generate poisons. But their method is computationally expensive; it requires surrogate model to solve a bi-level objective. A0 propose generation of an unlearnable dataset using neural tangent kernels. Their method also requires training surrogate model, takes a long time to generate, and does not scale easily to large datasets. In contrast, our approach simple and does not require surrogate models. [23) propose an invertible transformation to control learnability of a dataset for authorized users, while ensuring the data 35 showed that data poisoning methods can be broken using adversarial training. [30] and 37 propose variants of error-minimizing noise to defend against adversarial training. Our AR poisons do not focus on adversarial training. While adversarial training remains strong defense, our AR poisons show competitive performance. We discuss adversarial training in detail in Section 4.3.2 A thorough overview of data poisoning methods, including those that do not perturb the entire training dataset, can be found in [[2]
remains unlearnable for other users.
3 Autoregressive Noises for Poisoning
3.1 Problem Statement
We formulate the problem of creating a clean-label poison in the context of image classification with DNNs, following [1]. For a K-class classification task, we denote the clean training and test datasets as De and D., respectively. We assume De, De ~ D. We let fe represent a classification DNN with parameters 0. The goal is to perturb De into poisoned set D, such that when DNNs are trained on Dp. they perform poorly on test set D,.
Suppose there are n samples in the clean training set, i.c. D. ((x,))))) where I; F are the inputs and E {I,..., K} are the labels. We denote the poisoned dataset as D,- = { y1) ) where x/. = x; + 5, is the poisoned version of the example x; E DD and where 6; E A C IRd is the perturbation. The set of allowable perturbations, A, usually defined by [|0||p < where || Ilp the lp norm and e is set to be small enough that it does not affect the utility of the example. In this work. we use the εg norm to constrain the size of our perturbations for reasons we describe in Section 3.4
Poisons are created by applying a perturhation to a clean image in either a class-wise or sample-wise manner. When a perturbation is applied class-wise, every sample of given class is perturbed in the same way. That is, xr{ = I; + 5g. and 6y. E Ac = {51, -..,..., ōx) between the perturbation and the true label, it should not be surprising that class-wise poisons appear to trick the model to learn the perturbation over the image content, subsequently reducing generalization to the clean test set. When a poison is applied sample-wise, every sample of the training set is perturbed independently. That is, x = x; + ō; and d₂ E As perturbations can be recovered by taking the average image of a class, these should therefore be easy to remove. Hence, we focus our study on sample-wisg wise instead of class-win wiie poisons. We still compare simple, randomly generated class-wise noises shown by [[]] to further demonstrate the effectiveness of our method.
). Due to the explicit correlation
..., Because class-wise
All indiscriminate poisoning aims to solve the following levll objective:
max maxE E(,,))D,, [C(f(x), 0(6))
(I)
SEA
0(6) = arg min E [C(f(x; + 8.), (1:0))))
(2)
3

---

Xe-1
Azvd Xhoi
X1
Gaussian start signal
Use AR process window compute next value
Sliding window
Crop scale
Figure 2: To generate 2D noise for single channel using 3 3 sliding window, first sample Gaussian noise for the first columns and rows. Then, we use an AR(8) process within a sliding window to compute the next value. Finally, we crop and scale the perturbation to be within & constraint.
Eq. 2 describes the process training a network on poisoned data; i.e. H; perturbed by 6, Eq. states that the poisoned network should maximize the loss, and thus perform poorly, clean test data.
Different approaches have been proposed to construct 6,. Both error-maximizing [0] and error- minimizing [8] poisoning approaches use surrogate network, trained on clean training data, to optimize perturbations. We denote surrogate network parameters as l". maroo-mmximizzin poisoning 10 proposes constructing that maximize the loss of the surrogate network on clean training data:
max B (F [C(f(2; 8,), ye: y+) 8*)] SEA
(3)
whereas error-minimizing poisoning [8] solve the following objective to construct 6, that minimize the loss of the surrogate network clean training data:
min E (x,,))D [C[[(x; + 61), ye; 0z)
(4)
SEA
In both crror-maximizing and error error-mininen minimizing poisoning the adversary intends for a network, f. trained the poison to perform poorly the test distribution D,, from which De was also sampled. But the way in which both methods achieve the same goal is distinct.
3.2 Generating Autoregressive Noise
Autoregressive (AR) perturbations have particularly useful structure where local regions throughout the perturbation are Markovian, exposing a linear dependence on neighboring pixels B8). This property is eritical it allows for particular filter perfectly detect noise from specific AR process, indicating the noise is simple and potentially easily learned.
We develop sample wise poison where clean images are perturbed using additive noise. For each a;; the clean training dataset, our algorithm crafts a &, where ||0:112 S 6, so that the resulting poison image is x Iz + 6; The novelty of our method is in how we find and use autoregressive (AR) processes to generate 6, In the following, let refer to the tth entry within sliding window of 6;-
An autoregressive (AR) process models the conditional mean of x,, function of past observations X X,-1, X1 4-22..., Xt-p the following way:
X 31x,111 + 32x1-2 +... BpXt-p +
(5)
where & is an uncorrelated process with mean zero and 3, are the AR process coefficients. For simplicity, we set = 0 in our work. An AR process that depends on past observations is called an AR model of degree p. denoted AR(p). For any AR(p) process, we can construct a size p filter where the elements are 3p,..., 31 and the last entry of the filter is -1. This filter produces a zero response for any signal generated by the AR process with coefficients 3p..., 33 We refer to this filter AR filter, the utility of which is explained in Section 3 and Appendix A.I A.I

---

Process Process Process
Sample 3 Sample 2 Sample
Clean
AR
Error-Min Error-Max Regions-4 Regions-16 Random
Figure 3: Left: Normalized samples of perturbations generated by 3 AR processes. Right: Poisoned
images and the corresponding perturbation for randomly selected CIFAR- image.
Suppose we have a K class classification problem of x W x C dimensional images. For each class label Vi+ construct set Ay, of AR processes, one for each of the C' channels. For each of the C' channels, we will be applying an AR process from Ay. inside a V x V sliding window. Naturally, using an AR process requires initial observations, so we populate the perturbation vector 5, with Gaussian noise for the first V - columns and rows. The V x V sliding window starts at the top left corner of o,. Within this sliding window, we apply the AR(V2 1) process: the first 1 entries in the sliding window are considered previously generates (or randomly initialized) entries in the 2D array 6,. and the yd entry is computed by Eq. 5 The window is slid left to right, top to bottom until the first channel of 6, is filled. We then proceed to use the next AR(V2 - 1) process in Ay, for the remaining C channels. Finally, we discard the random Gaussian rows and columns used for initialization, and scale 6; to be of size in the fz-norm. Note that this sliding window procedure resembles that of a convolution. That is by design, and we explain why it important in Additional details are
Section 3.3, A high-level overview of this algorithm is illustrated in Figure 2 in Appendix A.3.2 While we describe our use of AR processes on C-channel images, our method could, in principle, be applied to data other than images. Note that these AR perturbations are fast to generate, do not require pre-trained surrogate model, and can generated independently from the datu.
3.3 Why do Autoregressive Perturbations Work?
Perturbations that easy to learn have been shown to be more effective at data poisoning [2 Intuitively, a signal that is easily interpolated by a network will be quickly identified and used as a "shorteut," whereas complex and unpredictable patterns may not be learned until after network has already extracted useful content-based features 29 Thus, we seek imperceptible perturbations that are easy learn. We propose simple hypothesis: if there exists a simple CNN that can classify autoregressive signals perfectly, then these signals will be easy to learn. The signals can then be applied to clean images and serve as shorteut for learning by commonly-used CNNs.
Autoregressive perturbations, despite looking visually complex, are actually very simple. To demon- strate their separability, manually specify the parameters of simple CNN that classifies AR perturbations perfectly by using AR filters. In the following, we prove AR filters satisfy an important property.
Lemma 3.1. Given AR perturbation 8, generated from AR(p) with coefficients B1, Po there exists a linear, shift invariant filter where the cross-correlation operator produces a zero response.
We provide a proof in Appendix A.I The construction of an AR filter that produces a zero response for any noise generated from the corresponding AR process is useful because which makes use of solely these AR filters to classify signals. That is, given any AR perturbation, the AR filter with the zero response correctly designates the AR process from which the perturbation was generated. We verify this claim in Appendix A.2 by specifying the 3-layer CNN that can perfectly classify AR perturbations.
construct CNN
Crucially, we are not interested in learning classes of AR signals. Rather, we are interested in how quickly model can learn classes of clean data perturbed by AR signals. Nevertheless, the
5

---

characterization of our AR perturbations as easy to learn, demonstrated by the manual specification of 3-layer CNN, is certainly an indication that, when applied to clean data, AR perturbations can serve as bait for CNNs. Our experiments will seek to answer the following question: If we perturb each sample in the training dataset with an imperceptible, yet easily learned AR perturbation, can we induce learning "shortcut" that minimizes the training loss but prevents generalization?
3.4 Finding AR Process Coefficients
We generate AR processes using a random search that promotes diversity. We generate processes one-at-a-time by starting with a random Gaussian vector of coefficients. We then scale the coefficients so that they sum to one. We then append -1 to the end of the coefficients to produce the associated AR filter, and convolve this filter with previously generated perturbations. We use the norms of the resulting convolution outputs norms is below a cutoff T. then we deem the AR process too coherent with previously generated perturbations the coefficients are discarded and we try again with different random vector.
measure of similarity between processes. If the minimum of these
Once the AR process coefficients are identified for a class, we use them to produce a perturbation 6; for each image in the class. This perturbation is scaled to be exactly of size in the f2-norm. To level the playing field among all poisoning methods, we measure all perturbations using an l norm in this work. A more detailed description of this process can be found in Appendix A.3.1
Experiments
We demonstrate the generality of AR poisoning by creating poisons across four datasets, including different image sizes and number of classes. Notably, we use the same set of AR processes to poison SVHN [222, S0L-10 (6, and CIFAR-10 20 since all of these datasets are 10 class classification problems. We demonstrate that despite the victim's 's choice of network architecture, AR poisons can degrade a network's 's accuracy on clean test data. We show that while strong data augmentations are an effective defense against all poisons we consider, AR poisoning is largely resistant. Adversarial training and diluting the poison with clean data remain strong defenses, but our AR poisoning method competitive with other poisons we consider. All experiments follow the same general pattern: train a network on a poisoned dataset and then evaluate the trained network's 's performance on clean test data. A poison is effective if it can cause the trained network to have poor test accuracy clean data, so lower numbers are better throughout our results.
Experimental Settings. We train a number of ResNet- 18 (RN-18) 14] models different poisons with cross-entropy loss for 100 epochs using a batch size of 128. For our optimizer, we use SGD with momentum of 0.9 and weight decay of 5 x 10-4. We use initial learning rate of 0.1, which decays by factor of 10 on epoch 50. In Table 2 we use use the same settings with different network architectures.
4.1 Error-Max, Error-Min, and other Random Noise Poisons
SVHN CIFAR-10, and CIFAR-100 100 20 poisons considered in this work contain perturbations of size c 1 in lg, unless stated otherwise. For STL-10 [6], all poisons use perturbations of size = 3 in & due to the larger size of STL-10 images. In all cases, perturbations
normalized and scaled to be of size e in f2, are additively applied to clean data, and are subsequently clamped to be in image space. Dataset details can be found in Appendix A.4 A sampling of poison images and their corresponding normalized perturbation can be found in Figure 3 and Appendix [.8] In our results,
class-wise poisons are marked with and sample-win wise poisons are marked with
Error-Max and Error-Min Noise. To generate marommiaimiiiin poisons, we use the open-source implementation of [0). In particular, we use a 250-step Eg PGD attack to optimize Eq. 3 To
generate error-minee moiommiiiiizznggg poisons, we use the open- spen-source implementation of [18, where 20-step l PGD attack is used to optimize Eq. (4h. For error- monommmiiiizing poisoning, we find that moving in &
normalized gradient directions is ineffective at reaching the required universal stop error [8] so we move in signed gradient directions instead (as is done for lox PGD attacks).
Regions-4 and Regions- -6 Noise. Synthetic, random noises are also dataset and network independent. Thus, to demonstrate the strength of our method, include three class-wise random noises in our

---

Table 1: Dataset Independence. AR noises are effective across variety of datasets. For SVHN, STL-10, 10, and CIFAR- 10, we use the same 10 AR processes to generate sample- wise noises, We display clean test accuracy of RN-18 when trained using standard augmentations on different poisons. Class-wise poisons are marked with and sample-wise poisons are marked with ..
SVHN
STL-10
CIFAR-10
CIFAR-100
Clean
96.40
83.65
93.62
75.30
Error-Max 10
80.80
17.26
26.94
4.87
Error-Min 18
96.82
80.71
16.84
74.58
Regions-4
9.80
41.36
20.75
9.14
Regions- 16
6.39
31.21
15.75
2.99
Random Noise
9.68
72.07
18.45
73.90
Autoregressive (Ours)
6.77
11.65
11.75
4.24
Table Architecture Independence. CIFAR-10 test accuracy for a variety of model architectures trained on different poisons, Error-Max and Error-Min poisons are crafted using RN-18.
RN-18
VGG-19
GoogleNNo MobileNet EfficientNet
DenseNet
ViT
Error-Max [0
16.84
20.34
15.31
15.38
16.21
18.02
41.70
Error-Min [18]
26.94
22.39
32.18
21.36
23.86
28.21
40.95
Regions-4
20.75
25.39
25.25
22.23
18.64
24.21
35.18
Regions-16 16
15.75
19.86
12.97
19.67
16.94
24.16
25.49
Random Noise
18.45
81.98
12.53
46.61
86.89
9.96
74.47
AR (Ours)
11.75
12.35
9.24
15.17
13.47
14.90
19.66
experiments. To generate what we call a Regions-p noise, we follow (
we sample
RGB vectors of size from a Gaussian distribution and repeat each vector along height and width dimensions, resulting in a grid-like pattern of p uniform cells or regions. Assuming a square image of side length L. Regions-p noise contains patches of size VP
VP
Random Noise. We also consider class-wise random noise poison, where perturbations for each class are sampled from Gaussian distribution.
4.2 AR Perturbations Dataset and Architecture Independent
Unlike error-maximizing and error-minimizing poisons, AR poisons are not dataset-specific. One cannot simply take the perturbations from an error-maximizing or error-minimizing poison and apply the same perturbations to images of another dataset. Perturbations optimized using PGD are known to be relevant features, necessary for classification [10] [9 Additionally, for both these methods, crafting network trained on clean data is needed to produce reasonable gradient information. In contrast, AR perturbations are generated from dataset-independee AR processes can be used to generate the same kinds of noise for images of new datasets. Building from this insight, one could potentially collect a large set of K AR processes to perturb any dataset of K or fewer classes, further showing the generality of our method.
AR processes. The same set of
In Table I we use the same AR processes to generate noise for images SVHN, STL-10, and CIFAR-10. AR poisons are, in all cases, either competitive the most effective poison poison-trained RN-18 reaches nearly chance accuracy on STL-10 and CIFAR-10. and being the second-best SVHN and CIFAR-100. The generality of AR perturbations to different kinds of datasets suggests that AR poisoning induces the most easily learned correlation between samples and their corresponding label.
We also evaluate the effectiveness of our AR poisons when different architectures are used for training. Recall that maxiommaxiiizing and error- arror-mmnmmizing poisoning use a crafting network to optimize the input perturbations. Because it may be possible that these noises are specific the network architecture, we perform evaluation of test set accuracy on CIFAR- 10 after poison training VGG-19
[2]. GoogLeNet [K]. MobileNet (6)), EfficientNet Md DenseNet [17. and VIT K Our ViT uses
patch size of 4. In Table 2 show that Error-Mar and Error-Min poisons generalize relatively

---

Table 3: Strong Data Augmentations. CIFAR-10 10 test accuracy RN-18 when training using standard augmentations plus Cutout, CutMix, or Mixup on different poisons, where clean images are perturbed within an & ball of size
Standard Aug
+Cutout
+CutMix
+Mixup
Error-Max
25.04
25.38
29.72
38.07
Error-Min [
48.07
44.97
53.77
53.81
Regions-4
57.48
67.47
67.72
66.80
0.5
o Regions-16
47.35
39.64
32.67
49.02
Random Noise
93.76
93.66
91.80
94.16
Autoregressive (Ours)
14.28
12.36
18.02
14.59
Error-Max TO
16.84
18.86
21.45
30.52
Error-Min [18
26.94
29.38
25.04
43.36
Regions-4
20.75
21.89
28.61
40.60
o Regions-16
15.75
19.61
16.67
20.81
Random Noise
18.45
12.61
12.63
23.64
Autoregressive (Ours)
11.75
11.90
11.23
11.40
well across range related CNNs, but struggle with ViT, which is a transformer architecture. In contrast, our AR poison is effective across all CNN architectures and is the most effective poison against VIT. Our AR poison is much more effective over other poisons in almost all cases. achieving improvements over the next best poison of 4% on RN-18, 5.8% on VIT, and 7.5% on GoogLeNet. The design of AR perturbations meant to target the convolution operation, so it surprising to see transformer network be adversely affected. We believe our AR poison is particularly effective on GoogLeNet due the presence of Inception modules that incorporate convolutions using various filter sizes. While our AR perturbations are generated using 3 x 3 window, the use of various filter sizes may exaggerate their separability, as described Section 3.3
4.3
AR Perturbations Against Common Defenses
4.3.1 Data Augmentations and Smaller Perturbations
Our poisoning method relies on imperceptible AR perturbations, so it is conceivable that one could modify the data to prevent the learning of these perturbations. One way of modifying data is by using data augmentation strategies during training. In addition to standard augmentations like random crops and horizontal flips, we benchmark our AR poison against stronger augmentations like Cutout 77. CutMix [41. and Mixup [42 in Table S Generally, Mixup seems to be the most effective at disabling poisons. A RN-18 18 poison- poison-trained trained using standard augmentations plus Mixup can achieve boosts in test set performance of 13.68% on Error-Max, 16.42% on Error-Min, 19.85% on Regions-4, 5.05% on Regions-16, and 19% on Random Noise. However, RN-18 poison-traie trained on our AR poison (e = 1) using standard augmentations plus Cutout, CutMix, Mixup cannot achieve any boost in test set performance.
We also present results for poisons using perturbations of size = 0.5 to explore just how small perturbations can be made while still maintaining poisoning performance. Under standard augmenta- tions, going from larger to smaller perturbations (c = to 0.5). poison effectiveness drops by 8.2% for Error-Max, 21.13% for Error-Min, 36.73% for Regions-4, and 31.6% for Regions- 16. Our AR poison achieves the smallest drop in effectiveness: only 2.53%. Random noise can no longer be considered a poison at 0.5 it completely breaks for small perturbations. Under all strong data augmentation strategies E = 0.5, AR poisoning dominates. For example, under Mixup, the best runner-up poison is Error-Max with an effectiveness that is more than 23% lower than AR. Unlike all other poisons, AR poisoning is exceptionally effective for small perturbations.
Note that in all three augmentation strategies pixels are either dropped scaled. Our method is unaffected by these augmentation strategies, unlike error- mroommmxxiizing. error-minimizing, and other randomly noise poisons. Scaling an perturbation does not affect how the corresponding matching AR filter will respond?? and thus, the patterns remain highly separable regardless of perturbation size.
See condition outlined in Lemma

---

Table Adversarial Training. CIFAR-10 test aceuracy after adversarially training with different radii Top row shows performance of adversarial training on clean data. AR poisons remain effective for small Pa-
0.125
0.25
0.50
0.75
Clean Data
87.07
84.75
81.19
77.01
Error-Max
=0.10
12.27
81.15
78.73+4.20
Error-Min 18
20.66
84.80
83.04
79.11
Regions-4
75.05
81.23
79.71
76.47 10.47
Regions-16
17.9
71.43
80.47
76.65
Random Noise
36.31
84.17
80.11
76.26 10.07
Autoregressive (Ours)
33.22
57.08
81.27
79.07
Table 5: Mixing Poisons with Clean Data. CIFAR-IO test accuracy when proportion of clean
data is used in addition to a poison. Top row shows test accuracy when training on only the clean proportion of the data; iee. poisoned data is used.
Clean Proportion
40%
30%
20%
10%
5%
Clean Only
90.84
89.92
87.90
81.01
74.97
Error-Max 18
87.83
86.83.0.44
84.70.
81. 21.63
76.48.1172
Error-Min 10
18.32
87 20.88
84.56
78.76.1.81
67. 67.82
o Regions-4
88,94
86. 20.5
83. 83.52:.0.20
78.23
70.19
Regions-16
88.03
16.23
83.01.0.44
76.52
67.24
Random Noise
86.40 ::
86.99 19
84.98 1.85
78.08
70.60 20.87
AR (Ours)
87.63
85. 85.62 0..22
83.28
76.13 2.34
62.69 1.......
Additionally, AR filters contain values which sum to 0, so uniform regions of an image also produce a zero response.
4.3.2 Adversarial Training
Adversarial training has also been shown to an effective counter strategy against f,-norm con- Using adversarial training. model trained on the poisoned data can achieve nearly the same performance as training on clean data (30) However, adversarial training is computationally more expensive than standard training and leads to decrease in test accuracy 3 when the perturbation radius, Pa. of the adversary is large. In Table we include adversarial training results clean data to outline this trade-off where training at large Pa comes at the cost of test accuracy. A recent line of work has therefore focused on developing better data poisoning methods that robust against adversarial training 10 3
strained data poisons [18 10
35
at larger adversarial training radius Pa-
In Table we compare performance of different poisons against adversarial training. We perform & adversanal training with different perturbation radii, Pr,, using 7-step PGD attack with step-size of p./4. We report error-bars by training three independent models for each run. We also show the performance of adversarial training on clean data. Data poisoning methods are fragile to adversarial training even when the training radius Pa smaller than poisoning radius (30] 37]
It is desirable
for poisons to remain effective for larger Pa. because the trade-off between standard test accuracy and robust test accuracy would be exaggerated further. As shown in the Table 4 when the adversarial training radius Pa increases, the poisons are gradually rendered ineffective. All poisons nearly ineffective at Pa = 0.5. Our proposed AR perturbations remain more effective at smaller radius, i.e. Pa 0.125 and Pa 0.25 compared to all other poisons.
4.3.3 Mixing Poisons with Clean Data
Consider the scenario when not all the data can be poisoned. This setup is practical because, to practitioner coming into control of poisoned data, additional clean data may be available through other sources. Therefore, it is common to evaluate poisoning performance using smaller proportions

---

of randomly selected poison training samples 10 18 30 A poison can be considered effective
the addition of poisoned data hurts test accuracy compared to training on only the clean data. In Table 5 we evaluate the effectiveness of poisons using different proportions of clean and poisoned data. The top row of Table 5 shows test accuracy after training on only the subset of clean data, with no poisoned data present. We report error-bars by training four independent models for each run. Our AR poisons remain effective compared to other poisons even when clean data is mixed in. AR poisons are much more effective when small portion of the data is clean. For example, when 5% of data is clean, a model achieves -75% accuracy when training on only the clean proportion, but using an additional 95% of AR data leads to -9% decrease in test set generalization. Our results on clean data demonstrate that AR poisoned data is worse than useless for training a network, and practitioner with access to the data would be better off not using it.
Conclusion
Using the intuition that simple noises casily Iearned, we proposed the design of AR perturbations, noises that are so simple they can be perfectly classified by a 3-layer CNN where all parameters are We demonstrate that these AR perturbations effective poisons for the purpose of preventing a network from generalizing to the clean test distribu- tion. Unlike other effective poisoning techniques that optimize error-man prror-maximizing noises, AR poisoning does not need access to a broader dataset or surrogate network parameters. We are able use the same set of 10 AR processes to generate imperceptible noises able degrade the test performance of networks trained on three different 10 class datasets. Unlike randomly generated poisons, AR poisons are more potent when training using a new network architecture or strong data augmentations like Cutout, CutMix, and Mixup. Against defenses like adversarial training, AR poisoning competitive or among the best for a range of attack radii. Finally, we demonstrated that AR poisoned data is worse than useless when it is mixed with clean data, reducing likelihood that a practitioner would want to include AR poisoned data their training dataset.
manually-speciffen
immediately useful and make
pror-mmnnmizzig
Acknowledgments and Disclosure of Funding
This material is based upon work supported by the National Science Foundation under Grant No. I 12101122000 and Grant No. IIS-22121 82, and by DARPA's Guaranteeing AI Robustness Against Deception (GARD) program under #HR001 12020007. Pedro is supported by an Amazon Lab126 Di- versity in Robotics and AI Fellowship. Any opinions, findings, and conclusions or recommendations expressed this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
References
[1] Marco Barreno, Blaine Nelson, Anthony D Joseph, and Doug Tygar. The security of machine learning. Machine Learning, 81(2):121-144 148, 2010.
[2] Battista Biggio, Blaine Nelson, and Pavel Laskov. Poisoning attacks against support vector machines. arXiv preprint / 220666389, 2012.
[3] Abeba Birhane and Vinay Uday Prabhu. Large image datasets: A pyrrhic win for computer vision? In 2021 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 1546-1546. IEEE, 2021.
[4] George EP Box, Gwilym M Jenkins, Gregory Reinsel, and Greta Ljung. Time series analysis: forecasting and control. John Wiley & Sons, 2015.
[5] Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. Targeted backdoor attacks on deep learning systems using data poisoning, arXiv preprint arXiv:1712.05526. 2017.
[6] Adam Coates, Andrew Ng, and Honglak Lec. An analysis of single-layer networks in unsupervised feature learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pages 215-223. JMLR Workshop and Conference Proceedings, 2011.
[7] Terrance DeVries and Graham W Taylor. Improved regularization of convolutional neural networks with cutout. arXiv preprint 170.:17000044552,, 2017.
10

---

[8] Alexey Dosovitskiy. Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai. Thomas Unterthiner. Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition scale. arXiv preprint arXiv:2010.1
2020.
[9] Liam Fowl, Ping-yeh Chiang, Micah Goldblum, Jonas Geiping, Arpit Bansal, Wojtek Czaja, and Tom Goldstein. Preventing unauthorized use proprictary data: Poisoning for secure dataset release. arXiv preprint w=X.v:2103..266: 2021.
[10] Liam Fowl, Micah Goldblum, Ping-ych Chiang, Jonas Geiping, Wojeiech Czaja, and Tom Goldstein. Adversarial examples make strong poisons. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 30339-30351. Curran Associates, Inc., 2021.
[11] Jonas Geiping, Liam H Fowl, W. Ronny Huang, Wojciech Czaja, Gavin Taylor, Michael Moeller. and Tom Goldstein. Witches' brew: Industrial scale data poisoning via gradient matching, In International Conference on Learning Representations, 2021.
[12] Micah Goldblum, Dimitris Tsipras, Chulin Xie, Xinyun Chen, Avi Schwarzschild, Dawn Song, Aleksander Madry, Bo Li, and Tom Goldstein. Dataset security for machine learning: Data poisoning. backdoor attacks, and defenses. IEFF Transactions on Pattern Analysis and Machine Intelligence, 2022.
[13] Tianyu Gu, Kang Liu, Brendan attacks on deep neural networks. IEEE Access, 7:47230-47244, 2019.
and Siddharth Garg. Badnets: Evaluating backdooring
[14] Kaiming He, Xiangyu 7hang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pottern recognition, pages 770-778, 2016.
[15] Kashmir Hill. The secretive company that might end privacy as we know Jan 2020.
[16] Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andrectto, and Hartwig Adam. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arviv: 1oo.:.704.44661 2017.
[17] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks, In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4700 4708, 2017.
[18] Hanxun Huang, Xingjun Ma, Sarah Monazam Erfani, James Bailey, and Yisen Wang. Unlcamable examples: Making personal data unexploitable. In International Conference on Learning Representations, 2021.
[19] Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversarial examples are not bugs, they features. Advances in neural information processing systems, 32, 2019.
[20] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers features from tiny images. 2009.
[21] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. To- wards deep learning models resistant to adversarial attacks. International Conference on Learning Representations, 2018.
[22] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.
[23] Weiqi Peng and Jinghui Chen. Learnability lock: Authorized learnability control through adversarial invertible transformations, In International Conference on Learning Representations, 2022.
[24] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, pages 8748 8763, PMLR, 2021.
[25] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.0611 2022.
[26] Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International Conference on Machine Learning. pages PMLR, 2021.
11

---

[27] Pedro Sanuoval-Segura, Vasu Singla, Liam Fowl, Jonas Geiping, Micah Goldblum, David Jacobs. and Tom effective. arXiv preprint arXiv:2204.0881 006112,,, 2022.
Goldstein. Poisons that learned faster
[28] Ali Shafahi, W Ronny Huang, Mahyar Najibi, Octavian Suciu, Christoph Studer, Tudor Dumitras, and Tom Goldstein. Poison frogs! targeted clean-label poisoning attacks neural networks. Advances in neural information processing systems, 31, 2018.
[29] Harshay Shah, Kaustav Tamuly, Aditi Raghunathan, Prateek Jain, and Praneeth Netrapalli. The pitfalls of simplicity bias in neural networks. Advances Neural Information Processing Systems, 33:9573 9585, 2020.
[30] Shaopeng. He Fengxiang, Liu Yang, Shen Li, and Tao Dacheng, Robust unlearnable examples: Protect- ing data privacy against adversarial learning. In International Conference on Learning Representations. 2022.
[31] Juncheng Shen, Xiaolei Zhu, and De Ma. Tensorclog: An imperceptible poisoning attack on deep neural network applications. IEEE Access, 7:41498 41506, 2019.
[32] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recogni- tion. arXiv preprint arXiv:
2014.
[33] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan. Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1-9, 2015.
[34] Mingxing Tan and Quoe Le. Efficientnet; Rethinking model scaling for convolutional neural networks. In International conference machine learning, pages 6105-6114 PMLR, 2019.
[35] Lue Tao, Lei Feng. Jinfeng Yi, Sheng-Jun Huang, and Songcan Chen. Better safe than sorry: Preventing delusive adversaries with adversarial training. Advances in Neural Information Processing Systems, 34, 2021.
[36] Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robust- ness may at odds with accuracy. International Conference on Learning Representations, 2019.
[37] Zhirui Wang, Yifei Wang. and Yisen Wang. Fooling adversarial training with inducing noise. arXiv preprint xiiv:2111.11011 2021.
[38] Herman Wold. A study in the analysis of stationary time series. PhD thesis, Almqvist & Wiksell, 1938.
[39] Da Yu, Huishuai Zhang, Wei Chen, Jian Yin, and Tie- Yan Liu. Indiscriminate poisoning attacks are shortcuts, 2022.
[40] Chia-Hung Yuan and Shan-Hung Wu. Neural tangent generalization attacks. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 12230-12240. PMLR, 18-24 Jul 2021.
[41] Sangdoo Yun, Dongyoon Han, Seong Joon Oh. Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regularization strategy to train strong classifiers with localizable features. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6023-6032, 2019.
[42] Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. arXiv preprint arXiv: 111.:0171000912.... 2017.
[43] Chen Zhu, Ronny Huang, Hengduo Li, Gavin Taylor, Christoph Studer, and Tom Goldstein. Transferable clean-label poisoning attacks on deep neural nets. In International Conference on Machine Learning, pages 1624-7623, PMLR, 2019.
12

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Appendix A. 12
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See
Appendix A.113
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] See Section 3.
(b) Did you include complete proofs of all theoretical results? [Yes] See Section 3.5 and Appendix A.I
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either the supplemental material or as a URL)? [Yes] Full algorithm and details are in Appendix A.3
(b) Did you specify all the training details (c.g.. data splits, hyperparameters, how they were chosen)? [Yes] See Section 9 and Appendix A.4
(c) Did you report error bars (e.g.. with respect the random seed after running experi- ments multiple times)? [N/A]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, cloud provider)? [Yes] See Appendix A..5
4. If you using existing assets (e.g., code, data, models) or curating/releasing assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any assets either in the supplemental material as URL? [N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you're 're using/curating? [N/A]
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "Rkvj0Te2L0l", "reviewer": "Reviewer_wQLZ", "summary": "This paper proposes to use autoregressive processes to generate perturbations for data poisoning. The generated perturbations, despite looking complex, are actually very simple. One advantage of the proposed method is that its generated perturbations are dataset and architecture independent. The paper evaluates the proposed method on multiple datasets and networks, showing the effectiveness of the perturbations when the poison rate is high.", "questions": "1. Is the proposed method only applicable to $\\ell_2$ norm?\n2. The proposed attack requires high poison rate to be effective?\n3. Is the proposed method only applicable to computer vision tasks?", "limitations": "1. The paper only studies $\\ell_2$ norm.\n2. The poison rate is high. The lowest poison rate studied in the paper is 0.6.\n3. The theoretical analysis is not sufficient. The relation between Lemma 3.1 and the effectiveness of the proposed method in poisoning attacks is not obvious.\n", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The strengths of this paper include\n1. The proposed attack method is interesting.\n2. The proposed method has good transferability.\n\nBut I still have the following concerns:\n\n1. Is the proposed method only applicable to $\\ell_2$ norm? The paper uses the sentence \"We measure AR perturbations\nin $\\ell_2$ because measuring in $\\ell_\\infty$ would underestimate the extent to which these perturbations are less perceptible than purely  $\\ell_\\infty$ random noise\" to explain why it uses $\\ell_2$ norm. But this sentence is hard to follow, and this short explanation is not convincing. The paper should provide more clear and convincing explanation about why it *only* uses $\\ell_2$ norm.\n\n2. The proposed attack requires high poison rate to be effective? In most experiments, the paper uses poison rate 1. In Table, the lowest poison rate is 0.6. The assumption of high poison rate is very strong. In practice, if the data is collected from multiple sources, then the attack is not effective? In the case that the data is collected from one source (the adversary), the entity who trains the model would be more cautious about the quality of data due to the high risk when the data only comes from one source.\n\n3. Section 3.3 is not easy to follow, and the logic is not very clear. I think Section 3.3 is one of the most important parts in the paper since it explains why the proposed method works. After reading Section 3.3, I am still very confused. The relation between Lemma 3.1 and the effectiveness of the proposed method in *poisoning attacks* is not obvious. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "CnrzvU3QDMJ", "reviewer": "Reviewer_GeFk", "summary": "This paper proposed autoregressive poisoning techniques to protect data from being exploited by unauthorized machine learning models. The proposed method does not rely on optimizations while generic towards different model architectures and different datasets. This paper also provides insight into why the proposed method is effective. \n", "questions": "- In experiments section line210:  \"We say that poisoning effectiveness drops from setup A to setup B if the network from poison-trained on setup B has higher test set accuracy than the network poison-trained on setup A. \" I find this is confusing.\n- For experiments in Table 4, for clean only, is it the same subset of data as in mixing poisons/clean? ", "limitations": "Please address the potential limitations in the Strengths And Weaknesses section. ", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The proposed method is efficient and technically sound. Existing works rely on optimizations which is the bottleneck. The proposed method does not rely on optimizations, and the parameters for AR are easy to find.  \n- The existing works are also shown that do not transfer well between model architectures or datasets. Experiment results show that one set of AR is generic across different architectures or datasets. \n- The efficient and generic can be very practical considering real-world applications.\n- Experiment results also demonstrated AR generated unlearnable examples are more robust towards augmentations.\n\n---\nLimitations:\n- Once the data is released, the defender may not modifies the data anymore, and the model trainer can retroactively apply new models/methods [1]. The adaptive case should be carefully examined. Consider that if the parameters for AR are leaked, can it be used to recover the original image? Or if a portion of the clean images are leaked, using pair of clean and unlearnable versions, is it possible to reverse the AR process? \n- In section 3.3, the assertion that the noises are easy to learn is more effective for poisoning, this could also mean they are easy to detect. Such as calculating sample-specific loss at the end of each training epoch. Although only detecting such samples does not make them \"learnable,\" but adaptive method (if there are any) can be applied to these samples. Or the model trainer could wait for future advancement for the recovery method as mentioned in [1]. \n\n[1] Data Poisoning Won’t Save You From Facial Recognition, ICML 2021 Workshop AML\n\n---\nAfter the author's response, I increased my rating score to 7. My main concerns over possible reverse operation if parameters are leaked have been well addressed. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "9y3pjgOA3cN", "reviewer": "Reviewer_Jf4X", "summary": "Paper considers a setting of `unlearnable examples' where a given dataset is perturbed in a way to make it hard to learn the true task. In essence, data here gets perturbed with correlated noise such that when learning is attempted, models learn to focus on the noise rather than on the true features useful for generalisation. While prior work focused on generating class-wise poisons, in this work the noise is generated per sample using a Markov process, producing linearly separatable noise. Paper thoroughly evaluates the setting and demonstrates that the approach effectively stops generalisation when the whole dataset is poisoned and struggles in a similar way in presence of adversrail training or dilution with clean data. ", "questions": "Thank you very much for the paper, it is a very interesting read! I only have a handful of questions:\n\n1. Are tables 4 and 5 computed over a number of models? Given how close the numbers are, it would be great to know if the differences are observed on distributional level, not just per model\n2. Given that we can produce arbitrary correlated noise of different flavors, how should one think about it? What is the fundamental difference between the noises in the related literature and the one produced in the paper? This naturally leads to my final question.\n3. Given the argument of easier learnability of different noises, it turns the question to how much observed behaviour is an artifact of the optimisation procedure itself? Did you try running the experiments with different lr/optimiser options? \n\nMinor:\n* Punctuation missing around eqs in some places", "limitations": "N/a", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "\nStrengths:\n+ Interesting setting\n+ Idea of hardness of learning is rather fascinating\n\nWeaknesses:\n+ Unclear how much of the evaluation is an artifact of chosen optimisation hyperparameters\n+ Unclear how one compares performance of different correlated noises", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "67TlknM7kNa", "reviewer": "Reviewer_FXVx", "summary": "This paper proposes a new data poisoning attack to prevent data scraping. The proposed method adds class conditional autoregressive (AR) noise to training data to prevent people from using the data for training, and the method is data and model independent, which means that the same noise can be used to poison different datasets and models of different architectures.\n\nThe intuition behind the idea is that easy to learn noise is more effective at data poisoning, and AR noise generated in the proposed way is easy for neural network to learn. The authors show that a manually specified 3-layer CNN with AR filter can easily learn class information from the AR noise. Experiments on four benchmark datasets (CIFAR10, STL10, SVHN, CIFAR100) show that the proposed method performs better than other four baselines (Error-min, Error-max, Regions, Random noise).", "questions": "About the process of AR noise generation:\n\n- It is clear how to generate AR noise at the beginning inside the sliding window. How about the subsequent steps? Take the example in Figure 2 as an example, if the sliding window slides one step to the right, there are three values to be generated. Are $x_{t-7}$ up to $x_t$ used to generate the next one ($x_{t+1}$)? Then $x_{t-6}$ up to $x_{t+1}$ are used to generate $x_{t+2}$, and so on. ", "limitations": "The author points out that the method does not perform well against adversarial training and experiments show that when evaluated using a mix of poisoned and clean data, the performance is also not good.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n- The proposed method is novel as autoregressive process hasn't been used before to do data poisoning. The method is easy to implement and the same AR coefficients can be used for different datasets and architectures as long as the numbers of classes are the same. Though code is not available, pseudo code (algorithms) and implementation details are provided. It is better that if actual code can be provided for reproduction of the results.\n\n- The paper is well-written and easy to follow. Empirical results on four different datasets show that the method performs better than other baselines, both under normal setting and defense settings.\n\nWeakness:\n\n- Though the proposed method performs better than other baselines compared in the paper, when tested against adversarial training, the performance is not satisfactory. It performs similarly to other baselines under this setting and the poisoning effect is not good, especially when the radii is large.\n\n- As pointed out in the paper, assuming that all the data can be poisoned is not realistic. In section 4.3.3, the poisoning methods are evaluated using a mix of poisoned and clean data. Under this setting, the performance of the proposed method is not good and similar to those of other baselines.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept