One-Pixel Shortcut: On the Learning Preference of Deep Neural Networks
Authors: Shutong Wu
Sizhe Chen
Cihang Xie
Xiaolin Huang
Venue: ICLR 2023 notable top 25% (2023)
Keywords: unlearnable examples
shortcut learning
one-pixel feature
deep neural network

Abstract: Unlearnable examples (ULEs) aim to protect data from unauthorized usage for training DNNs. Existing work adds $\ell_\infty$-bounded perturbations to the original sample so that the trained model generalizes poorly. Such perturbations, however, are easy to eliminate by adversarial training and data augmentations. In this paper, we resolve this problem from a novel perspective by perturbing only one pixel in each image. Interestingly, such a small modification could effectively degrade model accuracy to almost an untrained counterpart. Moreover, our produced \emph{One-Pixel Shortcut (OPS)} could not be erased by adversarial training and strong augmentations. To generate OPS, we perturb in-class images at the same position to the same target value that could mostly and stably deviate from all the original images. Since such generation is only based on images, OPS needs significantly less computation cost than the previous methods using DNN generators. Based on OPS, we introduce an unlearnable dataset called CIFAR-10-S, which is indistinguishable from CIFAR-10 by humans but induces the trained model to extremely low accuracy. Even under adversarial training, a ResNet-18 trained on CIFAR-10-S has only 10.61% accuracy, compared to 83.02% by the existing error-minimizing method.

Published as a conference paper at ICLR 2023
ONE-PIXEL SHORTCUT: ON THE LEARNING PREFER- ENCE OF DEEP NEURAL NETWORKS
Shutong Wu " Sizzee Chen " Cihang Xie & Xiaolin Huang
Department of Automation, Shanghai Jiao Tong University 2 CCmputer Science and Engineering, University of California, Santa Cruz
ABSTRACT
Unlearnable examples (ULEs) aim to protect data from unauthorized usage for train- ing DNNs. Existing work adds &x- -bouned perturbations to the original sample that the trained model generalizes poorly. Such perturbations, however, are casy to eliminate by adversarial training and data augmentations. In this paper, we resolve this problem from a novel perspective by perturbing only one pixel in each image. Interestingly, such a small modification could effectively degrade model accuracy to almost an untrained counterpart. Moreover, our produced One-Pixel Shortcut (OPS) could not be erased by adversarial training and strong augmentations. To generate OPS, we perturb in-class images at the same position to the same target value that could mostly and stably deviate from all the original images. Since such generation is only based on images, OPS needs significantly less computational cost than the previous methods using DNN generators. Based on OPS, we intro- duce unlearnable dataset called CIFAR-1O-S, which is indistinguishable from CIFAR-IO by humans but induces the trained model to extremely low accuracy. Even under adversarial training, ResNet- 18 trained on CIFAR-10-S has only 10.61% % accuracy, compared to 83.02% by the existing error- prror-mnnmizzig method.
INTRODUCTION
Deep neural networks (DNNs) have successfully promoted the computer vision field in the past decade. As DNNs are scaling up unprecedentedly (Brock et al., 2018 Huang et al. 2019 Riquelme et al., Zhang et al. 2022), data becomes increasingly vital. For example, ImageNet (Russakovsky
202T
ct al. 2015) fostered the development of AlexNet (Krizhevsky et al. 2017). Besides, people or organizations also collect online data to train DNNs, e.g., IG-3.5B-17k 17k (Mahajan al., 2018 and
JFT-300M (Sun et al. (017). This practice, however, raises the privacy concerns of Internet users. In this concern, researchers have made substantial efforts to protect personal data from abuse in model learning without affecting user experience (Feng et al. 2019: Huang et al. |220aa] Fowl et al. 2021
Yuan & Wu 2021: 1: Yu et al. (022). Among those proposed methods, unlearnable examples (ULEs) (Huang et al.) 2020a) take great step to inject original images with protective but imperceptible
perturbations from li-le error minimization (EM). DNNs trained on ULEs generalize very poorly on normal images. However, such perturbations could be completely canceled out by adversarial training, which fails the protection, limiting the practicality of ULFS.
We view the data protection problem from the perspective of shorteut learning (Geirhos et al.
2020).
which shows that DNN training is "lazy" (Chizat et al.] 2019 Caron & Chrétien 2020), ie., converges
to the solution with the minimum norm when optimized by gradient descent (Wilson al. 2017 7
Shah et al. 2018 Zhang et al. 2021). In this case, a DNN would rely on every accessible feature to minimize the training loss, no matter whether it is semantic or not (Ilyas et al.) 2019 Geirhos et al. 2018 Baker et al. 2018). Thus, DNNs tend to ignore semantic features if there are other casy-to-learn shortcuts that are sufficient for distinguishing examples from different classes. Such shortcuts exist naturally or manually. In data collection, e.g., cows may mostly appear with grasslands, misleading DNN to predict cows by large-area green, because the color is easier to learn than those semantic features and also sufficient to correctly classify images of cows during training, Such natural shortcuts
"quull contribution; correspondence to Xiaolin Huang (xiaolinhuang @ @jtu.cdd.cc)). "ode available at alhttps: / whw.. cneene- xe1 Shot.ct

---

Published as a conference paper at ICLR 2023
train
extratt
DNN
Clean
One Pixel Shortcut
train
extract
DNN
features
Figure 1: Effect of One-Pixe Shortcut. We visualize the features (after the first convolution) of the ResNet-18 18 (He et al. 2016) models trained by clean and OPS samples. Even at such shallow layer, the DNN trained on OPS extracts much fewer semantic features and is less activated.
have been illustrated detail datasets such as ImageNet-A (Hendryeks al., 2021) and ObjectNet (Barbu et al. 2019). Besides, shortcuts could also be manually crafted. For instance, EM-based ULEs (Huang et al. 2020a) mislead DNNs to learn features belonging to the perturbations, which falls in the category of shorteut learning
Yu et al. 2021).
In this paper, we are surprised to find that shortcuts could be so small in the area that it can even be simply instantiated as single pixel. By perturbing pixel of each training sample, namely One-Pixel Shortcut (OPS), degrades the model accuracy on clean data to almost an untrained counterpart. Moreover,
method,
generated unbounded small noise could not be erased by adversarial training (Madry et al. 2018), which is effective in mitigating existing ULEs (Huang et al. 2020a). To make the specific pixel stand out in view of DNNs, OPS perturbs in-class images the same position to the same target value that, if changed to a boundary value, could mostly and stably deviate from all original images. Specifically, the difference between the perturbed pixel and the original one in all in-class images should be large with low variance. Since such generation is only based on images, OPS needs significantly less computational cost than the previous methods based on DNN generators. We evaluate OPS and its counterparts in 6 architectures, 6 model sizes, 8 training strategies on CIFAR-10 (Krizhevsky et al.) 2009) and ImageNet (Russakovsky et al. 20155) subset, and find that OPS is always superior in degrading model's testing accuracy than EM ULEs. In this regard, we introduce new unlearnable dataset named CIFAR-10-S, which combines the EM and OPS to craft stronger imperceptible ULEs. Even under adversarial training, ResNet-18 (He et al. 2016) trained on CIFAR- 10-S has 10.61% % test accuracy, compared to 83.02% by the existing error-minimizing method. Different from the existing datasets like ImageNet-A (Hendrycks al. 202T) or ObjectNet (Barbu et al. 2019), which place objects into special environments to remove shortcuts, CIFAR- 10-S injects shortcuts to evaluate the model's resistance them. Altogether, our contributions are included as follows:
We analyze unlearnable examples from the perspective of shortcut learning, and demonstrate that strong shortcut for DNNs could be as small as a single pixel.
We propose a novel data protection method named One-Pixel Shorteut (OPS), which perturbs in-class images in the pixel that could mostly and stably deviate from the original images. OPS is a model-free method that is significantly faster than previous work.
We extensively evaluate OPS various models and training strategies, and find it outper- forms baselines by a large margin in the ability to degrade DNN training. Besides, we introduce CIFAR-10-S assess the model's ability learn essential semantic features.
RELATED WORK
2.1
ADVERSARIAL ATTACK AND DATA POISONING
Adversarial examples are perturbed by small perturbations, which are indistinguishable from the original examples by humans but make DNNs give wrong predictions (Szegedy al. 2014).
2

---

Published as a conference paper at ICLR 2023
Many different adversarial attacks are proposed in recent years. Generally, most adversarial attacks aim to perturb the whole image with a constrained intensity (usually bounded by l norm), e.g., PGD (Madry et al. 2018), C&W (Carlin & Wagner (0177) and Autoattack (Croce & Hein 2020). Besides, there are also other methods that only perturb a small part of an image (Croce & Hein (2019): Dong et al. (2020)) or even single pixel (Su al. (2019). The existence of adversarial examples and their transferability (Chen et al. (2022) indicates that DNNs do not sufficiently learn critical semantic information as we wish, but more or less depend on some non-robust features.
Data poisoning aims to modify the training data in order to affect the performance of models. Usually, the poisoned examples are notably modified and take only a part of the whole dataset (Yang et al. 2017 Koh & Liang 2017). But those methods cannot degrade the performance of models to a low enough level, and the poisoned examples are easily distinguishable. Recently researchers have paid great attention to imperceptible poisoning which modifies examples slightly and does not damage their semantic information (Huang et al.) 2020a Fowl et al. 2021 Huang al.) 2020b] Doan et al.) 2023 Fowl et al. (2021) uses adversarial perturbations
2021 Geiping et al. 2020; Chen et al.
which contain information of wrong labels to poison the training data, which is equivalent to random label fitting. On the contrary, Huang et al.
(2020a) attack the training examples inversely, i.e., using
error- minimiiiig perturbation, to craft unlearnable examples.
2.2 SHORTCUT LEARNING
Recently, researches on deep neural networks indicate that under the sufficiency of correct clas- sification, DNNs tend to learn easier features instead of semantic features which make the object itself. To be more specific, for example, the same object in different environments will get different predictions, which means the DNN overly relies on features that do not belong to the object (Beery al. 20188) Geirhos et al. (220)) investigate this phenomenon in different fields of deep learning and explain why shortcuts exist and how to understand them. Lapuschkin et al. (019)) also observe this problem and attribute to the unsuitable performance evaluation metries that we generally use. The existence of natural ad adversarial examples (Hendrycks et al.) [202]) also indicates that DNNs do not sufficiently learn the real semantic information during training. Instead, they may learn use the background or texture of an image to predict. Unlearnable examples (ULFs) (Huang et al., 2020a), which are manually crafted by error-mininn proor-miiiiizing. noises and able to lead the models trained on them to obtain terrible generalization on test data, are believed to be some kind of shortcut that provides some textures that are easy to learn YYu et al., 202T). Generally, if we get enough data, the interconnection of different features will be enhanced so that those shorteuts may not be sufficient for classification tasks, i.e., the model will have to use more complicated composed features in order to minimize the risk. However, when the data we collect obtains some specific bias (e.g., similar backgrounds), shortcut learning will not be mitigated effectively.
2.3 DATA AUGMENTATION
Data augmentation aims to enhance the generalization ability of models. This is usually implemented by applying some transformations to the training data, e.g., random stretching, random cropping, or color changing. Nowadays different kinds of data augmentation policies (Zhang et al. 2018 De 2017,Cubuk of DNNS. Sometimes adversarial training Madry et al. (2018) Li et al. (2022) also regarded as kind of data augmentation (Shorten& & Khoshgoftaar 2019 Xie et al.) 2020). Dao et al. (2019) deem data augmentation as a kind of data deteddepend regularization term. Since data augmentations are believed to improve the generalization ability of DNNs, we use different augmentations to evaluate the effectiveness of different data protection methods.
& Taylor 2017, Cubuk et al. 2019 2020) are proven to effectively boost the generalization ability
3 ONE-PIXEL SHORTCUT
3.1 PRELIMINARIES
Unlearnable examples (ULEs) data protection method created by error-minimizing (EM) noises (Huang et al. 2020a). Models trained on examples that are perturbed by those noises will get almost zero training error, but perform like random guessing clean test data. Due to the imperceptibilitt
3

---

Published as a conference paper at ICLR 2023
of the noise, this method can prevent the abuse of data by some unauthorized users who attempt to train deep models for improper purposes, without affecting normal usage. This bi-level problem can be solved by optimizing the inner minimization and the outer minimization alternately. It is proved that the perturbations belonging to the same class are well clustered and linearly separable Yu et al. 202T]. Thus, EM provides easy-to-learn features which closely interconnected with labels.
We design a shuffling experiment to demonstrate that DNNs learn the shortcuts instead of images trained on data that have shortcuts. Denote D = ((r, + y.)} as the perturbed training set, where is the perturbation associated with the i-th example. After shuffling. the training set becomes D' = [(z + 6, yt)}. We are curious about how the DNNs trained on different data (with or without shortcuts) would predict the shuffled perturbed training set. As shown in Table the DNN trained on clean data performs like a random guess on shuffled perturbed data while keeping high accuracy on unshuffled data, indicating that it successfully learns the representations from images { rather than giving predictions according to the perturbations {6:] from EM or OPS. In contrast, the DNN trained on EM training data tends to memorize significant proportion of perturbations, because it has 48.97% accuracy on the shuffled EM training data. Moreover, OPS training data produces a DNN with 72.43% accuracy the shuffled set, reflecting it forcing the DNN to learn the perturbations to a greater extent. Our study illustrates the learning characteristics of DNNs trained with or without shortcuts, and also shows that OPS is a more effective shortcut than EM.
Table 1: The testing accuracy of ResNet-18 models trained on unshuffled and shuffled data.
Testing / Training
Clean
EM
OPS
Clean
99.96
25.16
15.48
EM
94.00
99.99
15,54
Shuffled EM
9.97
48.97
10.15
OPS
99.52
25.07
99.97
Shuffled OPS
9.82
10.35
72.43
3.2 FOOLING DNN TRAINING BY A PIXEL
Following the discussion above, for the purpose of data protection, we need to craft shortcuts that are easy enough to learn and thus fool the network training. According to previous studies, shortcuts can come from background environments that naturally exist inside our datasets (Beery et al. 2018). or be manually crafted like EM (Huang et al. 2020a). Unlike those shortcuts which might occupy the whole image or a notable part, we investigate how a single pixel, which is the minimum unit of digital images, can affect the learning process of deep neural networks. Thus, we propose One-Pixel Shortcut (OPS), which modifies only a single pixel of each image. Images belonging to the same category perturbed the same position, which means the perturbed pixel is interconnected with the category label. Although so minuscule, it is efficient enough to fool the training of deep learning models. We use heuristic but effective method to generate perturbations for images belonging each category. We search the position and value of the pixel which can result in the most significant change for the whole category. Denoting D as the clean dataset and Dj as the clean subset containing all the examples of class h. the problem can be formulated as:
arg ( 19 (x, o.e Ex)] on.hk
s.t. |10xllo = 1, Cov(i.)) 1,
(I)
i.j
where d% E Hx represents the perturbed position mask and o (i.j) ) is the element at the i-th row and j-th column, & E RC stands for the perturbed target color (C = 3 for RGB images), and G the objective function. Since the optimization above is an NP-hard problem, we cannot solve directly. Thus constrain the feasible region to limited discrete searching space, where search the boundary value of each color channel, i.e.fx E {0, 1)3, 3, at every point of an image. Specifically, for CIFAR-10 images, the discrete searching space will contain 32 x 32 x 23 8192 elements. To
4

---

Published as a conference paper at ICLR 2023
ensure that the pixel is stably perturbed, we also hope that the variance of the difference between them is small. Accordingly, we design the objective function Ok for class & as:
B (aa)eD EE oelle Eks
Gk
(. Ejalllay oxlle Exyl)
where x; R#xW denotes the i-th channel of a. and Exj E IR is the i-th channel of Ex. After solving the position map and color, we get perturbation 8 for each example (2, y) as:
= 1y - By. Sy20y - oy.... Eycoy sc oy]
(3)
Details can be found in Algorithm
The resulting One-Pixel Shortcut is illustrated in Figure -
Algorithm 1 Model-Free Searching for
Shortcut
Input: Clean dataset D Di
U....UUM
Output: One-Pixel Shortcut dataset D = -..UU..UDM
1: for k = 2.3, .... M do
2: solve Eq and 2 get and &s
# calculate the best perturbed point for class
3: for each
D3 do
4: for i 1, 2, 3 do
5:
à T; (I σk)
Eki 0%
# modify the optimal pixel for every image class k
6: end for
7: end for
8: Dj. {2)
9: end for
10: return D =
clean
bounded dense noise
EMULE
train
Low-Acc DNN
train
Lower-Acc DNN
unbounded sparse noise
One Onx-Pixel Shortcut
The effects of Mirim-iinn noise and One-Pixel Shortcut. Models trained on either EM or OPS perturbed data get abnormally low accuracy on clean test data. To ensure imperceptibiliity EM adds t, bounded noise to the whole image, while OPS applies unbounded but sparse perturbation which only affects small part of the image.
Figure 2:
3.3 PROPERTIES OF ONE-PIXEL SHORTCUT
Since all empirically believe that convolutional networks tend capture textures (Hermann ctt al. 2020) or shapes (Gerhos et al. 20018] Zhang & Zhu 2019), it surprising that convolutional networks can be affected severely by just pixel. As illustrated Figure T the network indeed
5

---

Published as a conference paper at ICLR 2023
tends to learn those less complicated nonsemantic features brought by One-Pixel Shortcut. Besides convolutional networks, we observe that compact vision transformers (Hassani et al. 2021) are also attracted by One-Pixel Shortcut and ignore other semantic features. This indicates that shorteuts are not particularly learned by some specific architecture. We also visualize the loss landscape of ResNet- 18 trained clean CIFAR-10 data and One-Pixel Shorteut data. Illustrated Figure 3 while trained on OPS data, the loss surface is much flatter, which means that these minima found by the network are more difficult escape. Even we use ResNet-18 pretrained on clean CIFAR-I0 and then fine-tune it on the OPS data, the network will still fall into this badly generalized minima.
1.00
1.00
0.75
0.75
0.50
0.50
0.25
0.5
0.00
0.00
-0.55
-0.25
-0.0
-0.50
-0.75
-0.75
1.00
1.00
Figure 3: Loss landscape visualization (Li et al. 2018) of ResNet- 18 trained on clean CIFAR-10 -0 data and our One-Pixel Shortcut data. The landscape of THE OPS-trained model flatter, making minima harder to escape.
100
Trainnng Accuracy
75
Clean
EM
25
30
40
50
60
70
80
Norm of Parameter Difference
Figure 4: Training accuracy and the Frobenius norm of parameter difference (i.e..l0 0o when ResNet- 18 trained different training data. While training EM OPS data, the network tends to find an optimum closer to the initialized point. This is consistent with the view of shortcut learning.
In addition, we record the trajectories of training accuracy and the Frobenius norm of parameter difference, ||0 folFF, which can reflect the magnitude of network parameter change. Here and d respectively indicate the parameters after training and at the initialized point. We draw the relation curve between training accuracy and ||0 6 ll which can be found in Figure 4 When training accuracy rises up to 90% for the first time, the model trained OPS data has much smaller ||0 -0ll r than that trained on clean data, which indicates that the OPS-trained model gets stuck an optimum closer to the initialization. It has been widely known that overparameterized DNNs optimized by gradient descent will converge to the solution that is close to the initialization. i.e. with the minimum norm of parameter difference (Wilson al., 2017; Shah al.) 20188] Li & Liang ZhangZhang et al. (02T). Since OPS only perturbs single pixel, the original representations of images are not damaged, and the model trained on clean data can still keep great performance on solution far from the initialization still exists but not reached due to the tendency for close solution. The close solution believed to obtain better generalization ability. Nevertheless, this argument is true only under the assumption that the training data and test data are from the exact same distribution and have the exact same features. The existence of OPS forces the model to converge to optimum where the model generalizes well on OPS features, which are not contained test data. From our experiment results in Table 2 OPS can degrade test accuracy lower level. This because EM requires generator model, and thus may
OPS data, which indicates that the

---

Published as a conference paper at ICLR 2023
contain features more less depending it, which constrains the effectiveness of other models. On the other hand, OPS is universal model-free method, and the shortcuts are crafted based on the inherent learning preference of DNNs.
4 EXPERIMENTS
4.1 SETTING
Our experiments are implemented on CIFAR-I0 and ImageNet subset, using 4 NVIDIA RTX 2080Ti GPUs. We investigate how the One-Pixel Shortcut affect the training of different models (including different architectures and different capacities). We evaluate our method on different models including convolutional networks (He et al. 2016 Zagoruyko & Komodakis 2016 Huang
et al. 2017) and the recently proposed compact vision transformers (Hassani et al. 2021). For all SGD optimizer with learning rate
the convolutional networks, set to 0.9, and weight decay set to 5e - 4. For all the compact vision transformers, we use AdamW 5e 4, and weight decay Batch size is set to 128 for all the models except WideResNet-288 10, where it is set to 64.
0.1, momentum
optimizer with 81 0.9, 82 0.999, learning rate
3e
Table Clean test accuracy on CIFAR-10 models of different architectures.
Besides, We also try different training strategies including adversarial training (Madry al. 2018) and different data aug- mentations (Mixup (Zhang et al.) (0018). Cutout (DeVries & Taylor 2017) and Ran- dAugment (Cubuk et al.) 2020)). For ad- versarial training, we use 10-step PGD attack, setting step size to 2/255 and foc bound to 8/255. For Mixup, Cutout, and RandAugment,,
Training EM
Data
Model
Clean
OPS (Ours)
LeNet-5
70.27
26.98
22.19
CVT-7-4
87.46
27.60
18.21
CCT-7-3x1
88.98
27.06
17.95
DenseNet-121
94.10
23.72
11.45
the default settings from their original papers. All the models are trained for 200 epochs with multi-step learning rate schedule, and the training accuracy of each model is guaranteed to reach near 100%.
ResNet-18
94.01
19.58
15.56
WideResNNe-22-10
96.08
23.96
12.76
We additionally tested our method on the ImageNet subset (the first 100 classes). We center-crop all the images to 224 x 224 and train common DNNs with results in Table We adopt an initial learning rate of 0.1 with multi-step learning rate scheduler and train models for 200 epochs. Our One-Pixel Shortcut can still be effective in protecting large-scale datasets. The networks trained on OPS data will get much lower clean test accuracy than those trained on clean data.
Table 3: WideResNets of different capacities trained on One-Pixel Shorteut data. Test Acc. stands for accuracy on the clean CIFAR-10 testset.
WRN-28-1
WRN-28-2
WRN-28-4
WRN-28-8
WRN-28-16
WRN-28-20
Size 0.37M
1.47M
5,85M 21.75
23.36M 18.01
93.35M 15.36
145,8 145,84M
Test Acc.
21.20
14.74
18.04
4.2 EFFECTIVENESS ON DIFFERENT MODELS
We train different convolutional networks and vision transformers on the One-Pixel Shorteut CIFAR-I0 10 training set, and evaluate their performance on the unmodified CIFAR-10 test set. Details are shown in Table 2 Every model reaches a very high training accuracy after only several epochs, which is much faster than training on clean data. Meanwhile, they all get really low test accuracy (about 15%) on clean test data, indicating that they do not generalize at all. Although the perturbed image looks virtually the same as the original image, and all the models get 100% training accuracy quickly, they do not capture any semantic information but just
Table 4: Clean test accuracy on ImageNet models of different architectures.
Training Data
Model
Clean
OPS (Ours)
ResNet-13 18
76.18
9.68
ResNet-50
64.26
10.38
DenseNet-121
75.14
11.48
7

---

Published as a conference paper at ICLR 2023
the pixels we modify in the images. We also train models on EM training which is generated by a ResNet-18 using the official implementation of Huang et al. (202aa). The Eoo bound of EM noises is set 8/255. The generation of OPS costs only about 30 seconds, which is much faster than EM costing about half an hour. For different networks, OPS can degrade their test accuracy to lower level than EM. EM works the best ResNet-18 (19.58% test accuracy). which has the same architecture as the generator. On other models, they get higher test accuracy than ResNet- 18. Meanwhile, since OPS is model-free method that takes advantage of the natural learning preference of neural networks, its transferability is better across different models. Besides different architectures, we also explore the impact on models with the same architecture but different capacities. We trained several WideResNets (Zagoruyko & Komodakis 2016) with different sizes. The experiment results can be found in T From our observation, overparameterization, which is generally believed to enhance the ability to capture complicated features, does not circumvent the shortcut features.
Moreover, we observe that vision transformers are easily affected by manually crafted shorteuts, even though it is believed that their self-attention mechanism makes them less sensitive to data distribution shifts Shao et al. 202I Bhojanapalli et al., 202Ip. For CCT-7-3xI I and CVT-7-4 (Hassani et al. degrade their test accuracy below 30% and 20%. This indicates that vision data as well as our expectations. If the training data is largely biased, i.e., has notable shortcuts, and vision transformers will not perform much better than convolutional networks.
2021), EM and OPS
transformers may not generalize on out-of-distribuu
Table 5: Effectiveness of One-Pixel Shorteut Frror-Minimizing on ResNet-18 18 under different training strategies. Here loo AT stands for adversarial training with bound 8/ 8/255.
Training Data
Training Strategy
Clean
EM
OPS
CIFAR-IO-S
Standard
94.01
19.58
15.56
16.67
Mixup
94.75
38.18
33.13
23.23
Cutout
94.77
25.83
61.68
24.38
Rugment
94.91
51.66
71.18
39.62
RandAugment Cutout
94.86
36.26
79.70
33.85
to AT Loc + + Mixup
82.72
83.02
11.08
10.61
87.90
86.71
10.97
13.77
for AT + Cutout
84.58
84.24
24.60
23.78
Eo AT RandAugment
85.50
85.06
44.86
46.23
4.3 EFFECTIVENESS UNDER DIFFERENT TRAINING STRATEGIES
To evaluate the effectiveness of OPS under different training strategies, we train models on OPS perturbed data using adversarial training and different data augmentations such as Mixup ZZang ct al. 2018 Cutout (DeVsies & Taylor 2017) and RandAugment (Cubuk et al. 2020). Simple augmentations like random crop and flip used by default in standard training. Models
also we can observe that both EM and OPS have good performance on data protection, which degrade test accuracy 19.58% and 15.56%. As mentioned in previous works (Huang et al. 2020aa Fu et al. 2021I). EM can not work so effectively under adversarial training, and the model can reach an even higher accuracy than adversarially trained on clean data. Meanwhile, OPS can still keep effective under adversarial training. However, when it comes data augmentation, EM seems more impervious, while OPS more sensitive, especially to Cutout and RandAugment, This is due to the fact that EM injects global noises into images, while OPS only modifies a single pixel, which is equivalent to adding a very local perturbation. Adversarial training, which can be regarded on global shortcuts. On the other hand, local data augmentations like Cutout make models less sensitive to local shortcuts.
trained on EM perturbed data. As shown Table 55
kind of global augmentation, is able attenuate the dependence
Naturally, for the purpose of complementing each other, we can combine EM and our proposed OPS together to craft kind of ensemble shortcut. Since OPS only modified single pixel, after being applied to EM perturbed images, the imperceptibility can still be guaranteed. We evaluate the effectiveness of this ensemble method under different training strategies and find that can always
8

---

Published as a conference paper at ICLR 2023
keep effective. Even if we adversarial training and strong data augmentation like RandAugment, is still able to degrade test accuracy to relatively low level. Based on this property, we introduce CIFAR-100 ----, where all the images perturbed by the AMM-OPS-copposss noises. It can serve new henchmark to evaluate the ability to learn critical information under the disturbance composed non-semantic representations.
We also extend our method to multi-pixel scenarios. According to T as the number of perturbed pixels increases, the test accuracy can be degraded to lower level. Nevertheless, the more pixels are perturbed, the imperceptibilityy gets weaker, as illustrated in Figure 5 From our experiment on ResNet-18, 3-Pixel Shortcut can easily degrade the accuracy pixels alleviate the sensitivity to different data augmentations. For RandAugment one more perturbed pixel can degrade the test accuracy to 46.45%, which much lower than 71.18% of OPS.
7.74.. Moreover, more perturbed
Table 6: Effectiveness of Multi-Pixel Shortcut on ResNet-18 under different training strategies
Number Perturbed Pixels
Training Strategy
3
Standard
94.01
15.56
15.24
9.74
9.71
Mixup
94.75
33.13
9.99
9.94
10.02
Cutout
94.77
61.68
39.99
25.42
20.15
Rand/ugmeent
94.91
71.18
46.45
34.66
33.51
0 Pixel (Clean)
1 Pixel
2 Pixels
3 Pixels
4 Pixels
Figure 5: Multi-Pixel Shortcut examples. More perturbed pixels lead to lower imperceptibility.
DISCUSSION AND CONCLUSION
In this paper, we study the mechanism recently proposed unlearnable examples (ULEss) which use error-minimizing (EM) noises. We figure out that instead of semantic features contained by images themselves, features from EM noises are mainly learned by DNNs after training. These kinds of easy-to-learn representations work as shortcuts, which could naturally exist be manually crafted. Since DNNs optimized by gradient descent always find the solution with the minimum norm, shortcuts take precedence over those semantic features during training.
We find that shortcuts can be as small as even a single pixel. Thus, we propose One-Pixel Shortcut (OPS), which generator model and therefore needs very little computational cost and has better transferability between different models. Besides, OPS is less sensitive adversarial training compared EM ULEs. We investigate the effectiveness of OPS and EM under different training strategies. We find that EM and OPS have their respective advantages and disadvantages. While EM cannot keep effective under global data augmentations like adversarial training. OPS is sensitive to local data augmentations like Cutout. Based on our investigation, we combine EM and OPS together craft kind of stronger unlearnable examples, which can still keep imperceptible but more impervious, and consequently introduce CIFAR-10-S, which can be a new benchmark. Besides, we have also discussed our method in multi scenarios.
imperceivable and effective data protection method. OPS does require
There are still questions that need to be discussed in the future. Besides shortcuts that are crafted deliberately for the purpose of data protection, there are also shortcuts that naturally exist due to the inevitable bias during data collection. They can be the crux network generalization on unseen data. How identify and avoid them (e.g.. design data-dependent augmentation) is challenging problem. We believe our work will shed light on the important impacts shortcuts, and provide inspiration to harness them for more practical applications.

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGEMENT
This work is partl supported by National Natural Science Foundation of China (61977046, 61876107, U1803261), Shanghai Science and Technology Program (2251 1105600), and Shanghai Municipal Science and Technology Major Project (2021SHZDZX01 Cihang Xie is supported by gift from Open Philanthropy. In addition, we sincerely thank the reviewers for their valuable discussions.
REFERENCES
Nicholas Baker, Hongjing Lu, Gennady Erlikhman, and Philip Kellman. Deep convolutional networks do not classify based global object shape. PLoS computational biology, 14(12): c1006613, 2018.
Andrei Barbu, David Mayo, Julian Alverio, William Luo, Christopher Wang, Dan Gutfreund, Josh Tenenbaum, and Boris Katz. Objectnet: A large-scale bias-controlled dataset for pushing the limits of object recognition models. Advances in Neural Information Processing Systems, 32, 2019.
Sara Beery, Grant Van Horn, and Pietro Perona. Recognition in terra incognita. In Proceedings of the European Conference on Computer Vision, pp. 456-473, 2018.
Srinadh Bhojanapalli, Ayan Chakrabarti, Daniel Glasner, Daliang Li, Thomas Unterthiner, and Andreas Veit. Understanding robustness of transformers for image classification. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp.
2021.
Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale gan training for high fidelity natural image synthesis. In International Conference on Learning Representations, 2018.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In IEEE Symposium on Security and Privacy, pp. 39-57. IEEE, 2017.
Emmanuel Caron and Stéphane Chrétien. A finite sample analysis of the benign overfitting phe- nomenon for ridge function estimation. arXiv preprint arXiv:2007. 128882 2020.
Sizhe Chen, Zhengbao He, Chengjin Sun, and Xiaolin Huang. Universal adversarial attack on attention and the resulting dataset DAmageNet. In IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), pp. 2188-288 2188-2197, 2022.
Sizhe Chen, Geng Yuan, Xinwen Cheng, Yifan Gong, Minghai Qin, Yanzhi Wang, and Xiaolin Huang. Self-ensemble protection: Training checkpoints are good data protectors. International Conference Learning Representations, 2023.
Lenaic Chizat, Edouard Oyallon, and Francis Bach. On lazy training differentiable programming. Advances in Neural Information Processing Systems, 32, 2019.
Francesco Croce and Matthias Hein. Sparse and imperceivable adversarial attacks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4724-4732, 2019.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-fi free attacks. In International Conference on Machine Learning, pp. 2206-2216. PMLR, 2020.
Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V Le. Autoaugment: Learning augmentation strategies from data. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pottern Recognition, pp. 113-123, 2019.
Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V Le. Randaugment: Practical automated data augmentation with reduced search space. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 702-703, 2020.
Tri Dao, Albert Gu, Alexander Ratner, Virginia Smith, Chris De Sa, and Christopher Ré. A kernel theory modern data augmentation. In International Conference on Machine Learning, pp. 1528-1537. PMLR, 2019.
10

---

Published as a conference paper at ICLR 2023
Terrance VeVries and Graham W Taylor. Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv:1708.00555, 2017.
Khoa Doan, Yingjie Lao, Weijie Zhao, and Ping Li. Lira: Learnable, imperceptible and robust backdoor attacks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 11966-11976, 2021.
Xiaoyi Dong, Dongdong Chen, Jianmin Bao, Chuan Qin, Lu Yuan, Weiming Zhang, Nenghai Yu, and Dong Chen. Greedyfool: Distortion-aware sparse adversarial attack. Advances in Neural Information Processing Systems, 33:11226-11221 2020.
Ji Feng, Qi-Zhi Cai, and Zhi-Hua Zhou. Learning to confuse: generating training time adversarial data with auto-encoder. Advances in Neural Information Processing Systems, 32, 2019.
Liam H Fowl, Micah Goldblum, Ping-yeh Chiang, Jonas Geiping, Wojeiech Czaja, and Tom Goldstein. Adversarial examples make strong poisons. In Advances in Neural Information Processing Systems, 2021.
Shaopeng Fu, Fengxiang He, Yang Liu, Li Shen, and Dacheng Tao. Robust unlearnable examples: Protecting data privacy against adversarial learning. In International Conference on Learning Representations, 2021.
Jonas Geiping, Liam H Fowl, W Ronny Huang, Wojciech Czaja, Gavin Taylor, Michael Moeller, and Tom Goldstein. Witches' brew: Industrial scale data poisoning via gradient matching. In International Conference on Learning Representations, 2020.
Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness. In International Conference on Learning Representations, 2018.
Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(1 1):6656673, 2020.
Ali Hassani, Steven Walton, Nikhil Shah, Abulikemu Abuduweili, Jiachen Li, and Humphrey Shi. Escaping the big data paradigm with compact transformers. arXiv preprint arXiv:2104.05704, 2021.
Kaiming He. Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEF/CVF Conference on Computer Vision and Putlern Recognition, pp. 770-778, 2016.
Dan Hendrycks, Kevin Zhao. Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adver- sarial examples. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 15262-15271 2021.
Katherine Hermann, Ting Chen, and Simon Korbblith. The origins and prevalence of texture bias in convolutional neural networks. Advances in Neural Information Processing Systems, 33: 19000-19015, 2020.
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4700-4708, 2017.
Hanxun Huang, Xingjun Ma, Sarah Monazam Erfani, James Bailey, and Yisen Wang. Unlearn- able examples: Making personal data unexploitable. In International Conference on Learning Representations, 2020a.
W Ronny Huang, Jonas Geiping, Liam Fowl, Gavin Taylor, and Tom Goldstein. Metapoison: Practical general-purpoot clean-label data poisoning. Advances in Neural Information Processing Systems, 33:1 18:12080-1209) 2020b.
11

---

Published as a conference paper at ICLR 2023
Yanping Huang, Youlong Cheng, Ankur Bapna, Orhan Firat, Dehao Chen, Mia Chen, HyoukJoong Lee, Jiquan Ngiam, Quoe V Le. Yonghui Wu, et al. Gpipe: Efficient training of giant neural networks using pipeline parallelism. Advances in Neural Information Processing Systems, 32. 2019.
Andrew Hyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversarial examples are not bugs, they are features. Advances in Neural Information Processing Systems, 32, 2019.
Pang Wei Koh and Perey Liang. Understanding black-box predictions via influence functions. In International Conference on Machine Learning, pp. 1885-1894, PMLR, 2017.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009,
Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton. Imagenet classification with deep convolu- tional neural networks. Communications of the ACM, 60(6):84-90, 2017.
Sebastian Lapuschkin, Stephan Wäldchen, Alexander Binder, Grégoire Montavon, Wojeiech Samek, and Klaus-Robert Müller. Unmasking clever hans predictors and assessing what machines really learn. Nature communications, 10(1):1-8, 2019.
Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. Visualizing the loss landscape of neural nets. Advances Neural Information Processing Systems, 31, 2018.
Tao Li, Yingwen Wu, Sizhe Chen, Kun Fang, and Xiaolin Huang. Subspace adversarial training. In the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13409-13418, 2022.
Yuanzhi Li and Yingyu Liang. Learning overparameterized neural networks via stochastic gradient descent structured data. Advances in neural information processing systems, 31, 2018.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference Learning Representations, 2018.
Dhruv Mahajan, Ross Girshick, Vignesh Ramanathan, Kaiming He, Manohar Paluri, Yixuan Li, Ashwin Bharambe, and Laurens Van Der Maaten. Exploring the limits of weakly supervised pretraining. In Proceedings of the European Conference on Computer Vision, pp. 181-196, 2018.
Carlos Riquelme, Joan Puigcerver, Basil Mustafa, Maxim Neumann, Rodolphe Jenatton, André Susano Pinto, Daniel Keysers, and Neil Houlsby. Scaling vision with sparse mixture of experts. Advances in Neural Information Processing Systems, 34, 2021.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 115(3))211-25) 2015.
Vatsal Shah, Anastasios Kyrillidis, and Sujay Sanghavi. Minimum norm solutions do not always generalize well for over-parameterized problems. stat, 1050:16, 2018.
Rulin Shao, Zhouxing Shi, Jinfeng Yi, Pin-Yo Yu Chen, and Cho-Jui Hsieh. On the adversarial robustness of visual transformers. arXiv e-prints, pp. arXiv-2103, 2021.
Connor Shorten and Taghi M Khoshgoftaar. A survey on image data augmentation for deep learning. Journal of Big Data, 6(1):1-48, 2019.
Jiawei Su, Danilo Vasconcellos Vargas, and Kouichi Sakurai. One pixel attack for fooling deep neural networks. IEEE Transactions on Evolutionary Computation, 23(5):828-88) 8811,44,1111 2019.
Chen Sun, Abhinav Shrivastava, Saurabh Singh, and Abhinav Gupta. Revisiting unreasonable effec- tiveness of data in deep learning era. In Proceedings of the IEEE/CVF International Conference Computer Vision pp. 843-852. 2017.
12

---

Published as a conference paper at ICLR 2023
Christian Szegedy, Wojeiech Zaremba, llya Sutskever, Joan Bruna, Dumitru Erhan, lan Goodfellow, and Rob Fergus. Intriguing properties of neural networks. International Conference on Learning Representations, 2014.
Ashia C Wilson, Rebecca Roelofs, Mitchell Stern, Nati Srebro, and Benjamin Recht. The marginal value of adaptive gradient methods in machine learning. Advances in Neural Information Process- ing Systems, 30, 2017.
Cihang Xie, Mingxing Tan, Boqing Gong, Jiang Wang, Alan Yuille, and Quoe V Le. Adversarial examples improve image recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 819-828, 2020.
Chaofei Yang, Qing Wu, Hai Li, and Yiran Chen. Generative poisoning attack method against neural networks. arXiv preprint arXiv:1703.01333 2017.
Da Yu, Huishuai Zhang, Wei Chen, Jian Yin, and Tie-Yan Liu. Indiscriminate poisoning attacks are shortcuts. arXiv preprint arXiv:2111.008% 2021.
Chia-Hung Yuan and Shan-Hung Wu. Neural tangent generalization attacks. In International Conference on Machine Learning, pp. 12230-12240. PMLR, 2021.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In British Machine Vision Conference 2016. British Machine Vision Association, 2016.
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning (still) requires rethinking generalization. Communications of the ACM, 64(3):107-115, 2021.
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. In International Conference Learning Representations, 2018.
Qiming Zhang, Yufei Xu, Jing Zhang, and Dacheng Tao. Vitaev2: Vision transformer advanced by exploring inductive bias for image recognition and beyond. arXiv preprint arXiv:2202. 10108, 2022.
Tianyuan Zhang and Zhanxing Zhu. Interpreting adversarially trained convolutional neural networks. In International Conference on Machine Learning, pp. 7502-7511. PMLR, 2019.
13

---

Published as a conference paper at ICLR 2023
APPENDIX
A FURTHER INVESTIGATION ON SHORTCUT GENERATION
In Sec 3.2. craft OPS by perturbing fixed position fixed color target for each class. What if we only constrain one of these two properties? In other words, if perturbing fixed position to different color targets, or perturbing different positions a fixed color target, will strong shortcut be created? With curiosity, we further explore these two properties individually.
We use OPS-Position and OPS-Color represent the former and the latter setting. For OPS-Position, after finding the optimal position for each class via Algorithm [ random colors. For OPS-Color, we assign each class predefined color (we add two colors since there are only 8 boundary colors) and perturb pixels at different positions to this color. Note that there might be many positions where the original color is already the same as the predefined color. We perturb the position where the original color is the furthest from the predefined color (measured by the l2 norm) for each sample. As shown in Table A the results indicate that if we only constrain one property, the resulting shortcuts will be harder to be captured. Besides, random perturbations at fixed position can serve as a stronger shortcut than a fixed color at different positions.
we perturb pixels at this position to
Table A: One-Pixel Shortcut generated by different constraints. If perturbing a fixed position to different color targets or perturbing different positions to fixed color target, the ability to degrade DNNs training will be less strong.
Training Data
Clean
OPS
OPS-Position
OPS-Color
Clean Test Acc.
94.01
15.56
46.22
85.34
B BASELINE OF A RANDOM PIXEL
For the purpose of evaluating our searching algorithm proposed in Sec. 3.2 we compare it with the baseline, which randomly chooses fixed position and fixed color (denoted as RandPix) for each class to craft the shortcut training set, and report the clean accuracy of 10 runs in Table B Since 15.56% is out of the three standard deviation intervals, our OPS design is validated to be meaningful.
Table B: Comparison with the randomly chosen perturbed position and target color.
Training Data
Clean
OPS
RandPix
Clean Test Acc.
94.01
15.56
50.67 + 11.29
C SEARCHING OPS ON SMALL PROPORTION OF DATA
In Algorithm 0 we search the entire dataset to generate OPS. What if the data is gradually added to the dataset? In other words, will the perturbed position and target color of OPS that are found small proportion of data serve as strong shortcut on the entire dataset? Moreover, users might want to add their own noise before uploading their data to the dataset. From practical standpoint, we further explore the potential of OPS by designing the following experiments.
Firstly, we take out different proportions of training data from the CIFAR-10 10 training set to search for the perturbed position and target color. Then we perturbed the whole training set based on the results we get from the selected small proportion of data, and train DNNs the perturbed data. Secondly, considering the scenario that users add their own noise before uploading data to the dataset, based on the setting of the first experiment, we additionally inject random noise sampled from the uniform distribution [-e, e| (here we set e = 8/255, following the common setting of adversarial learning) to the unselected data.
14

---

Published as a conference paper at ICLR 2023
The results shown in Table C where denote the first setting Clean-Uploat and the second setting as Noisy-Upload. Even we only use 1% of the whole dataset to search the perturbed position and target color, OPS is still able to degrade the clean accuracy of the trained DNN to 32.87%. Besides, additional noise does not degrade the efficiency of OPS. To some degree, the experiments prove the generalization capability and robustness of OPS from practical standpoint.
Table C: We different proportions of data search the perturbed position and target we only use 1% of data, OPS can still largely degrade the clean accuracy of the trained network.
OPS. Even
Searching Proportion
Updating Type
1%
10%
100%
Clean-Upload Noisy-Upload
32.87 32.10 10
17.07 17.30
15.56
15.56
D EXPLORATIONS ON L2 ADVERSARIAL TRAINING
In Sec 4.3 we have studied the effectiveness of different types of shortcuts under different types of training strategies, including lo adversarial training and various data augmentations. For a more comprehensive exploration, we additionally evaluate them on l2 adversarial training. Besides the commonly used setting (e 0.5), also try larger attack budgets, as shown in Table D
Compared to loo, l2 is proved to be a more efficient AT manner for alleviating OPS. Nevertheless, when the perturbation budget is not large enough, the network will still be affected by the shorteuts. If we further enlarge c, it will hurt the clean test accuracy to a greater extent, As local shortcut, OPS displays stronger inertia under (2 AT, which can be viewed as a global data augmentation. As for EM, although generated by loc perturbations, is still sensitive to l2 AT. These results conform to our discussion about the properties of local&global shortcuts and local&global data augmentations in Sec 4.3
Table D: ResNet- 18 trained on different training data using adversarial training with diffcrent attack budgets.
Training Data OPS
Training Strategy
Clean
EM
Standard
94.01
15.56
19.58
12 AT (c=0.5)
87.68
43.26
71.00
l2 AT (c=1)
82.58
50.29
81.75
12 AT (e=2)
73.45
73.70
74.51
15


Reviews:
[{"review_id": "vcJbyQf30u", "reviewer": "Reviewer_qd1o", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper aims at preventing data from being used (without authorization) for training DNNs. They propose the One-Pixel Shortcut (OPS) method where all the images of a same class get the same pixel location replaced with a same color pixel. This method fools the network during training as the network will use this replaced color pixel shared within the class as a shortcut during training to learn the class instead of using actual discriminative features. The authors evaluate this method on CIFAR-10 on several architectures from ResNet to Vision Transformers.", "summary_of_the_review": "The paper is interesting, easy to read and proposes an easy strategy to prevent the unauthorized use of images for training. My concern is that modern architectures such as ViT use considerable amount of data augmentations whereas here data augmentations are studied separately and even individually they seem to reduce the impact of the proposed method. Furthermore, the authors only study l-inf adversarial perturbations which by construction will fail against the proposed method. Studying l2 perturbations should be more relevant. ", "strength_and_weaknesses": "Strengths:\n1) The authors evaluate their method on several architectures.\n2) The authors evaluate various training strategies as defenses against their proposed attack. They investigate data augmentations and adversarial training.\n\nWeaknesses:\n1) Subsection 3.1 is unclear and could be re-worked. Table 1 is not clear and could be better explained: the metric could be stated more clearly.\n2)  In Table 5, CutOut and RandAugment seem individually quite effective against the one pixel shortcut method. This table is missing the combination of CutOut and RandAugment. It seems that such a combination could strongly affect the proposed one pixel shortcut method (as the individual components are already effective). Furthermore, modern architectures such as ViT use considerable amount of data augmentations so it makes sense to study combination of augmentations.\n3) As easy missing training strategy in this study as mean of defense against the proposed one pixel shortcut, I would like to point to median filters and gaussian blurring.\n4) Using l-inf perturbations for adversarial training seems unfair as by construction such perturbations will not be able to compensate the pixel replacement of the proposed OPS method. Using l2 perturbations with medium-strong perturbation radiuses seems much more appropriate and interesting in this setting. Especially as adversarial training with l-inf perturbations already lowers the clean test accuracy due to the robustness/accuracy tradeoff observed with adversarial training. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "lGOtZRQ2JY", "reviewer": "Reviewer_bTQi", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a one-pixel shortcut (OPS) method to generate unlearnable examples that would render a trained model perform not better than a random network on test examples. This performs better than the existing unlearning example generation method by error minimization (EM) at different settings. EM and OPS are combined to create a dataset named CIFAR-10-S, which injects shortcuts in the dataset to evaluate different models' resistance to them.", "summary_of_the_review": "The one-Pixel shortcut method provides a new way of generating unlearnable examples. The paper is well-motivated and well-explained. The experiments are satisfactory. Although I would still argue that from a practical standpoint, EM unlearnable examples generation is more usable, since its not easily detectable upon an initial inspection. From a practitioner's standpoint, seeing that training losses are reaching 0, while the model gets poor test error, they would inspect the dataset first. OPS-based augmentations are easier to detect and remedy, once detected. \n\nHowever, I like the idea and think that this paper can lead the way for mitigating some of the shortcomings of the method.", "strength_and_weaknesses": "Strengths:\n- The paper provides a new perspective on learning a per-class noise that minimizes test accuracy while achieving good training accuracy. Shortcuts like these are easy to pick up by a network, especially if they are consistent across images in the same class.\n- Evaluation is very thorough, with different architectures showing that the method is not dependent or sensitive to architecture choices, augmentations, or training strategies. \n\nWeaknesses:\n- The method is rather complex in the sense that since neural networks are good at picking up shortcuts, it should be possible to perturb *any* pixel with *any* color consistently across all images in the same class - as long as the values of $\\sigma_k, \\xi_k$ are different for different k. This should be an initial baseline to compare against OPS which is obtained by minimizing Eqn (2). \n- One major downside of using OPS is that regardless of the claims about the perceptibility of the augmented images, it is extremely easy for a human (the reviewer in this case) to see exactly which pixel is perturbed. Unlike the EM method which has the capability to learn a per-image noise, the OPS method performs a class-wise perturbation. This is problematic for few reasons. First, the easy perceptibility of the augmented pixel can lead to manual solutions to fix this problem (one way to combat OPS - if done across the dataset, is to just find the pixel such that the standard deviation of the pixel is 0 across the images of that class - and replace it with a mean of neighboring pixels, etc.) - and finding an analogous way to \"undo the noise\" in EM is not easy. Secondly, if part of the information is perturbed, visual inspection is enough to discard the perturbed data and then learn a network, but in EM, this is not possible.\n\nThere are no comments about the \"generalization capability\" of this noise location and color. How robust is this in settings where more data is gradually added to the dataset (like continual learning, etc.)? The OPS shortcut is learned on the entire dataset, which is not what is done from a practical standpoint (an end user would want to add their own noise before uploading a picture to the internet). This issue need not be solved in the paper, but should be addressed.\n\nMoreover, it would be interesting to see how a method like Noise2Void [1] can be modified to use as an adversarial method to combat OPS - which performs selective masking on the image.\n\n[1] https://arxiv.org/pdf/1811.10980.pdf\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "irh_610QGOi", "reviewer": "Reviewer_vxV5", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper presents a method to generate unlearnable examples (ULE), using which if one trains a neural network, that network will not perform well (sometimes random performance) on an unclean test. The authors show that even changing just a small pixel value, given that that change is done consistently at the same location in all the images of the same class, can lead to surprisingly effective unlearnable examples. The application of such unlearnable examples is shown in many image classification settings (e.g., CIFAR-10) and superior performance over the existing paradigm of creating ULEs is shown.\n", "summary_of_the_review": "The findings of the paper are surprising. The community of machine learning for privacy will definitely find the work useful. The method introduced is simple and effective for an important task. The authors have shown the generalization ability of the method through multiple lens (different tasks, different network architectures). I hence recommend acceptance.\n", "strength_and_weaknesses": "The method is well motivated, very simple, and most importantly, surprisingly effective at creating unlearnable examples.\n\nThe results are not simply empirical, but the authors also analyze what goes on with the created ULEs. For example, the authors visualize what happens to the feature computed through ULEs and show that with just one pixel change, all the intermediate features change their properties. This result will be surprising to the community. That being said, it also needs to be studied a bit more. In other words, in what way have the community understood the convolutional layers incorrectly which led to us assuming something like this (Fig. 1) happens. \n\nThe phenomenon discovered by the authors seems to generalize well enough; in that it not only affects the traditional convolutional architectures (e.g. ResNet) but also to transformer based architectures (ViT). This corroborates the fact that this result is indeed not an exploitation of a specific weakness, but is a bug (or feature) of neural networks in general.\n\nComments/Questions\n\n“Images belonging to the same category are perturbed at the same position”. If I understand correctly, if you have two images of dogs, both will be perturbed at the same location with the same perturbation. How are these two properties related? In other words, what happens if you enforce one of the constraints (need for the perturbation to be in the same location) but not the other, and vice versa. This property should be studied better.\n\nMathematical notations are confusing sometimes. For example, in Eq. 1, it is not clear what c (in D_c, k) and i, j (in sigma_k(i, j)) mean. \n\nIs there any trend in the discovered location for perturbation? In other words, is it the case that for certain classes, the perturbed locations turn out to be in center whereas for some other they are on the edge?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: notable-top-25%