Original: O NE -P IXEL S HORTCUT : O N THE L EARNING P REFERENCE OF D EEP N EURAL N ETWORKS
Revised: O NE -P IXEL S HORTCUT : O N THE L EARNING P REFERENCE OF D EEP N EURAL N ETWORKS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Unlearnable examples (ULEs) aim to protect data from unauthorized usage for training DNNs.
Revised: Unlearnable examples (ULEs) aim to protect data from unauthorized usage for training DNNs.

Original: Existing work adds ℓ ∞ -bounded perturbations to the original sample so that the trained model generalizes poorly.
Revised: Existing work adds ℓ ∞ -bounded perturbations to the original sample so that the trained model generalizes poorly.

Original: Such perturbations, however, are easy to eliminate by, e.g., adversarial training and data augmentations.
Revised: Such perturbations, however, are easy to eliminate by, e.g., adversarial training and data augmentations.

Original: In this paper, we resolve this problem from a novel perspective by perturbing only one pixel in each image.
Revised: In this paper, we resolve this problem from a novel perspective by perturbing only one pixel in each image.

Original: Interestingly, such a small modification could effectively degrade model accuracy to almost an untrained counterpart.
Revised: Interestingly, such a small modification could effectively degrade model accuracy to almost an untrained counterpart.

Original: Moreover, our produced One-Pixel Shortcut (OPS) could not be erased by adversarial training and strong augmentations.
Revised: Moreover, our produced One-Pixel Shortcut (OPS) could not be erased by adversarial training and strong augmentations.

Original: To generate OPS, we perturb in-class images in the same pixel that, if changed to a 0 or 1, could mostly and stably deviate from all original images.
Revised: To generate OPS, we perturb in-class images in the same pixel that, if changed to a 0 or 1, could mostly and stably deviate from all original images.

Original: Since such calculation is only based on images, OPS needs significantly less computation than the previous methods based on model training.
Revised: Since such calculation is only based on images, OPS needs significantly less computation than the previous methods based on model training.

Original: By OPS, we introduce an unlearnable dataset called CIFAR-10-S, which is indistinguishable from CIFAR-10 by humans but induces the trained model to extremely low accuracy.
Revised: By OPS, we introduce an unlearnable dataset called CIFAR-10-S, which is indistinguishable from CIFAR-10 by humans but induces the trained model to extremely low accuracy.

Original: Even under adversarial training, a ResNet-18 trained on CIFAR-10-S has only 10.61% accuracy, compared to 83.02% by the existing error-minimizing method.
Revised: Even under adversarial training, a ResNet-18 trained on CIFAR-10-S has only 10.61% accuracy, compared to 83.02% by the existing error-minimizing method.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Deep neural networks (DNNs) have successfully promoted computer vision field in the past decade.
Revised: Deep neural networks (DNNs) have successfully promoted computer vision field in the past decade.

Original: As DNNs are scaling up unprecedentedly (Brock et al., 

Revised: As DNNs are scaling up unprecedentedly (Brock et al.,

Original: ; Huang et al., 2019;
Revised: ; Huang et al., 2019;

Original: Riquelme et al., 2021; Zhang et al., 2022), data becomes increasingly vital.
Revised: Riquelme et al., 2021; Zhang et al., 2022), data becomes increasingly vital.

Original: For example, ImageNet (Russakovsky et al., 2015) fostered the development of AlexNet (Krizhevsky et al., 2017).
Revised: For example, ImageNet (Russakovsky et al., 2015) fostered the development of AlexNet (Krizhevsky et al., 2017).

Original: Besides, people or organizations also collect online data to train DNNs, e.g. , IG-3.5B-17k (Mahajan et al., 2018) and
Revised: Besides, people or organizations also collect online data to train DNNs, e.g. , IG-3.5B-17k (Mahajan et al., 2018) and

Original: JFT-300M (Sun et al., 2017).
Revised: JFT-300M (Sun et al., 2017).

Original: This practice, however, raises the privacy concerns of Internet users.
Revised: This practice, however, raises the privacy concerns of Internet users.

Original: In this concern, researchers have made substantial efforts to protect personal data from abuse in model learning without affecting user experience (Feng et al., 2019;
Revised: In this concern, researchers have made substantial efforts to protect personal data from abuse in model learning without affecting user experience (Feng et al., 2019;

Original: Huang et al., 2020a; Fowl et al., 2021;
Revised: Huang et al., 2020a; Fowl et al., 2021;

Original: Yuan & Wu, 2021; Yu et al., 2021).
Revised: Yuan & Wu, 2021; Yu et al., 2021).

Original: Among those proposed methods, unlearnable examples (ULEs) (Huang et al., 2020a) take a great step to inject original images with protective but imperceptible perturbations from bi-level error minimization (EM).
Revised: Among those proposed methods, unlearnable examples (ULEs) (Huang et al., 2020a) take a great step to inject original images with protective but imperceptible perturbations from bi-level error minimization (EM).

Original: DNNs trained on ULEs generalize very poorly on normal images.
Revised: DNNs trained on ULEs generalize very poorly on normal images.

Original: However, such perturbations could be completely canceled out by adversarial training, which fails the protection, limiting the practicality of ULEs.
Revised: However, such perturbations could be completely canceled out by adversarial training, which fails the protection, limiting the practicality of ULEs.

Original: We view the data protection problem from the perspective of shortcut learning (Geirhos et al., 2020), which shows that DNN training is “lazy” (Chizat et al., 2019; Caron & Chrétien, 2020), i.e. , converges to the solution with minimum norm when optimized by gradient descent (Wilson et al., 2017; Shah et al., 2018; Zhang et al., 2021).
Revised: We view the data protection problem from the perspective of shortcut learning (Geirhos et al., 2020), which shows that DNN training is “lazy” (Chizat et al., 2019; Caron & Chrétien, 2020), i.e. , converges to the solution with minimum norm when optimized by gradient descent (Wilson et al., 2017; Shah et al., 2018; Zhang et al., 2021).

Original: In this case, a DNN would rely on every accessible feature to minimize the training loss, no matter whether it is semantic or not (Ilyas et al., 2019; Geirhos et al., 2018; Baker et al., 2018).
Revised: In this case, a DNN would rely on every accessible feature to minimize the training loss, no matter whether it is semantic or not (Ilyas et al., 2019; Geirhos et al., 2018; Baker et al., 2018).

Original: Thus, DNNs tend to ignore semantic features if there are other easy-to-learn shortcuts that are sufficient for distinguishing samples in different classes.
Revised: Thus, DNNs tend to ignore semantic features if there are other easy-to-learn shortcuts that are sufficient for distinguishing samples in different classes.

Original: Such shortcuts exist naturally or manually.
Revised: Such shortcuts exist naturally or manually.

Original: In data collection, e.g. , cows may mostly appear with grasslands, misleading DNN to predict cows by large-area green, because the color is easier to learn than cow’s semantic features and also sufficient to classify cows in training.
Revised: In data collection, e.g. , cows may mostly appear with grasslands, misleading DNN to predict cows by large-area green, because the color is easier to learn than cow’s semantic features and also sufficient to classify cows in training.

Original: Such natural shortcuts have been illustrated in detail in datasets
Revised: Such natural shortcuts have been illustrated in detail in datasets

Original: such ImageNet-A
Revised: such ImageNet-A

Original: (Hendrycks et al., 2021) and
Revised: (Hendrycks et al., 2021) and

Original: ObjectNet (Barbu et al., 2019).
Revised: ObjectNet (Barbu et al., 2019).

Original: Besides, shortcuts could also be manually crafted (Huang et al.,
Revised: Besides, shortcuts could also be manually crafted (Huang et al.,

Original: [Figure]
Revised: [Figure]

Original: n
Revised: n

Original: [Figure]
Revised: [Figure]

Original: i
Revised: i

Original: [Figure]
Revised: [Figure]

Original: l
Revised: l

Original: S ho
Revised: S ho

Original: [Figure]
Revised: [Figure]

Original: u t
Revised: u t

Original: DNN
Revised: DNN

Original: DNN extract features train train extract features
Revised: DNN extract features train train extract features

Original: 2020a).
Revised: 2020a).

Original: EM-based ULEs (Huang
Revised: EM-based ULEs (Huang

Original: , 2020a) mislead DNNs to learn features belonging to the perturbations, which falls in the category of shortcut learning (Yu et al., 2021).
Revised: , 2020a) mislead DNNs to learn features belonging to the perturbations, which falls in the category of shortcut learning (Yu et al., 2021).

Original: In this paper, we are surprised to find that shortcuts could be so small in the area that it can even be simply instantiated as a single pixel .
Revised: In this paper, we are surprised to find that shortcuts could be so small in the area that it can even be simply instantiated as a single pixel .

Original: By perturbing a pixel of each training sample, our method, namely One-Pixel Shortcut (OPS), degrades the model accuracy on clean data to almost an untrained counterpart.
Revised: By perturbing a pixel of each training sample, our method, namely One-Pixel Shortcut (OPS), degrades the model accuracy on clean data to almost an untrained counterpart.

Original: Moreover, our generated unbounded small noise could not be erased by adversarial training (Madry et al., 2018), which is effective in mitigating existing ULEs (Huang et al., 2020a).
Revised: Moreover, our generated unbounded small noise could not be erased by adversarial training (Madry et al., 2018), which is effective in mitigating existing ULEs (Huang et al., 2020a).

Original: To make the specific pixel stand out in view of DNNs, OPS perturbs in-class images in the same pixel that, if changed to a 0 or 1, could mostly and stably deviate from all original images.
Revised: To make the specific pixel stand out in view of DNNs, OPS perturbs in-class images in the same pixel that, if changed to a 0 or 1, could mostly and stably deviate from all original images.

Original: Specifically, the difference between the perturbed pixel and the original one in all in-class images should be large with low variance.
Revised: Specifically, the difference between the perturbed pixel and the original one in all in-class images should be large with low variance.

Original: Since such calculation is only based on images, OPS needs significantly less computational cost than the previous methods based on model training.
Revised: Since such calculation is only based on images, OPS needs significantly less computational cost than the previous methods based on model training.

Original: We evaluate OPS and its counterparts in 6 architectures, 6 model sizes, 8 training strategies on
Revised: We evaluate OPS and its counterparts in 6 architectures, 6 model sizes, 8 training strategies on

Original: CIFAR-10 (Krizhevsky et al., 2009) and ImageNet (Russakovsky et al., 2015) subset, and find that
Revised: CIFAR-10 (Krizhevsky et al., 2009) and ImageNet (Russakovsky et al., 2015) subset, and find that

Original: OPS is always superior in degrading model’s testing accuracy than EM ULEs.
Revised: OPS is always superior in degrading model’s testing accuracy than EM ULEs.

Original: Even under adversarial training, a ResNet-18 (He et al., 2016) trained on CIFAR-10-S has 10.61% testing accuracy, compared to 83.02% by the existing error-minimizing method.
Revised: Even under adversarial training, a ResNet-18 (He et al., 2016) trained on CIFAR-10-S has 10.61% testing accuracy, compared to 83.02% by the existing error-minimizing method.

Original: In this regard, we introduce a new unlearnable dataset named CIFAR-10-S, which combines the EM and OPS to craft stronger imperceptible ULEs.
Revised: In this regard, we introduce a new unlearnable dataset named CIFAR-10-S, which combines the EM and OPS to craft stronger imperceptible ULEs.

Original: Different from the existing datasets like ImageNet-A (Hendrycks et al., 2021) or ObjectNet (Barbu et al., 2019), which place objects into special environments to remove shortcuts, CIFAR-10-S injects shortcuts to evaluate the model’s resistance to them.
Revised: Different from the existing datasets like ImageNet-A (Hendrycks et al., 2021) or ObjectNet (Barbu et al., 2019), which place objects into special environments to remove shortcuts, CIFAR-10-S injects shortcuts to evaluate the model’s resistance to them.

Original: Contributions
Revised: Contributions

Original: • We analyze unlearnable examples from the perspective of shortcut learning, and demonstrate the DNN shortcut could be as small as a single pixel.
Revised: • We analyze unlearnable examples from the perspective of shortcut learning, and demonstrate the DNN shortcut could be as small as a single pixel.

Original: • We propose a novel data protection method named One-Pixel Shortcut (OPS), which perturbs in-class images in the pixel that could mostly and stably deviate from the original images.
Revised: • We propose a novel data protection method named One-Pixel Shortcut (OPS), which perturbs in-class images in the pixel that could mostly and stably deviate from the original images.

Original: OPS is a model-free method that is significantly faster than previous work.
Revised: OPS is a model-free method that is significantly faster than previous work.

Original: • We extensively evaluate OPS and find OPS outperforms baselines by a large margin in the ability to degrade accuracy, resist adversarial training, and data augmentations.
Revised: • We extensively evaluate OPS and find OPS outperforms baselines by a large margin in the ability to degrade accuracy, resist adversarial training, and data augmentations.

Original: Thus, we introduce CIFAR-10-S to assess the model’s ability to learn semantic features.
Revised: Thus, we introduce CIFAR-10-S to assess the model’s ability to learn semantic features.

Original:  R ELATED W ORK 2. A DVERSARIAL ATTACK AND DATA POISONING
Revised:  R ELATED W ORK 2. A DVERSARIAL ATTACK AND DATA POISONING

Original: Adversarial examples are perturbed by small perturbations, which are indistinguishable from the original examples by humans but can make DNNs give wrong predictions (Szegedy et al., 2014).
Revised: Adversarial examples are perturbed by small perturbations, which are indistinguishable from the original examples by humans but can make DNNs give wrong predictions (Szegedy et al., 2014).

Original: Many different adversarial attacks are proposed in recent years.
Revised: Many different adversarial attacks are proposed in recent years.

Original: Generally, most adversarial attacks aim to perturb the whole image with a constrained intensity (usually bounded by ℓ p norm), e.g. ,
Revised: Generally, most adversarial attacks aim to perturb the whole image with a constrained intensity (usually bounded by ℓ p norm), e.g. ,

Original: PGD (Madry et al., 2018), C&W (Carlini & Wagner, 2017) and Autoattack (Croce & Hein, 2020).
Revised: PGD (Madry et al., 2018), C&W (Carlini & Wagner, 2017) and Autoattack (Croce & Hein, 2020).

Original: Besides, there are also other methods that only perturb a small part of an image (Croce & Hein (2019); Dong et al.
Revised: Besides, there are also other methods that only perturb a small part of an image (Croce & Hein (2019); Dong et al.

Original: (2020)) or even a single pixel (Su et al.
Revised: (2020)) or even a single pixel (Su et al.

Original: The existence of adversarial examples indicates that DNNs do not sufficiently learn critical semantic information as we wish, but more or less depending on some non-robust features.
Revised: The existence of adversarial examples indicates that DNNs do not sufficiently learn critical semantic information as we wish, but more or less depending on some non-robust features.

Original: Data poisoning aims to modify the training data in order to affect the performance of models.
Revised: Data poisoning aims to modify the training data in order to affect the performance of models.

Original: Usually, the poisoned examples are notably modified and take only a part of the whole dataset (Yang et al., 2017; Koh & Liang, 2017).
Revised: Usually, the poisoned examples are notably modified and take only a part of the whole dataset (Yang et al., 2017; Koh & Liang, 2017).

Original: But those methods cannot degrade the performance of models to a low enough level, and the poisoned examples are easily distinguishable.
Revised: But those methods cannot degrade the performance of models to a low enough level, and the poisoned examples are easily distinguishable.

Original: Recently researchers have paid great attention to imperceptible poisoning which modifies examples slightly and does not damage their semantic information (Huang et al., 2020a;
Revised: Recently researchers have paid great attention to imperceptible poisoning which modifies examples slightly and does not damage their semantic information (Huang et al., 2020a;

Original: Fowl et al., 2021;
Revised: Fowl et al., 2021;

Original: Huang et al., 2020b; Doan et al., 2021;
Revised: Huang et al., 2020b; Doan et al., 2021;

Original: Geiping et al., 2020).
Revised: Geiping et al., 2020).

Original: Fowl et al.
Revised: Fowl et al.

Original: (2021) uses adversarial perturbations which contain information of wrong labels to poison the training data, which is equivalent to random label fitting.
Revised: (2021) uses adversarial perturbations which contain information of wrong labels to poison the training data, which is equivalent to random label fitting.

Original: On the contrary,
Revised: On the contrary,

Original: Huang et al.
Revised: Huang et al.

Original: (2020a) attack the training examples inversely, i.e. ,using error-minimizing perturbation, to craft unlearnable examples.
Revised: (2020a) attack the training examples inversely, i.e. ,using error-minimizing perturbation, to craft unlearnable examples.

Original: 2. S HORTCUT LEARNING
Revised: 2. S HORTCUT LEARNING

Original: Recently, researches on deep neural networks indicate that under the sufficiency of correct classification, DNNs tend to learn easier features instead of semantic features which make the object itself.
Revised: Recently, researches on deep neural networks indicate that under the sufficiency of correct classification, DNNs tend to learn easier features instead of semantic features which make the object itself.

Original: To be more specific, for example, the same object in different environments will get different predictions, which means the DNN overly relies on features that do not belong to the object (Beery et al., 2018).
Revised: To be more specific, for example, the same object in different environments will get different predictions, which means the DNN overly relies on features that do not belong to the object (Beery et al., 2018).

Original: Geirhos et al.
Revised: Geirhos et al.

Original: (2020) investigate this phenomenon in different fields of deep learning and explain why shortcuts exist and how to understand them.
Revised: (2020) investigate this phenomenon in different fields of deep learning and explain why shortcuts exist and how to understand them.

Original: Lapuschkin et al.
Revised: Lapuschkin et al.

Original: (2019) also observe this problem and attribute it to the unsuitable performance evaluation metrics that we generally use.
Revised: (2019) also observe this problem and attribute it to the unsuitable performance evaluation metrics that we generally use.

Original: The existence of natural adversarial examples (Hendrycks et al., 2021) also indicates that DNNs do not sufficiently learn the real semantic information during training.
Revised: The existence of natural adversarial examples (Hendrycks et al., 2021) also indicates that DNNs do not sufficiently learn the real semantic information during training.

Original: Instead, they may learn to use the background or texture of an image to predict.
Revised: Instead, they may learn to use the background or texture of an image to predict.

Original: Unlearnable examples (ULEs) (Huang et al., 2020a), which are manually crafted by error-minimizing noises and able to lead the models trained on them to obtain terrible generalization on test data, are believed to be some kind of shortcut that provides some textures that are easy to learn (Yu et al., 2021).
Revised: Unlearnable examples (ULEs) (Huang et al., 2020a), which are manually crafted by error-minimizing noises and able to lead the models trained on them to obtain terrible generalization on test data, are believed to be some kind of shortcut that provides some textures that are easy to learn (Yu et al., 2021).

Original: Generally, if we get enough data, the interconnection of different features will be enhanced so that those shortcuts may not be sufficient for classification tasks, i.e. , the model will have to use more complicated composed features in order to minimize the risk.
Revised: Generally, if we get enough data, the interconnection of different features will be enhanced so that those shortcuts may not be sufficient for classification tasks, i.e. , the model will have to use more complicated composed features in order to minimize the risk.

Original: However, when the data we collect obtains some specific bias ( e.g. , similar backgrounds), shortcut learning will not be mitigated effectively.
Revised: However, when the data we collect obtains some specific bias ( e.g. , similar backgrounds), shortcut learning will not be mitigated effectively.

Original: 2. D ATA AUGMENTATION
Revised: 2. D ATA AUGMENTATION

Original: Data augmentation aims to enhance the generalization ability of models.
Revised: Data augmentation aims to enhance the generalization ability of models.

Original: This is usually implemented by applying some transformations to the training data, e.g. , random stretching, random cropping, or color changing.
Revised: This is usually implemented by applying some transformations to the training data, e.g. , random stretching, random cropping, or color changing.

Original: Nowadays different kinds of data augmentation policies (Zhang et al., 2018;
Revised: Nowadays different kinds of data augmentation policies (Zhang et al., 2018;

Original: DeVries & Taylor, 2017; Cubuk et al., 2019; 2020) are proven to effectively boost the generalization ability of DNNs.
Revised: DeVries & Taylor, 2017; Cubuk et al., 2019; 2020) are proven to effectively boost the generalization ability of DNNs.

Original: Sometimes adversarial training is also regarded as a kind of data augmentation (Shorten & Khoshgoftaar, 2019; Xie et al., 2020).
Revised: Sometimes adversarial training is also regarded as a kind of data augmentation (Shorten & Khoshgoftaar, 2019; Xie et al., 2020).

Original: Dao et al.
Revised: Dao et al.

Original: (2019) deem data augmentation as a kind of data-dependent regularization term.
Revised: (2019) deem data augmentation as a kind of data-dependent regularization term.

Original: Since data augmentations are believed to improve the generalization ability of DNNs, we use different augmentations to evaluate the effectiveness of different data protection methods.
Revised: Since data augmentations are believed to improve the generalization ability of DNNs, we use different augmentations to evaluate the effectiveness of different data protection methods.

Original:  O NE -P IXEL S HORTCUT 3. P RELIMINARIES
Revised:  O NE -P IXEL S HORTCUT 3. P RELIMINARIES

Original: Unlearnable examples (ULEs) are a data protection method created by error-minimizing (EM)
Revised: Unlearnable examples (ULEs) are a data protection method created by error-minimizing (EM)

Original: noises (Huang et al., 2020a).
Revised: noises (Huang et al., 2020a).

Original: Models trained on examples that are perturbed by those noises will get almost zero training error, but perform like random guessing on clean test data.
Revised: Models trained on examples that are perturbed by those noises will get almost zero training error, but perform like random guessing on clean test data.

Original: Due to the imperceptibility
Revised: Due to the imperceptibility

Original: [Table]
Revised: [Table]

Original: of the noise, this method can prevent the abuse of data by some unauthorized users who attempt to train deep models for improper purposes, without affecting normal usage.
Revised: of the noise, this method can prevent the abuse of data by some unauthorized users who attempt to train deep models for improper purposes, without affecting normal usage.

Original: This bi-level problem can be solved by optimizing the inner minimization and the outer minimization alternately.
Revised: This bi-level problem can be solved by optimizing the inner minimization and the outer minimization alternately.

Original: It is proved that the perturbations belonging to the same class are well clustered and linearly separable (Yu et al., 2021).
Revised: It is proved that the perturbations belonging to the same class are well clustered and linearly separable (Yu et al., 2021).

Original: Thus, EM provides easy-to-learn features which are closely interconnected with labels.
Revised: Thus, EM provides easy-to-learn features which are closely interconnected with labels.

Original: We design a shuffling experiment to demonstrate that DNNs learn the shortcuts instead of images if trained on data that have shortcuts.
Revised: We design a shuffling experiment to demonstrate that DNNs learn the shortcuts instead of images if trained on data that have shortcuts.

Original: Denote ˆ D
Revised: Denote ˆ D

Original: =
Revised: =

Original: ( x i + δ i , y i ) } as the perturbed training set, where δ i is the perturbation associated with the i -th example.
Revised: ( x i + δ i , y i ) } as the perturbed training set, where δ i is the perturbation associated with the i -th example.

Original: After shuffling, the training set becomes ˆ
Revised: After shuffling, the training set becomes ˆ

Original: D ′
Revised: D ′

Original: = { ( x j + δ i , y i ) } .
Revised: = { ( x j + δ i , y i ) } .

Original: We are curious about how the DNNs trained on different data (with or without shortcuts) would predict the shuffled perturbed training set.
Revised: We are curious about how the DNNs trained on different data (with or without shortcuts) would predict the shuffled perturbed training set.

Original: As shown in Table 1, the DNN trained on clean data performs like a random guess on shuffled perturbed data while keeping high accuracy on unshuffled data, indicating that it successfully learns the representations from images { x i } rather than giving predictions according to the perturbations { δ i } from EM or OPS.
Revised: As shown in Table 1, the DNN trained on clean data performs like a random guess on shuffled perturbed data while keeping high accuracy on unshuffled data, indicating that it successfully learns the representations from images { x i } rather than giving predictions according to the perturbations { δ i } from EM or OPS.

Original: In contrast, the DNN trained on EM training data tends to memorize a significant proportion of perturbations, because it has 48.97% accuracy on the shuffled EM training data.
Revised: In contrast, the DNN trained on EM training data tends to memorize a significant proportion of perturbations, because it has 48.97% accuracy on the shuffled EM training data.

Original: Moreover, OPS training data produces a DNN with 72.43% accuracy on the shuffled set, reflecting it forcing the DNN to learn the perturbations to a greater extent.
Revised: Moreover, OPS training data produces a DNN with 72.43% accuracy on the shuffled set, reflecting it forcing the DNN to learn the perturbations to a greater extent.

Original: Our study illustrates the learning characteristics of DNNs trained with or without shortcuts, and also shows that OPS is a more effective shortcut than EM.
Revised: Our study illustrates the learning characteristics of DNNs trained with or without shortcuts, and also shows that OPS is a more effective shortcut than EM.

Original: 3. F OOLING DNN TRAINING BY A PIXEL
Revised: 3. F OOLING DNN TRAINING BY A PIXEL

Original: Following the discussion above, for the purpose of data protection, we need to craft shortcuts that are easy enough to learn and thus fool the network training.
Revised: Following the discussion above, for the purpose of data protection, we need to craft shortcuts that are easy enough to learn and thus fool the network training.

Original: According to previous studies, shortcuts can come from background environments that naturally exist inside our datasets (Beery et al., 2018), or be manually crafted like EM (Huang et al., 2020a).
Revised: According to previous studies, shortcuts can come from background environments that naturally exist inside our datasets (Beery et al., 2018), or be manually crafted like EM (Huang et al., 2020a).

Original: Unlike those shortcuts which might occupy the whole image or a notable part, we investigate how a single pixel, which is the minimum unit of digital images, can affect the learning process of deep neural networks.
Revised: Unlike those shortcuts which might occupy the whole image or a notable part, we investigate how a single pixel, which is the minimum unit of digital images, can affect the learning process of deep neural networks.

Original: Thus, we propose One-Pixel
Revised: Thus, we propose One-Pixel

Original: Shortcut (OPS), which modifies only a single pixel of each image.
Revised: Shortcut (OPS), which modifies only a single pixel of each image.

Original: Images belonging to the same category are perturbed at the same position, which means the perturbed pixel is interconnected with the category label.
Revised: Images belonging to the same category are perturbed at the same position, which means the perturbed pixel is interconnected with the category label.

Original: Although so minuscule, it is efficient enough to fool the training of deep learning models.
Revised: Although so minuscule, it is efficient enough to fool the training of deep learning models.

Original: We use a heuristic but effective method to generate perturbations for images belonging to each category.
Revised: We use a heuristic but effective method to generate perturbations for images belonging to each category.

Original: We search the position and value of the pixel which can result in the most significant change for the whole category.
Revised: We search the position and value of the pixel which can result in the most significant change for the whole category.

Original: Denoting D as the clean dataset and D k as the clean subset containing all the examples of class k , the problem can be formulated as:
Revised: Denoting D as the clean dataset and D k as the clean subset containing all the examples of class k , the problem can be formulated as:

Original: [Equation]
Revised: [Equation]

Original: where σ k ∈ R H × W represents the perturbed position mask and σ k ( i, j ) is the element at the i -th row and j -th column, ξ k ∈ R C stands for the perturbed target color ( C = 3 for RGB images), and G is the objective function.
Revised: where σ k ∈ R H × W represents the perturbed position mask and σ k ( i, j ) is the element at the i -th row and j -th column, ξ k ∈ R C stands for the perturbed target color ( C = 3 for RGB images), and G is the objective function.

Original: Since the optimization above is an NP-hard problem, we cannot solve it directly.
Revised: Since the optimization above is an NP-hard problem, we cannot solve it directly.

Original: Thus we constrain the feasible region to a limited discrete searching space, where we search the boundary value of each color channel, i.e. , ξ k ∈ { 0 , 1 } 3 , at every point of an image.
Revised: Thus we constrain the feasible region to a limited discrete searching space, where we search the boundary value of each color channel, i.e. , ξ k ∈ { 0 , 1 } 3 , at every point of an image.

Original: Specifically, for CIFAR-10 images, the discrete searching space will contain 32 × 32 × 2 3 = 8192 elements.
Revised: Specifically, for CIFAR-10 images, the discrete searching space will contain 32 × 32 × 2 3 = 8192 elements.

Original: To ensure that the pixel is stably perturbed, we also hope that the variance of the difference between them is small.
Revised: To ensure that the pixel is stably perturbed, we also hope that the variance of the difference between them is small.

Original: Accordingly, we design the objective function G k for class k as:
Revised: Accordingly, we design the objective function G k for class k as:

Original: [Equation]
Revised: [Equation]

Original: where x i ∈
Revised: where x i ∈

Original: R H
Revised: R H

Original: × W denotes the i -th channel of x , and ξ kj ∈ R is the i -th channel of ξ k .
Revised: × W denotes the i -th channel of x , and ξ kj ∈ R is the i -th channel of ξ k .

Original: After solving the position map and color, we get perturbation δ for each example ( x, y ) as:
Revised: After solving the position map and color, we get perturbation δ for each example ( x, y ) as:

Original: [Equation]
Revised: [Equation]

Original: Details can be found in Algorithm 1.
Revised: Details can be found in Algorithm 1.

Original: The resulting One-Pixel Shortcut is illustrated in Figure 2.
Revised: The resulting One-Pixel Shortcut is illustrated in Figure 2.

Original: Algorithm do solve the best perturbed do do 5: x i x i · ( I − σ k ) ξ k # modify the optimal for every k 6: end for 7: end for 8: ˆ D k
Revised: Algorithm do solve the best perturbed do do 5: x i x i · ( I − σ k ) ξ k # modify the optimal for every k 6: end for 7: end for 8: ˆ D k

Original: = { ˆ x } 9: end for 10: return
Revised: = { ˆ x } 9: end for 10: return

Original: D M
Revised: D M

Original: EM ULE
Revised: EM ULE

Original: O ne- P ixel
Revised: O ne- P ixel

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: 3. P ROPERTIES OF O NE -P IXEL S HORTCUT
Revised: 3. P ROPERTIES OF O NE -P IXEL S HORTCUT

Original: Since we all empirically believe that convolutional networks tend to capture textures (Hermann et al., 2020) or shapes (Geirhos et al., 2018; Zhang & Zhu, 2019), it is surprising that convolutional networks can be affected so severely by just one pixel.
Revised: Since we all empirically believe that convolutional networks tend to capture textures (Hermann et al., 2020) or shapes (Geirhos et al., 2018; Zhang & Zhu, 2019), it is surprising that convolutional networks can be affected so severely by just one pixel.

Original: As illustrated by Figure 1, the network indeed tends to learn those less complicated nonsemantic features brought by One-Pixel Shortcut.
Revised: As illustrated by Figure 1, the network indeed tends to learn those less complicated nonsemantic features brought by One-Pixel Shortcut.

Original: Besides convolutional networks, we observe that compact vision transformers (Hassani et al., 2021) are also attracted by One-Pixel Shortcut and ignore other semantic features.
Revised: Besides convolutional networks, we observe that compact vision transformers (Hassani et al., 2021) are also attracted by One-Pixel Shortcut and ignore other semantic features.

Original: This indicates that shortcuts are not particularly learned by some specific architecture.
Revised: This indicates that shortcuts are not particularly learned by some specific architecture.

Original: We also visualize the loss landscape of
Revised: We also visualize the loss landscape of

Original: ResNet-18 trained on clean CIFAR-10 data and One-Pixel Shortcut data.
Revised: ResNet-18 trained on clean CIFAR-10 data and One-Pixel Shortcut data.

Original: Illustrated as Figure 3, while trained on OPS data, the loss surface is much flatter, which means that these minima found by the network are more difficult to escape.
Revised: Illustrated as Figure 3, while trained on OPS data, the loss surface is much flatter, which means that these minima found by the network are more difficult to escape.

Original: Even if we use a ResNet-18 pretrained on clean CIFARand then fine-tune it on the OPS data, the network will still fall into this badly generalized minima.
Revised: Even if we use a ResNet-18 pretrained on clean CIFARand then fine-tune it on the OPS data, the network will still fall into this badly generalized minima.

Original: [Figure]
Revised: [Figure]

Original: In addition, we record the trajectories of training accuracy and the Frobenius norm of parameter difference,
Revised: In addition, we record the trajectories of training accuracy and the Frobenius norm of parameter difference,

Original: ∥ θ − θ 

Revised: ∥ θ − θ

Original: ∥ F , which can reflect the magnitude of network parameter change.
Revised: ∥ F , which can reflect the magnitude of network parameter change.

Original: Here θ and θrespectively indicate the parameters after training and at the initialized point.
Revised: Here θ and θrespectively indicate the parameters after training and at the initialized point.

Original: We draw the relation curve between training accuracy and ∥ θ − θ 

Revised: We draw the relation curve between training accuracy and ∥ θ − θ

Original: ∥ F , which can be found in Figure 4.
Revised: ∥ F , which can be found in Figure 4.

Original: When training accuracy rises up to 90% for the first time, the model trained on OPS data has a much smaller ∥ θ − θ 

Revised: When training accuracy rises up to 90% for the first time, the model trained on OPS data has a much smaller ∥ θ − θ

Original: ∥ F than that trained on clean data, which indicates that the OPS-trained model gets stuck in an optimum closer to the initialization point.
Revised: ∥ F than that trained on clean data, which indicates that the OPS-trained model gets stuck in an optimum closer to the initialization.

Original: It has been widely known that deep models optimized by gradient descent willalways converge to the solution  with  minimum norm  (Wilson et al., 2017; Shah et al., 2018;  Zhang et al., 2021).
Revised: It has been widely known that overparameterized DNNs optimized by gradient descent will converge to the solution that is close to the initialization, i.e. with the minimum norm of parameter difference (Wilson et al., 2017; Shah et al., 2018; Li & Liang, 2018; Zhang et al., 2021).

Original: Since OPS only perturbs a single pixel, the original representations of images are not damaged, and the model trained on clean data can still keep great performance on OPS data, which indicates that the corresponding solution  still exists, but  due to the tendency for minimum norm solution, the well-generalized solution with lager norm is not reached.
Revised: Since OPS only perturbs a single pixel, the original representations of images are not damaged, and the model trained on clean data can still keep great performance on OPS data, which indicates that the well-generalized solution far from the initialization still exists but is not reached due to the tendency for a close solution.

Original: Minimum-norm solution is believed to obtain better generalization ability.
Revised: The close solution is believed to obtain better generalization ability.

Original: Nevertheless, this argument is true only under the assumption that the training data and test data are from the exact same distribution and have the exact same features.
Revised: Nevertheless, this argument is true only under the assumption that the training data and test data are from the exact same distribution and have the exact same features.

Original: The existence of OPS forces the model to converge to an optimum where the model generalizes well on OPS features, which are not contained in test data.
Revised: The existence of OPS forces the model to converge to an optimum where the model generalizes well on OPS features, which are not contained in test data.

Original: From our experiment results in Table 2, OPS can degrade test accuracy to a lower level.
Revised: From our experiment results in Table 2, OPS can degrade test accuracy to a lower level.

Original: This is because EM requires a generator model, and thus may contain features more or less depending on it, which constrains the effectiveness of other models.
Revised: This is because EM requires a generator model, and thus may contain features more or less depending on it, which constrains the effectiveness of other models.

Original: On the other hand, OPS is a universal model-free method, and the shortcuts are crafted based on the inherent learning preference of DNNs.
Revised: On the other hand, OPS is a universal model-free method, and the shortcuts are crafted based on the inherent learning preference of DNNs.

Original:  E XPERIMENTS 4. S ETTING
Revised:  E XPERIMENTS 4. S ETTING

Original: Our experiments are implemented on CIFAR-10 and ImageNet subset, using 4 NVIDIA RTX 2080Ti
Revised: Our experiments are implemented on CIFAR-10 and ImageNet subset, using 4 NVIDIA RTX 2080Ti

Original: GPUs.
Revised: GPUs.

Original: We investigate how the One-Pixel Shortcut can affect the training of different models (including different architectures and different capacities).
Revised: We investigate how the One-Pixel Shortcut can affect the training of different models (including different architectures and different capacities).

Original: We evaluate our method on different models including convolutional networks (He et al., 2016; Zagoruyko & Komodakis, 2016; Huang et al., 2017) and the recently proposed compact vision transformers (Hassani et al., 2021).
Revised: We evaluate our method on different models including convolutional networks (He et al., 2016; Zagoruyko & Komodakis, 2016; Huang et al., 2017) and the recently proposed compact vision transformers (Hassani et al., 2021).

Original: For all the convolutional networks, we use an SGD optimizer with a learning rate set to 0 .
Revised: For all the convolutional networks, we use an SGD optimizer with a learning rate set to 0 .

Original: 1 , momentum set to 0 .
Revised: 1 , momentum set to 0 .

Original: 9 , and weight decay set to 5 e − 4 .
Revised: 9 , and weight decay set to 5 e − 4 .

Original: For all the compact vision transformers, we use AdamW optimizer with β 1 = 0 .
Revised: For all the compact vision transformers, we use AdamW optimizer with β 1 = 0 .

Original: 9 , β 2 = 0 .
Revised: 9 , β 2 = 0 .

Original: 999 , learning rate set to 5 e − 4 , and weight decay set to 3 e − 2 .
Revised: 999 , learning rate set to 5 e − 4 , and weight decay set to 3 e − 2 .

Original: Batch size is set to 128 for all the models except WideResNet-28-10, where it is set to 64 .
Revised: Batch size is set to 128 for all the models except WideResNet-28-10, where it is set to 64 .

Original: [Table]
Revised: [Table]

Original: Besides, We also try different training strategies including adversarial training (Madry et al., 2018) and different data augmentations (Mixup (Zhang et al., 2018),
Revised: Besides, We also try different training strategies including adversarial training (Madry et al., 2018) and different data augmentations (Mixup (Zhang et al., 2018),

Original: Cutout (DeVries & Taylor, 2017) and RandAugment (Cubuk et al., 2020)).
Revised: Cutout (DeVries & Taylor, 2017) and RandAugment (Cubuk et al., 2020)).

Original: For adversarial training, we use a 10-step PGD attack, setting step size to 2 / 255 and ℓ ∞ bound to 8 / 255 .
Revised: For adversarial training, we use a 10-step PGD attack, setting step size to 2 / 255 and ℓ ∞ bound to 8 / 255 .

Original: For Mixup, Cutout, and
Revised: For Mixup, Cutout, and

Original: RandAugment, we use the default settings from their original papers.
Revised: RandAugment, we use the default settings from their original papers.

Original: All the models are trained for 200 epochs with a multi-step learning rate schedule, and the training accuracy of each model is guaranteed to reach near 100%.
Revised: All the models are trained for 200 epochs with a multi-step learning rate schedule, and the training accuracy of each model is guaranteed to reach near 100%.

Original: We additionally tested our method on the ImageNet subset (the first 100 classes).
Revised: We additionally tested our method on the ImageNet subset (the first 100 classes).

Original: We center-crop all the images to 224 × 224 , and train common DNNs with results in Table 4.
Revised: We center-crop all the images to 224 × 224 , and train common DNNs with results in Table 4.

Original: We adopt an initial learning rate of 0.1 with a multi-step learning rate scheduler and train models for 200 epochs.
Revised: We adopt an initial learning rate of 0.1 with a multi-step learning rate scheduler and train models for 200 epochs.

Original: Our
Revised: Our

Original: One-Pixel Shortcut can still be effective in protecting large-scale datasets.
Revised: One-Pixel Shortcut can still be effective in protecting large-scale datasets.

Original: The networks trained on
Revised: The networks trained on

Original: OPS data will get much lower clean test accuracy than those trained on clean data.
Revised: OPS data will get much lower clean test accuracy than those trained on clean data.

Original: [Table]
Revised: [Table]

Original: 4. E FFECTIVENESS ON DIFFERENT MODELS
Revised: E FFECTIVENESS ON DIFFERENT MODELS

Original: [Table]

Original: We train different convolutional networks and vision transformers on the One-Pixel Shortcut CIFARtraining set, and evaluate their performance on the unmodified CIFAR-10 test set.
Revised: We train different convolutional networks and vision transformers on the One-Pixel Shortcut CIFAR10 training set, and evaluate their performance on the unmodified CIFAR-10 test set.

Original: Details are shown in Table 2.
Revised: Details are shown in Table 2.

Original: Every model reaches a very high training accuracy after only several epochs, which is much faster than training on clean data.
Revised: Every model reaches a very high training accuracy after only several epochs, which is much faster than training on clean data.

Original: Meanwhile, they all get really low test accuracy (about 15%) on clean test data, indicating that they do not generalize at all.
Revised: Meanwhile, they all get really low test accuracy (about 15%) on clean test data, indicating that they do not generalize at all.

Original: Although the perturbed image looks virtually the same as the original image, and all the models get near 100% training accuracy quickly, they do not capture any semantic information but just the pixels we modify in the images.
Revised: Although the perturbed image looks virtually the same as the original image, and all the models get near 100% training accuracy quickly, they do not capture any semantic information but just the pixels we modify in the images.

Original: We also train models on EM training set, which is generated by a ResNetusing the official implementation of Huang et al.
Revised: We also train models on EM training set, which is generated by a ResNet-18 using the official implementation of Huang et al.

Original: (2020a).
Revised: (2020a).

Original: The ℓ ∞ bound of EM noises is set to 8 / 255 .
Revised: The ℓ ∞ bound of EM noises is set to 8 / 255 .

Original: The generation of OPS costs only about 30 seconds, which is much faster than EM costing about half an hour.
Revised: The generation of OPS costs only about 30 seconds, which is much faster than EM costing about half an hour.

Original: For different networks, OPS can degrade their test accuracy to a lower level than EM.
Revised: For different networks, OPS can degrade their test accuracy to a lower level than EM.

Original: EM works the best on ResNet-18 (19.58% test accuracy), which has the same architecture as the generator.
Revised: EM works the best on ResNet-18 (19.58% test accuracy), which has the same architecture as the generator.

Original: On other models, their get higher test accuracy than ResNet-18.
Revised: On other models, their get higher test accuracy than ResNet-18.

Original: Meanwhile, since OPS is a model-free method that takes advantage of the natural learning preference of neural networks, its transferability is better across different models.
Revised: Meanwhile, since OPS is a model-free method that takes advantage of the natural learning preference of neural networks, its transferability is better across different models.

Original: Besides different architectures, we also explore the impact on models with the same architecture but different capacities.
Revised: Besides different architectures, we also explore the impact on models with the same architecture but different capacities.

Original: We trained several WideResNets (Zagoruyko & Komodakis, 2016) with different sizes.
Revised: We trained several WideResNets (Zagoruyko & Komodakis, 2016) with different sizes.

Original: The experiment results can be found in Table 3.
Revised: The experiment results can be found in Table 3.

Original: We observe that overparameterization, which is generally believed to enhance the ability to capture complicated features, does not circumvent the shortcut features.
Revised: We observe that overparameterization, which is generally believed to enhance the ability to capture complicated features, does not circumvent the shortcut features.

Original:  Moreover, we observe that vision transformers are easily affected by manually crafted shortcuts, even though it is believed that their self-attention mechanism makes them less sensitive to data distribution shifts (Shao et al., 2021; Bhojanapalli et al., 2021).
Revised: [Table] Moreover, we observe that vision transformers are easily affected by manually crafted shortcuts, even though it is believed that their self-attention mechanism makes them less sensitive to data distribution shifts (Shao et al., 2021; Bhojanapalli et al., 2021).

Original: For CCT-7-3 × 1 and CVT-7-4 (Hassani et al., 2021), EM and OPS can degrade their test accuracy below 30% and 20%.
Revised: For CCT-7-3 × 1 and CVT-7-4 (Hassani et al., 2021), EM and OPS can degrade their test accuracy below 30% and 20%.

Original: This indicates that vision transformers may not generalize on out-of-distributions data as well as our expectations.
Revised: This indicates that vision transformers may not generalize on out-of-distributions data as well as our expectations.

Original: If the training data is largely biased, i.e. , has notable shortcuts, and vision transformers will not perform much better than convolutional networks.
Revised: If the training data is largely biased, i.e. , has notable shortcuts, and vision transformers will not perform much better than convolutional networks.

Original: [Table]
Revised: [Table]

Original: 4. E FFECTIVENESS UNDER DIFFERENT TRAINING STRATEGIES
Revised: 4. E FFECTIVENESS UNDER DIFFERENT TRAINING STRATEGIES

Original: To evaluate the effectiveness of OPS under different training strategies
Revised: To evaluate the effectiveness of OPS under different training strategies

Original: [Table]
Revised: [Table]

Original: we train models on OPS perturbed data using adversarial training and different data augmentations such as Mixup (Zhang et al., 2018), Cutout (DeVries & Taylor, 2017) and RandAugment (Cubuk et al., 2020).
Revised: we train models on OPS perturbed data using adversarial training and different data augmentations such as Mixup (Zhang et al., 2018), Cutout (DeVries & Taylor, 2017) and RandAugment (Cubuk et al., 2020).

Original: Simple augmentations like random crop and flip are used by default in standard training.
Revised: Simple augmentations like random crop and flip are used by default in standard training.

Original: Models are also trained on EM perturbed data.
Revised: Models are also trained on EM perturbed data.

Original: As shown in Table
Revised: As shown in Table

Original: [Table]
Revised: [Table]

Original: we can observe that both EM and OPS have a good performance on data protection, which degrade test accuracy to 19.58% and 15.56%.
Revised: we can observe that both EM and OPS have a good performance on data protection, which degrade test accuracy to 19.58% and 15.56%.

Original: As mentioned in previous works (Huang et al., 2020a; Fu et al., 2021)
Revised: As mentioned in previous works (Huang et al., 2020a; Fu et al., 2021)

Original: [Table]
Revised: [Table]

Original: EM can not work so effectively under adversarial training, and the model can reach an even higher accuracy than adversarially trained on clean data.
Revised: EM can not work so effectively under adversarial training [Table] and the model can reach an even higher accuracy than adversarially trained on clean data.

Original: Meanwhile, OPS can still keep effective under adversarial training.
Revised: Meanwhile, OPS can still keep effective under adversarial training.

Original: However, when it comes to data augmentation, EM seems more impervious, while OPS is more sensitive, especially to
Revised: However, when it comes to data augmentation, EM seems more impervious, while OPS is more sensitive, especially to

Original: Cutout and RandAugment.
Revised: Cutout and RandAugment.

Original: This is due to the fact that EM injects global noises into images, while
Revised: This is due to the fact that EM injects global noises into images, while

Original: OPS only modifies a single pixel, which is equivalent to adding a very local perturbation.
Revised: OPS only modifies a single pixel, which is equivalent to adding a very local perturbation.

Original: Adversarial training, which can be regarded as a kind of global augmentation, is able to attenuate the dependence on global shortcuts.
Revised: Adversarial training, which can be regarded as a kind of global augmentation, is able to attenuate the dependence on global shortcuts.

Original: On the other hand, local data augmentations like Cutout make models less sensitive to local shortcuts.
Revised: On the other hand, local data augmentations like Cutout make models less sensitive to local shortcuts.

Original: Naturally, for the purpose of complementing each other, we can combine EM and our proposed
Revised: Naturally, for the purpose of complementing each other, we can combine EM and our proposed

Original: OPS together to craft a kind of ensemble shortcut.
Revised: OPS together to craft a kind of ensemble shortcut.

Original: Since OPS only modified a single pixel, after being applied to EM perturbed images, the imperceptibility can still be guaranteed.
Revised: Since OPS only modified a single pixel, after being applied to EM perturbed images, the imperceptibility can still be guaranteed.

Original: We evaluate the effectiveness of this ensemble method under different training strategies and find that it can always keep effective.
Revised: We evaluate the effectiveness of this ensemble method under different training strategies and find that it can always keep effective.

Original: Even if we use adversarial training and strong data augmentation like RandAugment, it is still able to degrade test accuracy to a relatively low level.
Revised: Even if we use adversarial training and strong data augmentation like RandAugment, it is still able to degrade test accuracy to a relatively low level.

Original: Based on this property, we introduce
Revised: Based on this property, we introduce

Original: CIFAR-10-S, where all the images are perturbed by the EM-OPS-composed noises.
Revised: CIFAR-10-S, where all the images are perturbed by the EM-OPS-composed noises.

Original: It can serve as a new benchmark to evaluate the ability to learn critical information under the disturbance of composed non-semantic representations.
Revised: It can serve as a new benchmark to evaluate the ability to learn critical information under the disturbance of composed non-semantic representations.

Original: We also extend our method to multi-pixel scenarios.
Revised: We also extend our method to multi-pixel scenarios.

Original: According to Table 6, as the number of perturbed pixels increases, the test accuracy can be degraded to a lower level.
Revised: According to Table 6, as the number of perturbed pixels increases, the test accuracy can be degraded to a lower level.

Original: Nevertheless, the more pixels are perturbed, the imperceptibility gets weaker, as illustrated in Figure 5.
Revised: Nevertheless, the more pixels are perturbed, the imperceptibility gets weaker, as illustrated in Figure 5.

Original: From our experiment on
Revised: From our experiment on

Original: ResNet-18, 3-Pixel Shortcut can easily degrade the test accuracy to 9.74%.
Revised: ResNet-18, 3-Pixel Shortcut can easily degrade the test accuracy to 9.74%.

Original: Moreover, more perturbed pixels alleviate the sensitivity to different data augmentations.
Revised: Moreover, more perturbed pixels alleviate the sensitivity to different data augmentations.

Original: For RandAugment, one more perturbed pixel can degrade the test accuracy to 46.45%, which is much lower than 71.18% of OPS.
Revised: For RandAugment, one more perturbed pixel can degrade the test accuracy to 46.45%, which is much lower than 71.18% of OPS.

Original: [Table]
Revised: [Table]

Original:  D ISCUSSION AND C ONCLUSION
Revised:  D ISCUSSION AND C ONCLUSION

Original: In this paper, we study the mechanism of recently proposed unlearnable examples (ULEs) which use error-minimizing (EM) noises.
Revised: In this paper, we study the mechanism of recently proposed unlearnable examples (ULEs) which use error-minimizing (EM) noises.

Original: We figure out that instead of semantic features contained by images themselves, features from EM noises are mainly learned by DNNs after training.
Revised: We figure out that instead of semantic features contained by images themselves, features from EM noises are mainly learned by DNNs after training.

Original: These kinds of easy-to-learn representations work as shortcuts, which could naturally exist or be manually crafted.
Revised: These kinds of easy-to-learn representations work as shortcuts, which could naturally exist or be manually crafted.

Original: Since DNNs optimized by gradient descent always find the solution with the minimum norm, shortcuts take precedence over those semantic features during training.
Revised: Since DNNs optimized by gradient descent always find the solution with the minimum norm, shortcuts take precedence over those semantic features during training.

Original: We find that shortcuts can be as small as even a single pixel.
Revised: We find that shortcuts can be as small as even a single pixel.

Original: Thus, we propose One-Pixel Shortcut (OPS), which is an imperceivable and effective data protection method.
Revised: Thus, we propose One-Pixel Shortcut (OPS), which is an imperceivable and effective data protection method.

Original: OPS does not require a generator model and therefore needs very little computational cost and has better transferability between different models.
Revised: OPS does not require a generator model and therefore needs very little computational cost and has better transferability between different models.

Original: Besides, OPS is less sensitive to adversarial training compared to EM
Revised: Besides, OPS is less sensitive to adversarial training compared to EM

Original: ULEs.
Revised: ULEs.

Original: We investigate the effectiveness of OPS and EM under different training strategies.
Revised: We investigate the effectiveness of OPS and EM under different training strategies.

Original: We find that EM and OPS have their respective advantages and disadvantages.
Revised: We find that EM and OPS have their respective advantages and disadvantages.

Original: While EM cannot keep effective under global data augmentations like adversarial training, OPS is sensitive to local data augmentations like Cutout.
Revised: While EM cannot keep effective under global data augmentations like adversarial training, OPS is sensitive to local data augmentations like Cutout.

Original: Based on our investigation, we combine EM and OPS together to craft a kind of stronger unlearnable examples, which can still keep imperceptible but more impervious, and consequently introduce CIFAR-10-S, which can be a new benchmark.
Revised: Based on our investigation, we combine EM and OPS together to craft a kind of stronger unlearnable examples, which can still keep imperceptible but more impervious, and consequently introduce CIFAR-10-S, which can be a new benchmark.

Original: Besides, we have also discussed our method in multi-pixel scenarios.
Revised: Besides, we have also discussed our method in multi-pixel scenarios.

Original: There are still questions that need to be discussed in the future.
Revised: There are still questions that need to be discussed in the future.

Original: Besides shortcuts that are crafted deliberately for the purpose of data protection, there are also shortcuts that naturally exist due to the inevitable bias during data collection.
Revised: Besides shortcuts that are crafted deliberately for the purpose of data protection, there are also shortcuts that naturally exist due to the inevitable bias during data collection.

Original: They can be the crux of network generalization on unseen data.
Revised: They can be the crux of network generalization on unseen data.

Original: How to identify and avoid them ( e.g. , design data-dependent augmentation) is a challenging problem.
Revised: How to identify and avoid them ( e.g. , design data-dependent augmentation) is a challenging problem.

Original: We believe our work will shed light on the important impacts of shortcuts, and provide inspiration to harness them for more practical applications.
Revised: We believe our work will shed light on the important impacts of shortcuts, and provide inspiration to harness them for more practical applications.

Original: R
Revised: R

Original: A F URTHER INVESTIGATION ON SHORTCUT GENERATION
Revised: A F URTHER INVESTIGATION ON SHORTCUT GENERATION

Original: In Sec 3.2, we craft OPS by perturbing a fixed position to a fixed color target for each class.
Revised: In Sec 3.2, we craft OPS by perturbing a fixed position to a fixed color target for each class.

Original: What if we only constrain one of these two properties?
Revised: What if we only constrain one of these two properties?

Original: In other words, if perturbing a fixed position to different color targets, or perturbing different positions to a fixed color target, will a strong shortcut be created?
Revised: In other words, if perturbing a fixed position to different color targets, or perturbing different positions to a fixed color target, will a strong shortcut be created?

Original: With curiosity, we further explore these two properties individually.
Revised: With curiosity, we further explore these two properties individually.

Original: We use OPS-Position and OPS-Color to represent the former and the latter setting.
Revised: We use OPS-Position and OPS-Color to represent the former and the latter setting.

Original: For OPS-Position, after finding the optimal position for each class via Algorithm 1, we perturb pixels at this position to random colors.
Revised: For OPS-Position, after finding the optimal position for each class via Algorithm 1, we perturb pixels at this position to random colors.

Original: For OPS-Color, we assign each class a predefined color (we add two colors since there are only 8 boundary colors) and perturb pixels at different positions to this color.
Revised: For OPS-Color, we assign each class a predefined color (we add two colors since there are only 8 boundary colors) and perturb pixels at different positions to this color.

Original: Note that there might be many positions where the original color is already the same as the predefined color.
Revised: Note that there might be many positions where the original color is already the same as the predefined color.

Original: We perturb the position where the original color is the furthest from the predefined color (measured by the l 2 norm) for each sample.
Revised: We perturb the position where the original color is the furthest from the predefined color (measured by the l 2 norm) for each sample.

Original: As shown in Table A, the results indicate that if we only constrain one property, the resulting shortcuts will be harder to be captured.
Revised: As shown in Table A, the results indicate that if we only constrain one property, the resulting shortcuts will be harder to be captured.

Original: Besides, random perturbations at a fixed position can serve as a stronger shortcut than a fixed color at different positions.
Revised: Besides, random perturbations at a fixed position can serve as a stronger shortcut than a fixed color at different positions.

Original: [Table]
Revised: [Table]

Revised: B B ASELINE OF A RANDOM PIXEL

Revised: For the purpose of evaluating our searching algorithm proposed in Sec.

Revised: 3.2, we compare it with the baseline, which randomly chooses a fixed position and fixed color (denoted as RandPix) for each class to craft the shortcut training set, and report the clean accuracy of 10 runs in Table B.

Revised: Since 15.56% is out of the three standard deviation intervals, our OPS design is validated to be meaningful.

Revised: randomly chosen perturbed position and target color.

Revised: Training Data

Revised: Clean

Revised: OPS

Revised: RandPix

Revised: Clean Test Acc.

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: C S EARCHING OPS ON A SMALL PROPORTION OF DATA

Revised: In Algorithm 1, we search the entire dataset to generate OPS.

Revised: What if the data is gradually added to the dataset?

Revised: In other words, will the perturbed position and target color of OPS that is found on a small proportion of data serve as a strong shortcut on the entire dataset?

Revised: Moreover, users might want to add their own noise before uploading their data to the dataset.

Revised: From a practical standpoint, we further explore the potential of OPS by designing the following experiments.

Revised: Firstly, we take out different proportions of training data from the CIFAR-10 training set to search for the perturbed position and target color.

Revised: Then we perturbed the whole training set based on the results we get from the selected small proportion of data, and train DNNs on the perturbed data.

Revised: Secondly, considering the scenario that users add their own noise before uploading data to the dataset, based on the setting of the first experiment, we additionally inject random noise sampled from the uniform distribution [ − ϵ, ϵ ]

Revised: (here we set ϵ = 8 / 255 , following the common setting of adversarial learning) to the unselected data.

Revised: The results are shown in Table C, where we denote the first setting as Clean-Upload and the second setting as Noisy-Upload .

Revised: Even if we only use 1% of the whole dataset to search the perturbed position and target color, OPS is still able to degrade the clean accuracy of the trained DNN to 32 .

Revised: Besides, additional noise does not degrade the efficiency of OPS.

Revised: To some degree, the experiments prove the generalization capability and robustness of OPS from a practical standpoint.

Revised: [Table]

Original: B E XPLORATIONS ON L 2 ADVERSARIAL TRAINING
Revised: D E XPLORATIONS ON L 2 ADVERSARIAL TRAINING

Original: In Sec 4.3, we have studied the effectiveness of different types of shortcuts under different types of training strategies, including l ∞ adversarial training and various data augmentations.
Revised: In Sec 4.3, we have studied the effectiveness of different types of shortcuts under different types of training strategies, including l ∞ adversarial training and various data augmentations.

Original: For a more comprehensive exploration, we additionally evaluate them on l 2 adversarial training.
Revised: For a more comprehensive exploration, we additionally evaluate them on l 2 adversarial training.

Original: Besides the commonly used setting ( ϵ = 0 . 5 ), we also try larger attack budgets, as shown in Table B.
Revised: Besides the commonly used setting ( ϵ = 0 . 5 ), we also try larger attack budgets, as shown in Table D.

Original: Compared to l ∞ , l 2 is proved to be a more efficient AT manner for alleviating OPS.
Revised: Compared to l ∞ , l 2 is proved to be a more efficient AT manner for alleviating OPS.

Original: Nevertheless, when the perturbation budget is not large enough, the network will still be affected by the shortcuts.
Revised: Nevertheless, when the perturbation budget is not large enough, the network will still be affected by the shortcuts.

Original: If we further enlarge ϵ , it will hurt the clean test accuracy to a greater extent.
Revised: If we further enlarge ϵ , it will hurt the clean test accuracy to a greater extent.

Original: As a local shortcut, OPS displays stronger inertia under l 2 AT, which can be viewed as a global data augmentation.
Revised: As a local shortcut, OPS displays stronger inertia under l 2 AT, which can be viewed as a global data augmentation.

Original: As for
Revised: As for

Original: EM, although generated by l ∞ perturbations, is still sensitive to l 2 AT.
Revised: EM, although generated by l ∞ perturbations, is still sensitive to l 2 AT.

Original: These results conform to our discussion about the properties of local&global shortcuts and local&global data augmentations in
Revised: These results conform to our discussion about the properties of local&global shortcuts and local&global data augmentations in

Original: Sec 4.3.
Revised: Sec 4.3.

Original: [Table]
Revised: [Table]