Disrupting Model Training with Adversarial Shortcuts
Abstract
When data is publicly released for human consumption, it is unclear how to prevent its unauthorized usage for machine learning purposes.
Successful model training may be preventable with carefully designed dataset modiﬁcations and we present a proof-of-concept approach for the image classiﬁcation setting.
We propose methods based on the notion of adversarial shortcuts , which encourage models to rely on non-robust signals rather than semantic features and our experiments demonstrate that these measures successfully prevent deep learning models from achieving high accuracy on real, unmodiﬁed data examples.
1. Introduction
Datasets are publicly released with a diverse set of use cases (e.g., posting images for friends, promoting photography work), but not all stakeholders will consent to the data’s usage for machine learning (ML).
Different parties may desire that their copyright be respected, or they may wish to avoid potentially harmful uses such as deepfakes, facial recognition systems, or other biometric models.
To help manage such situations, we consider the problem of modifying datasets to ensure that they are unusable for
ML purposes.
By developing a disruptive modiﬁcation that preserves the data’s original semantics, we hope to provide an orthogonal approach to traditional privacy preservation avenues such as anti-scraping technology and legal agreements, which in recent years have proved insufﬁcient at protecting user data (Hill et al., 2020).
Because this is a broad and challenging problem, our initial focus is the canonical setting of multi-class image classiﬁcation.
Our goal is to modify a clean dataset so that ML models, and primarily deep neural networks (DNNs), achieve and 2021 by the author(s). [Figure] high training accuracy while failing to generalize to unmodiﬁed test examples.
As a mechanism for such modiﬁcations, we explore adversarial shortcuts , a method that encourages
DNNs to lazily rely on spurious signals rather than robust, semantic features.
While recent work focuses on the perils of shortcuts in deep learning (DeGrave et al., 2021), we identify a potential use-case as a security measure.
Our contributions in this work are as follows:
• We introduce the notion of adversarial shortcuts and propose three dataset modiﬁcation techniques to prevent DNNs from learning useful classiﬁcation functions.
• We evaluate each technique on the popular CIFAR-
(Krizhevsky et al.)
and ImageNet (Russakovsky et al., 2015) datasets and ﬁnd that the proposed techniques severely limit the test set accuracy of state-of-the-art models.
We also verify that our techniques are robust to certain simple countermeasures.
• We compare our approach to a concurrent proposal (Fowl et al., 2021a) and show that our simpler approach based on adversarial shortcuts proves more effective at disrupting model training.
2. Related Work
Our work focuses on making datasets unusable for training ML models.
Concurrent work that considers the same problem (Fowl et al., 2021a) proposed a poisoning approach based on second-order gradients.
Similarly, Shumailov et al.
(2021) proposed a method to disrupt training with a data ordering attack, but our setting is more challenging in that we do not assume control over model optimization.
Other closely related research directions are trojaning and backdooring (Schwarzschild et al., 2020; Goldblum et al., 2020), where the adversary can inﬂuence the inputs at both training and inference time, and in some cases the training procedure.
Data poisoning (Biggio et al., 2011; 2012) is another direction where subtle changes to the classiﬁcation’s decision boundary are induced by modifying a few training examples.
Targeted data poisoning attacks focus on inﬂuencing model behavior on speciﬁc inference examples, either in a transfer learning setting (Shafahi et al., 2018;
Zhu et al., 2019; Aghakhani et al., 2020) or when training from scratch (Mu˜noz-Gonz´alez et al., 2017; 2019; Huang et al., 2020a;
Geiping et al., 2020; Huang et al., 2021; Fowl et al., 2021b)
We explore shortcuts as a mechanism to disrupt model training, an idea with some precedent in prior work.
Research on the learning tendencies of DNNs has found that conventionally trained models often rely on non-robust, localized, texture-based features (Zhang et al., 2016; Jo & Bengio, 2017;
Madry et al., 2017; Geirhos et al., 2018; Ilyas et al., 2019), and, when available, confounders or shortcuts (Zech et al., 2018;
Badgeley et al., 2019; DeGrave et al., 2021).
Rather than attributing model failures to naturally occurring shortcuts, we purposely introduce shortcuts to discourage a model’s reliance on robust, semantic features.
Finally, our aim is the reverse of instance hiding (Huang et al., 2020b; Carlini et al., 2020), which tries to create a dataset that is useful for ML but uninterpretable to humans, and our approach is similar in spirit to image watermarking (Podilchuk & Delp, 2001; Singh & Chadha, 2013; Dekel et al., 2017), which aims to prevent unauthorized usage of publicly released image data.
Setup and Goals
Assume that we have a dataset D train = { ( x i , y i ) } ni = where x i ∈ R w × h × c are RGB images and y i ∈ { 1 , ..., K } are the corresponding labels for a classiﬁcation task.
D train is assumed to be drawn from a data generating distribution
D , and as in standard supervised learning, we assume that models trained on D train are used to classify test samples from D test , which is drawn independently from D .
Rather than releasing D train directly, which can be used to train a model that achieves high accuracy on D test , our goal is to create a modiﬁed dataset D (cid:48) train = { ( x (cid:48)
i , y i ) } ni =1 with the following properties:
• Semantics in D (cid:48) train are preserved.
The modiﬁed images x (cid:48)
i should differ from the original images x i minimally, ideally being visually indistinguishable, but at least retaining the important semantics (objects, shapes, colors, etc).
The labels y
i are unmodiﬁed, and we assume that a party obtaining our modiﬁed dataset may reconstruct y i even if labels are not provided.
• Models trained on D (cid:48) train achieve low accuracy on D test .
When DNNs are trained to predict the labels y i given images x (cid:48)
i from the modiﬁed dataset, the models should be unable to generalize to unmodiﬁed examples, ensuring low accuracy on the test dataset D test .
4. Protective Dataset Modiﬁcations
In this section, we introduce dataset modiﬁcations that encourage DNNs to rely on spurious signals rather than robust, generalizable features.
We propose three approaches: a sparse pixel-based pattern, a visible watermark, and a brightness modulation.
All three generate modiﬁcations that are unique to each class k ∈ { 1 , . .
, K } , creating a shortcut that the DNN can use to quickly achieve high accuracy on the training data while failing to generalize to unmodiﬁed examples.
We refer to such modiﬁcations as adversarial shortcuts , and each technique is tuneable, allowing us to control the tradeoff between disrupting training and preserving visual features in the data.
[Figure]
The image depicted here is available at https://www.flickr.
com/photos/volvob12b/9797687423 , was accessed on
June 3, 2021, and is distributed in the public domain.
This would have class index 263 for “Pembroke, Pembroke Welsh corgi.”
4.1. Sparse pixel-based patterns
The ﬁrst approach we introduce is a sparse, pixel-based modiﬁcation.
We generate random perturbation masks ∆ k ∈ { 0 , 1 } w × h × c for each class k ∈ { 1 , . .
, K } with entries determined as follows: for each value, we sample δ ∼ N ( µ, σ ) and set the value to one if δ exceeds the middle of the pixel brightness range (e.g., 0.5 with 0-1 normalization), and zero otherwise.
In practice, we ﬁx σ = 0 . 2 and experiment with different µ values.
With the masks ﬁxed, we then modify images in the pixels indicated by their corresponding perturbation masks.
Assuming that the maximum pixel value in the dataset is given by x max ∈ R , we generate the modiﬁed image x (cid:48)
i with label y i
=
k using the formula
[Equation]
An example of this perturbation is shown in Figure 2, where a small portion of pixels are set to the maximum brightness.
4.2. Visible watermarks
The next approach we introduce is a visible, class-speciﬁc watermark.
If the watermark is prominent and easy to detect, a DNN can use it as a shortcut to achieve high accuracy without relying on robust features.
To generate shapes with a sufﬁcient degree of variation, which is known to make watermarks more difﬁcult to remove (Dekel et al., 2017), we create watermarks by enumerating the class indices using digits from the MNIST dataset (LeCun, 1998).
For example, in CIFAR-10 (Krizhevsky et al.)
the “airplane” class has index 0, so we create a watermark for each airplane example by randomly sampling a zero from the
MNIST dataset.
For ImageNet, which has 1000 classes, the watermarks require up to three randomly selected digits.
The watermark generation process for each class k ∈ { 1 , . . .
, K } can be understood as sampling a binary image M ∈ { 0 , 1 } w × h × c from a random variable M ( k ) , which we then blend with the original image x
i using a parameter α ∈ [0 , 1] as follows:
[Equation]
The blending parameter α controls how visible the watermark is, with α =
having no effect and α = 1 overlaying the watermark on the original image.
An example with α = 0 .
5 is shown in Figure 2, with index 263 for the “Pembroke, Pembroke Welsh corgi” class.
4.3. Brightness modulation
While the two previous approaches provide shortcuts that can successfully disrupt model training, they may prove easy to remove with basic countermeasures.
Our next approach is designed to be more difﬁcult to circumvent.
Rather than creating a localized, visually distinguishable perturbation, we now modify images using a randomized brightness modulation that either brightens or darkens pixels identically for images in each class.
The brightness modulation for each class is generated as follows.
At the start, we randomly sample a location in the image that serves as the center of a square; we then decide, with equal probability, whether to darken or brighten the corresponding pixels.
Given a parameter γ ∈ [0 . 5 , 1] , we darken pixels by multiplying them by γ or brighten them by multiplying by 2 − γ .
We perform T iterations of this process with T distinct squares, which can and do overlap, resulting in a checkerboard-type pattern.
This process is equivalent to sampling a class-speciﬁc mask
B k ∈ R w × h × c , where an image x i with class y i
= k is modiﬁed using the following formula:
[Equation]
An example of this modiﬁcation is shown in Figure 2 with the parameter γ = 0 .
9 and T = 600 iterations.
For all experiments with ImageNet we set T = 600 , and for CIFAR10 we use T = 32 .
5. Evaluation
In this section, we evaluate our proposed techniques for disrupting model training; see Appendix A for more details on the setup.
Although we do not reach state-of-the-art training accuracy on CIFAR-10 and ImageNet, either due to computational constraints or insufﬁcient hyperparameter tuning, we ensure a fair comparison by using identical training procedures across all experiments.
5.1. Training on modiﬁed CIFAR-
We ﬁrst test our dataset modiﬁcations on CIFAR-10.
Figure 3 summarizes the results from training a ResNet18 architecture with various dataset modiﬁcations.
Figures 6, 7, and 8 provide more ablations and details, including different model architectures and more parameter settings for each adversarial shortcut.
The best achievable validation accuracy after 50 epochs with the unmodiﬁed version of CIFARis above 70% accuracy, while all of our modiﬁcations, even at the weakest settings we tested, have a signiﬁcant impact on model performance.
Relatively small pixel-based perturbations are enough to nearly halve the accuracy: with µ = 0 .
01 , CIFAR-10 classiﬁers achieve at best no more than 40% accuracy.
This setting corresponds to modifying only 22 out of 3,072 pix- U n m o d i f i e d d a t a s e t
F o w l e t a l .,
P i x e l - b a s e d 0 .
P i x e l - b a s e d 0 . V i s u a l w a t e r m a r k 0 . V i s u a l w a t e r m a r k 0 . B r
i g h t n e ss m o d ..
B r
i g h t n e ss m o d .. 5 0.
B e s t
A cc .
O n C l e a n T e s t S e t els, on average.
Similarly, visible watermark perturbations with a blend factor of α = 0 .
5 and brightness modulations with parameter γ = 0 .
9 succeed in reducing the validation accuracy to less than 40%.
Fowl et al.
(2021a) shared a version of CIFAR-10 protected with their proposed approach, which we compared with our methods.
1 While the validation accuracy does not reach 70%, as with training on the unprotected version of the dataset, models trained on the dataset with Fowl et al.
(2021a) protections manage to achieve up to 60% accuracy, which is signiﬁcantly higher than our methods (also see Figure 9).
We also tested the stability of our proposed modiﬁcations for disrupting training when certain countermeasures are in place.
For this purpose, we considered two categories of countermeasures: aggressive training set augmentations and the addition of Gaussian noise.
Results for the pixelbased approach with these countermeasures are shown in
Figures 11, 12 and 13.
Our ﬁndings generalize, and the best achievable accuracy across the same set of hyperparameters and random seeds remains the same.
However, for the brightness modulation method with γ = 0 . 9 , aggressive augmentations are effective at undoing the modiﬁcations and allowing effective training (see Figure 13).
We suggest using a stronger setting of γ = 0 .
70 that makes the brightness modulation more visible.
5.2. Training on modiﬁed ImageNet
Next, we perform experiments on ImageNet.
Although we do not have the computational resources to train with a va- riety of hyperparameter and random seed choices, several takeaways are apparent from Figure 4 and additional results in Appendix A (Figure 5).
First, sparse pixel-based pattern protections and visible watermark protections remain effective.
In both cases, the best achievable validation accuracy is less than 30%, whereas training on the unprotected version of ImageNet easily achieves more than 50% accuracy with the same setup.
This is again achievable with fairly minor modiﬁcations, such as pixel-based with µ = 0 .
01 and visible watermarking with α = 0 .
Furthermore, the plots of accuracy on the clean validation set and the protected training set in Figure 1 reveal an interesting dynamic.
While the model can ﬁt the training set extremely well, achieving up to 90% accuracy, it does not generalize to the validation set.
This suggests that our objective of disrupting training with a non-robust shortcut is successful, and that the models ﬁts the simple class-speciﬁc pattern as opposed to the true semantics.
This divergence in training and validation accuracy, or the rapid increase in the generalization gap, does not manifest when training with the unmodiﬁed ImageNet data (Figure 1).
6. Discussion
Our experiments show that it is possible to disrupt DNN training by modifying datasets with simple patterns, such as our adversarial shortcuts, that discourage models from relying on robust, generalizable features.
These modiﬁcations can reduce model accuracy on clean data while having minimal impact on the image semantics.
Our work focuses on the narrow setting of multi-class image classiﬁcation, but there is great potential for future work that considers more effective dataset modiﬁcations, attempts to undo protective modiﬁcations, and develops new approaches for different
ML tasks, e.g., preventing the unauthorized development of deepfakes or facial recognition systems. Acknowledgements
The authors would like to thank Gabriel Ilharco, Kaiming Cheng and Samuel Ainsworth for helpful discussions on the subject matter and for their thoughtful feedback. This work was supported in part by the University of Washington Tech Policy Lab, which receives support from: the
William and Flora Hewlett Foundation, the John D. and Catherine T. MacArthur Foundation, Microsoft, the Pierre and Pamela Omidyar Fund at the Silicon Valley Community Foundation; it was also supported by the US National Science Foundation (Award 156525).
A. Experimental Setup
We experiment with two standard image classiﬁcation datasets:
CIFAR-10 (Krizhevsky et al.) and ImageNet (Deng et al., 2009).
In both cases, we apply our protective modiﬁcations to the training set and keep the validation set intact.
By measuring accuracy on the validation set, we can observe how well the trained models solve their intended classiﬁcation task.
If our protections are successful, the best achievable accuracy should be considerably lower than when training on an unprotected dataset.
For the experiments with CIFAR- we run training forepochs at a batch size of 1024 and vary the learning rate, the model architecture and the random seed.
Speciﬁcally, we experiment with learning rates 0 .
001 , 0 . 0001 ; with the ResNet18 (He et al., 2016), DenseNet201 (Huang et al., 2017), VGG11 (Simonyan & Zisserman, 2014), and
SqueezeNet (Iandola et al., 2016) architectures; and with seeds 3525462, 15254521, 63246662, 32542462.
We then report the best achievable accuracy across all the runs for several different settings of each of our intensity parameters ( µ for pixel-based patterns, α for watermarks, and γ for brightness modulations).
All models are as implemented in torchvision (Paszke et al., 2019) and are trained from scratch.
For ImageNet, we use the training script available at https://github.com/pytorch/examples/ tree/master/imagenet and the ResNet18 (He et al., 2016) architecture.
We only train using the default hyperparameter values due to our limited computational resources.
For testing the stability of our approach with aggressive augmentations, we employ random cropping of 28 byimages, random horizontal ﬂips, random Afﬁne transformations (translation by up to 0.1 and rotation between -30 and 30 degrees), random color jitter (brightness adjustment by up to 0.8 and contrast adjustment between 0.9 and 1.08).
Each augmentation is sampled at random for each new training image and for each epoch during training.
When testing the stability of our approach with Gaussian noise, we add noise with mean 0 . 0 and σ = 0 .
B. Additional Figures
Here, we include ﬁgures that do not ﬁt in the page limit.