What Can We Learn from Unlearnable Datasets?
Authors: Pedro Sandoval-Segura
Vasu Singla
Jonas Geiping
Micah Goldblum
Tom Goldstein
Venue: NeurIPS 2023 poster (2023)
Keywords: data poisoning
poisons
unlearnable dataset
data protection
imperceptible perturbations
adversarial machine learning

Abstract: In an era of widespread web scraping, unlearnable dataset methods have the potential to protect data privacy by preventing deep neural networks from generalizing. But in addition to a number of practical limitations that make their use unlikely, we make a number of findings that call into question their ability to safeguard data. First, it is widely believed that neural networks trained on unlearnable datasets only learn shortcuts, simpler rules that are not useful for generalization. In contrast, we find that networks actually can learn useful features that can be reweighed for high test performance, suggesting that image protection is not assured. Unlearnable datasets are also believed to induce learning shortcuts through linear separability of added perturbations. We provide a counterexample, demonstrating that linear separability of perturbations is not a necessary condition. To emphasize why linearly separable perturbations should not be relied upon, we propose an orthogonal projection attack which allows learning from unlearnable datasets published in ICML 2021 and ICLR 2023. Our proposed attack is significantly less complex than recently proposed techniques.

What Can We Learn from Unlearnable Datasets?
Pedro Sandoval-Segura'
Vasu Singla' Jonas Geiping'
Micah Goldblum
Tom Goldstein'
University of Maryland
New York University
(psando, vsingla, jgeiping, (omm) edu goldblumenyu.edi
Abstract
In an era of widespread web scraping, unlearnable dataset methods have the poten- tial to protect data privacy by preventing deep neural networks from generalizing. But in addition to a number of practical limitations that make their use unlikely, make number of findings that call into question their ability to safeguard data. First, is widely believed that neural networks trained on unlearnable datasets only learn shorteuts, simpler rules that find that networks actually can learn useful features that can reweighed for high test performance, suggesting that image protection is not assured. Unlearnable datasets are also believed to induce learning shortcuts through linear separability of added perturbations. We provide a counterexample, demonstrating that linear separability of perturbations is not necessary condition. To emphasize why lin- early separable perturbations should not be relied upon, we propose orthogonal projection attack which allows learning from unlearnable datasets published in ICML 2021 and ICLR 2023. Our proposed attack significantly less complex than recently proposed techniques)
not useful for generalization. In contrast, we
Introduction
Deep learning is fueled by an abundance of data, the collection of which is largely unregulated [I7 20 [4 32]. Images of human faces (9) artwork [T], and text 24] are increasingly scraped at scale without consent. In an attempt to prevent the unauthorized use of data, unlearnable dataset methods make small perturbations to data that deep neural networks (DNNs) trained on the modified data result in poor test accuracy [IO. 7 6 36 27). The idea is that if generalization performance is harmed by incorporating the modified, "unlearnable" data, third parties will be disincentivized from scraping it. Unleamable dataset methods answer the following question: How should one imperceptibly modify the clean training set cause the largest generalization gap?
In this paper, we analyze properties of unlearnable dataset methods in order to assess their future viability and security promises. Currently, unlearnable dataset methods unlikely to be used for safeguarding public data because they require perturbing a large portion (e.g. more than 50%) of the training set [10 27 and adversarial training is principled attack (3I) Additionally, published data is immutable and must withstand current and future attacks 23 While there are number of
reasons why these unlearnable dataset methods are unlikely be employed, our results shed light on privacy implications and challenge common hypotheses about how they work. We make several findings by analyzing number of unlearnable datasets developed from diverse objectives and theory. In particular,
We demonstrate that, in many cases, neural networks can learn generalizable features from unlearnable datasets. Our results imply that while resulting test accuracy may be low, DNNs may still learn useful features from protected data.
'Code available at
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Can be used learn features which generalize to...
Clean Dataset
Perturbation
Unlearnable Dataset
May be recovered from...
Need not Linearly Seperable!
Figure I: We study unlearnable datasets constructed from additive perturbations. Although unlearn- able datasets are created to prevent the exploitation of the clean dataset, we show that unlearnable datasets can be used to learn features which generalize to clean test data. We demonstrate that unlearnable datasets can created without linearly separable perturbations. And we develop a novel method for recovering class-wise, linearly separable perturbations from unlearnable datasets.
We challenge the common belief that unlearnable datasets work due to linearly separable perturbations. We construct counterexample, suggesting that there is room for new unlearnable dataset methods that evade common attacks.
Inspired by our results on linear separability of perturbations, we present new Orthogonal Projection attack which allows learning from unlcarnable datasets perturbed by class- wise, linearly separable perturbations. Our attack demonstrates that class-wise applied perturbations do not protect data from being learned. Our attack is often more effective than adversarial training, fraction of the computational cost.
Terminology Data poisoning methods that perturb the entire training dataset, and which we refer
to as "unlearnable datasets" or simply "poisons", are also known as availability attacks 7 (6). generalization attacks (37), delusive attacks [31), or simply unlearnable examples [35]. Throughout this work, unlearnable datasets are considered defenses, given that their primary use case is to prevent the exploitation of data. Aiming to learn from unlearnable datasets is considered an attack.
2 Related Work
Unlearnable Datasets One of the earliest instances of a data poisoning attack intended to reduce overall test performance is from [2]. who optimize single sample that corrupts SVM training. Poisoning a convex model like SVM or logistic regression can be solved exactly, but poisoning
deep neural networks is more challenging. A number of approaches have been proposed, validated
primarily by empirical evidence due to the complexity of the objective (See Eq. [ and 2}. [5] propose using an auto-encoder-liie network generate perturbations against victim classifier, (37 use
NTKs to understand network predictions and optimize perturbations to produce misclassifications,
36 generate linearly separable perturbations, [10 28 optimize error-minimizing noise whereas [
optimize maximiiax noise, and [27] generate autoregressive patterns, among other methods. The diversity of approaches motivates us understand whether these unlearnable datasets share any properties in common, as we explore in Section 4.3
Privacy Vulnerabilities In the context of image classification, unlearnable datasets said to protect privacy by ensuring the modified data is not used or included as part of a larger dataset. That is, training on only poisoned data should prevent test generalization and training on dataset consisting of clean and poisoned data should be no better than training just on the clean portion. poisoned training data is rendered useless for test set generalization, the data will not be used. In this way, only the initial owner can train on the original, unperturbed data and achieve high generalization performance. In recent work, [9] introduce the ability to "lock" and "unlock" training data by leveraging a class-wise perturbation, removable only by someone with knowledge of the exact perturbation. Unfortunately, as we empirically demonstrate in Section 4.4 class-wise perturbations can give a false sense of security. An argument against the use of poisoning for data privacy is the unavoidable reality that published data immutable, and thus must withstand current and future methods which may exploit the data
(23]. While it remains possible to recover relatively high test

---

accuracy from unlearnable datasets, our experiments also suggest that there is not yet a way to restore test accuracy to levels seen during clean training. Unlearnable datasets are known to vulnerable to adversarial training I0 31 By formalizing unlearnable datasets as finding the worst-case training data within oc- ac-Wasserstein ball, [1] find that adversarial training [6] is a principled defense and can recover high degree of accuracy by preventing models from relying on non-robust features.
(28) challenge this assertion and optimize perturbations designed to degrade model performance under adversarial training. In Section 4.4 we show that adversarial training remains strong defense against unlearnable datasets. In fact, we find that hyperparameter changes to the adversarial training recipe allows learning from Robust Unlearnable Examples [28). demonstrating the brittleness of unlearnable datasets.
How Unlearnable Datasets Work There are many explanations for how unlearnable datasets prevent networks from test set generalization: aroo-minimizing perturbations cause overfitting (10). error-maximizing noise stimulates learning non-robust features [7]. convolutional layers are receptive to autoregressive patterns [27]. and more. There are a variety of explanations because different methods arose from different optimization objectives and theory. But the leading explanation comes from Yu et al. [36], who find near perfect linear separability of perturbations for all the unlearnable datasets they consider. They explain that unlearnable datasets cause learning shortcuts due to linear separability of perturbations. In Section 4.3 we find counterexample, demonstrating that while linear separability of perturbations may certainly be a property that helps unlearnable datasets function, the property is not necessary.
3 Problem Setting
We consider the problem of creating clean-label unlearnable dataset in the context of a K K-way image classification task, following [10]. We denote the clean training and test datasets as Dtrain and Diest, respectively.
Suppose there are n samples in clean training set, i.e. Dirain are the inputs and yi E (1,..,, K} are the labels. We consider a unlearnable dataset, denoted Derain = {(11,, ) where x² I + 8; is the perturbed or poisoned version of the example I; Dirain and where & E A C Rd is the perturbation. The set of allowable perturbations, A, is typically an balll with radius e where is small enough that is imperceptible.
[)) where æ; E Rd
Unlearnable datasets are created by applying a perturbation to a clean image in either a class-wise sample-wise manner. When perturbation is applied class-wise, every sample of given class is perturbed in the same way. That is, x = x, + 6g. and 6y, E Ac = (6,,..., 5x). When samples are perturbed in sample-wise manner, every sample has a unique perturbation.
All unlearnable dataset methods aim to solve the following bi-level maximization:
max E ar. Dreaa [fff()) y: 0(8)]
(1)
AEA
0(8) arg min E (r..g [C(
82), yes 0c)
(2)
Eq. 2 describes the process of training model on unlearnable data, where 0 denotes the model parameters. Eq. [ states that the unlearnable data should be chosen so that the trained network has high test loss, and thus fails to generalize to the test set.
4 Experiments
4.1 Datasets and Training Settings
For our experiments, we make use of open-source unlearnable datasets: From Unlearnable Examples [00, we use their sample-wise and class-wise poisons. From Adversarial Poisoning 7 we use their targeted PGD attack poison. From Autoregressive Poisoning (AR) [27). Neural Tangent
Generalization Attacks (NTGA) (37, Robust Unlearnable Examples [28). and Linearly Separable
3

---

Table I: Generalizable features be learned from unlearnable datasets. While published unlearnable datasets cause DNNs to train to low test accuracy, the features learned by DNNs can be reweighted using DFR to high test accuracy in many cases. For each unlearnable dataset, we report test accuracy and test loss for the best performing checkpoint after DFR. In gray, we indicate test accuracy improvemenn/deterrioaattoo over DFR on randomly initialized RN-18. DFR uses 5, 000 clean samples for finetuning. For the majority of unlearnable datasets, peak performance seems to occur early training. High DFR Test Accuracy, and Low DFR Test Loss, indicates that useful features are learned during training.
TRAINING DATA
MAX DER TEST ACCURACY
MIN DFR TEST Loss
NONE
35.97
2.379
UNLEARNABLE EXAMPLES [T
39.56 (+3.59)
1.798
ADVERSARIAL POISONING
69.99 (+34.02)
1.036
(f2) [27
58.73 (+22.76)
1.531
NTGA Ki
57.12 (+21.15)
1.391
ROBUST UNLEARNABLE
41.02 (+5.05)
1.790
LSP Rd
43.31 (+7.34)
1.675
OPS+EM 351
38.98 (43.01)
1.869
9 OPS
B5
47.70 (+11.73)
1.697
о UNLEARNABLE EXAMPLES 10
34.09 (-1.88)
2.037
o REGIONS-4 26
48.09 (+12.12)
1.706
RANDOM NOISE
66.79 (+30.82)
1.352
their main poison. For One-pixel Shortcut 335 we their OPS and CIFAR-10-S poisons, but we refer to their CIFAR-10-S poison as OPS+EM for ease of attribution. We also include two class-wise Regions-4 and Random Noise poisons which contain randomly sampled perturbations, following [26]. To poison a K-class dataset in a class-wise manner, need K perturbation vectors. For Regions-4, each perturbation independently created by sampling 4 vectors of size 3 from Bernoulli distribution. Each vector is then scaled to lie in the range |- 255 255 Finally, each of the vectors are repeated along height and width dimensions to create patches of size 16 x 16, which are then arranged side by side to achieve shape of 32 x 32, For Random Noise perturbations, we sample an i.i.d. vector, one entry per pixel, from Bernoulli and scale perturbations to fit the imperceptibility constraint. See Appendix A.5 for image samples for all unlearnable datasets we consider.
Perturbations (LSP) 36
Results in this section are for CIFAR-I0 10 113, and additional results for SVHN [8], CIFAR- 100 [13] and ImageNet (25) Subset are in Appendix A..2. and A.4.2 Unlearnable datasets constructed with class-wise perturbations are prefixed with o, otherwise the perturbations were added sample-wise.
For the imperceptibilityy constraint, the AR (fz) dataset contains perturbations 8 of size ||0|| S
LSP has perturbations of size ||ō|| S d'vd, where is the image dimension and 255 OPS and OPS+FM contain unbounded perturbations. All other datasets contain perturbations of size || < 255
We primarily use the ResNet-18 (RN-18) S architecture for training on the above datasets. We include results for additional network architectures including VGG-16 29), GoogleNet [30], and ViT
4 in Appendix A.2.1 and 1.4.1 Training hyperparameters can be found in Appendix A.1
4.2 DNNs Can Learn Useful Features From Unlearnable Datasets
Unlearnable datasets are meant to be unlearnable; models trained on unlearnable datasets should not be capable generalizing to clean test data. Presumably, the modified, unlearnable dataset protects the clean version of the dataset from unauthorized use in this fashion. But by reweighting deep features, we find that DNNs are still able to recover some generalizable features from unlearnable datasets.
The technique of training new linear classification layer on top of fixed feature extractor known as Deep Feature Reweighting (DFR) [12 and a method for learning from data with spurious correlations, where predictive features in the train distribution not present in the test
DFR Method

---

100
100
90
(8)
Adhennnal Poianig Advaanal
90
(b)
Bansom
CIFAR- Test Accuricy
80
Randen
CIFAR-11 10 Test Accuracy
50
60
60
50
50
30
30
20
20
10
10
0
10
20
30
40
50
10
20
30
10
50
60
Epoch
Epoch
100
100
(c)
(d)
90
90
CIFAR- 10 Test Accumecy
80
Mandion
CIFAR-10 10 Test Accuracy
Rardem
70
60
60
50
50
30
30
20
20
10
10
01
0
20
20
30
40
50
60
10
20
50
40
50
60
Epoch
Epoch
Figure 2: Features learned during poison training can be reweighed for high test accuracy. (a) When training on Adversarial Poisoning, RN-18 test accuracy (solid line) drops below random chance test accuracy. For the dashed line, each data point at epoch i represents the final test accuracy obtained after DFR using the poisoned checkpoint from epoch of standard training. Using DFR, the final RN-18 poisoned checkpoint can achieve nearly 70% test accuracy (dashed line). Using DFR on randomly initialized RN-18 only achieves 40% test accuracy (dotted line). (b-d) For different unlearnable datasets, at different points during poison training (marked by star), DFR test accuracy more than 20% above the accuracy of randomly initialized feature extractor, suggesting that features learned from unlearnable data are relevant for clean test set generalization.
distribution. We borrow the DFR method to better understand the utility of features learned during poison training. The higher the test accuracy after DFR, the more likely is that the model has learned features present in the original clean data. DFR works well on poison-trained networks, this would suggest that even poisoned weights contain semblance of relevant information for generalization.
To evaluate the extent that generalizable features are learned, we start by saving network weights at every epoch of poison training. In the context of image classification, network weights consist of a feature extractor followed by a fully-connected classification layer mapping feature vectors to class logits. Next, for each checkpoint, training samples (10% of the original training set) to train a new classification head (and keep the feature extractor weights fixed). Finally, we plot test accuracy in Figure and evaluate the maximum CIFAR-10 10 test accuracy achieved through DFR for each dataset and report results in Table | As baseline, we train classification head randomly initialized RN- 18 feature extractor and find that these random features can be reweighted to achieve 35.97% test accuracy.
utilize random subset of 5, 000 clean CIFAR-10 10
DER Results We find that, to different extents, DNNs do learn generalizable features from unlearn- able datasets. Surprisingly, on Adversarial Poisoning and NTGA, RN-18 features actually improve throughout training, as we show Figure 2 (a) and (d), despite low test accuracy during poison training (solid lines in Figure (a) and (d)). On AR and o Random Noise. test accuracy peaks early training before dropping to random chance accuracy (solid lines in Figure (b) and (c)). Still, when early checkpoints are used, one can use features from the poison-trained feature extractors
5

---

Table 2: Linearly separable perturbations not necessary to create unlearnable datasets. We train a linear logistic regression model on perturbations and report train accuracy. High train accuracy indicates linear separability of perturbations. Unlike perturbations from other unlearnable datasets, Autoregressive Perturbations (AR) are not linearly separable and are, in fact, less separable than clean CIFAR-10 images.
TRAINING DATA
TRAIN ACCURACY
CLEAN
53.88
UNLEARNABLE EXAMPLES 10
100
ADVERSARIAL POISONING
100
AR (82) 27
39.58
NTGA 37
100
ROBUST UNLEARNABLE 28
100
LSP 36
100
OPS+EM 35
100
OPS 35
99.93
UNLEARNABLE EXAMPLES [0]
100
REGIONS-4 (26
100
RANDOM NOISE
100
to achieve high accuracy (dashed lines Figure 2 (b) and (c)). Accuracy curves for Adversarial Poisoning unique because images are perturbed with error-maximizing noise, which correspond to actual features models use during classification [T]] In this case, DFR reweighting useful, existing features for classification, leading higher test accuracy. On the other hand, Random Noise and AR poisons do not perturb images with useful features; instead, both perturb with synthetic noise. In these cases, useful features still learned during poison training, but only in the first epochs. As training progresses, the model checkpoints are continually corrupted by synthetic noise features which cannot be useful for classification despite reweighting.
As we show Table T o Unlearnable Examples are very effective at corrupting network weights during training; using DFR on checkpoints from poison training only yields maximum test accuracy of 34.09%. nearly 2% worse than a randomly initialized feature extractor. Six of the eleven unlearn- able datasets consider yield checkpoints which achieve DFR test accuracies that are 10% more higher than a randomly initialized feature extractor. Generally, it is interesting that unlearnable dataset methods with the lowest max DFR test accuracy are those which some capacity, e-g. Unlearnable Examples, Robust Unlearnable Examples, and OPS+EM F.M. Analyzing test loss follows trends from test accuracy: Adversarial Poisoning, AR, NTGA, and Random Noise achieve lowest losses and those poisoned checkpoints also have the highest DFR test accuracy in Table T More interestingly, we find that all poisoned models have lower loss than the randomly initialized model This reinforces our claim that models learned useful features from poisoned data.
mirimmiiiiizzing perturbations in
Overall, our results suggest that network weights during poison training not fully corrupted in many cases. While poison-traine trained networks may be evaluated to have low test accuracy, we show that, for some unlearnable datasets, the networks have learned generalizable features which can be reweighted for high test performance. It is entirely possible that these features, which encode the original data amount to useful image information that the original data owner did not want incorporated into an ML system.
Privacy concerns are often cited primary motivation for unlearnable datasets, yet our results raise concerns about this framing. model trained on an apparently unlearnable dataset might have low test error, but this does not imply, as we show, that it does not contain usable information about the original data, and might not keep promises to protect data privacy.
4.3 Linearly Separable Perturbations Are Not Necessary
The work of Yu et al. (36] demonstrated that unlearnable datasets contain linearly separable perturba- tions and suggested that linear separability is necessary property of unlearnable datasets. However,
On 10 class classification task, the expected random chance loss is -In ( 2.302.

---

Algorithm 1 Orthogonal Projection
Input: Unlearnable Dataset, (X, Y) E D train Output: Recovered Dataset, (X,,, Y)
I: while not converged do
2: Sample batch (a, 3) from Dersia
3: W < W - v/ w (W
4: end while
5: Perform QR decomposition on W to obtain Q matrix
6: X, X QQ""X
by finding a counterexample, we confirm that linear separability is not necessary to have an effective unleamnalle dataset.
Following [36], we train linear logistic regression models on image perturbations by subtracting the clean image from the perturbed image. Given the publication of new unlearnable datasets, we add to Yu et al.'s analysis by including AR, Robust Unleurnable Examples, and OPS poisons. It is possible to obtain image perturbations given our access to clean CIFAR-10. 10, but should not be possible in practice if the clean version of the unlearnable dataset is unavailable to the public. We optimize the logistic regression weights (3072 x 10 parameters for CIFAR- 10) using L-BFGS (15) for 500 steps using learning rate of 0.5. For reference, we also train a logistic regression model on clean CIFAR- 10 image pixels and report the train accuracy in the first row of Table 2 For results the linear separability of the perturbed images, see Appendix[A.33
While most current unlearnable datasets contain linearly separable perturbations, as was initially shown by Yu et al. (6), AR perturbations stand out as counterexample in Table After training on AR perturbations, a logistic regression model can only achieve 39.58% train accuracy while all other unlearnable datasets we tested contain perturbations that are almost perfectly linearly separable. Because most unlcarnable dataset methods developed from diverse optimization objectives result in lincarly separable perturbations, we posit that linear separability is an easy solution for optimization of Eq. and to find. For unlearnable datasets that are not optimized, class-wise perturbations are trivially linearly separable, so long as no two class perturbations are the same. Thus, although AR per- turbations are not linearly separable, they remain separable by simple 2-layer CNN [27), suggesting that simple perturbations may be more accurate way of defining effective unlearnable perturbations.
Being the first counterexample of its kind, we believe there may be underexplored unlearnable dataset methods. This finding sheds light on the complexity of unlearnable datasets, whose behavior depends on choices of loss function, optimizer, and even network architecture. An important detail about AR perturbations is that they are not optimized - they are generated [27]. We leave to future work an investigation to why most optimized perturbations in unlearnable datasets to date result in linear separability.
4.4 Orthogonal Projection for Learning From Datasets with Class-wise, Linear Perturbations
In this section, we develop a method to learn from unlearnable datasets. Given our understanding of linear separability from Section |4.3 learning the most predictive features in an unlearnable dataset can amount to learning the perturbation itself. We leverage this intuition in our Orthogonal Projection attack, which works by learning a linear model, then projecting the data to be orthogonal to linear weights. In other words, we remove the most predictive dimensions of the data. For unlearnable datsets perturbed by class-wise, linearly separable perturbations, the most predictive dimensions are easily found by linear model. Attacks for learning from class-wise perturbed unlearnable datasets have included using diffusion models (3), adversarial training [10), and parro-maximizing augmen- tations [21]. In contrast to these techniques, our method is simpler and computationally cheaper.
4.4.1 Orthogonal Projection Method
Unlearnable datasets with class-wise perturbations are so simple that visualizing the average image of a class can expose the class perturbation (see Figure5). In order to adaptively learn these perturbations and remove them from poisoned data, we propose a method that first learns the simple perturbations and then orthogonally projects samples to omit them. Our Orthogonal Projection method is designed
7

---

LSP Perturbation
Batch
(a)
+
Class
Class
Class
OPS Perturbation
OPS Batch
Class
Class
Class
Class
Class
Class
Class
Class 7
Class 8
Class
LSP
(b)
Clean
Figure Our Orthogonal Projection attack learns linearly separable features. In (a), visualize how LSP (36) and OPS (35) unlcarnable data is created by taking clean image and adding perturba- tion. In (b), visualize learned weights (W in Algorithm| [ after training a linear model LSP and OPS images; we also include learned weights after training on clean CIFAR-10 for reference. Learned weights from LSP and OPS unlearnable data nearly match the original perturbation, while learned weights from clean data resemble the corresponding class. In our Orthogonal Projection attack, we project each perturbed image to be orthogonal each these learned weights (Algorithm) Line
exploit class-wise perturbations by design and is meant emphasize why class-wise perturbations should not be relied on for creating unlearnable datasets.
Given unlearnable dataset, train simple logistic regression model poison image pixels, in an attempt to optimize the most predictive linear image features. The result is a learned feature matrix W. which perform QR decomposition such that W QR, where consists of orthonormal columns. Image samples from the unlearnable dataset can be orthogonally projected using the matrix transformation 1 - QQ" Orthogonal projection ensures that the dot product of row of X with every column of Q is zero, so that learned perturbations from the logistic regression model not seen during the final training run on recovered data. The recovered image samples are then written as X, = (1 QQ")) X. Finally, we train the final network on recovered data, X,., A detailed algorithm is shown in Algorithm [ We include additional intuition in Appendix A.4.3
The added computational cost of Orthogonal Projection over standard training is only in training the logistic regression model and projecting the dataset, which can be done once. For CIFAR-10, train the logistic regression model for 20 epochs using SGD with an initial learning rate of 0.1, which decays factor of 10 epochs and We find that training the linear model for longer is often detrimental, potentially because of overfitting.
4.4.2 Orthogonal Projection Results
We compare our attack against fx adversarial training, as previous work has shown that adversarial training principled defense [3T] against unlearnable datasets. For adversarial training, we use a 3-step PGD adversary with a perturbation radius of 255 -orm.. Note that large adversarial training perturbation radii harm clean accuracy B3]. We use SGD with momentum of 0.9 and an initial learning rate of 0.1, which decays by a factor of 10 on epoch 75, 90, and 100.
We find our method is more effective than adversarial training when learning from unlearnable datasets corrupted in class-wise manner. Table 3 our Orthogonal Projection attack leads to greater test accuracy all class-wise poisons prefixed by By visualizing the learned weights from

---

Table 3: Orthogonal Projection can make class-wise unlearnable data learnable. Especially for unlearnable datasets with class-wise, linearly separable perturbations, our Orthogonal Projection attack improves C1 test accuracy over Eo Adversarial Training fraction of the computa- tional cost. We train RN-18 using different kinds of unlearnable training data and, for every attack, we indicate accuracy improvement/éetrrooaiion over standard training (no attack) in gray. For every dataset, we bold the highest test accuracy achieved.
ATTACK
TRAINING DATA
NONE
ADV TRAINING
ORTHO PROJ (OuRS)
CLEAN
94.37
87.16 (-7.21)
90.16 (-4.21)
UNLEARNABLE EXAMPLES
10
24.56
85.64 (+61.08)
65.17 (+40.61)
ADVERSARIAL POISONING 7
7.96
85.32 (+77.36)
14.74 (+6.78)
AR (f2) 27
13.77
84.09 (+70.32)
13.03 (-0.74)
NTGA S7
40.78
84.85 (+44.07)
82.21 (441.43)
ROBUST UNLEARNABLE 28
26.86
87.07 (+60.21)
25.83 (-1.03)
LSP 36
25.75
86.18 (+60.43)
87.99 (462.24)
OPS+EM
20.11
12.22 (-7.89)
39.43 (+19.32)
o OPS [35]
15.35
11.77 (-3.58)
87.94 (472.59)
o UNLEARNABLE EXAMPLES 10
14.49
85.56 (+71.07)
89.98 (+75,49)
REGIONS-4 26
10.32
85.74 (+75.42)
86.87
(476.55)
RANDOM NOISE
10.03
86.36 (+76.33)
90.37 (480.34)
the linear model in Figure 3 we see that the class-wise, linearly separable perturbations are recovered. Projecting orthogonal to these vectors effectively removes those features from the unlearnable dataset, allowing training to proceed without predictive, but semantically meaningless features. It should not be surprising that Orthogonal Projection is also effective against OPS (ICLR 2023), given that all images of a class have the same pixel modified. In Figure 3 we visualize the linear model weights and show that it is able to pinpoint the modified class pixel, whose influence is effectively removed after projection in our method. We visualize additional linear model weights for other datasets in Appendix [A.45 Orthogonal Projection also surprisingly effective against NTGA (ICML 2021), such that an RN- 8 model can achieve 82.21% test accuracy when trained on orthogonally projected NTGA images. For seven of the eleven unlcarnable datasets we consider, Orthogonal Projection can achieve above 80% test accuracy without adversarial training. Unlike the linear model from 4.3 which is trained on only perturbations, the linear model that we train for Orthogonal Projection trains on perturbed images and the resulting data is no longer linearly separable. For this reason, Orthogonal Projection struggles against Adversarial Poisoning, AR, and Unlearnable Examples; the training distribution of the linear model is more complex than just perturbations. Given that OPS+EM is a combination of OPS and class-wist wise error minimizing noise from Unlearnable Examples, both Orthogonal Projection and adversarial training are ineffective. For both OPS and OPS+EM, adversarial training achieves approximately 12% accuracy, a dismal result that is expected given the unbounded OPS perturbations. A limitation of our approach is that, by orthogonally projecting data, we effectively remove K dimensions from the data manifold, where K is the number of classes in dataset. While this may not be a problem for high-resolution images with tens of thousands of dimensions, this detail could impact applicability for low row-rusolution datasets.
Section
The susceptibility of using class-wise perturbations to craft unlearnable datasets should be expected, given that every image of a class contains the same image feature that is perfectly predictive of the label. Class- wise unlearnable datasets contain simple features that are perfectly correlated with the label and the linear logistic regression model is able to optimize features resembling the original perturbations. By design, our orthogonal projection attack is most effective against class-wise perturbed unlearnable datasets. It serves as evidence that class-wise, linearly separable perturbations cannot be relied upon for protecting data.
5 Conclusion
We design experiments to test prevailing hypotheses about unlearnable datasets, and our results have practical implications for their use. First, while all unlearnable datasets cause low test accuracy for

---

trained models, we demonstrate that generalizable features can still be learned from data. In many cases, high test accuracy can be achieved from feature extractor trained on unlearnable datasets. If data were modified for protection, an adversary may still be able to train a reasonable feature extractor, a harm for data thought to be secure. We advocate for our evaluation framework to be used for auditing whether new unlearnable datasets prevent useful features from being learned. Second, we find that the reason unlearnable datasets cause low test accuracy is not as simple as previously thought. AR perturbations serve as a counterexample to the hypothesis that unlearnable datasets contain linearly separable perturbations. Our finding that AR perturbations are not linearly separable suggests there could be underexplored methods for protecting data using imperceptible perturbations. Although unlearnalle datasets need not have linearly separable perturbations, if they do have them applied in class-wise manner, we present Orthogonal Projection attack that can effectively remove them. While learning the perturbations from perturbed images is challenging, our results imply class-wise perturbations cannof not be relied upon for data protection. We can learn a significant amount from unlearnable datasets. DNNs can learn generalizable features from unlearnable datasets assumed to be protected, linear separability is not necessary condition for preventing generalization from unlearnable datasets, and class-wise perturbations can be optimized against and effectively removed.
Acknowledgments and Disclosure of Funding
This material is based upon work supported by the National Science Foundation under Grant No. 198-1910132 and Grant No. IIS-22 2212122, and by DARPA's Guaranteeing AI Robustness Against Deception (GARD) program under #HR001 12020007. Tom Goldstein was supported by the ONR MURI program, the AFOSR MURI program, and the National Science Foundation (IS-22112822 & 2229885). Vasu Singla was supported in part by the National Science Foundation under grant number 11$-2213335. Pedro Sandoval-Segura is supported by a National Defense Science and Engineering Graduate (NDSEG) Fellowship. Any opinions, findings, and conclusions recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
References
[1] Andy Baio. Exploring 12 million of the 2.3 billion images used to train stable diffusion's 's image generator, Sep 2022.
[2] Battista Biggio. Blaine Nelson, and Pavel Laskov. Poisoning attacks against support vector machines. In Proceedings of the 29th International Coference International Conference on Machine Learning. ICML' 12, page 1467-1474, Madison, WI, USA, 2012. Omnipress.
[3] Hadi M. Dolatabadi, Sarah Erfani, and Christopher Leckie. The devil's advocate: Shattering the illusion of unexploitable data using diffusion models, 2023.
[4] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai Thomas Unterthiner. Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.1 20.200000//9/ /929, 2020.
[5] Ji Feng. Qi-Zhi with auto-encoder. Advances in Neural Information Processing Systems. 32, 2019.
Zhi Cai, and tua-Hua Zhou Learning to confuse: generating training time adversarial data
[6] Liam Fowl, Ping-yeh Chiang, Micah Goldblum, Jonas Geiping, Arpit Bansal, Wojtek Czaja, and Tom Goldstein. Preventing unauthorized use of proprietary data: Poisoning for secure dataset release. arXiv preprint arXiv:2103.02653 2021.
[7] Liam Fowl, Micah Goldblum, Ping-ych Chiang, Jonas Geiping, Wojeiech Czaja, and Tom Goldstein. Adversarial examples make strong poisons. M. Ranzato, A. Beygelzimer, Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 30339-30351, Curran Associates, Inc., 2021.
[8] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778 2016.
[9] Kashmir Hill. The secretive company that might end privacy as we know it, Jan 2020.
[10] Hanxun Huang. Xingjun Ma, Sarah Monazam Erfani, James Bailey, and Yisen Wang. Unleamable examples: Making personal data unexploitable. In International Conference on Learning Representations. 2021.
10

---

[11] Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversarial examples not bugs, they features. Advances in neural information processing xystems, 32, 2019.
[12] Polina Kirichenko, Pavel Izmailov, and Andrew Gordon Wilson. Last layer re-training is sufficient for robustness to spurious correlations, 2022.
[13] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[14] Alina Kuznetsova, Hassan Rom, Neil Alldrin, Jasper Uijlings, Ivan Krasin, Jordi Post-Tuset. Shahab Ka- mali, Stefan Popov, Matteo Malloii, Alexander Kolesnikov, et al. The open images dataset v4. International Journal of Computer Vision, 1956))99961181, 2020.
[15] Dong C Liu and Jorge Nocedal. On the limited memory bígs method for large scale optimization. Mathematical programming. 45(1-3):503- 528, 1989.
[16] Aleksander Madry. Aleksandar Makelov. Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. To- wards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations. 2018.
[17] Clare McGlynn, Erika Rackley, and Ruth Houghton. Beyond 'revenge porn': The continuum of image- based sexual abuse. Feminist legal studies, 25(1):25-46, 2017.
[18] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.
[19] Weiqi Peng and Jinghui Chen. Learnability lock: Authorized learnability control through adversarial invertible transformations. In International Conference on Learning Representations, 2022.
[20] Vinay Uday Prabhu and Abeba Birhane. Large image datasets: A pyrrhic win for computer vision? 202/ IEEE Winter Conference on Applications of Computer Vision (WACV), pages 1536-1546, 2020.
[21] Tianrui Qin, Xitong Gao, Juanjuan Zhao, Kejiang Ye, and Cheng-7hong Xu. Learning the unlearnable: Adversarial augmentations suppress unlearnable example attacks. arXiv preprint wwXiv2330311127. 2023.
[22] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog. 188)99 2019.
[23] Evani Radiya-Dixit, Sanghyun Hong, Nicholas Carlini, and Florian Tramer. Data poisoning won't 't save you from facial recogmition. In International Conference on Learning Representations, 2022.
[24] Colin Raffel, Noam Shazcer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J Liu, et al. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21(140):1-67. 2020.
[25] Olga Russakovsky, Jia Deng. Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, I 115:211-252, 2015.
[26] Pedro Sandoval-Segurs Vasu Singla, Liam Fowl, Jonas Geiping, Micah Goldblum, David Jacohs, and Tom Goldstein. Poisons that are learned faster are more effective. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pages 198-205, June 2022.
[27] Pedro Landoval-Segura. Vasu Singla, Jonas Geiping, Micah Goldblum, Tom Goldstein, and David Jacobs. Autoregressive perturbations for data poisonnn.. Advances in Neural Information Processing Systems. 35:27374-2783 2022.
[28] Fu Shaopeng, He Fengxiang, Liu Yang, Shen Li, and Tao Dacheng. Robust unleamable examples: Protect- ing data privacy against adversarial learning. International Conference on Learning Representations. 2022.
[29] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recogni- tion. arXiv preprint arXiv: 1409, 2014.
[30] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1-9, 2015.
[31] Lue Tao, Lei Feng. Jinfeng Yi. Sheng-Jun Huang, and Songcan Chen. Better safe than sorry: Preventing delusive adversaries with adversarial training. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances Neural Information Processing Systems, volume 34, pages 16209-16225. Curran Associates, Inc., 2021.
[32] Andrew Tarantola. Getty images the maker of ai art generator stable diffusion over data scraping allegations, Jan 2023.
[33] Dimitris Tsipras. Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robust- may be at odds with accuracy. In International ConfeeerennnCCCCeceneeee on Learning Representations, 2019.
11

---

[34] Rui Wen, Zhengyu Zhao, Zhuoran Liu. Michael Backes, Tianhao Wang, and Yang Zhang. Is adversarial training really silver bullet for mitigating data poisoning? In The Eleventh International Conference on Learning Reprexentatinss 2023.
[35] Shutong Wu, Sizhe Chen, Cihang Xie, and Xiaolin Huang. One-pixel shortcut: On the learning preference of deep neural networks. In The Eleventh International Conference on Learning Representations, 2023.
[36] Da Yu, Huishuai Zhang, Wei Chen, Jian Yin, and Tie- Yan Liu. Availability attacks create shortcuts. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. ACM, aug 2022.
[37] Chia Hung Yuan and Shan-Hung Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 12230-12240. PMLR, 18-24 Jul 2021.
Hung Wu, Neural tangent generalization attacks. In Marina Mcila and Tong
12

---

A Appendix
A.1 Training Hyperparameters
DNNs Can Learn Useful Features From Unlearnable Datasets In Section 4.2 we train number of ResNet-18 (RN-18) 8 models on different unlearnable datasets with cross-entropy loss for 60 epochs using batch size of 128. We save checkpoints at every epoch of training. For our optimizer, we use SGD with momentum of 0.9 and weight decay of 5 x 10 -4 We use an initial learning rate of 0.1 which decays using a cosine annealing schedule.
For training new classification layer feature extractor checkpoints, use 0.0 random clean images from the original training set data. Note that 5, 000 images is 10% of CIFAR-10 and CIFAR-100, but 3.9% of Imagenet subset, etc. Following [2]. using the feature extractor, we extract embeddings from this clean subset of data and preprocess the embeddings to have mean zero and unit standard deviation. To retrain the last layer, we use the logistic regression implementation from scikit-learn (sklearn. linear Liinar_modell wog.el..eeesciieeessiioooo
Linearly Separable Perturbations Are Not Necessary In Section for every unlearnable dataset, we first gather the set of perturbations by subtracting the clean image from the perturbed image. We zero-one normalize the perturbations before training a linear layer using L.-BFGS [5] for 500 steps using learning rate of 0.5.
Orthogonal Projection for Learning From Datasets with Class-wise, Linear Perturbations In Section 4.4, for CIFAR-IO, we train the logistic regression model for 20 epochs using SGD with an initial learning rate of 0.1, which decays by factor of 10 on epochs and (at epochs which are 0.5 and 0.75 through training). For CIFAR-100, we train the logistic regression model for 60 epochs due to the higher number of classes. After we orthogonally project the unlcarnable data using the optimized weights, train networks using the hyperparameters from checkpoint-trainine of Section
A.2 Additional Section 4.2 Results: DNNs Can Learn Useful Features From Unlearnable Datasets
A.2.1 More Model Architectures for Section 4.2
We consider three more model architectures: VGG-16 22), GoogLeNet [30, and ViT 4 Our ViT uses patch size of 4. For RN-18 8 and 16,-16, feature vectors are din-dimensional. Feature vectors for GoogleNet are 1024-dimensional. The ViT class token is 384-dimensional.
Table 4: Generalizable features can be learned from unlearnable datasets, using a variety of network architectures. We report Max DFR Test Accuracy for each CIFAR-10 unlearnale dataset. In gray, we indicate test accuracy improvemenn/deteriioaiion over DFR on the corresponding randomly initialized model architecture.
MODEL ARCHITECTURE GOOGLENET
VGG-16
VIT
CIFAR-10 TRAINING DATA
NONE
35.69
48.08
37.40
UNLEARNABLE EXAMPLES 10
37.84 (+2.
41.08 (-7.00)
49.57
(+12.17)
ADVERSARIAT. POISONING 2
64.73 (+29.04)
71.70 (+23.62)
68.97
(+31.57)
AR (82) 21
36.98 (+1.29)
40.12 (-7.96)
60.53 (423.13)
NTGA
56.03 (+20.34)
61.24 (+13.16)
60.53 (+23.13)
ROBUST UNLEARNABLE 28
343.4443444)
40.59 (-7.49)
49.10 (+11.70)
LSP [36]
40.86 (+5.17)
(+10.141
50.95
(+13.55)
OPS+EM 35
31.31 (-4.38)
49.73
(+12.33)
OPS 3
39.63 (+3.94)
52.02 (+3.94)
56.04 (+18.64)
UNLEARNABLE EXAMPLES 10
30.47 (-5.22)
(-11.76)
44.90 (+7.50)
REGIONS-4 26
43.29 (47.60)
48.65 (40.57)
52.60 (+15.20)
RANDOM NOISE
72.08 (436.39)
62.19 (+14.11)
55.58 (418.18)
13

---

In T we find that across architectures, Adversarial Poisoning data is easiest to extract general- izable features from. Surprisingly. ViT is most effective at learning generalizable features from all unlearnable datasets, achieving more than 7% test accuracy improvement over randomly initialized ViT all cases. For example, using only 5, 000 clean CIFAR-10 samples can be used achieve nearly 69% test accuracy, while using the same clean samples can only achieve 37. 37.40% test accuracy on a randomly initialized ViT. The GoogleNet architecture weights are seemingly more easily cor- rupted during training; Max DFR Test Accuracy for Unlearnable Examples, AR, Robust Unlearnable, and other datasets is much lower than test accuracy from a finetuned randomly initialized GoogleNet. Interestingly, the randomly initialized GoogleNet feature extractor achieves the highest DFR test accuracy.
A.2.2 More Datasets for Section 4.2
We consider three additional base datasets for four unlearnable dataset methods. We use an ImageNet [25 subset of the first 100 classes, following [10]
The train split consists of 129, 395 images, while
the test split consists of 5, 000 images.
Our SVHN [18] CIFAR 100 13], and Adversarial Poisoning ImageNet subset datasets contain pertur- bations of size ||8| 0c
255 Unlearnable Examples ImageNet (25 subset contains perturbations of
size ||ó]|oo I 255 following their open-source repository. We generate the Adversarial Poisoning [ ImageNet subset from published source code using PGD restart, opposed to 8 due to computation time. Our SVHN and CIFAR-100 100 Adversarial Poisoning datasets use 3 PGD restarts. On clean SVHN, CIFAR-100 100, and ImageNet subset, RN-18 achieves 96.33%, 74. 74.14% 78.92% test accuracy respectively.
Table 5: Generalizable features can be learned from unlearnable datasets different underlying distributions. We report Max DFR Test Accuracy for each unlearnable dataset. RN-18 checkpoints are trained on SVHN, CIFAR- 100, and ImageNet subset unlearnable datasets, and DFR is performed using 5, 000 clean samples from the corresponding base dataset.
FINETUNE DATA CIFAR- 100
SVHN
IMAGENET
TRAINING DATA
NONE
32.05
8.13
3.64
UNLEARNABLE EXAMPLES 10]
26.76
16.12
8.44
ADVERSARIAL POISONING 7
87.06
44.37
20.22
UNLEARNABLE EXAMPLES (10
22.48
10.32
7.88
RANDOM NOISE
27.35
47.41
23.30
In Table we again show that Adversarial Poisoning unlearnable data can be easily used to extract generalizable features regardless the underlying distribution (base dataset SVHN, CIFAR- 100, or ImageNet). For SVHN, Adversarial Poisoning is the only dataset from which the trained feature extractor performs better in Max DFR Test Accuracy (87.00 (87.06%) over a randomly initialized RN-18 (32.05%). As mentioned in Section 4.2 crror-minimizing perturbations of Unlearnable Examples tend to most effective at corrupting weights during training, regardless underlying finetune data.
A.2.3 Additional Plots for Section 4.2
We add results to the experiment from Figure Figure unlearnable datasets sufficiently corrupt RN- 18 weights during training and prevent DFR from recovering test accuracy.
14

---

[4]
Unieamable Exauples
Robust Unleamable
66
LSP
CIRAK 10 Toa A000IA
DFR Unlcamable Examples
10 Toa Asoumay
DER Rodund Unlcamablc
DER LSP
Randon
Random
Randona
Spacd
Fpadd
Egods
Unlcarmable Examples
CINAR 10TOL NOVAR2
DFR OPS+EM
DFR OPS
DER Unlearaable Examples
Rundon Init
Renden Init
Ramdiom Indt
00
Epach
Epaad
Bpods
Figure 4: Representations learned by other poisons consider better than random. (a-b) Reweighting deep features from sample-wise error minimizing noises provide no benefit over random features. DFR on a randomly initialized RN-18 only achieves 40% test accuracy (dotted line). (c-f) Other class-wise perturbations are very effective at corrupting network representations during training- effective that even DFR unable recover test accuracy from random features (red dotted line).
A.3
Additional Results for Section
4.3 Linearly Separable Perturbations Are Not Necessary
A.3.1 Other Background
related unlearnable dataset introduces entangled features (EntF) 34 which motivated by the separability of recent poisoning perturbations. However, separability is qualitatively evaluated through t-SNE visualizations, which is different from the separability experiment we perform. More specifically, t-SNE cluster separability should not be equated to the linear separability we measure possible to have linearly separable data that, when plotted using t-SNE, appears not separable. In other words, the EntF poison from (34 could still contain linearly separable perturbations.
Table 2 because
A.3.2 Evaluating Linear Separability of Poison Images
In Section we document the linear separability of perturbations from various poisons, as in 36
Poison images, the other hand, behave slightly differently. In Table we report logistic regression train accuracy on various CIFAR-10 poison images. We find that Unlearnable Examples, LSP, EMS and class-wise poisons have linearly separable poison images, but the remaining poisons we consider do not.
A.4 Additional Section 4.4 Results: Orthogonal Projection for Learning From Datasets with Class-wise, Linear Perturbations
A.4.1 VIT for Section 4.4
To evaluate recovered data from Orthogonal Projection, we consider an additional architecture: ViT. In Table 7 we train ViT with patch size of 4 on CIFAR-10 unlearnale datasets using different attacks. Our Orthogonal Projection method competitive with adversarial training for all class-wise perturbed unlearnable datasets and most sample-wise wise perturbed datasets. Orthogonal Projection is the best performing method for EPS and OPS (ICLR 2023). Note that EPS and OPS are most difficult for adversarial training.
15

---

Table 6: NTGA contains the least linearly separable images, while (R images most parable to the clean distribution. Other unlearnable datasets become more linearly separable. We train linear logistic regression model poison images and report train accuracy. High train accuracy indicates linear separability of poison images. Mean and one standard deviation computed from 10 independent runs.
TRAINING DATA
TRAIN ACCURACY
CLEAN
53.94 0.02
UNLEARNABLE EXAMPLES 10
100.00 0.00
ADVERSARIAL POISONING
62.40 0.01
AR (82) 27
53.97 + 0.02
NTGA 37
31.48 + 0.02
ROBUST
28
77.21 + 0.01
LSP 36
100.00 0.00
OPS+EM 35
100.00 0.00
OPS 35
100.00 = 0.00
UNLEARNABLE EXAMPLES [0]
100.00 0.00
REGIONS-4 26
100.00 0.00
RANDOM NOISE
100.00 0.00
Table 7: Orthogonal Projection can make wise unlcarnable data learnable for ViT. Es- pecially for unlcarnable datasets with class-wise, linearly separable perturbations, Orthogonal Projection attack is competitive with foo Adversarial Training at fraction of the computational cost.
ATTACK
CIFAR-10 TRAINING DATA
NONE
ADV TRAINING
ORTHO PROJ (OURS)
CLEAN
84.99
76.38
74.14
UNLEARNABLE EXAMPLES 10
25.39
75,44
60.15
ADVERSARIAL POISONING
31.33
75.15
41.49
AR (f2) 27
17.13
75.12
35.11
NTGA 37
32.67
71.95
66.19
ROBUST UNLEARNABLE 28
28.24
78.03
37.34
LSP 36
29.40
75.45
74.77
OPS+EM 35
20.73
11.79
51.94
OPS BS1
21.58
10.17
72.80
o UNLEARNABLE EXAMPLES 10
12.19
76.35
76.01
REGIONS-4 26
15.00
75,96
67.20
RANDOM NOISE
29.66
76.23
73.05
A.4.2 CIFAR-100 Dataset for Section 4.4
We consider an additional dataset, CIFAR-100, to evaluate Orthogonal Projection. We train a RN-18 four unlearnable dataset methods. During the first step of Orthogonal Projection, train the logistic regression model for 60 epochs using SGD with an initial learning rate of 0. which decays by factor of 10 on epochs 30 and 45.
Table 8: Orthogonal Projection is competitive on CIFAR-100 class-wise unlearnable data.
ATTACK
CIFAR-100 TRAINING DATA
NONE
ADV TRAINING
ORTHO PROJ (OURS)
CLEAN
74.14
59.23
26.78
UNLEARNABLE EXAMPLES
10
8.11
58.29
28.44
ADVERSARIAL POISONING
7
5.93
57.60
25.24
a UNLEARNABLE EXAMPLES 10
1.72
60.31
41.83
RANDOM NOISE
1.30
58.83
51.23
16

---

In Table 8 we find that Orthogonal Projection performs better on class-wise unlearnable data than sample-wist perturbed data, as expected. At approximately the cost of standard training (Attack: None), Orthogonal Projection achives gains of more than 20% test accuracy for sample-wise perturbed data and more than 40% test accuracy for class-wise perturbed data.
A.4.3 Additional Intuition for Orthogonal Projection
Assume CIFAR-IO images of shape (3, 32, 32). Each column i of W (optimized in Alg Lines 1-4) is a 3072-dimensional vector that represents the most predictive image feature for class This step serves recovery of the perturbation. After the QR decomposition of W, Q consists of orthonormal columns that form basis for the column space of W. When we say Orthogonal Projection "ensures that the dot product that every recovered image vector does not contain any linearly separable component (i.c., does not contain any column of as component). Alg. Line ensures image vectors and columns of are orthogonal and so the dot product is O. The "recovered" data thus has 10 dimensions (approximations of the 10 perturbations) removed.
row of X with every column of Q is zero," (i.e., X, Q 0) this means
A.4.4 Subtracting Class-wiss Image
Given that the goal of Orthogonal Projection is to extract perturbations from poison images, is reasonable to consider visualizing the average image of a class for class-wise poisons like LSP and OPS. In Figure we that class-wise average images somewhat reveal class-wise perturbations, but the results are not clear enough to be useful. In contrast, class-wise patters and clearly present in learned weights from logistic regression.
Class
Class
Class 2
Class 3
Class
Class
Class
Class
Class Class
LSP
OPS
Class Wise
Learned Weights (Orthogonal Projection)
Figure 5: Average class images display class-wise perturbations, but not as clearly as learned weights from logistic regression. We compare the average image of class (Left) and learned weights of a logistic regression classifier (Right) trained on image pixels (the first step of Orthogonal Projection method) to for LSP and OPS Poisons. While average image of class does reveal the class-wise perturbation (block pattern for LSP and one highlighted pixel for OPS), the result is blurry and contains other semantic image features. Learned weights from Orthogonal Projection properly isolate the perturbation.
For training models on class-wise perturbed data, one might consider subtracting the class-wise average image from each class. However, simply subtracting this average class image from each image does not remove the poisoning effect. Additionally, because we not know the true class at inference time, cannot subtract the class image, resulting in distribution mismatch between train and test sets. This trivial method subtracting average class images compared to Orthogonal Projection in Table
A.4.5 Visualizing Additional Logistic Regression Weights
We visualize additional linear model weights (from the first step of Orthogonal Projection) for sample-win wise perturbed unlearnable datasets in Figure 6 and for class-wise perturbed unlearnable datasets in Figure We find that for Adversarial Poisoning, AR, and Robust Unlearnable the linear model learns features comparable to when trained on clean data. We posit that because the diversity of perturbations in these datasets is higher, the linear model struggles to find predictive features to project away. contrast, for class-wise perturbed data, Figure demonstrates that the linear model can recover features that resemble the original class- wise perturbation.
17

---

Table 9: Subtracting the average class image from training images is not effective. For LSP and OPS, datasets with class-wise perturbations, our Orthogonal Projection attack improves CIFAR-10 test accuracy simply subtracting the average class image training time.
ATTACK
TRAINING DATA
CLASS-AVG SUBTRACT
ORTHO PROJ (OURS)
LSP 36
13.05
87.99
OPS 35
12.62
87.94
Unlearnable Examples
Adversarial
Polsoning
(E2)
NTGA
Robust Unlearnable
LSP
Figure 6: Learned Weights from first step of Orthogonal Projection on sample-wise perturbed CIFAR-10 unlearnable datasets. We visualize learned weights (W in Algorithm 1 after training linear model on unlearnable datasets. Learned weights from Adversarial Poisoning, AR, and Robust Unlearnable resemble the learned weights from clean data (See Figure 3). In our Orthogonal Projection attack, we project each perturbed image to be orthogonal to each of these learned weights (Algorithm Line 6).
A.5 Samples from Unlearnable Datasets
We visualize samples from sample-wise perturbed CIFAR-10 10 unlearnable datasets in Figure g and from class-wise perturbed unlearnable datasets (prefixed by throughout results) in Figure 9 NTGA omitted due data ordering of the publicly available poison. We also visualize SVHN samples in Figure (0 and CIFAR-100 in Figure
A.6 Broader Impact Statement
Our findings test prevailing hypothesis about unlearnable datasets and results have practical implications for their use. Two our three main conclusions relate to privacy vulnerabilities when employing unlearnable datasets for data protection. In one experiment, features can be learned from unlcarnable data. In another, we demonstrate how one can effectively remove class-wise perturbation. Our findings highlight the need for extra caution when it comes to using unlearnable datasets. By making this information available to the public, the capabilities and vulnerabilities of unlearnable datasets can be better understood.
demonstrate useful
18

---

Class
Class
Class 2
Class 3
Class
Class
Class
Class
Class 8
Class 9
Unlearnale
Examples
Regions-4
Random
Noise
Figure 7: Learned Weights from first step of Orthogonal Projection on class-wise perturbed CIFAR-10 unlearnable datasets. We visualize learned weights (W in Algorithm I after training linear model on unlearnable datasets. Learned weights appear to recover the added class-wise perturbation for all datasets. In our Orthogonal Projection attack, we project each perturbed image be orthogonal each of these learned weights (Algorithm) Line
Clean
Unieamable Examples
Adversarial Poisoning
(fx)
Robust Unlearnable
LSP
OPS+EM
Figure 8: Samples from CIFAR-10 unlearnable datasets. We visualize the first 10 images from each sample-wise perturbed unleamable dataset.
19

---

Clean
Unlearnable Examples
Reglons-4
Random
Nolse
Figure 9: Samples from CIFAR-10 unlearnable datasets. We visualize the first 10 images from each class-wise perturbed unlearnable dataset.
1919
2
3
9393
3
31
Clean
Unlearnable Examples
1919
2
3
9393
3
311
Adversarial Polsoning
1919
2
3
332
327
Class-wise Unlearnable Examples
1919
2
3
9393
33
31
Random Noise
1919
2
3
9393
311
Figure 10: Samples from SVHN unlearnable datasets.
Clean
Unlearnable
Examples
Adversarial
Polsoning
Class-wise
Unlearnable
Examples
Noise
Figure 11: Samples from CIFAR-100 unlearnable datasets.
20


Reviews:
[{"review_id": "khEv5QeltX", "reviewer": "Reviewer_TH5w", "summary": "This paper studies the problem of the actual learnability of unlearnable datasets. Specifically, the authors have demonstrated that unlearnable datasets that are generated by existing methods can actually be used to learn generalizable features. In addition, the authors show that it is not necessary to make poisons linearly separable for achieving effective poisoning effects. Furthermore, based on the fact that most existing poisoning methods rely on linear separability, the authors propose a simple yet effective countermeasure to recover clean images for training.", "questions": "N/A", "limitations": "Yes.", "rating": 7, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- It is valuable to revisit existing work in a popular field like data poisoning.\n- The paper is very well written, with a thorough review of related work and sufficient descriptions of technical details. \n- Experiments are extensive in terms of the number of models, datasets, settings, compared baselines, and so on. \n- Several visualizations are provided to help support new findings.", "weaknesses": "I very much appreciate the paper, and there are no major weaknesses but minor ones about tuning down some claims:\n\n1. This paper is not the first to find a counterexample to the linear separability hypothesis considering that previous work has intentionally relied on ideas beyond linear separability for generating unlearnable datasets [a].\n\n2. The orthogonal projection method is not the only simple yet effective countermeasure considering [b], where the simple image compression-based method, ISS, is applied to remove poisoning perturbations. About the results, it seems that ISS is more globally effective than the orthogonal projection method. In addition, It would also be appreciated if diffusion models [3] and error-maximizing augmentation [20] can be compared. These two methods are also conceptually simple because they rely on either a per-trained (diffusion) model for pre-processing or data augmentation (that may not necessarily be adversarial). \n\n[a] Is Adversarial Training Really a Silver Bullet for Mitigating Data Poisoning? ICLR 2023\n\n[b] Image Shortcut Squeezing: Countering Perturbative Availability Poisons with Compression. ICML 2023", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their time and feedback. We appreciate your mentioning that our “experiments are extensive'' and that the paper \"is very well written.” \n\n> previous work has intentionally relied on ideas beyond linear separability for generating unlearnable datasets [a]\n\nThank you for letting us know about the work of [a], which is relevant to our findings in Section 4.2. While the authors of [a] motivate their approach using separability of recent poisoning perturbations, they only use t-SNE visualizations, which is different from the separability experiment we perform.\n\nMore specifically, t-SNE cluster separability should not be equated to the linear separability we measure in Table 2 because it is possible to have linearly separable data that, when plotted using t-SNE, appears not separable. In other words, the poison from [a] could still contain linearly separable perturbations. \n\n> ISS is more globally effective than the orthogonal projection method\n\nOur Orthogonal Projection method is designed to exploit class-wise perturbations by design. It is not meant to be a general solution like ISS [b]. Nearly all Unlearnable Dataset papers focus on optimizing sample-wise noise [10,7,26,35,27,34,33], as opposed to class-wise noise, with little explanation why. We designed Orthogonal Projection to answer, for ourselves, why class-wise perturbations should not be relied on. Compared to ISS [b], our Orthogonal Projection method produces higher test accuracy on the recently published class-wise OPS poison (87.94% (ours) vs 85.16% [b]). \n\n[...] remaining citations are from paper references"}, {"review_id": "P8VsnGbUdc", "reviewer": "Reviewer_kEiu", "summary": "This paper conducts an analysis of the properties of unlearnable dataset methods to evaluate their potential for future viability and security assurances. It is demonstrated that neural networks possess the ability to learn generalizable features from unlearnable datasets, while also suggesting that image privacy may not be effectively preserved. Additionally, a counterexample is provided to challenge the widely held belief that unlearnable datasets induce learning shortcuts through the linear separability of added perturbations. To address this issue, an orthogonal projection attack is proposed, which enables learning from various unlearnable datasets. The results of this approach demonstrate that linearly separable perturbations should not be relied upon.", "questions": "1. The correlation between the method proposed in Section 4.4 and the findings in Sections 4.3 and 4.2, particularly 4.2, is of interest. \n2. Can additional sources be provided to corroborate the assertion that \"the higher the test accuracy after DFR, the more likely it is that the model has earned private image features present in the original clean data\"? \n3. The efficacy of the proposed method on other datasets merits investigation. It is recommended that further experimentation be conducted. \n4. The dissimilar trends displayed by (a) (d) and (b) (c) in Figure 2 require explanation.", "limitations": "Yes, the authors adequately addressed the limitations.\nHowever, in order to further enhance and augment the credibility and validity of their work, it is suggested that they provide a more elaborate and comprehensive explanation regarding the various methodologies and motivations that were employed in their research process. Additionally, it would be highly beneficial if they could include a greater number of experiments and references within their work, as this would serve to substantially bolster and fortify the robustness and soundness of their findings and conclusions.", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The originality of the paper is good, as it gives us a different view to unlearnable examples and poisoning attacks by\n    (a). Demonstrating that neural networks possess the ability to learn generalizable features from unlearnable datasets \n    (b). Giving a counterexample is provided to challenge the widely held belief that unlearnable datasets induce learning shortcuts through \n      the linear separability of added perturbations\n    (c). Proposing a new evaluation framework and a novel attack method to assess the viability and security promises of unlearnable datasets.\n   Overall, The paper's findings challenge some widely held beliefs about unlearnable datasets and provide insights into their limitations and \n   potential vulnerabilities\n2. The proposed method can achieve good results on some of unlearnable tasks (class-wise) and is more effective than adversial training.\n3. This paper is well-writen", "weaknesses": "1. The motivation for the paper does not align to me. I hope that the proposed method is motivated by the findings (Sections 4.2 and 4.3), however, the authors only illustrate the relationship between linearly separable perturbations and the proposed orthogonal projection method, making Section 4.2 a superfluous. There should be more discussion on the relationship between the findings and the methods presented in Section 4.4.\n2. The author uses the DFR method to prove that DNN can learn useful features, but it is not convincing to me. The improvement in accuracy may simply come from using cleaner samples. (eg Figure 2, pictures (b), (c)). Also, I think the act of loss should also be involved in this part. There should be more references to why \"the higher the test accuracy after DFR, the more likely the model will pick up on private image features present in the original clean data\".\n3. Although the authors give a counterexample to challenge the commonly held view that non-learnable datasets induce learning shortcuts by adding perturbed linear separability, it lacks theoretical analysis.\n4. The method only achieves satisfactory results on Class-Wise perturbation, but the generalization of the method is not enough. In contrast, adversarial training has better generalization ability.\n5. Limited evaluation: Evaluation requires more datasets.\n\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their careful review, for mentioning that our work “gives us a different view to unlearnable examples,” and for recognizing that our results “challenge some widely held beliefs about unlearnable datasets.” Those were our goals in this work.\n\n> The author uses the DFR method to prove that DNN can learn useful features, but it is not convincing to me. The improvement in accuracy may simply come from using cleaner samples\n\nBy finetuning a randomly initialized network (first row of Table 1) on a random subset of 5k clean samples, we can measure the effect of the clean samples. By doing the same procedure on poisoned networks and measuring the improvement **relative to** the finetuned randomly initialized network, we can measure the utility of poisoned-network features. Looking at Figure 2, plot (b), suppose we obtain checkpoints by training on class-wise random noise at epochs 3 and epoch 40. If we finetune the epoch 3 checkpoint (with 5k clean samples) we get test accuracy of 66.8%, while if we finetune epoch 40 checkpoint (with the same 5k clean samples) we get test accuracy of 19.2%. Relative to a finetuned random-init checkpoint (35.9% test acc), epoch 3 features obtained by training on the perturbed unlearnable data, yield more than a 30% increase in test accuracy. Useful features, which help test set generalization, are therefore present at epoch 3 but not at epoch 40. We found it surprising that despite both checkpoints being trained on unlearnable data, the learned features could still rise to such high performance.\n\n> The method only achieves satisfactory results on Class-Wise perturbation, but the generalization of the method is not enough\n\nOur Orthogonal Projection method is designed to exploit class-wise perturbations by design. It is not meant to be a general solution. Nearly all Unlearnable Dataset papers focus on optimizing sample-wise noise [10,7,26,35,27,34,33], as opposed to class-wise noise, with little explanation why. We designed Orthogonal Projection to answer, for ourselves, why class-wise perturbations should not be relied on. In [10], the authors note that despite class-wise error-minimizing noise being “superior to random noise” and “more efficiently and more flexibly in practical usage,” the noise “may get more easily exposed.” In [26], the authors state that “because class-wise perturbations can be recovered by taking the average image of a class, these should therefore be easy to remove” but do not provide evidence. If class-wise noises are more easily exposed, how does one remove them? We found that although one can average a class image, it is difficult to train a new network to reasonably high test accuracy by removing the average image. Instead, our solution is to project image features orthogonal to features optimized by logistic regression. Our Orthogonal Projection method is specifically for class-wise perturbations, which have recently been used to lock and protect data [18]. \n\n[10] Unlearnable Examples: Making Personal Data Unexploitable, ICLR 2021\n[18] Learnability lock: Authorized learnability control through adversarial invertible transformations, ICLR 2022\n[26] Autoregressive Perturbations for Data Poisoning, NeurIPS 2022\n[...] remaining citations are from paper references\n\n> \"the higher the test accuracy after DFR, the more likely it is that the model has learned private image features present in the original clean data\"\n\nUnlearnable dataset perturbations are supposed to stop the model from learning useful features, and if the model gets high test accuracy by training on them, that demonstrates that it has learned useful features. Thank you for pointing out our ambiguous language.  We have updated our draft to clarify and to remove the confusing “private image features” terminology.  We will include this update in our camera ready version.\n\n> The efficacy of the proposed method on other datasets merits investigation. It is recommended that further experimentation be conducted.\n\nIn the Supplementary Material, we include additional experiments on SVHN, CIFAR-100, and an ImageNet subset (Appendix 2.2) to enhance the results of Section 4.2. Moreover, we include additional experiments on CIFAR-100 in Appendix 3.2 to augment Section 4.4. We also provide additional results for different model architectures in Appendix 2.1 and Appendix 3.1.\n\n>The dissimilar trends displayed by (a) (d) and (b) (c) in Figure 2 require explanation.\n\nThank you for pointing this out. We have added additional explanation to the text following Figure 2. To clarify: This behavior is due to the different poison generation methods. More specifically, adversarial poisoning perturbs images with error-maximizing noise (and this noise has been shown to be actual features models use during classification [36]). In this case, DFR is reweighting useful, existing features for classification, leading to higher test accuracy. On the other hand, Random Noise and AR poisons do not perturb images with useful features; instead, both perturb with synthetic noise. In these cases, useful features are still learned during poison training, but only in the first epochs of training. As training progresses, the model checkpoints are continually corrupted by synthetic noise features which cannot be useful for classification despite reweighting. By “useful,” we mean the technical definition in Eq. 1 [36].\n\n[36] Adversarial Examples Are Not Bugs, They Are Features, NeurIPS 2019\n\nThank you again for your thoughtful review. We made a significant effort to address your feedback including multiple paper edits, and we would appreciate it if you would consider raising your score in light of our response. Do you have any additional questions we can address?\n"}, {"review_id": "4BeNbiGEuM", "reviewer": "Reviewer_vqjk", "summary": "This paper suggests that DNNs can learn useful features from unlearnable datasets and provides a counterexample, demonstrating that linear separability of perturbations is not a necessary condition. They propose the Orthogonal Projection method to recover unlearnable datasets.", "questions": "1. About weakness 1:\n1.1 In Figure 2, the performance of DFR on different unlearnable datasets is not consistent. At the last epoch, DFR in (b, d) is even worse than randomly initialized baseline. For certain datsets, such as Random noise and AR, the peak in early epochs is a property of these datsets, not your DFR method.\n1.2 I don't think the increase of test accuracy caused by DFR can support you claim because you retrain the linear classifier using clean data. Don't forget that your DFR method improve the accuracy of randomly initialized baseline from $10\\%$ to $35.79\\%$. However, you can not conclude that a randomly initialized feature extractor learns useful features.\n2. About weakness 2:\n2.1 In the algorithm of Orthogonal Projection, the recovered data $X_r$ is ensured that $W^T X_r=0$. What does this equation mean? How does it work for recovery?\n2.2 What is the reason that Orthogonal Projection is not good at recovering sample-wise perturbations?\n2.3 It is mentioned in Line 262 that perturbed images of sample-wise Unlearnable Examples and Adversarial Poisoning are not linearly separable though perturbation themselves are linearly separable. Does this illustration contradict to the claim in Line 216 \"learning the most predictive features in an unlearnable dataset can amount to learning the perturbation itself\"?\n2.4 Please give experimental evidence of linear (in)separability of images perturbed by those class-wise and sample-wise attacks.\n", "limitations": "As mentioned in the paper, orthogonally projection removes K dimensions from the data manifold, where K is the number of classes in a dataset. While this may not be a problem for high-resolution images with tens of thousands of dimensions, this detail could impact applicability for low-resolution datasets.\nAlso see the weakness.", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. A new method called Orthogonal Projection is proposed to recover unlearnable datasets. On class-wise unlearnable datasets, this method achieves similar (or even betters) performance compared to adversarial training. Moreover, it is much faster than adversarial training.\n2. This work suggests the risk of using class-wise perturbations to craft unlearnable datasets.", "weaknesses": "1. For the claim \"DNNs can learn useful features from unlearnable datasets\", your experimental evidence is not sufficient and the logic is not very solid. See Question 1.\n2. Though the intuition of Orthogonal Projection is clear, the purpose of its mathematical form is still ambiguous. Especially it cannot explain why this method is good at treating class-wise perturbations rather than sample-wise ones. See Question 2.\n3. Sections 4.2, 4.3 and 4.4 have weak connections with each other, appearing as if they are independent from one another. The results in sections 4.2 and 4.3 are not interesting enough.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their thorough feedback and for recognizing that our work “suggests the risk of using class-wise perturbations.” That was one of our goals in this work.\n\n> About weakness 1: 1.1 In Figure 2, the performance of DFR on different unlearnable datasets is not consistent. At the last epoch, DFR in (b, d) is even worse than randomly initialized baseline. For certain datasets, such as Random noise and AR, the peak in early epochs is a property of these datasets, not your DFR method.\n\n\nThis behavior is due to the different poison generation methods. More specifically, adversarial poisoning perturbs images with error-maximizing noise (and this noise has been shown to inject actual features models use during classification [36]). In this case, DFR reweights useful existing features for classification. On the other hand, Random Noise and AR poisons do not perturb images with image features; instead, both perturb with synthetic noise. In these cases, useful features are still learned during poison training, but only in the first epochs of training. In Figure 2, plots (a) and (d), we find that checkpoint features progressively get better despite test accuracy of the model remaining low. In plots (b) and (c), we find that checkpoint features are better around the test accuracy peak. In both cases, features learned during poison training can be reweighted for high test accuracy – calling into question the ability for any of these unlearnable dataset methods to safeguard data.\n\n> 2.1…What does this equation mean? How does it work for recovery? 2.2 What is the reason that Orthogonal Projection is not good at recovering sample-wise perturbations?\n\nAssume CIFAR-10 images of shape (3,32,32). Each column i of W (optimized in Alg. 1, Lines 1-4) is a 3072-dim vector that represents the most predictive image feature for class i. This step serves as recovery of the perturbation. After the QR decomposition of W, Q consists of orthonormal columns that form a basis for the column space of W. When we say Orthogonal Projection “ensures that the dot product of a row of X with every column of Q is zero,” (i.e., $X_r \\cdot Q = 0$) this means that every recovered image vector does not contain any linearly separable component (i.e., does not contain any column of Q as a component). Alg. 1, Line 6 ensures image vectors and columns of Q are orthogonal and so the dot product is 0. The “recovered” data thus has 10 dimensions (approximations of the 10 perturbations) removed. We have now updated Sec. 4.1.1 to be more clear on these steps, and we will include these updates in our camera ready version.\n\nOrthogonal Projection sometimes works on sample-wise perturbations, but is not meant for this case since perturbations can vary greatly within a class. This can make it difficult to optimize a representative set of vectors (W) which are predictive of the label.\n\n> 2.3 It is mentioned in Line 262 that perturbed images of sample-wise Unlearnable Examples and Adversarial Poisoning are not linearly separable though perturbation themselves are linearly separable. Does this illustration contradict to the claim in Line 216 \"learning the most predictive features in an unlearnable dataset can amount to learning the perturbation itself\"? \n\nWe don’t claim that learning the most predictive features will “amount to learning the perturbation itself”, only that it can amount to that. More specifically, the many poisons we consider in our work all behave differently, having been generated or optimized with a variety of objectives. Despite this, we find that for class-wise noises, a logistic regression classifier can learn the most predictive features and thereby isolate the perturbation itself (Figure 3).\n\n> 2.4 Please give experimental evidence of linear (in)separability of images perturbed by those class-wise and sample-wise attacks\n\nIn the paper, we document the linear separability of **perturbations** from various poisons, as in [34]. Perturbed images, on the other hand, behave differently. Prompted by your feedback, we report logistic regression train accuracy on various CIFAR-10 poison images below, and will include these results in our camera ready version:\n\n|   Data                    |   Train Accuracy |\n|---------------------------|------------------|\n|   Clean                   |   53.88          |\n|   Unlearnable Examples    |    100        |\n|   Adversarial Poisoning   |     49.47       |\n|   AR (L2)                 |      37.99     |\n|   Robust Unlearnable      |       67.21     |\n| LSP                       | 100            |\n| OPS+EM                    | 100            |\n| (CW) OPS                  | 100          |\n| (CW) Unlearnable Examples | 100           |\n|   (CW) Random Noise       | 100          |\n\nIn the table above, we find that Unlearnable Examples, LSP, OPS, and class-wise (CW) poisons have linearly separable poison images, but the remaining poisons we consider do not.\n\nThank you again for your thoughtful review. We made a significant effort to address your feedback including experiments and multiple paper edits, and we would appreciate it if you would consider raising your score in light of our response. Do you have any additional questions we can address?"}, {"review_id": "b1TzVb7Zr6", "reviewer": "Reviewer_zKV8", "summary": "This paper comprehensively evaluated existing unlearnable examples and showed a surprising result. Unlearnable examples aim to prevent the model from learning useful features from the data. However, results show that several methods that networks actually can learn useful features. This is revealed by applying an existing feature reweighting method. Such results indicate that \"privacy\" might not be preserved. This paper also demonstrated that the recent findings of linear separation in unlearnable examples are not a necessary condition and an orthogonal projection attack can break class-wise linear separable unlearnable noise. ", "questions": "See the weaknesses section.", "limitations": "The authors adequately addressed the limitations and potential negative societal impact.", "rating": 7, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "- By using an existing method of Deep Feature Reweighting (DFR), the finding is very interesting and new in this field. This paper has comprehensively evaluated the existing different methods for generating unlearnable examples. It is surprising to see model can \"actually\" learn useful features on several unlearnable methods. Interestingly, the error-minimizing does not. Although the exact cause remains unknown, such findings provide valuable insights for future works. Additionally, this also indicates that future works should consider DFR as a reasonable evaluation method for unlearnable examples. \n- Based on recent work, Yu et al. [34] show that unlearnable examples rely on linearly separable noise. This paper extended the analysis to a wide range of different generation methods. Results show that although unlearnable examples are commonly induced linear separable noise, one particular method, Autoregressive perturbations, does not fall into this category. This indicates that linear separable noise is not a necessary condition. These results provide insights for future works that there could be more effective non-linear separable unlearnable noise. \n- The presentations in this work are very good. It covered comprehensive related works, motivations, challenges and limitations in unlearnable examples.", "weaknesses": "My main concern is the Orthogonal Projection attack. Although it is technically sound, it is unclear what is the main contribution or the insight of this proposed method. \n- Class-wise noise is known to be easily detected in unlearnable examples. Considering an additive noise, Yu et al. [34] show averaging across classes can easily expose the class-wise noise. One trivial solution that might consider is to subtract the noise for each class. Does an Orthogonal Projection attack more effective compared to this trivial solution? For Orthogonal Projection attacks, does it have any effect on the sample-wise setting?\n- Compared to adversarial training (AT), the main benefit is efficiency. However, AT has other desired properties, such as adversarial robustness and learning robust features. As for effectiveness, it seems to be no significant benefit of the Orthogonal Projection attack. For OPS or OPS+EM, the constraint is a single pixel, while AT evaluated in this paper is $L_\\infty$, changing AT constraint might be effective against the OPS-based method. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their thorough review, for mentioning that our findings are “very interesting and new in this field,” and for writing that our paper provides “valuable insights for future works.”\n\n> Class-wise noise is known to be easily detected…Yu et al. [34] show averaging across classes can easily expose the class-wise noise. One trivial solution that might consider is to subtract the noise for each class. Does an Orthogonal Projection attack more effective compared to this trivial solution?\n\nWe did perform experiments averaging images of a given class to recover class-wise perturbations, but simply subtracting this class image from each image doesn’t remove the poisoning effect. Additionally, because we do not know the true class at inference time, we cannot subtract the class image (and so there is also a distribution mismatch between train/test). This trivial method of subtracting average class images is compared to Orthogonal Projection below: \n\n| Poison | Class-Avg Subtract | Orthogonal Projection |\n|--------|--------------------|-----------|\n| LSP    | 13.05              | 87.99     |\n| OPS    | 12.62              | 87.94     |\n\nHere, we report CIFAR-10 test accuracy for LSP and OPS poisons. We will be sure to add these results to the Appendix for completeness. Also, we revisited Yu et al. [34] but did not find their experiment where they average images of a class. Please let us know if there is a different work you were referring to, and we would be happy to comment.\n\n> For Orthogonal Projection attacks, does it have any effect on the sample-wise setting?\n\nOrthogonal Projection sometimes works on sample-wise perturbations (for example, NTGA and Unlearnable Examples in Table 3), but is not meant for this case since perturbations can vary greatly within a class. This can make it difficult to optimize a representative set of vectors (columns of W in Alg. 1 Line 3) which are predictive of the label.\n\n> Compared to adversarial training (AT), the main benefit is efficiency…AT has other desired properties, such as adversarial robustness and learning robust features.\n\nWe agree there are desirable properties of adversarial training (AT). For learning from unlearnable data, however, the goal is only to improve clean test performance – to somehow learn from corrupted data and generalize to clean data. Aside from being ~5x faster than AT,   Orthogonal Projection is also more interpretable: we can visualize the most predictive features (Figure 3 (b)) and choose whether to remove these features. On the other hand, it is difficult to understand which features AT is ignoring.\n\nAdditionally, the OPS poison is a case where AT will fail due to the unbounded perturbation, but Orthogonal Projection can achieve 87.94% test accuracy, which is higher than 14.41% of AT, and higher than 85.16% of ISS [37]. \n\n[37] Image Shortcut Squeezing: Countering Perturbative Availability Poisons with Compression, ICML 2023\n\n> changing AT constraint might be effective against the OPS-based method\n\nThe authors of OPS [33] indeed conduct an experiment where they perform $L_2$ AT as opposed to $L_{\\infty}$ and find that the poisoning effect can be mitigated: Using $\\epsilon=2$ $L_2$ AT yields a model with 73.7% test accuracy. But this is at the cost of clean test accuracy which degrades from 94% to 73.45%. Orthogonal Projection is more effective as it does not degrade clean accuracy much (90% test accuracy on clean train data), and achieves high test accuracy (87.94%) on OPS poison.\n"}]

Author rebuttal:
We'd like to thank everyone again for their reviews. A few reviewers mentioned wanting to see average images of a class to compare them to the learned weights from the first step of Orthogonal Projection. In the attached PDF, we include an additional figure which performs this visualization for two class-wise poisons. We will include this update in our camera ready version.

Meta-review: This paper looks at different proposals for making unlearnable datasets, and shows that they fail (i.e., it remains possible to learn generalizable features from these datasets).
The reviewers had mixed opinions on the generalizability and novelty of the paper's contributions.
I encourage the authors to follow the reviewers' recommendations to more clearly compare their approach's strengths and weaknesses with those of adversarial training.
Decision: Accept (poster)