Self-Ensemble Protection: Training Checkpoints Are Good Data Protectors
Authors: Sizhe Chen
Geng Yuan
Xinwen Cheng
Yifan Gong
Minghai Qin
Yanzhi Wang
Xiaolin Huang
Venue: ICLR 2023 poster (2023)
Keywords: data protection
poisoning attack
self-ensemble
deep neural network

Abstract: As data becomes increasingly vital, a company would be very cautious about releasing data, because the competitors could use it to train high-performance models, thereby posing a tremendous threat to the company's commercial competence. To prevent training good models on the data, we could add imperceptible perturbations to it. Since such perturbations aim at hurting the entire training process, they should reflect the vulnerability of DNN training, rather than that of a single model. Based on this new idea, we seek perturbed examples that are always unrecognized (never correctly classified) in training. In this paper, we uncover them by model checkpoints' gradients, forming the proposed self-ensemble protection (SEP), which is very effective because (1) learning on examples ignored during normal training tends to yield DNNs ignoring normal examples; (2) checkpoints' cross-model gradients are close to orthogonal, meaning that they are as diverse as DNNs with different architectures. That is, our amazing performance of ensemble only requires the computation of training one model. By extensive experiments with 9 baselines on 3 datasets and 5 architectures, SEP is verified to be a new state-of-the-art, e.g., our small $\ell_\infty=2/255$ perturbations reduce the accuracy of a CIFAR-10 ResNet18 from 94.56% to 14.68%, compared to 41.35% by the best-known method. Code is available at https://github.com/Sizhe-Chen/SEP.

Published as a conference paper at ICLR 2023
SELF-ENSEMBLE PROTECTION: TRAINING CHECK- POINTS ARE GOOD DATA PROTECTORS
Sizhe Chen Geng Yuan", Xinwen Cheng', Yifan Gong", Minghai Qin', Yanzhi Wang", Xiaolin 1 Department of Automation, Shanghai Jiao Tong University
Xiaolin Huang"
2 Department of Electrical and Computer Engineering, Northeastemn University
ABSTRACT
As data becomes increasingly vital, company would be very cautious about releas- ing data, because the competitors could use it to train high- ser-pperformanc models, thereby posing tremendous threat to the company's commercial competence. To prevent training good models on the data, we could add impenceptible perturbations to it. Since such perturbations aim at hurting the entire training process, they should reflect the vulnerability of DNN training, rather than that of a single model. Based on this new idea, we seek perturbed examples that are always unrecognized (never correctly classified) in training. In this paper, we uncover them by model check- points' gradients, forming the proposed self- ensemblle protection (SEP), which is very effective because (1) learning on examples ignored during normal training tends to yield DNNs ignoring normal examples; (2) checkpoints' cross-model gradients are close to orthogonal, meaning that they are as diverse DNNs with different architectures. That is, our amazing performance of ensemble only requires the computation of training one model. By extensive experiments with baselines on 3 datasets and 5 architectures, SEP is verified to be a new state-of-thee-ar e.g., our small fo 2/255 perturbations reduce the accuracy of a CIFAR-10 ResNet18 from 94.56% to 14.68% 68, compared to 41.5% by the best-known method. Code is available https: //gtttbtttttt//gg//g com/ Si zhe Chen/SEP
INTRODUCTION
Large-scale datasets have become increasingly important in training high-performance deep neural networks (DNNs). Thus, it is a common practice to collect data online (Mahajan et al.) 2018 Sun almost unlimited data source. This poses great threat to the commercial competence of data owners such as social media companies since the competitors could also train good DNNs from their data. Therefore, great efforts have been devoted protecting data from unauthorized use in model training. The most typical way is to add imperceptible perturbations to the data, so that DNNs trained on it have poor generalization (Huang et al.) 2020aa) Fowl et al.. 2021b).
et al.]]
Existing data protection methods use a single DNN to generate incorrect but DNN-sensitive features (Huang et al. 2020a Fu et al. 2021 Fowl et al. 2021b) for training data by, e.g., adversarial attacks (Goodfellow et al.) 2015). However, the data protectors cannot know what DNN and what training strategies the unauthorized users will adopt. Thus, the protective examples should aim at hurting the DNN training, a whole dynamic process, instead of static DNN. Therefore, it would be interesting to study the vulnerubility of DNN truining. Recall that the vulnerability of DNN is revealed by the adversarial examples which are similar to clean ones but unrecognized by the model (Madry et al., 2018). Similarly, we depict the vulnerability of training by the perturbed training samples that are never predicted correctly during training. Learning on examples ignored during normal training tends to yield DNNs ignoring normal examples.
Such examples could be easily uncovered by the gradients from the ensemble of model training checkpoints. However, ensemble methods have never been explored in data protection to the best of our knowledge, it natural wonder
Can we use these intermediate checkpoint models for data protection in self-ensemble manner?
"Correspondence Xiaolin Huang (xiaolinhuang @sjtu.cdu.cn).

---

Published as a conference paper at ICLR 2023
An effective ensemble demands high diversity of sub-models, which is generally quantified by their gradient similarity (Pang et al. 2019: Yang et al. (021). i.., the gradients on the same image from different sub-models should be orthogonal. Surprisingly, we found that checkpoints' gradients are as orthogonal as DNNs with different architectures in the conventional ensemble. In this regard, we argue that intermediate checkpoints very diverse to form the proposed self-ensemble protection (SEP), challenging existing beliefs on their similarity (Li et al. 2022).
By SEP, effective ensemble protection is achieved by the computation of training only one DNN. Since the scale of data worth protecting is mostly very large, SEP avoids tremendous costs by training multiple models. Therefore, our study enables practical ensemble for large-scale data, which may help improve the generalization, increase the attack transferabilitty, and study DNN training dynamics.
Multiple checkpoints offer us a pool of good features for an input. Thus, we could additionally take the advantage of diverse features besides diverse gradients at no cost. Inspired by neural collapse theory (Papyan et al. 2020). which demonstrates that the mean feature of samples in class is a highly representative depiction of this class, we bring about a novel feature alignment loss that induces sample's last-layer feature collapse into the mean of incorrect-class features. With features from multiple checkpoints, FA robustly injects incorrect features so that DNNs are deeply confounded.
Equipping SEP with FA, our method achieves astonishing performance by revealing the vulnerability of DNN training: (1) our examples are mostly mis-classified in any training processes compared to et al. 2022), and (2) clean samples are always much closer to each other than to protected samples, indicating that the latter belong another distribution that could not be noticed by normal training. By setting &x = 2/255, a very small bound, SEP perturbations on the CIFAR-IO 10 training set reduce the testing accuracy of a ResNet18 from 94.56% to 14.68%, while the best-known results could only reach 41 11.55% with the same amount of overall calculation to craft the perturbations. The superiority of our method is also observable in the study on CIFAR-IO0 100 and ImageNet subset on 5 architectures. We also study perturbations under different norms, and found that mixing So and lo perturbations (Wu et al., (023) is the only effective way resist & adversarial training, which could recover the accuracy for all other types of perturbations. Our contributions could be summarized below.
recent method (Sandoval-Seguu
We propose that protective perturbations should reveal the vulnerability of the DNN training process, which we depict by the examples never classified correctly in training.
We uncover such examples by the self-ensemble of model checkpoints, which are found to be surprisingly diverse data protectors.
Our method very effective even using the computation of training one DNN. Equipped with novel feature alignment loss, our lov 8/255 perturbations lead DNNs to have < 5.7% / 3.2% / 0.6% accuracy on CIFAR-10 / CIFAR-100 ImageNet subset.
2 RELATED WORK
Small perturbations are known to be able to fool DNNs into incorrect predictions (Szegedy et al. 2014). Such test-time adversarial perturbations are crafted effectively by adversarially updating samples with model gradients (Carlini & Wagner (017)). and the produced adversarial examples (AEs) transfer to hurt other DNNs as well (Chen et al. 2022). Similarly, training- -time adversarial perturbations, i.e., poisoning examples, are also obtainable by adversarialyy modify training samples using DNN gradients (Koh & Liang] 2017 Fowl et al. 2021b). All DÑNs trained on poisoning examples generalize poorly on clean examples. making poisoning methods helpful in protecting data from unauthorized use of training. Besides adversarial noise, it has been demonstrated that
error-minimization (Huang et al., (2020a), gradient alignment (Fowl et al.] (021a) and influence functions (Fang et al. (2200) are also useful in protecting data. However, current methods only use one DNN because the scale of data worth protection is very large for training multiple models.
Ensemble is validated as panacea for boosting adversarial attacks (Liu et al.) 2017 Dong et al. 2018). By aggregating the probabilities (Liu et al. 2017). logits or losses (Dong et al. 2018) of multiple models, ensemble attacks significantly increase the black-box attack success rate. Ensemble attacks could be further enhanced by reducing the gradient variance of sub-models (Xiong et al. 2022), and such an optimization way is also adopted in our method. Besides, ensemble has also
2

---

Published as a conference paper at ICLR 2023
been shown effective defense method by inducing low diversity across sub-models (Pang et al. 2019; Yang et al. 2020 202I) or producing diverse AEs in adversarial training (Tramer et al. 2018
Wang & Wang 2021). Despite the good performance of ensemble in attacks and defenses, has not been introduced to protect datasets due to its inefficiency. In this regard, we adopt the self-ensemble strategy, which only requires the computation of training one DNN. Its current applications focused on semi-supervised learning (Zhao et al. 2019 Liu et al. 2022)
Two similar but different tasks besides poisoning- backdoor attacks. Adversarial training (Madry et al., 2018)) Zhang et al. 2019 Stutz et al. 2020)
data protection are adversarial training and
continuously generates AEs with current checkpoint gradients improve the model's 's robustness towards worst-case perturbations, In contrast, data protection produces fixed poisoming examples so that unauthorized training yields low clean accuracy. Backdoor attacks (Geiping et al., 2020 Huang et al. 2020b) perturb a small proportion of the training set to make the DNNN mispredict certain samples, but remain well-functional on other clean samples. While data protectors perturb the whole training set to degrade the model's 's performance on all clean samples.
3 THE PROPOSED METHOD
3.1 PRELIMINARIES
We first introduce the threat model and problem formulation of the data protection task. The data owner company wishes to release data for users while preventing unauthorized appropriator to collect data for training DNNs. Thus, the data protector would add imperceptible perturbations to samples, that humans have obstacle seeing the data, while the appropriator cannot train DNNs to achieve an acceptable testing accuracy. Since the protector has access to the whole training set, can craft perturbations for each sample for effective protection (Shen al. [2019] Feng et al., 2019 Huang et al. 2020a Fowl et al. 2021b). Mathematically, the problem can be formulated as
max SEll.
Σ
Cc
s.t. 0" E arg min
Σ (T.
LeB (Ja (ar + 8,0),0),
(E.
where the perturbations d bounded by are added to the training set T so that appropriator model trained on fa(-: 0*) have a low accuracy on the test set D, ie.,, high cross-entropy loss LCE('s) ( )
The ö could be effectively calculated by targeted attacks (Fowl et al. 2021b), which use well-trained protecting DNN fp to produce targeted adversarial examples (AEs) that have the non-robust features in the incorrect class g(v) as
Lc+1 II, (z, - a sign (Grelfp.
Gce(fp: x) VeLCE (fp (2)))(v))
(2)
where II, clips the sample into the -orm bound after each update with a step size a. g()) stands for permutation on the label space. Here our method also adopts the optimization in
3.2 DEPICTING THE VULNERABILITY OF DNN TRAINING
Current methods (Shen et al. 2019 Feng et al. 2019 Huang et al. 2020a Fowl et al. 202 craft protective perturbations that are supposed generalize to poison different unknown architectures by a single DNN fp. However, the data protectors cannot know what DNN and what training strategies the unauthorized users will adopt. Thus, the protective examples should aim at hurting the DNN training, a whole dynamic process, instead of a static DNN. Therefore, it would be interesting to study the vulnerability DNN training.
Recall that the vulnerability of a DNN is represented by AEs (Madry et al. 2018), because they are slightly different from clean testing samples but are totally unrecognizable by a static model. Similarly, the vulnerability of DNN training could be depicted by examples that slightly different from clean training samples but are always unrecognized in the training process, i.c., the perturbed data never correctly predicted by the training model. If we view the training process the generation of checkpoint models, the problem becomes finding the examples that are adversarial to checkpoints, which could be easily solved by the ensemble attack (Dong et al. 2018).
Let us investigate whether the training checkpoints, which are similar (Li et al. 2022) in architecture and parameters, could be diverse sub-models for effective self-ensemble. To measure the diversity,
3

---

Published as a conference paper at ICLR 2023
0.10
self-ensemble
0.08
model set
0.035
0.036
0.36
01155
0.005
1.400
0.056
0.034
moxds training
0.04
0.024
0.305
0.085
1.000
0.042
0.02
0.024
0.0
0.034
0,0
1.000
0.30
0.10
0.08
0.036
0.037
0.086
0:08
1,000
0.001
ensemble
0.04
0.038
0.037
0.030
1,000
0.009
model set
0.036
0.036
0.031
0.036
1,000
0.02
0.00
Figure I: Ensemble and Self- Eniembnee Ensemble trains multiple models (left to right and top to bottom), while self-ensemble trains once and collects intermediate checkpoints (only top to bottom). Thus, self-ensemble costs much less training calculation. Moreover, checkpoints could provide diverse gradients. The right figures show the average absolute cosine value of gradients (on 1K CIFAR-10 1-0 images) for each model pair in ensemble (bottom) and self-ensemble (top) sub-models. In ensemble, models 0 to stand for ResNet18, SENet18, VGGI6, DenseNet121, and GoogLeNet, respectively, and in self-ensemble, they mean the ResNet18 after training for 24, 48, 72, 96, 120 epochs. Gradient analysis on CIFAR-100 and ImageNet are in Appendix
we adopt the common gradient similarity metric (Pang et al. 2019 Yang al. 2021). In Fig,
(upper right), we plot the average of absolute cosine value for gradients in different checkpoints, and the low value indicates that gradients on images are close orthogonal for different checkpoints like in the ensemble of different architectures (bottom right). This means, surprisingly. intermediate checkpoints are very diverse and sufficient to form the proposed self-ensemble protection (SEP) as
GSEP (pp x)
= Σ GCE(Sp x),
(3)
k=
where fo is the hthh equidistant intermediate checkpoint and GSEP is the gradients for update 2 As illustrated in (left, SEP (vertical box) requires the computation of training only one DNN compared to the conventional ensemble (horizontal box) that needs time- and resource-consuming training processes to obtain large number of ensemble models. This efficiency is especially important for data protection. Because only a large amount of data, stolen, could be used to train competitive DNNs. In this regard, the scale of data requiring particular protection would be large, and saving the calculation of training extra models makes a significant difference.
SEP is differently motivated compared to conventional ensemble. Ensemble attacks aim to produce architecture- invariant adversarial examples, and such transferable examples reveal the common vulnerability of different architectures (Chen et al., 2022). SEP, in contrast, targets the vulnerability of DNN training. By enforcing consistent misclassification SEP produces examples ignored during normal training, and learning them would thus yield DNNs ignoring normal examples.
3.3 PROTECTING DATA BY SELF-ENSEMBLE
Multiple checkpoints offer us pool of features for an input. Those representations, though distinctive, all contribute to accurate classification. Thus, we could additionally take the advantage of diverse features besides diverse gradients at no cost. Motivated by this, we resort to the neural collapse theory (Papyan et al. 2020) because unravels the characteristics of DNN features.
Neural collapse has four manifestations for deep classifier. (1) In-class variability of last-layer activation collapse to class means. (2) Class means converge to simplex equiangular tight frame. (3) Linear classifiers approach class means. (4) Classifier converges to choose the nearest class mean. They demonstrate that the last-layer features of well-trained DNNs center closely on class means. In this regard, the mean feature of in-class samples is highly representative depiction of this class.
Based on this, we develop the feature alignment loss to jointly use different but good representations of class from multiple checkpoints. Specifically, for every checkpoint, encourage the last-layer feature of a sample to approximate the mean feature of target-class samples. In this way, FA promotes neural collapse incorrect centers that sample has the exact high-dimensional feature of the
4

---

Published as a conference paper at ICLR 2023
target-class samples. Therefore, non-robust features of that target class could be robustly injected into data so that DNNs are deeply confounded. Mathematically, FA in SEP can be expressed as
n-1 CAEE.FA(Mp = = (i
n-1
Vallip (a)) - (o((())) 15(() ΣØET, h (2:
Σ
(4)
ITyl
k=0
k=
where ie stands for the feature extractor (except for the last linear layer) of fp. and h (g(v)) is for the mean (center) feature in the target class g(v) calculated by hy. || means the MSE loss.
Our overall algorithm summarized in Alg. I where we use a stochastic variance reduction (VR) gradient method (Johnson & Zhang 2013) to avoid bad local minima in optimization. Our method first calculates the FA gradients gs. by each training checkpoint (line 4). Then before updating the sample by the accumulated gradients (line 11), we reduce the variance of ensemble gradients (from Jens to uupa) in a predictive way by M inner virtual optimizations on Em. which has been verified to boost ensemble adversarial attacks (Xiong et al. )022).
Algorithm 1 Self Ensemble Protection with Feature Alignment and Variance Reduction
Input: Dataset = {(x y) for bound E, step size a, number of protection iterations T, number of training iterations N, number of checkpoints in self-ensemble number of inner updates M Output: Protected dataset x'
I: Train DNN for N epochs and save equidistant checkpoints
2: 2o = Ø
3: for = 0 - T 1 do
4: for k = + - do 92. = GrA (b)) # get the gradients by each checkpoints as
5: 9ens
9x. Jupd 0, to xt initialize variables for inner optimization for = M do
1 -
"
6:
7: Pick random index stochastic variance reduction (Johnson Zhang 2013)
8:
9apd = 9upd G (hp 2m) - (gx 9ens) # #ccumullte gradients with variance sggn) # virtual update onIm
9:
Øm+11 II. @im-
10: end for
Tt+11 II, (xt - a sign(gupa)) update samples with variance- rerduce--reduced gradients
12: end for
13:
return x' xT-1
In word, the main part of our method is to use checkpoints to craft targeted AEs for the training set (Line 4-5 Alg. [T) in a self-ensemble protection (SEP) manner. And SEP could be boosted by FA loss (Line 4) and VR optimization (Line 6-11). In this way, our overall method only requires N training epochs, and T' x (n + M) times of backward calculation to update samples.
4 EXPERIMENTS
4.1 SETUP
We evaluate SEP along with 7 data protection baselines, including adding random noise, TensorClog aiming to cause gradient vanishing (Shen et al., 2019), Gradient Alignment to target-class gradients
(Fowl et al.) 2021a), DeepConfuse that protects by an autoencoder (Feng al. 2019). Unlearnable Examples (ULEs) using error- cotor--miiizzaion noise (Huang et al. 2020a).. Robust ULEs (RULEs) that
use adversarial training (Fu et al. (02T)) Adversarial Poison (AdvPoison) resorting to targeted attacks
(Fowl et al. b)21b), and RutoRegeessive (AR) Poison (Sandoval-Segura -Seuura et al. 2022) using Markov
chain. Hyperparameters of baselines are shown in Appendix We use the reproduced results in
(Fowl et al. 2021b) in Table 5 and
For our method, we optimize class-y samples to have the mean feature of target incorrect class g(v). where g(v) = (y + 51%10 for CIFAR-10 (Krizhevsky et al. 2209) and ImageNet (Krizhevsky et al. 2017) protected classes, and g(y) (y + 50)%100 for CIFAR-I00. For the ImageNet subset, we train and f, with the first 100 classes in ImageNet-IK. 1K but only protect samples in 10 significantly different classes, which are African chameleon, black grouse, electric ray, hammerhead, hen, house finch, king snake, ostrich, tailed frog, and wolf spider. This establishes the class-wise data protection
5

---

Published as a conference paper at ICLR 2023
setting, and the reported accuracy is caleulated on the testing samples in these 10 classes. We train
a ResNet18 for N = 120 epochs as fp following (Huang et al., 2020a Fowl et al. Ib). 15 equidistant intermediate checkpoints (epoch 8, 16, .... 120) are adopted with M = 15, I 30 if not otherwise stated. Experiments are conducted on an NVIDIA Tesla A 100 GPU but could be run on GPUs with 4GB+ memory because store checkpoints on hardware.
The data protection methods are assessed by training models with 5 architectures, which are ResNet18 (He et al. 2016), SENe(18 (Hu et al., 2018), VGGI6 (Simonyan & Zisserman (015), DenseNet121 (Huang et al. 2017)) and GoogLeNet (Szegedy et al. 2013). They are implemented from Pytorch vision (Paszke al., 2019). We train appropriator DNNs fa for 120 epochs by an SGD optimizer with an initial learning rate of 0.1, which is divided by 10 in epochs 75 and 90. The momentum item in training is 0.9 and the weight decay is 5e-4. In this setting, DNNs trained on clean data could get great accuracy, i..., 95% / 75% / 78% for CIFAR-10 CIFAR- 100 / ImageNet subset. Training configurations are the same for fa and fp- In the ablation study, we denote the pure self-ensemble (3) SEP, SEP with feature alignment (4) SEP-FA, and SEP-FA with variance reduction as SEP-FA- VR (Alg. D. In other experiments, "ours" stands for our final method, i.e., SEP-FA-VR. We put the confusion matrix of fa in Appendix to provide class wise analysis.
4.2
UNCOVERING THE VULNERABILITY OF DNN TRAINING
100
0.00 2.6
0:0
0.23
0,00
0.00
0.00
5.00
140
1.0
0.00
1.60
0:20
0.00
0.00
6.00
140
28
66
0:
0.00
0.00
0.00
0:99
021 6.00
0.00
6.00
1.0 000
Accuracy
60
AR (VGG16)
0000
000
3.5
025
0.00
2.60
0.00
26
0:29
0.00
6.00
0.00
0.00
240 0:0
ours (VGG16)
000
000
0:00
2.60
0.00
0.20
0.00
6:0
0.00
0.00
240 0.00
AR (DerseNe:121)
6000
0:0
3:2
0:09
0.00
0:60
0.00
90
0.00
6:0
0.00
0.00
2:00 0:00
(DenseNet122
0000
6:0
333
0:41 0:50 0:00
0:60
0.00
0:00
6:00
0.00
0.00
2:00 0:7
20
000
355
0:59 0:00
0:65
0.00
99 0:00
8:00
0.00
0.00
2.00 037
352
0:00
6:60
0.00
0:00
8:0
0.00
0:00
2.00 0:0
6:0
pm
0:00
60
0:09
m:
5:59 pm 600 000
5:0 0:0
20
40
60
80
100
120
Epoch
(9)
Figure 2: (a) Our examples generated by ResNet18 are mostly mis-classified during the training of DenseNet121 and VGG16 compared to AR poison & = 1. (b) The confusion matrix (X-axis for predicted classes) of training with clean (classes 0-4) and protective examples (classes 5-9). (c) The confusion matrix of training with clean (class 0) and protective examples (classes 1-9).
We first investigate whether our examples successfully reveal the vulnerability of DNN training. so, we should be able to hurt different training processes regardless of the model architecture. In this regard, we test the "cross-training transferability" of our protective data. We report the results in Fig. 2 (a), where one could see that SEP samples generated by ResNet18 training tend to be mispredicted in training DenseNet121 and VGG16. In contrast, AR samples behave like clean training data and can be well recognized. This demonstrates that our method well depicts the vulnerability of DNN training compared to the recent baseline (Sandoval-Segura et al. 2022).
We also perform class wise study to illustrate how DNNs treat clean and protective examples. We first train a CIFAR- 10 DNN with clean samples (classes 0-4) and protective ones (classes 5-9, with features of classes 0-4). The DNN performs well in classes 0-4 but misclassifies all clean samples in classes 5-9 to classes 0-4, seeing Fig. 2 (b). This indicates in DNN's view, clean samples (from different classes) are much closer to each other than to protective ones (which look very similar). More extremely, if we inject features of class 0 to classes 1-9 examples, and use them with clean samples (class 0) to train, the DNN would classify all testing samples to class O, seeing Fig. (c).
4.3 PROTECTIVE PERFORMANCE
By depicting the vulnerability of DNN training, our method achieves amazing protective performance
on 3 datasets and 5 architectures against various baselines. In table [ our method surpasses existing otate-of-the--at baselines by large margin, leading DNNs to have < 5.7% / 3.2% / 0.6% accuracy on CIFAR-10 / CIFAR-100 / ImageNet subset. Comparison with weaker baselines is shown in Table 5
The great performance enables us to set extremely small bound of = 2/255, even

---

Published as a conference paper at ICLR 2023
for high-resolution ImageNet samples. The perturbations would be invisible under meticulous inspection, seeing Appendix A However, the appropriator can only reach no more than 30% accuracy in most cases, seeing Table
Table 1: Model testing accuracy trained on the protected dataset (lx. = 8/255).
Dataset
CIFAR-10
CIFAR-100
ImageNet subset
Model
RN18
VGG16
DN121
RN18
VGG16
DN121
RN18
VGGI6
DN121
ULEs
19.93
28.34
20.25
14.81
17.56
13.71
12.20
11.14
15.44
RULEs
27.09
28.17
24.96
10.14
14.39
13.96
13.74
12.77
14.36
AdvPoison
6.25
6.88
6.22
3.49
4.46
3.57
2.30
5.40
4.80
Ours
4.73
5.61
3.76
2.65
3.15
2.43
0.00
0.60
0.20
Table 2: Model testing accuracy trained on slight perturbed dataset (l 2/255).
Dataset
RN18
SENet18
VGG16
DN121
GoogLeNet
CIFAR-10
14.68
15.93
23.66
15.02
17.99
CIFAR-I00
21.16
19.48
66.73
21.85
27.49
ImageNet subset
8.80
10.70
27.00
9.40
30.80
Adversarial training (AT) has been validated as the most effective strategy recover the accuracy of training with protective perturbations. does not hinder the practicability of data protection methods because AT significantly decreases the accuracy and requires several-fold training computation. However, it would be interesting to study the effect of different types of perturbations in different AT settings. Here study with AR Poison, which is claimed to resist AT. We set the perturbation bound as l = 1 (step size = 0.2) (Sandoval-Segun latter means perturbing one pixel without other restrictions. We keep the AT bound the same as the perturbations bound, and find that in this case, both for and 6 ATs could recover accuracy of locs fz. and foo + lo perturbations. The only type of perturbations able to resist foo AT the mixture of fox and lo perturbations. Besides, our method is significantly better than AR Poison in normal training.
et al. 2022) and to 1 (Wu et al. (003),, and the
Table 3: Performance of perturbations under different norms fo 8/255, & 1, in CIFAR-10 10 adversarial training of ResNet18. too la means mixing (adding) perturbations.
Pert.
l2
fo0 fg
foo + Eu
AT
None
foc
E
None
8.0
8
None
80o
8
None
foc
8y
AR
20.49
84.73
82.66
12.99
82.03
82.86
12.29
82.87
83.47
15.25
12.28
70.26
ours
4.73
82.51
81.91
3.67
81.52
81.89
5.05
81.54
82.25
3.43
13.19
71.01
4.4
ABLATION STUDY
We study the performance improvements from SEP, FA, and VR separately along with the best baseline AdvPoison (Fowl et al. (021b), seeing Table We first control the overall computation the same for all experiments. Then we vary the number of sub-models to see its effect on our method.
In Table 4 we maintain our methods to have comparative computation with AdyPoison, which trains the protecting DNN for 40 epochs and craft perturbations by 250 steps 8 times (we modify to 4). In SEP and SEP-FA,
train for N 120 epochs and use 30 checkpoints to update samples for = 30 times, aligning the computation with AdvPoison. SEP-FA- VR with inner update M = 15. alter the number of checkpoints the default setting for all experiments as in Sec. [4. We use ResNet 8 as fg on CIFAR-10 10 dataset here. In the conventional ensemble, 30 DNNs with 5 architectures trained with 6 seeds.
15 that the overall computation is the same, which is also
As shown in Table 4 SEP is able to halve the accuracy of AdyPoison within the same computation budget, indicating that knowledge of multiple models is much more important than additional update
7

---

Published as a conference paper at ICLR 2023
steps. In comparison with the conventional ensemble, which requires x training computation, SEP performs just a little worse. Moreover, equipped with FA, which consumes no additional calculation, efficient SEP-FA could be effective the conventional ensemble. With VR, SEP-FA-VE FA-VR is stably better, and reduces the accuracy from 45.10% to 17.47% on average.
Table 4: Ablation study on Self-Ensembie Protection, Feature Alignment, and Variance Reduction.
Method
Train (Epoch)
Crafting (Step)
CIFAR-I0 Model Testing Accuracy (L)
(c = 2/255)
RN18
SENet18
VGG16
DN121
GoogLeNet
AdvPoison
40
1000
41.35
40.54
52.22
43,28
48.04
Ensemble
30x 120
30x30
16.32
16.91
29.19
16.74
20.34
SEP
120
30x30
19.92
17.81
28.12
18.07
22.48
SEP-FA
120
30x30
16.91
17.01
26.82
16.88
21.15
SEP-FA-VR (ours)
120
30x(15+15)
14.68
15.93
23.66
15.02
17.99
90
80
70
None (Test)
None (Train)
Accuracy
60
AdvPoison (Test)
AdvPoison (Train)
SEP (Test)
SEP (Train)
50
SEP+FA (Test)
SEP+FA (Train)
SEP VR (Test)
SEP+FA+ VR (Train)
30
20
10
80
Epoch
Figure 3: The accuracy trend of training CIFAR-10 ResNet18 using different data protection methods (s = 2/255, None means no protection). All methods have high training accuracy v.s. low testing accuracy, but SEP, equipped with FA and VR, reduces model's generalization effectively,
None (Irain)
90
(Valid)
80
None (Test)
70
Linfty (Train)
Accuracy
60
Linfty (Valid)
Linfty ('lest)
50
(Train)
L2 (Valid)
20
Linfly-L2 (Train)
Linfty+L2 (Valid)
10
Linfty (lest)
0
10
20
30
40
60
80
100
110
120
Epoch
Figure 4: The validation (using 2500 samples separated from training data) performance different types of perturbations by our method (CIFAR-10. lo 8//255, l = 1, fp fa ResNet18).
We illustrate the training process of Table experiments in Fig. Compared to the training on clean data (purple line), data protection methods accelerate the model's convergence on training data, but the DNN's testing accuracy would suddenly drop at the initial stage of training. After the learning rate decay at epoch 75, the protection performance of different methods could be clearly observed. SEP accounts for the majority of performance improvements, and FA and VR could also further decrease the accuracy, making our method finally outperform the conventional inefficient ensemble. We also show the validation accuracy (on unlearned protective examples) of different perturbations
8

---

Published as a conference paper at ICLR 2023
in Fig. where it is obvious that early stopping could not be good defense because validation accuracy is mostly close to training accuracy. However, a huge and unusual gap between them may be signal for the existence of protective examples.
We also vary the number of intermediate checkpoints n used in self-ensemble to perform ablation study under different computation budgets. We set hyper-parameters in self-ensemble, and plot the results in Fig. [ Similar results could seen, i... FA, and VR could stably contribute the performance. We also discover that although n is increasing exponentially, the resulting performance increase would not be that significant as n is large. Most prominently, raising n from 30 to 120 is not bound to yield better results, meaning that the performance would saturate around = 30, and it would be unnecessary to use all checkpoints.
10, 30, 120 without changing other
ResNet18 Accuracy
SENet18 Accuracy
VGG16 Accuracy
DenseNet121 Accuracy 45
GoogLeNet Accuracy
45
45
45
SEP
SEP-FA
40
40
40
40
40
SEP-FA-VR
SEP-FA-VR
SEP-FA-VR
SEP-FA-VR
35
35
35
35
30
30
30
30
30
20
20
20
20
20
SEP-FA
SEP-FA-VR
15
15
15
15
102
102
10a
10
10a
Ens
# Ens
Ens
# Ens
Ens
Figure 5: Ablation study on FA and VR with a different number of sub-models in self-ensemble. Results are produced on CIFAR-10 models with the bound of protective perturbations 2/255..
DISCUSSION AND CONCLUSION
In this paper, we propose that data protection should target the vulnerability of DNN training, which we successfully depict as the examples never classified correctly in training. Such examples could be easily calculated by model checkpoints, which are found to have surprisingly diverse gradients. By self-ensemble, effective ensemble performance could be achieved by the computation of training one model, and we could also take advantage of the diverse features from checkpoints further boost performance by the novel feature alignment loss. Our method exceeds current baselines significantly, reducing the appropriator model's ' accuracy from 45, 10% (best-known results) to 17.47% on average.
Our method could also serve potential benchmark to evaluate the DNN's learning process, e.g., how to prevent DNNs from learning non-robust features (shortcuts) instead semantic ones. And it would be interesting to study the poisoning task in self-supervised learning and stable diffusion. Since our method implemented as the targeted ensemble attack, is also applicable to non- classification tasks where the adversarial attack has also been developed and neural collapse also exists for pre-trained feature extractors.
ACKNOWLEDGEMENT
This work is partly supported by the National Natural Science Foundation China (61977046), Shanghai Science and Technology Program nology Major Project 2021SHZDZX0102, and National Science Foundation (CCF-1937500), The authors grateful Prof. Sijia Liu for his valuable discussions.
Shanghai Municipal Science and Tech-

---

Published as a conference paper at ICLR 2023
REFERENCES
Martin Abadi, Andy Chu, lan Goodfellow, Brendan McMahan, llya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS), pp. 308-318, 2016.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In IEEE Symposium Security and Privacy (S&P), pp. 39-57. IEEE, 2017.
Sizhe Chen, Zhengbao He, Chengjin Sun, and Xiaolin Huang. Universal adversarial attack on attention and the resulting dataset DAmageNet. In JEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), pp. 2188 2188-2197, 2022.
Terrance Veiiies and Graham W Taylor. Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv: /70X, 04552, 2017.
Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversarial attacks with momentum. In the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 9185-9193, 2018.
Minghong Fang, Neil Zhenqiang Gong, and Jia Liu. Influence function based data poisoning attacks to top-n recommender systems. In Proceedings of The Web Conference (WWW), pp. 3019-3025, 2020.
Feng, Qi-Zhi Cai, and Zui-Hua Zhou. Learning to confuse: generating training time adversarial data with auto-encoder. In Advances in Neural Information Processing Systems (NeurIPS), pp. 11971-11981, 2019.
Liam Fowl, Ping-yeh Chiang, Micah Goldblum, Jonas Geiping, Arpit Bansal, Wojtek Czaja, and Tom Goldstein. Preventing unauthorized use of proprietary data: Poisoning for secure dataset release. arXiv preprint Ttt::21030026833 2021a.
Liam H Fowl, Micah Goldblum, Ping-yeh Chiang, Jonas Geiping, Wojeiech Czaja, and Tom Goldstein. Adversarial examples make strong poisons. In Advances in Neural Information Processing Systems (NeurIPS), pp. 30339-30351, 2021b.
Shaopeng Fu, Fengxiang He, Yang Liu, Li Shen, and Dacheng Tao. Robust unlearnable examples: Protecting data privacy against adversarial learning. In International Conference on Learning Representations (ICLR), 2021.
Jonas Geiping. Liam H Fowl, W Ronny Huang, Wojeiech Czaja, Gavin Taylor, Michael Moeller, and Tom Goldstein. Witches' brew: Industrial scale data poisoning via gradient matching. In International Conference on Learning Representations (ICLR), 2020.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations (ICLR), 2015.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In the JEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016.
Jie Hu, Li Shen, and Gang Sun. Squeezz-andd-exiiaatin networks. In the IEEE/CVF Conference on Computer Vision and Patlern Recognition (CVPR), pp. 7132-7141, 2018
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4700-4708, 2017.
Hanxun Huang, Xingjun Ma. Sarah Monazam Erfani, James Bailey, and Yisen Wang. Unleamn- able examples: Making personal data unexploitable. In International Conference on Learning Representations (ICLR), 2020a.
W Ronny Huang, Jonas Geiping, Liam Fowl, Gavin Taylor, and Tom Goldstein. Metapoison: Practical general- general-purpose purpose Jabell data poisoning. In Advances in Neural Information Processing Systems (NeurIPS), pp. 2080-12091, 2020b.
10

---

Published as a conference paper at ICLR 2023
Rie Johnson and Tong Zhang. Accelerating stochastic gradient descent using predictive variance reduction. Advances in Neural Information Processing Systems (NeurIPS), pp. 315-323. 2013.
Pang Wei Koh and Perey Liang. Understanding black-box predictions via influence functions. In International Conference on Machine Learning (ICML), pp. 1885-1894, 2017.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009,
Alex Krizhevsky, nya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolu- tional neural networks. Communications of the ACM (CACM), pp. 84-90, 2017.
Tao Li, Lei Tan, Zhehao Huang, Qinghua Tao, Yipeng Liu, and Xiaolin Huang. Low dimensional trajectory hypothesis is true: Dnns can be trained tiny subspaces. In IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), pp. 1-1, 2022.
Jiabin Liu, Zhiquan Qi, Bo Wang. YingJie Tian, and Yong Shi. Self-Ilp: Self-supervised learning from label proportions with self-ensemble. Pattern Recognition, pp. 108767, 2022.
Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song. Delving into transferable adversarial examples and black-box attacks. In International Conference on Learning Representations (ICLR). 2017.
Aleksander Madry. Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. International Conference on Learning Representations (ICLR), 2018.
Dhruv Mahajan, Ross Girshick, Vignesh Ramanathan, Kaiming He, Manohar Paluri, Yixuan Li, Ashwin Bharambe, and Laurens Van Der Maaten. Exploring the limits of weakly supervised pretraining. In European Conference on Computer Vision (ECCV). pp. 181-196, 2018.
Tianyu Pang, Kun Xu, Chao Du, Ning Chen, and Jun Zhu. Improving adversarial robustness via promoting ensemble diversity. In International Conference on Machine Learning (ICML., pp- 4970-4979, 2019.
Vardan Papyan, XY Han, and David L Donoho. Prevalence of neural collapse during the terminal phase of deep learning training. In Proceedings of the National Academy of Sciences (PNAS), pp. 14652-246633 2020.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems (NeurIPS), pp. 8024-8035, 2019.
Pedro Sandoval-Segura, Vasu Singla, Jonas Geiping, Micah Goldblum, Tom Goldstein, and David W Jacobs. Autoregressive perturbations for data poisoning. In Advances in Neural Information Processing Systems (NeurIPS). 2022.
Juncheng Shen, Xiaolei Zhu, and De Ma. Tensorelog: An imperceptible poisoning attack on deep neural network applications. In IEEE Access, pp. 415004 2019.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In International Conference on Learning Representations (ICLR), 2015.
David Stutz, Matthias Hein, and Bernt Schiele. Confidence-calibrate adversarial training: Generaliz- ing to unseen attacks. In International Conference on Machine Learning (ICML), pp. 9155-9166, 2020.
Chen Sun, Abhinav Shrivastava, Saurabh Singh, and Abhinav Gupta. Revisiting unreasonable effectiveness of data in deep learning era. In the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 843-852, 2017.
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. International Conference on Learning Representations (ICIR), 2014.
11

---

Published as a conference paper at ICLR 2023
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Du- mitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 1-9, 2015.
Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick Mc- Daniel. Ensemble adversarial training: Attacks and defenses. International Conference on Learning Representations (ICLR), 2018.
Hongjun Wang and Yisen Wang. Self-ensemble adversarial training for improved robustness. In International Conference on Learning Representations (ICLR), 2021.
Shutong Wu, Sizhe Chen, Cihang Xie, and Xiaolin Huang. One-pixel shortcut: on the learning preference of deep neural networks. In International Conference Learning Representations (ICLR), 2023.
Yifeng Xiong, Jiadong Lin, Min Zhang, John E Hoperoft, and Kun He. Stochastic variance re- duced ensemble adversarial attack for boosting the adversarial transferability. In the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pp. 14983-14992, 2022.
Huanrui Yang, Jingyang Zhang, Hongliang Dong, Nathan Inkawhich, Andrew Gardner, Andrew
Touchet, Wesley Wilkes, Heath Berry, and Hai Li. Dverge: diversifying vulnerabilities for enhanced robust generation of ensembles. pp. 5515,, 2020.
Zhuolin Yang, Linyi Li, Xiaojun Xu, Shiliang Zuo, Qian Chen, Pan Zhou, Benjamin Rubinstein, Ce Zhang, and Bo Li. TRS: Transferability reduced ensemble via promoting gradient diversity and model smoothness. In Advances in Neural Information Processing Systems (NeurIPS), pp.
17652-17655, 2021.
Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regularization strategy to train strong classifiers with localizable features. In the IEEE/CVF International Conference Computer Vision (ICCV), pp. 6023-6032, 2019.
Dinghuai Zhang, Tianyuan Zhang, Yiping Lu, Zhanxing Zhu, and Bin Dong- You only propagate once: Painless adversarial training using maximal principle. In Advances in Neural Information Processing Systems (NeurIPS), pp. 227-238. 2019.
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. In International Conference on Learning Representations (ICLR), 2018.
Yuan-Xina Xing Zhao, Yan-Ming Zhang, Ming Song, and Cheng-Lin Liu. Multi-view semi-supervised 3d
whole brain segmentation with a self enssmbns network. In International Conference on Medical Image Computing and Computer- Assiitted Intervention (MICCAI), pp. 256-265, 2019.
12


Reviews:
[{"review_id": "xrmQ-Pc3tjv", "reviewer": "Reviewer_JA31", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposed to utilize intermediate checkpoints from a single training process to protect data from unauthorized use. It also proposed a novel feature alignment (FA) technique that improves the accuracy of its proposed self-ensemble protection (SEP) method. FA uses an existing theory called neural collapse to align the last layer feature of a sample for each checkpoint to the feature mean of the target class samples. It has minor improvement in performance. But the method is simple which will make it useful. ", "summary_of_the_review": "The paper attempts to solve a compelling problem and proposes a simple solution. However, it could not support empirically enough. As mentioned in the weakness section, it needs a little more work and rewriting.\n\n***\nThe authors have clarified the raised doubts, updating the ratings accordingly. \n***", "strength_and_weaknesses": "**Strength: **\n\n- The objective is straightforward and the problem of data protection is exciting and vital. \n\n- Improving performance with intermediate checkpoints from a single training is an excellent idea to save time and resources. \n\n**Weaknesses:**\n\n- The feature alignment section could be explained a little better. Especially the neural collapse theory should be explained a little more since it is the primary tool used to develop FA.\n\n- The experiments are limited. More results on different datasets comparing different models should have been shown. Only one model and dataset are tested while the proposed method is compared with the previous methods in Table 1.\n\n- It is unclear what model and dataset are used to produce Table 3.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "ts3sCuhF8x9", "reviewer": "Reviewer_bFJ9", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper is about perturbing images such that it is not possible to use them for training a model which has good performance. The authors propose to build upon existing works which use adversarial perturbations to corrupt the training set. As contributions, this work proposes to ensemble the perturbation gradients coming different model snapshots of a same training deep network. The perturbation gradients come from a feature alignment loss such that perturbed samples when passed to the DNN result in feature maps which are close to the mean feature of other classes.", "summary_of_the_review": "The paper is enjoyable to read but it only marginally improves over existing methods both technically and experimentally. ", "strength_and_weaknesses": "Strength:\n1) The paper is clear and very well written.\n2) Lots of experiments to support the paper claims with interesting ablation studies.\n\nWeakness and suggestions:\n1) Adversarial training easily circumvents the data corruption. The proposed method suffers from the same drawback of other existing methods.\n2) L2 perturbation(not only L-inf) could complete the study. Maybe mixed perturbations could be helpful against adversarial training.\n3) What about exponential moving average as a comparison to the ensemble of snapshots? It would be a way to avoid storing all these snapshots.\n4) Typos in the abstract and other places for the perturbation bounds: with l_inf=8 instead of L_inf=8/255 for examples.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "LVigsmflN0", "reviewer": "Reviewer_FTbn", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a simple new ensemble method for protecting data from someone training on it.  The work shows the empirical success of their method on multiple datasets and compared to strong baselines like adversarial poisons.", "summary_of_the_review": "In general, while the writing could use some work and there is an additional baseline that would be nice to include, I currently lean towards acceptance.  The paper provides significant empirical contribution and extensive testing.", "strength_and_weaknesses": "The main strength of the paper is in the empirical success compared to strong baselines and defenses.\n\nIn Figure 3, the peak test accuracy of the proposed method is just as high as for adversarial poisons.  Is there a way to reduce this peak?  Otherwise, early stopping might be a highly effective defense.\n\nIt might be worth considering the AR poisons of “Autoregressive Perturbations for Data Poisoning” as they are a recent stronger baseline than adversarial poisons.\n\nHow did you tune the attack hyperparameters (e.g. step size) of adversarial poisons since you are using a smaller perturbation radius than they used in the experiments in their paper?\n\nIn general, the writing is a bit messy and hard to follow, but the overall structure and content is good, and I was still able to understand the writing nonetheless.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster