Original: D EEP P ARTITION A GGREGATION : P ROVABLE D EFENSES AGAINST G ENERAL P OISONING A
Revised: D EEP P ARTITION A GGREGATION : P ROVABLE D EFENSES AGAINST G ENERAL P OISONING A

Original: TTACKS
Revised: TTACKS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Adversarial poisoning attacks distort training data in order to corrupt the test-time behavior of a classiﬁer.
Revised: Adversarial poisoning attacks distort training data in order to corrupt the test-time behavior of a classiﬁer.

Original: A provable defense provides a certiﬁcate for each test sample, which is a lower bound on the magnitude of any adversarial distortion of the training set that can corrupt the test sample’s classiﬁcation.
Revised: A provable defense provides a certiﬁcate for each test sample, which is a lower bound on the magnitude of any adversarial distortion of the training set that can corrupt the test sample’s classiﬁcation.

Original: We propose two novel provable defenses against poisoning attacks: (i) Deep Partition Aggregation (DPA), a certiﬁed defense against a general poisoning threat model, deﬁned as the insertion or deletion of a bounded number of samples to the training set — by implication, this threat model also includes arbitrary distortions to a bounded number of images and/or labels; and (ii) Semi-Supervised DPA (SS-DPA), a certiﬁed defense against label-ﬂipping poisoning attacks.
Revised: We propose two novel provable defenses against poisoning attacks: (i) Deep Partition Aggregation (DPA), a certiﬁed defense against a general poisoning threat model, deﬁned as the insertion or deletion of a bounded number of samples to the training set — by implication, this threat model also includes arbitrary distortions to a bounded number of images and/or labels; and (ii) Semi-Supervised DPA (SS-DPA), a certiﬁed defense against label-ﬂipping poisoning attacks.

Original: DPA is an ensemble method where base models are trained on partitions of the training set determined by a hash function.
Revised: DPA is an ensemble method where base models are trained on partitions of the training set determined by a hash function.

Original: DPA is related to both subset aggregation , a well-studied ensemble method in classical machine learning, as well as to randomized smoothing , a popular provable defense against evasion (inference) attacks.
Revised: DPA is related to both subset aggregation , a well-studied ensemble method in classical machine learning, as well as to randomized smoothing , a popular provable defense against evasion (inference) attacks.

Original: Our defense against label-ﬂipping poison attacks, SS-DPA, uses a semi-supervised learning algorithm as its base classiﬁer model: each base classiﬁer is trained using the entire unlabeled training set in addition to the labels for a partition.
Revised: Our defense against label-ﬂipping poison attacks, SS-DPA, uses a semi-supervised learning algorithm as its base classiﬁer model: each base classiﬁer is trained using the entire unlabeled training set in addition to the labels for a partition.

Original: SS-DPA signiﬁcantly outperforms the existing certiﬁed defense for label-ﬂipping attacks (Rosenfeld et al., 2020) on both MNIST and CIFAR-10: provably tolerating, for at least half of test images, over 600 label ﬂips (vs. < 200 label ﬂips) on MNIST and over 300 label ﬂips (vs. 175 label ﬂips) on CIFAR-10.
Revised: SS-DPA signiﬁcantly outperforms the existing certiﬁed defense for label-ﬂipping attacks (Rosenfeld et al., 2020) on both MNIST and CIFAR-10: provably tolerating, for at least half of test images, over 600 label ﬂips (vs. < 200 label ﬂips) on MNIST and over 300 label ﬂips (vs. 175 label ﬂips) on CIFAR-10.

Original: Against general poisoning attacks where no prior certiﬁed defenses exists, DPA
Revised: Against general poisoning attacks where no prior certiﬁed defenses exists, DPA

Original: can certify ≥ 50% of test images against over 500 poison image insertions on MNIST, and nine insertions on CIFAR-10.
Revised: can certify ≥ 50% of test images against over 500 poison image insertions on MNIST, and nine insertions on CIFAR-10.

Original: These results establish new state-of-the-art provable defenses against general and label-ﬂipping poison attacks.
Revised: These results establish new state-of-the-art provable defenses against general and label-ﬂipping poison attacks.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Adversarial poisoning attacks are an important vulnerability in machine learning systems.
Revised: Adversarial poisoning attacks are an important vulnerability in machine learning systems.

Original: In these attacks, an adversary can manipulate the training data of a classiﬁer, in order to change the classiﬁcations of speciﬁc inputs at test time.
Revised: In these attacks, an adversary can manipulate the training data of a classiﬁer, in order to change the classiﬁcations of speciﬁc inputs at test time.

Original: Several poisoning threat models have been studied in the literature, including threat models where the adversary may insert new poison samples (Chen et al., 2017), manipulate the training labels (Xiao et al., 2012; Rosenfeld et al., 2020), or manipulate the training sample values (Biggio et al., 2012; Shafahi et al., 2018).
Revised: Several poisoning threat models have been studied in the literature, including threat models where the adversary may insert new poison samples (Chen et al., 2017), manipulate the training labels (Xiao et al., 2012; Rosenfeld et al., 2020), or manipulate the training sample values (Biggio et al., 2012; Shafahi et al., 2018).

Original: A certiﬁed defense against a poisoning attack provides a certiﬁcate for each test sample, which is a guaranteed lower bound on the magnitude of any adversarial distortion of the training set that can corrupt the test sample’s classiﬁcation.
Revised: A certiﬁed defense against a poisoning attack provides a certiﬁcate for each test sample, which is a guaranteed lower bound on the magnitude of any adversarial distortion of the training set that can corrupt the test sample’s classiﬁcation.

Original: In this work, we propose certiﬁed defenses against two types of poisoning attacks:
Revised: In this work, we propose certiﬁed defenses against two types of poisoning attacks:

Original: General poisoning attacks: In this threat model, the attacker can insert or remove a bounded number of samples from the training set.
Revised: General poisoning attacks: In this threat model, the attacker can insert or remove a bounded number of samples from the training set.

Original: In particular, the attack magnitude ρ is deﬁned as the cardinality of the symmetric difference between the clean and poisoned training sets.
Revised: In particular, the attack magnitude ρ is deﬁned as the cardinality of the symmetric difference between the clean and poisoned training sets.

Original: This threat model also includes any distortion to an image and/or label in the training set — a distortion of a training image is simply the removal of the original image followed by the insertion of the distorted image.
Revised: This threat model also includes any distortion to an sample and/or label in the training set — a distortion of a training sample is simply the removal of the original sample followed by the insertion of the distorted sample.

Original: (Note that an image distortion or label ﬂip therefore increases the symmetric difference attack magnitude by two.)[Figure]
Revised: (Note that a sample distortion or label ﬂip therefore increases the symmetric difference attack magnitude by two.)

Original: Label-ﬂipping poisoning attacks: In this threat model, the adversary changes only the label for ρ out of m training samples.
Revised: Label-ﬂipping poisoning attacks: In this threat model, the adversary changes only the label for ρ out of m training samples.

Original: Rosenfeld et al.
Revised: Rosenfeld et al.

Original: (2020) has recently provided a certiﬁed defense for this threat model, which we improve upon.
Revised: (2020) has recently provided a certiﬁed defense for this threat model, which we improve upon.

Original: In the last couple of years, certiﬁed defenses have been extensively studied for evasion attacks, where the adversary manipulates the test images, rather than the training data (e.g. Wong & Kolter (2018); Gowal et al.
Revised: In the last couple of years, certiﬁed defenses have been extensively studied for evasion attacks, where the adversary manipulates the test samples, rather than the training data (e.g. Wong & Kolter (2018); Gowal et al.

Original: Lecuyer et al.
Revised: Lecuyer et al.

Original: ; Li et al.
Revised: ; Li et al.

Original: ; Salman et al.
Revised: ; Salman et al.

Original: ; Levine &
Revised: ; Levine &

Original: Feizi (2020a; 2019);
Revised: Feizi (2020a; 2019);

Original: Cohen et al.
Revised: Cohen et al.

Original: (2019), etc.)
Revised: (2019), etc.)

Original: In the evasion case, a certiﬁcate is a lower bound on the distance from the image to the classiﬁer’s decision boundary: this guarantees that the image’s classiﬁcation remains unchanged under adversarial distortions up to the certiﬁed magnitude.
Revised: In the evasion case, a certiﬁcate is a lower bound on the distance from the sample to the classiﬁer’s decision boundary: this guarantees that the sample’s classiﬁcation remains unchanged under adversarial distortions up to the certiﬁed magnitude.

Original: Rosenfeld et al.
Revised: Rosenfeld et al.

Original: (2020) provides an analogous certiﬁcate for label-ﬂipping poisoning attacks: for an input image x , the certiﬁcate of x is a lower bound on the number of labels in the training set that would have to change in order to change the classiﬁcation of x . 1 Rosenfeld et al.
Revised: (2020) provides an analogous certiﬁcate for label-ﬂipping poisoning attacks: for an input sample x , the certiﬁcate of x is a lower bound on the number of labels in the training set that would have to change in order to change the classiﬁcation of x . 1 Rosenfeld et al.

Original: (2020)’s method is an adaptation of a certiﬁed defense for sparse ( L 0 ) evasion attacks proposed by Lee et al.
Revised: (2020)’s method is an adaptation of a certiﬁed defense for sparse ( L 0 ) evasion attacks proposed by Lee et al.

Original: The adapted method for label-ﬂipping attacks proposed by Rosenfeldet al.
Revised: The adapted method for label-ﬂipping attacks proposed by Rosenfeld

Original: (2020) is equivalent to randomly ﬂipping each training label with ﬁxed probability and taking a consensus result.
Revised: (2020) is equivalent to randomly ﬂipping each training label with ﬁxed probability and taking a consensus result.

Original: If implemented directly, this would require one to train a large ensemble of classiﬁers on different noisy versions of the training data.
Revised: If implemented directly, this would require one to train a large ensemble of classiﬁers on different noisy versions of the training data.

Original: However, instead of actually doing this, Rosenfeld et al.
Revised: However, instead of actually doing this, Rosenfeld et al.

Original: (2020) focuses only on linear classiﬁers and is therefore able to analytically calculate the expected result.
Revised: (2020) focuses only on linear classiﬁers and is therefore able to analytically calculate the expected result.

Original: This gives deterministic, rather than probabilistic, certiﬁcates.
Revised: This gives deterministic, rather than probabilistic, certiﬁcates.

Original: Further, because
Revised: Further, because

Original: Rosenfeld et al.
Revised: Rosenfeld et al.

Original: (2020) considers a threat model where only labels are modiﬁed, they are able to train an unsupervised nonlinear feature extractor on the (unlabeled) training data before applying their technique, in order to learn more complex features.
Revised: (2020) considers a threat model where only labels are modiﬁed, they are able to train an unsupervised nonlinear feature extractor on the (unlabeled) training data before applying their technique, in order to learn more complex features.

Original: Inspired by an improved provable defense against L 0 evasion attacks (Levine & Feizi, 2020a), in this paper, we develop certiﬁable defenses against general and label-ﬂipping poisoning attacks that signiﬁcantly outperform the current state-of-the-art certiﬁable defenses.
Revised: Inspired by an improved provable defense against L 0 evasion attacks (Levine & Feizi, 2020a), in this paper, we develop certiﬁable defenses against general and label-ﬂipping poisoning attacks that signiﬁcantly outperform the current state-of-the-art certiﬁable defenses.

Original: In particular, we develop a certiﬁable defense against general poisoning attacks called Deep Partition Aggregation (DPA) which is based on partitioning the training set into k partitions, with the partition assignment for a training sample determined by a hash function of the sample.
Revised: In particular, we develop a certiﬁable defense against general poisoning attacks called Deep Partition Aggregation (DPA) which is based on partitioning the training set into k partitions, with the partition assignment for a training sample determined by a hash function of the sample.

Original: The hash function can be any deterministic function that maps a training sample t to a partition assignment: the only requirement is that the hash value depends only on the value of the training sample t itself, so that neither poisoning other samples, nor changing the total number of samples, nor reordering the samples can change the partition that t is assigned to.
Revised: The hash function can be any deterministic function that maps a training sample t to a partition assignment: the only requirement is that the hash value depends only on the value of the training sample t itself, so that neither poisoning other samples, nor changing the total number of samples, nor reordering the samples can change the partition that t is assigned to.

Original: We then train k base classiﬁers separately, one on each partition.
Revised: We then train k base classiﬁers separately, one on each partition.

Original: At the test time, we evaluate each of the base classiﬁers on the test image x and return the plurality classiﬁcation c as the ﬁnal result.
Revised: At the test time, we evaluate each of the base classiﬁers on the test sample x and return the plurality classiﬁcation c as the ﬁnal result.

Original: The key insight is that removing a training sample, or adding a new sample, will only change the contents of one partition, and therefore will only affect the classiﬁcation of one of the k base classiﬁers.
Revised: The key insight is that removing a training sample, or adding a new sample, will only change the contents of one partition, and therefore will only affect the classiﬁcation of one of the k base classiﬁers.

Original: This immediately leads to robustness certiﬁcations against general poisoning attacks which, to the best of our knowledge, is the ﬁrst one of this kind.
Revised: This immediately leads to robustness certiﬁcations against general poisoning attacks which, to the best of our knowledge, is the ﬁrst one of this kind.

Original: If the adversary is restricted to ﬂipping labels only (as in Rosenfeld et al.
Revised: If the adversary is restricted to ﬂipping labels only (as in Rosenfeld et al.

Original: , we can achieve even larger certiﬁcates through a modiﬁed technique.
Revised: , we can achieve even larger certiﬁcates through a modiﬁed technique.

Original: In this setting, the unlabeled data is trustworthy: each base classiﬁer in the ensemble can then make use of the entire training set without labels, but only has access to the labels in its own partition.
Revised: In this setting, the unlabeled data is trustworthy: each base classiﬁer in the ensemble can then make use of the entire training set without labels, but only has access to the labels in its own partition.

Original: Thus, each base classiﬁer can be trained as if the entire dataset is available as unlabeled data, but only a very small number of labels are available.
Revised: Thus, each base classiﬁer can be trained as if the entire dataset is available as unlabeled data, but only a very small number of labels are available.

Original: This is precisely the problem statement of semi-supervised learning (Verma et al., 2019;
Revised: This is precisely the problem statement of semi-supervised learning (Verma et al., 2019;

Original: Luo et al., 2018; Laine & Aila, 2017; Kingma et al., 2014; Gidaris et al., 2018).
Revised: Luo et al., 2018; Laine & Aila, 2017; Kingma et al., 2014; Gidaris et al., 2018).

Original: We can then leverage these existing semi-supervised learning techniques directly to improve the accuracies of the base classiﬁers in DPA.
Revised: We can then leverage these existing semi-supervised learning techniques directly to improve the accuracies of the base classiﬁers in DPA.

Original: Furthermore, we can ensure that a particular image is assigned to the same partition regardless of label, so that only one partition is affected by a label ﬂip (rather than possibly two).
Revised: Furthermore, we can ensure that a particular (unlabeled) sample is assigned to the same partition regardless of label, so that only one partition is affected by a label ﬂip (rather than possibly two).

Original: The resulting algorithm, Semi-Supervised Deep Partition Aggregation (SS-DPA) yields substantially increased certiﬁed accuracy against label-ﬂipping attacks, compared to DPA alone and compared to the current state-of-the-art.
Revised: The resulting algorithm, Semi-Supervised Deep Partition Aggregation (SS-DPA) yields substantially increased certiﬁed accuracy against label-ﬂipping attacks, compared to DPA alone and compared to the current state-of-the-art.

Original: Furthermore, while our method is de-randomized (as Rosenfeld et al.
Revised: Furthermore, while our method is de-randomized (as Rosenfeld et al.

Original: (2020) is) and therefore yields deterministic certiﬁcates, our technique does not require that the classiﬁcation model be linear, allowing deep networks to be used.
Revised: (2020) is) and therefore yields deterministic certiﬁcates, our technique does not require that the classiﬁcation model be linear, allowing deep networks to be used.

Original: On MNIST, SS-DPA substantially outperforms the existing state of the art (Rosenfeld et al., 2020) in defending against label-ﬂip attacks: we certify at least half of images in the test set against attacks to over 600 (1.0%) of the labels in the training set, while still maintaining over 93% accuracy (See
Revised: On MNIST, SS-DPA substantially outperforms the existing state of the art (Rosenfeld et al., 2020) in defending against label-ﬂip attacks: we certify at least half of images in the test set against attacks to over 600 (1.0%) of the labels in the training set, while still maintaining over 93% accuracy (See

Original: Figure 1-a, and Table 1).
Revised: Figure 1, and Table 1).

Original: In comparison, Rosenfeld et al.
Revised: In comparison, Rosenfeld et al.

Original: (2020)’s method achieves less than 60% clean accuracy on MNIST, and most test images cannot be certiﬁed with the correct class against attacks of even 200 label ﬂips.
Revised: (2020)’s method achieves less than 60% clean accuracy on MNIST, and most test images cannot be certiﬁed with the correct class against attacks of even 200 label ﬂips.

Original: We are also the ﬁrst work to our knowledge to certify against general poisoning attacks, including insertions and deletions of new training images: in this domain, we can certify at least half of test images against attacks consisting of over 500 arbitrary training image insertions or deletions.
Revised: We are also the ﬁrst work to our knowledge to certify against general poisoning attacks, including insertions and deletions of new training images: in this domain, we can certify at least half of test images against attacks consisting of over 500 arbitrary training image insertions or deletions.

Original: On CIFAR-10, a substantially more difﬁcult classiﬁcation task, we can certify at least half of test images against label-ﬂipping attacks on over 300 labels using SS-DPA (versus 175 label-ﬂips for (Rosenfeld et al., 2020)), and can certify at least half of test images against general poisoning attacks of up to nine insertions or deletions using DPA. 
Revised: On CIFAR-10, a substantially more difﬁcult classiﬁcation task, we can certify at least half of test images against label-ﬂipping attacks on over 300 labels using SS-DPA (versus 175 label-ﬂips for (Rosenfeld et al., 2020)), and can certify at least half of test images against general poisoning attacks of up to nine insertions or deletions using DPA. To see how our method performs on datasets with larger numbers of classes, we also tested our methods on the German Trafﬁc Sign Recognition Benchmark (Stallkamp et al., 2012), a task with 43 classes and on average ≈ 1000 samples per class. Here, we are able to certify at least half of test images as robust to 177 label ﬂips, or 20 general poisoning attacks.

Original: These results establish new state-of-the-art in provable defenses against label-ﬂipping and general poisoning attacks.
Revised: These results establish new state-of-the-art in provable defenses against label-ﬂipping and general poisoning attacks.

Original:  R ELATED W ORKS
Revised:  R ELATED W ORKS

Original: Levine & Feizi (2020a) propose a randomized ablation technique to certiﬁably defend against sparse atatcks.
Revised: Levine & Feizi (2020a) propose a randomized ablation technique to certiﬁably defend against sparse atatcks.

Original: Their method ablates some pixels, replacing them with a null value.
Revised: Their method ablates some pixels, replacing them with a null value.

Original: Since it is possible for the base classiﬁer to distinguish exactly which pixels originate from x , this results in more accurate base classiﬁcations and therefore substantially greater certiﬁed robustness than Lee et al.
Revised: Since it is possible for the base classiﬁer to distinguish exactly which pixels originate from x , this results in more accurate base classiﬁcations and therefore substantially greater certiﬁed robustness than Lee et al.

Original: For example, on ImageNet, Lee et al.
Revised: For example, on ImageNet, Lee et al.

Original: (2019) certiﬁes the median test image against distortions of one pixel, while Levine & Feizi (2020a) certiﬁes against distortions of 16 pixels.
Revised: (2019) certiﬁes the median test image against distortions of one pixel, while Levine & Feizi (2020a) certiﬁes against distortions of 16 pixels.

Original: Our proposed method is related to classical ensemble approaches in machine learning, namely bootstrap aggregation and subset aggregation (Breiman, 1996; Buja & Stuetzle, 2006; B¨uhlmann, 2003;
Revised: Our proposed method is related to classical ensemble approaches in machine learning, namely bootstrap aggregation and subset aggregation (Breiman, 1996; Buja & Stuetzle, 2006; B¨uhlmann, 2003;

Original: Zaman & Hirose, 2009).
Revised: Zaman & Hirose, 2009).

Original: However, in these methods each base classiﬁer in the ensemble is trained on an independently sampled collection of points from the training set: multiple classiﬁers in the ensemble may be trained on (and therefore poisoned by)
Revised: However, in these methods each base classiﬁer in the ensemble is trained on an independently sampled collection of points from the training set: multiple classiﬁers in the ensemble may be trained on (and therefore poisoned by)

Original: the same sample point.
Revised: the same sample point.

Original: The purpose of these methods has typically been to improve generalization.
Revised: The purpose of these methods has typically been to improve generalization.

Original: Bootstrap aggregation has been proposed as an empirical defense against poisoning attacks (Biggio et al., 2011) as well as for evasion attacks (Smutz & Stavrou, 2016).
Revised: Bootstrap aggregation has been proposed as an empirical defense against poisoning attacks (Biggio et al., 2011) as well as for evasion attacks (Smutz & Stavrou, 2016).

Original: However, at the time of the initial distribution of this work, these techniques had not yet been used to provide certiﬁed robustness.
Revised: However, at the time of the initial distribution of this work, these techniques had not yet been used to provide certiﬁed robustness.

Original: Our unique partition aggregation variant provides deterministic robustness certiﬁcates against poisoning attacks.
Revised: Our unique partition aggregation variant provides deterministic robustness certiﬁcates against poisoning attacks.

Original: See Appendix D for further discussion.
Revised: See Appendix D for further discussion.

Original: Weber et al.
Revised: Weber et al.

Original: (2020) have recently proposed a different randomized-smoothing based defense against poisoning attacks by directly applying Cohen et al.
Revised: (2020) have recently proposed a different randomized-smoothing based defense against poisoning attacks by directly applying Cohen et al.

Original: (2019)’s smoothing L 2 evasion defense to the poisoning domain.
Revised: (2019)’s smoothing L 2 evasion defense to the poisoning domain.

Original: The proposed technique can only certify for clean-label attacks (where only the existing images in the dataset are modiﬁed, and not their labels), and the certiﬁcate guarantees robustness only to bounded L 2 distortions of the training data, where the L 2 norm of the distortion is calculated across all pixels in the entire training set.
Revised: The proposed technique can only certify for clean-label attacks (where only the existing samples in the dataset are modiﬁed, and not their labels), and the certiﬁcate guarantees robustness only to bounded L 2 distortions of the training data, where the L 2 norm of the distortion is calculated across all pixels in the entire training set.

Original: Due to well-known limitations of dimensional scaling for smoothing-based robustness certiﬁcates (Yang et al., 2020; Kumar et al., 2020; Blum et al., 2020), this yields certiﬁcates to only very small distortions of the training data.
Revised: Due to well-known limitations of dimensional scaling for smoothing-based robustness certiﬁcates (Yang et al., 2020; Kumar et al., 2020; Blum et al., 2020), this yields certiﬁcates to only very small distortions of the training data.

Original: (For binary
Revised: (For binary

Original: MNIST [13,007 images], the maximum reported L 2 certiﬁcate is 2 pixels.)
Revised: MNIST [13,007 images], the maximum reported L 2 certiﬁcate is 2 pixels.)

Original: Additionally, when using deep classiﬁers, Weber et al.
Revised: Additionally, when using deep classiﬁers, Weber et al.

Original: (2020) proposes a randomized certiﬁcate, rather than a deterministic one, with a failure probability that decreases to zero only as the number of trained classiﬁers in an ensemble approaches inﬁnity.
Revised: (2020) proposes a randomized certiﬁcate, rather than a deterministic one, with a failure probability that decreases to zero only as the number of trained classiﬁers in an ensemble approaches inﬁnity.

Original: Moreover, in Weber et al.
Revised: Moreover, in Weber et al.

Original: (2020), unlike in our method, each classiﬁer in the ensemble must be trained on a noisy version of the entire dataset.
Revised: (2020), unlike in our method, each classiﬁer in the ensemble must be trained on a noisy version of the entire dataset.

Original: These issues hinder Weber et al.
Revised: These issues hinder Weber et al.

Original: (2020)’s method to be an effective scheme for certiﬁed robustness against poisoning attacks.
Revised: (2020)’s method to be an effective scheme for certiﬁed robustness against poisoning attacks.

Original: P ROPOSED M ETHODS 3. N OTATION
Revised: After the initial distribution of this work, a recent revision of Rosenfeld et al. (2020) has suggested using randomised smoothing techniques on training samples, rather than just training labels, as a general approach to poisoning defense. Both this work and Weber et al. (2020) could be considered as implementations of this idea, although this generalized proposal in Rosenfeld et al.

Revised: (2020) does not include a derandomization scheme (unlike Rosenfeld et al.

Revised: ’s

Revised: proposed derandomized defense against label-ﬂipping attacks) .

Revised: Other prior works have provided distributional , rather than pointwise guarantees against poisoning attacks.

Revised: In these works, there is a (high-probability) guarantee that the classiﬁer will achieve a certain level of average overall accuracy on test data, under the assumption that the test data and clean (pre-poisoning) training data are drawn from the same distribution.

Revised: These works do not provide any guarantees that apply to speciﬁc test samples, however.

Revised: As mentioned above, Steinhardt et al.

Revised: (2017) provides such a distributional guarantee, speciﬁcally for a threat model of addition of poison samples.

Revised: Other such works include Sloan (1995), which considers label-ﬂipping attacks and determines conditions under which PAC-learning is possible in the presence of such attacks, and

Revised: Bshouty et al.

Revised: (2002) provides similar guarantees for replacement of samples.

Revised: Other works (Diakonikolas et al., 2016; Lai et al., 2016) provide distributional guarantees for unsupervised learning under poisoning attacks.

Revised: Mahloujifar et al.

Revised: (2019) proposes provably effective poisoning attacks with high-probability pointwise guarantees of effectiveness on test samples.

Revised: However, that work relies on properties of the distribution that the training set is drawn from.

Revised: Diakonikolas et al.

Revised: (2019) provide a robust training algorithm which provably approximates the clean trained model despite poisoning (rather than the behavior at a certain test point): this result also makes assumptions about the distribution of the clean training data.

Revised:  P ROPOSED M ETHODS 3. N OTATION

Original: Let S be the space of all possible unlabeled samples (i.e., the set of all possible images).
Revised: Let S be the space of all possible unlabeled samples (i.e., the set of all possible images).

Original: We assume that it is possible to sort elements of S in a deterministic, unambiguous way.
Revised: We assume that it is possible to sort elements of S in a deterministic, unambiguous way.

Original: In particular,  we can sort images lexicographically by pixel values: in general, any data that can be represented digitally will be sortable.
Revised: In particular, in the case of image data, we can sort images lexicographically by pixel values: in general, any data that can be represented digitally will be sortable.

Original: We represent labels as integers, so that the set of all possible labeled samples is 
Revised: We represent labels as integers, so that the set of all possible labeled samples is S L :

Original: S L := { ( x , c ) | x ∈ S , c ∈ N } .
Revised: = { ( x , c ) | x ∈ S , c ∈ N } .

Original: A training set for a classiﬁer is then represented as T ∈ P ( S L ) , where P ( S L ) is the power set of S L .
Revised: A training set for a classiﬁer is then represented as T ∈ P ( S L ) , where P ( S L ) is the power set of S L .

Original: For t ∈ S L , we let sample ( t ) ∈ S refer to the (unlabeled) sample, and label ( t ) ∈ N refer to the label.
Revised: For t ∈ S L , we let sample ( t ) ∈ S refer to the (unlabeled) sample, and label ( t ) ∈ N refer to the label.

Original: For a set of samples T ∈ P ( S L ) , we let samples ( T ) ∈ P ( S ) refer to the set of unique unlabeled samples which occur in T .
Revised: For a set of samples T ∈ P ( S L ) , we let samples ( T ) ∈ P ( S ) refer to the set of unique unlabeled samples which occur in T .

Original: A classiﬁer model is deﬁned as a deterministic function from both the training set and the sample to be classiﬁed to a label, i.e. f : P ( S L ) × S → N .
Revised: A classiﬁer model is deﬁned as a deterministic function from both the training set and the sample to be classiﬁed to a label, i.e. f : P ( S L ) × S → N .

Original: We will use f ( · ) to represent a base classiﬁer model (i.e., a neural network), and g ( · ) to refer to a robust classiﬁer (using DPA or SS-DPA).
Revised: We will use f ( · ) to represent a base classiﬁer model (i.e., a neural network), and g ( · ) to refer to a robust classiﬁer (using DPA or SS-DPA).

Original: A (cid:9) B represents the set symmetric difference between A and B : A (cid:9) B
Revised: A (cid:9) B represents the set symmetric difference between A and B : A (cid:9) B

Original: =
Revised: =

Original: ( A \ B ) ∪ ( B \ A ) .
Revised: ( A \ B ) ∪ ( B \ A ) .

Original: The number of elements in A is | A | , [ n ] is the set of integers 1 through n , and
Revised: The number of elements in A is | A | , [ n ] is the set of integers 1 through n , and

Original: (cid:98) z (cid:99) is the largest integer less than or equal to z .
Revised: (cid:98) z (cid:99) is the largest integer less than or equal to z .

Original: 1 represents the indicator function: 1 Prop = 1 if Prop is true; 1 Prop =otherwise.
Revised: 1 represents the indicator function: 1 Prop = 1 if Prop is true; 1 Prop =otherwise.

Original: For a set A of sortable elements, we deﬁne Sort ( A ) as the sorted list of elements.
Revised: For a set A of sortable elements, we deﬁne Sort ( A ) as the sorted list of elements.

Original: For a list L of unique elements, for l ∈ L , we will deﬁne index ( L, l ) as the index of l in the list L 
Revised: For a list L of unique elements, for l ∈ L , we will deﬁne index ( L, l ) as the index of l in the list L .

Original: . 3. DPA
Revised: 3. DPA

Original: The Deep Partition Aggregation (DPA) algorithm requires a base classiﬁer model f : P ( S L ) ×S → N , a training set T ∈ P ( S L ) , a deterministic hash function h : S L → N , and a hyperparameter k ∈ N indicating the number of base classiﬁers which will be used in the ensemble.
Revised: The Deep Partition Aggregation (DPA) algorithm requires a base classiﬁer model f : P ( S L ) ×S → N , a training set T ∈ P ( S L ) , a deterministic hash function h : S L → N , and a hyperparameter k ∈ N indicating the number of base classiﬁers which will be used in the ensemble.

Original: At the training time, the algorithm ﬁrst uses the hash function h to deﬁne partitions P 1 , ..., P k ⊆ T of the training set, as follows:
Revised: At the training time, the algorithm ﬁrst uses the hash function h to deﬁne partitions P 1 , ..., P k ⊆ T of the training set, as follows:

Original: [Equation]
Revised: [Equation]

Original: The hash function h can be any deterministic function from S L to N : however, it is preferable that the partitions are roughly equal in size.
Revised: The hash function h can be any deterministic function from S L to N : however, it is preferable that the partitions are roughly equal in size.

Original: Therefore we should choose an h which maps images to a domain of integers signiﬁcantly larger than k , in a way such that h ( . )
Revised: Therefore we should choose an h which maps samples to a domain of integers signiﬁcantly larger than k , in a way such that h ( . )

Original: (mod k ) will be roughly uniform over [ k ] .
Revised: (mod k ) will be roughly uniform over [ k ] .

Original: In practice,  we let h ( t ) be the sum of the pixel values in the image t .
Revised: In practice, for image data, we let h ( t ) be the sum of the pixel values in the image t .

Original: Base classiﬁers are then trained on each partition: we deﬁne trained base classiﬁers f
Revised: Base classiﬁers are then trained on each partition: we deﬁne trained base classiﬁers f

Original: i : S → N as:
Revised: i : S → N as:

Original: [Equation]
Revised: [Equation]

Original: Finally, at the inference time, we evaluate the input on each base classiﬁcation, and then count the number of classiﬁers which return each class:
Revised: Finally, at the inference time, we evaluate the input on each base classiﬁcation, and then count the number of classiﬁers which return each class:

Original: [Equation]
Revised: [Equation]

Original: This lets us deﬁne the classiﬁer which returns the consensus output of the ensemble:
Revised: This lets us deﬁne the classiﬁer which returns the consensus output of the ensemble:

Original: [Equation]
Revised: [Equation]

Original: When taking the argmax, we break ties deterministically by returning the smaller class index.
Revised: When taking the argmax, we break ties deterministically by returning the smaller class index.

Original: The resulting robust classiﬁer has the following guarantee:
Revised: The resulting robust classiﬁer has the following guarantee:

Original: Theorem 1.
Revised: Theorem 1.

Original: For a ﬁxed deterministic base classiﬁer f , hash function h , ensemble size k , training set T , and input x , let:
Revised: For a ﬁxed deterministic base classiﬁer f , hash function h , ensemble size k , training set T , and input x , let:

Original: [Equation]
Revised: [Equation]

Original: Then, for any poisoned training set U , if | T (cid:9) U | ≤ ¯ ρ ( x ) , then g dpa ( U, x ) = c .
Revised: Then, for any poisoned training set U , if | T (cid:9) U | ≤ ¯ ρ ( x ) , then g dpa ( U, x ) = c .

Original: All proofs are presented in Appendix A. Note that T and U are unordered sets: therefore, in addition to providing certiﬁed robustness against insertions or deletions of training data, the robust classiﬁer g dpa is also invariant under re-ordering of the training data, provided that f has this invariance (which is implied, because f maps deterministically from a set; see Section 3.2.1 for practical considerations).
Revised: All proofs are presented in Appendix A. Note that T and U are unordered sets: therefore, in addition to providing certiﬁed robustness against insertions or deletions of training data, the robust classiﬁer g dpa is also invariant under re-ordering of the training data, provided that f has this invariance (which is implied, because f maps deterministically from a set; see Section 3.2.1 for practical considerations).

Original: As mentioned in Section 1, DPA is a deterministic variant of randomized ablation (Levine &
Revised: As mentioned in Section 1, DPA is a deterministic variant of randomized ablation (Levine &

Original: Feizi, 2020a) adapted to the poisoning domain.
Revised: Feizi, 2020a) adapted to the poisoning domain.

Original: Each base classiﬁer ablates most of the training set, retaining only the samples in one partition.
Revised: Each base classiﬁer ablates most of the training set, retaining only the samples in one partition.

Original: However, unlike in randomized ablation, the partitions are deterministic and use disjoint samples, rather than selecting them randomly and independently.
Revised: However, unlike in randomized ablation, the partitions are deterministic and use disjoint samples, rather than selecting them randomly and independently.

Original: In Appendix C, we argue that our derandomization has little effect on the certiﬁed accuracies, while allowing for exact certiﬁcates using ﬁnite samples.
Revised: In Appendix C, we argue that our derandomization has little effect on the certiﬁed accuracies, while allowing for exact certiﬁcates using ﬁnite samples.

Original: We also discuss how this work relates to Levine & Feizi (2020b), which proposes a de-randomized ablation technique for a restricted class of sparse evasion attacks (patch adversarial attacks).
Revised: We also discuss how this work relates to Levine & Feizi (2020b), which proposes a de-randomized ablation technique for a restricted class of sparse evasion attacks (patch adversarial attacks).

Original: 3.2. DPA P RACTICAL I MPLEMENTATION D ETAILS
Revised: 3.2. DPA P RACTICAL I MPLEMENTATION D ETAILS

Original: One of the advantages of DPA is that we can use deep neural networks for the base classiﬁer f .
Revised: One of the advantages of DPA is that we can use deep neural networks for the base classiﬁer f .

Original: However, enforcing that the output of a deep neural network is a deterministic function of its training data, and speciﬁcally, its training data as an unordered set, requires some care.
Revised: However, enforcing that the output of a deep neural network is a deterministic function of its training data, and speciﬁcally, its training data as an unordered set, requires some care.

Original: First, we must remove dependence on the order in which the training samples are read in.
Revised: First, we must remove dependence on the order in which the training samples are read in.

Original: To do this, in each partition P i , we sort the training samples prior to training, taking advantage of the assumption that S is well-ordered (and therefore S L = S
Revised: To do this, in each partition P i , we sort the training samples prior to training, taking advantage of the assumption that S is well-ordered (and therefore S L = S

Original: × N is also well ordered).
Revised: × N is also well ordered).

Original: In the case of the image data, this is implemented as a lexical sort by pixel values, with the labels concatenated to the samples as an additional value.
Revised: In the case of the image data, this is implemented as a lexical sort by pixel values, with the labels concatenated to the samples as an additional value.

Original: The training procedure for the network, which is based on standard stochastic gradient descent, must also be made deterministic: in our PyTorch (Paszke et al., 2019) implementation, this can be accomplished by deterministically setting a random seed at the start of training.
Revised: The training procedure for the network, which is based on standard stochastic gradient descent, must also be made deterministic: in our PyTorch (Paszke et al., 2019) implementation, this can be accomplished by deterministically setting a random seed at the start of training.

Original: As discussed in
Revised: As discussed in

Original: Appendix F, we ﬁnd that it is best for the ﬁnal classiﬁer accuracy to use different random seeds during training for each partition.
Revised: Appendix F, we ﬁnd that it is best for the ﬁnal classiﬁer accuracy to use different random seeds during training for each partition.

Original: This reduces the correlation in output between base classiﬁers in the ensemble.
Revised: This reduces the correlation in output between base classiﬁers in the ensemble.

Original: Thus, in practice, we use the partition index as the random seed (i.e., we train base classiﬁer f i using random seed i .)
Revised: Thus, in practice, we use the partition index as the random seed (i.e., we train base classiﬁer f i using random seed i .)

Original: 3. SS-DPA
Revised: 3. SS-DPA

Original: Semi-Supervised DPA (SS-DPA) is a defense against label-ﬂip attacks.
Revised: Semi-Supervised DPA (SS-DPA) is a defense against label-ﬂip attacks.

Original: In SS-DPA, the base classiﬁer may be a semi-supervised learning algorithm: it can use the entire unlabeled training dataset, in addition to the labels for a partition.
Revised: In SS-DPA, the base classiﬁer may be a semi-supervised learning algorithm: it can use the entire unlabeled training dataset, in addition to the labels for a partition.

Original: We will therefore deﬁne the base classiﬁer to also accept an unlabelled dataset as input: f : P ( S ) × P ( S L ) × S → N .
Revised: We will therefore deﬁne the base classiﬁer to also accept an unlabelled dataset as input: f : P ( S ) × P ( S L ) × S → N .

Original: Additionally, our method of partitioning the data is modiﬁed both to ensure that changing the label of a sample affects only one partition rather than possibly two, and to create a more equal distribution of samples between partitions.
Revised: Additionally, our method of partitioning the data is modiﬁed both to ensure that changing the label of a sample affects only one partition rather than possibly two, and to create a more equal distribution of samples between partitions.

Original: First, we will sort the unlabeled data samples ( T ) :
Revised: First, we will sort the unlabeled data samples ( T ) :

Original: [Equation]
Revised: [Equation]

Original: For a sample t ∈ T , note that index ( T sorted , sample ( t )) is invariant under any label-ﬂipping attack to T , and also under permutation of the training data as they are read.
Revised: For a sample t ∈ T , note that index ( T sorted , sample ( t )) is invariant under any label-ﬂipping attack to T , and also under permutation of the training data as they are read.

Original: We now partition the data based on sorted index:
Revised: We now partition the data based on sorted index:

Original: [Equation]
Revised: [Equation]

Original: Note that in this partitioning scheme, we no longer need to use a hash function h .
Revised: Note that in this partitioning scheme, we no longer need to use a hash function h .

Original: Moreover, this scheme creates a more uniform distribution of samples between partitions, compared with the hashing scheme used in DPA.
Revised: Moreover, this scheme creates a more uniform distribution of samples between partitions, compared with the hashing scheme used in DPA.

Original: This can lead to improved certiﬁcates: see
Revised: This can lead to improved certiﬁcates: see

Original: Appendix E. This sorting-based partitioning is possible because the unlabeled samples are “clean”, so we can rely on their ordering, when sorted, to remain ﬁxed.
Revised: Appendix E. This sorting-based partitioning is possible because the unlabeled samples are “clean”, so we can rely on their ordering, when sorted, to remain ﬁxed.

Original: As in DPA, we train base classiﬁers on each partition, this time additionally using the entire unlabeled training set:
Revised: As in DPA, we train base classiﬁers on each partition, this time additionally using the entire unlabeled training set:

Original: [Equation]
Revised: [Equation]

Original: The inference procedure is the same as in the standard DPA:
Revised: The inference procedure is the same as in the standard DPA:

Original: [Equation]
Revised: [Equation]

Original: The SS-DPA algorithm provides the following robustness guarantee against label-ﬂipping attacks.
Revised: The SS-DPA algorithm provides the following robustness guarantee against label-ﬂipping attacks.

Original: Theorem 2.
Revised: Theorem 2.

Original: For a ﬁxed deterministic semi-supervised base classiﬁer f , ensemble size k , training set T (with no repeated samples), and input x , let:
Revised: For a ﬁxed deterministic semi-supervised base classiﬁer f , ensemble size k , training set T (with no repeated samples), and input x , let:

Original: [Equation]
Revised: [Equation]

Original: For a poisoned training set U obtained by changing the labels of at most ¯ ρ samples in T , g ssdpa ( U, x )
Revised: For a poisoned training set U obtained by changing the labels of at most ¯ ρ samples in T , g ssdpa ( U, x )

Original: = c .
Revised: = c .

Original: 3.3. S EMI -S UPERVISED L EARNING M ETHODS FOR SS-DPA
Revised: [Table] Median Certiﬁed Robustness is the attack magnitude (symmetric difference for DPA label ﬂips for SS-DPA) at which certiﬁed accuracy is 50%. Training times are on a single GPU; note that many partitions can be trained in parallel.

Revised: Note we observe some constant overhead time for training each classiﬁer, so on MNIST, where the training time per image is small, k has little effect on the training time.

Revised: For SS-DPA, training times do not include the time to train the unsupervised feature embedding (which must only be done once).

Revised: 3.3. S EMI -S UPERVISED L EARNING M ETHODS FOR SS-DPA

Original: In the standard DPA algorithm, we are able to train each classiﬁer in the ensemble using only a small fraction of the training data; this means that each classiﬁer can be trained relatively quickly: as the number of classiﬁers increases, the time to train each classiﬁer can decrease (see Table 1).
Revised: In the standard DPA algorithm, we are able to train each classiﬁer in the ensemble using only a small fraction of the training data; this means that each classiﬁer can be trained relatively quickly: as the number of classiﬁers increases, the time to train each classiﬁer can decrease (see Table 1).

Original: However, in a naive implementation of SS-DPA, Equation 8 might suggest that training time will scale with k , because each semi-supervised base classiﬁer requires to be trained on the entire training set.
Revised: However, in a naive implementation of SS-DPA, Equation 8 might suggest that training time will scale with k , because each semi-supervised base classiﬁer requires to be trained on the entire training set.

Original: Indeed, with many popular and highly effective choices of semi-supervised classiﬁcation algorithms, such as temporal ensembling (Laine & Aila, 2017), ICT (Verma et al., 2019), Teacher Graphs (Luo et al., 2018) and generative approaches (Kingma et al., 2014), the main training loop trains on both labeled and unlabeled samples, so we would see the total training time scale linearly with k .
Revised: Indeed, with many popular and highly effective choices of semi-supervised classiﬁcation algorithms, such as temporal ensembling (Laine & Aila, 2017), ICT (Verma et al., 2019), Teacher Graphs (Luo et al., 2018) and generative approaches (Kingma et al., 2014), the main training loop trains on both labeled and unlabeled samples, so we would see the total training time scale linearly with k .

Original: In order to avoid this, we instead choose a semi-supervised training method where the unlabeled samples are used only to learn semantic features of the data, before the labeled samples are introduced: this allows us to use the unlabeled samples only once, and to then share the learned feature representations when training each base classiﬁer.
Revised: In order to avoid this, we instead choose a semi-supervised training method where the unlabeled samples are used only to learn semantic features of the data, before the labeled samples are introduced: this allows us to use the unlabeled samples only once, and to then share the learned feature representations when training each base classiﬁer.

Original: In our experiments, we choose RotNet (Gidaris et al., 2018) for experiments on MNIST, and SimCLR (Chen et al., 2020) for experiments on CIFAR-10.
Revised: In our experiments, we choose RotNet (Gidaris et al., 2018) for experiments on MNIST, and SimCLR (Chen et al., 2020) for experiments on CIFAR-10 and GTSRB.

Original: Both methods learn an unsupervised embedding of the training set, on top of which all classiﬁers in the ensemble can be learned.
Revised: Both methods learn an unsupervised embedding of the training set, on top of which all classiﬁers in the ensemble can be learned.

Original: Note that Rosenfeld et al.
Revised: Note that Rosenfeld et al.

Original: (2020) also uses SimCLR for CIFARexperiments.
Revised: (2020) also uses SimCLR for CIFARexperiments.

Original: As discussed in Section 3.2.1, we also sort the data prior to learning (including when learning unsupervised features), and set random seeds, in order to ensure determinism.[Table]
Revised: As discussed in Section 3.2.1, we also sort the data prior to learning (including when learning unsupervised features), and set random seeds, in order to ensure determinism.

Original:  R ESULTS
Revised:  R ESULTS

Original: In this section [Table] we present empirical results evaluating the performance of proposed methods, DPA and SS-DPA, against poison attacks on MNIST and CIFAR-10 datasets.
Revised: In this section, we present empirical results evaluating the performance of proposed methods, DPA and SS-DPA, against poison attacks on MNIST, CIFAR-10, and GTSRB datasets.

Original: As discussed in Section 3.3.1, we use the RotNet architecture (Gidaris et al., 2018) for SS-DPA’s semi-supervised learning on MNIST.
Revised: As discussed in Section 3.3.1, we use the RotNet architecture (Gidaris et al., 2018) for SS-DPA’s semi-supervised learning on MNIST.

Original: Conveniently, the RotNet architecture is structured such that the feature extracting layers, combined with the ﬁnal classiﬁcation layers, together make up the Network-In-Network (NiN) architecture for the supervised classiﬁcation (Lin et al., 2013).
Revised: Conveniently, the RotNet architecture is structured such that the feature extracting layers, combined with the ﬁnal classiﬁcation layers, together make up the Network-In-Network (NiN) architecture for the supervised classiﬁcation (Lin et al., 2013).

Original: Therefore, on MNIST, We  use NiN for DPA’s supervised training, and RotNet for SS-DPA’s semi-supervised training.
Revised: Therefore, on MNIST, We [Figure] use NiN for DPA’s supervised training, and RotNet for SS-DPA’s semi-supervised training.

Original: We use training parameters, for both the DPA (NiN) and SS-DPA (RotNet), directly from Gidaris et al.
Revised: We use training parameters, for both the DPA (NiN) and SS-DPA (RotNet), directly from Gidaris et al.

Original: (2018), with a slight modiﬁcation: we eliminate horizontal ﬂips in data augmentation, because horizontal alignment is semantically meaningful for digits.
Revised: (2018), with a slight modiﬁcation: we eliminate horizontal ﬂips in data augmentation, because horizontal alignment is semantically meaningful for digits.

Original: On CIFAR-10, also use NiN (with full data augmentation) for DPA experiments.
Revised: On CIFAR-10 and GTSRB, we also use NiN (with full data augmentation for CIFAR-10, and without horizontal ﬂips for GTSRB) for DPA experiments.

Original: For  SS-DPA, we use SimCLR (Chen et al., 2020), as (Rosenfeld et al., 2020) does, for semi-supervised learning.
Revised: For semi-supervised learning in SS-DPA, we use SimCLR (Chen et al., 2020) for both datasets, as (Rosenfeld et al., 2020) does on CIFAR-10.

Original: Hyperparameters are provided in Appendix H. Note that for SimCLR we use linear classiﬁers as the ﬁnal, supervised classiﬁers for each partition.
Revised: SimCLR hyperparameters are provided in Appendix H, and additional details about processing the GTSRB dataset are provided in Appendix I. Note that for SimCLR we use linear classiﬁers as the ﬁnal, supervised classiﬁers for each partition.

Original: Results are presented in Figures 2 and 3,  and are summarized in Table 1.
Revised: Results are presented in Figures 2, 3, 4, and are summarized in Table 1.

Original: Our metric, Certiﬁed Accuracy as a function of attack magnitude (symmetric-difference or label-ﬂips), refers to the fraction of samples which are both correctly classiﬁed and are certiﬁed as robust to attacks of that magnitude.
Revised: Our metric, Certiﬁed Accuracy as a function of attack magnitude (symmetric-difference or label-ﬂips), refers to the fraction of samples which are both correctly classiﬁed and are certiﬁed as robust to attacks of that magnitude.

Original: Note that different poisoning perturbations, which poison different sets of training samples, may be required to poison each test sample; i.e. we assume the attacker can use the attack budget separately for each test sample Table 1 also reports Median Certiﬁed Robustness, the attack magnitude to which at least 50% of the test set is provably robust.
Revised: Note that different poisoning perturbations, which poison different sets of training samples, may be required to poison each test sample; i.e. we assume the attacker can use the attack budget separately for each test sample Table 1 also reports Median Certiﬁed Robustness, the attack magnitude to which at least 50% of the test set is provably robust.

Original: Our SS-DPA method substantially outperforms the existing certiﬁcate (Rosenfeld et al., 2020) on label-ﬂipping attacks: in median, 392 label ﬂips on CIFAR-10, versus 175; 647 label ﬂips on
Revised: Our SS-DPA method substantially outperforms the existing certiﬁcate (Rosenfeld et al., 2020) on label-ﬂipping attacks: in median, 392 label ﬂips on CIFAR-10, versus 175; 647 label ﬂips on

Original: MNIST, versus < 200.
Revised: MNIST, versus < 200.

Original: With DPA, we are also able to certify at least half of MNIST images to attacks of over 500 poisoning insertions or deletions, and can certify at least half of CIFARimages to 9 poisoning insertions or deletions
Revised: With DPA, we are also able to certify at least half of MNIST images to attacks of over 500 poisoning insertions or deletions, and can certify at least half of CIFAR-10 images to 9 poisoning insertions or deletions. On GTSRB, we can certify over half of images topoisoning insertions or deletions, or 177 label ﬂips.

Revised: Note that this represents a substantially larger fraction of each class (each class has > 1000 training images on average) compared to certiﬁcates on CIFAR-10 (5000 training images per class).

Original: The hyperparameter k controls the number of classiﬁers in the ensemble: because each sample is used in training exactly one classiﬁer, the average number of samples used to train each classiﬁer is inversely proportional to k .
Revised: The hyperparameter k controls the number of classiﬁers in the ensemble: because each sample is used in training exactly one classiﬁer, the average number of samples used to train each classiﬁer is inversely proportional to k .

Original: Therefore, we observe that the base classiﬁer accuracy (and therefore also the ﬁnal ensemble classiﬁer accuracy) decreases as k is increased; see Table 1.
Revised: Therefore, we observe that the base classiﬁer accuracy (and therefore also the ﬁnal ensemble classiﬁer accuracy) decreases as k is increased; see Table 1.

Original: However, because the certiﬁcates described in Theorems 1 and 2 depend directly on the gap in the number of[Figure] [Figure] classiﬁers in the ensemble which output the top and runner-up classes, larger numbers of classiﬁers are necessary to achieve large certiﬁcates.
Revised: However, because the certiﬁcates described in Theorems 1 and 2 depend directly on the gap in the number of classiﬁers in the ensemble which output the top and runner-up classes, larger numbers of classiﬁers are necessary to achieve large certiﬁcates.

Original: In fact, using k classiﬁers, the largest certiﬁed robustness possible is k/ 2 .
Revised: In fact, using k classiﬁers, the largest certiﬁed robustness possible is k/ 2 .

Original: Thus, we see in Figures 2 and 3  that larger values of k tend to produce larger robustness certiﬁcates.
Revised: Thus, we see in Figures 2, 3 and 4 that larger values of k tend to produce larger robustness certiﬁcates.

Original: Therefore k controls a trade-off between robustness and accuracy. 
Revised: Therefore k controls a trade-off between robustness and accuracy. [Figure] [Figure]

Original: Rosenfeld et al.
Revised: Rosenfeld et al.

Original: (2020) also reports robustness certiﬁcates against label-ﬂipping attacks on binary
Revised: (2020) also reports robustness certiﬁcates against label-ﬂipping attacks on binary

Original: MNIST classiﬁcation, with classes 1 and 7.
Revised: MNIST classiﬁcation, with classes 1 and 7.

Original: Rosenfeld et al.
Revised: Rosenfeld et al.

Original: (2020) reports clean-accuracy of 94.5% and certiﬁed accuracies for attack magnitudes up to 2000 label ﬂips (out of 13007), with best certiﬁed accuracy less than 70%.
Revised: (2020) reports clean-accuracy of 94.5% and certiﬁed accuracies for attack magnitudes up to 2000 label ﬂips (out of 13007), with best certiﬁed accuracy less than 70%.

Original: By contrast, using a specialized form of SS-DPA, we are able to achieve clean accuracy of 95 .
Revised: By contrast, using a specialized form of SS-DPA, we are able to achieve clean accuracy of 95 .

Original: 5% , with every correctly-classiﬁed image certiﬁably robust up to 5952 label ﬂips (i.e. certiﬁed accuracy is also 95 .
Revised: 5% , with every correctly-classiﬁed image certiﬁably robust up to 5952 label ﬂips (i.e. certiﬁed accuracy is also 95 .

Original: 5% at 5952 label ﬂips.)
Revised: 5% at 5952 label ﬂips.)

Original: See Appendix B for discussion.
Revised: See Appendix B for discussion.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper, we described a novel approach to provable defenses against poisoning attacks.
Revised: In this paper, we described a novel approach to provable defenses against poisoning attacks.

Original: Unlike previous techniques, our method both allows for exact, deterministic certiﬁcates and can be implemented using deep neural networks.
Revised: Unlike previous techniques, our method both allows for exact, deterministic certiﬁcates and can be implemented using deep neural networks.

Original: These advantages allow us to outperform the current state-ofthe-art on label-ﬂip attacks, and to develop the ﬁrst certiﬁed defense against a broadly deﬁned class of general poisoning attacks.
Revised: These advantages allow us to outperform the current state-ofthe-art on label-ﬂip attacks, and to develop the ﬁrst certiﬁed defense against a broadly deﬁned class of general poisoning attacks.

Original: A P ROOFS
Revised: A P ROOFS

Original: Theorem 1.
Revised: Theorem 1.

Original: For a ﬁxed deterministic base classiﬁer f , hash function h , ensemble size k , training set T  and input x , let:
Revised: For a ﬁxed deterministic base classiﬁer f , hash function h , ensemble size k , training set T , and input x , let:

Original: [Equation]
Revised: [Equation]

Original: Then, for any poisoned training set U , if | T (cid:9) U | ≤ ¯ ρ ( x ) , we have: g dpa ( U, x ) = c .
Revised: Then, for any poisoned training set U , if | T (cid:9) U | ≤ ¯ ρ ( x ) , we have: g dpa ( U, x ) = c .

Original: Proof.
Revised: Proof.

Original: We deﬁne the partitions, trained classiﬁers, and counts for each training set ( T and U ) as described in the main text:
Revised: We deﬁne the partitions, trained classiﬁers, and counts for each training set ( T and U ) as described in the main text:

Original: [Equation]
Revised: [Equation]

Original: Note that here, we are using superscripts to explicitly distinguish between partitions (as well as base classiﬁers and counts) of the clean training set T and the poisoned dataset U (i.e, P Ti is equivalent to P i in the main text).
Revised: Note that here, we are using superscripts to explicitly distinguish between partitions (as well as base classiﬁers and counts) of the clean training set T and the poisoned dataset U (i.e, P Ti is equivalent to P i in the main text).

Original: In Equation 15, as discussed in the main text, when taking the argmax, we break ties deterministically by returning the smaller class index.
Revised: In Equation 15, as discussed in the main text, when taking the argmax, we break ties deterministically by returning the smaller class index.

Original: Note that P Ti = P Ui unless there is some t , with h ( t ) ≡ i (mod k ) , in T (cid:9) U .
Revised: Note that P Ti = P Ui unless there is some t , with h ( t ) ≡ i (mod k ) , in T (cid:9) U .

Original: Because the mapping from t to h ( t ) (mod k ) is a deterministic function, the number of partitions i for which P T i (cid:54) =
Revised: Because the mapping from t to h ( t ) (mod k ) is a deterministic function, the number of partitions i for which P Ti (cid:54) =

Original: P U i is at most | T (cid:9) U | , which is at most ¯ ρ ( x ) .
Revised: P Ui is at most | T (cid:9) U | , which is at most ¯ ρ ( x ) .

Original: P
Revised: P

Original: Ti = P
Revised: Ti = P

Original: Ui implies f Ti ( x ) =
Revised: Ui implies f Ti ( x ) =

Original: f Ui ( x ) , so the number of classiﬁers i for which f
Revised: f Ui ( x ) , so the number of classiﬁers i for which f

Original: Ti ( x ) (cid:54) =
Revised: Ti ( x ) (cid:54) =

Original: f Ui ( x ) is also at most ¯ ρ ( x ) .
Revised: f Ui ( x ) is also at most ¯ ρ ( x ) .

Original: Then:
Revised: Then:

Original: [Equation]
Revised: [Equation]

Original: Let c :=
Revised: Let c :=

Original: g dpa ( T, x ) .
Revised: g dpa ( T, x ) .

Original: Note that g dpa ( U, x )
Revised: Note that g dpa ( U, x )

Original: =
Revised: =

Original: c iff:
Revised: c iff:

Original: [Equation]
Revised: [Equation]

Original: where the separate cases come from the deterministic selection of the smaller index in cases of ties in Equation 15: this can be condensed to ∀ c (cid:48)
Revised: where the separate cases come from the deterministic selection of the smaller index in cases of ties in Equation 15: this can be condensed to ∀ c (cid:48)

Original: (cid:54)
Revised: (cid:54)

Original: =
Revised: =

Original: c : n Uc ( x ) ≥ n Uc [Equation] ( x ) + 1 c (cid:48) <c .
Revised: c : n Uc ( x ) ≥ n Uc (cid:48) ( x ) + 1 c (cid:48) < c .

Original: Then by triangle inequality with Equation 16, we have that g dpa ( U, x 
Revised: Then by triangle inequality with Equation 16, we have that g dpa ( U, x )

Original: [Equation]
Revised: = c if ∀ c (cid:48) (cid:54) = c : n Tc ( x ) ≥ n Tc (cid:48) ( x )+2¯ ρ + 1 c (cid:48) <

Original: c 
Revised: c .

Original: [Equation]

Original: This condition is true by the deﬁnition of ¯ ρ ( x ) , so g dpa ( U, x )
Revised: This condition is true by the deﬁnition of ¯ ρ ( x ) , so g dpa ( U, x )

Original: = c .
Revised: = c .

Original: Theorem 2.
Revised: Theorem 2.

Original: For a ﬁxed deterministic semi-supervised base classiﬁer f , ensemble size k , training set T (with no repeated samples), and input x , let:
Revised: For a ﬁxed deterministic semi-supervised base classiﬁer f , ensemble size k , training set T (with no repeated samples), and input x , let:

Original: [Equation]
Revised: [Equation]

Original: For a poisoned training set U obtained by changing the labels of at most ¯ ρ samples in T , g ssdpa ( U, x )
Revised: For a poisoned training set U obtained by changing the labels of at most ¯ ρ samples in T , g ssdpa ( U, x )

Original: = c .
Revised: = c .

Original: Proof.
Revised: Proof.

Original: Recall the deﬁnition:
Revised: Recall the deﬁnition:

Original: [Equation]
Revised: [Equation]

Original: Because samples ( T ) = samples ( U ) , we have T sorted =
Revised: Because samples ( T ) = samples ( U ) , we have T sorted =

Original: U sorted .
Revised: U sorted .

Original: We can then deﬁne partitions and base classiﬁers for each training set ( T and U ) as described in the main text:
Revised: We can then deﬁne partitions and base classiﬁers for each training set ( T and U ) as described in the main text:

Original: [Equation]
Revised: [Equation]

Original: Recall that for any t ∈ T , index ( T sorted , sample ( t )) is invariant under label-ﬂipping attack to T .
Revised: Recall that for any t ∈ T , index ( T sorted , sample ( t )) is invariant under label-ﬂipping attack to T .

Original: Then, for each i , the samples in P Ti will be the same as the samples in P Ui , possibly with some labels ﬂipped.
Revised: Then, for each i , the samples in P Ti will be the same as the samples in P Ui , possibly with some labels ﬂipped.

Original: In particular, the functions f Ti ( · ) and f Ui ( · ) will be identical, unless the label of some sample with index ( T sorted , sample ( t ))
Revised: In particular, the functions f Ti ( · ) and f Ui ( · ) will be identical, unless the label of some sample with index ( T sorted , sample ( t ))

Original: ≡ i (mod k ) has been changed.
Revised: ≡ i (mod k ) has been changed.

Original: If at most ¯ ρ ( x ) labels change, at most ¯ ρ ( x ) ensemble classiﬁers are affected: the rest of the proof proceeds similarly as that of Theorem 1.
Revised: If at most ¯ ρ ( x ) labels change, at most ¯ ρ ( x ) ensemble classiﬁers are affected: the rest of the proof proceeds similarly as that of Theorem 1.

Original: B B INARY MNIST E XPERIMENTS
Revised: B B INARY MNIST E XPERIMENTS

Original: We perform a specialized instance of SS-DPA on the binary ‘1’ versus ‘7’ MNIST classiﬁcation task.
Revised: We perform a specialized instance of SS-DPA on the binary ‘1’ versus ‘7’ MNIST classiﬁcation task.

Original: Speciﬁcally, we set k
Revised: Speciﬁcally, we set k

Original: = m , so that every partition receives only one label.
Revised: = m , so that every partition receives only one label.

Original: We ﬁrst use 2-means clustering on the unlabeled data, to compute two means.
Revised: We ﬁrst use 2-means clustering on the unlabeled data, to compute two means.

Original: This allows for each base classiﬁer to use a very simple “semi-supervised learning algorithm”: if the test image and the one labeled training image provided to the base classiﬁer belong to the same cluster, then the base classiﬁer assigns the label of the training image to the test image.
Revised: This allows for each base classiﬁer to use a very simple “semi-supervised learning algorithm”: if the test image and the one labeled training image provided to the base classiﬁer belong to the same cluster, then the base classiﬁer assigns the label of the training image to the test image.

Original: Otherwise, it assigns the opposite label to the test image.
Revised: Otherwise, it assigns the opposite label to the test image.

Original: Formally: [Equation]
Revised: Formally: µ 1 , µ 2 :=

Original: Cluster centroids of samples ( T ) [Equation]
Revised: Cluster centroids of samples ( T ) assignment ( s ) :

Original: f i ( x ) =
Revised: = (cid:26) 1 , if (cid:107) µ 1 − s (cid:107) 2 ≤ (cid:107) µ 2 − s (cid:107)2 , if (cid:107) µ 1 − s (cid:107) 2 > (cid:107) µ 2 − s (cid:107)f i ( x ) =

Original: f
Revised: f

Original: ( samples ( T ) , { t i } , x ) = (cid:26) label ( t i ) , if assignment ( x ) = assignment ( t i ) 1 − label ( t i ) , if assignment ( x ) (cid:54) =
Revised: ( samples ( T ) , { t i } , x ) = (cid:26) label ( t i ) , if assignment ( x ) = assignment ( t i ) 1 − label ( t i ) , if assignment ( x ) (cid:54) =

Original: assignment ( t i )
Revised: assignment ( t i )

Original: Note that each base classiﬁer behaves exactly identically, up to a transpose of the labels: so in practice, we simply count the training samples which associate each of the two cluster centroids with each of the two labels, and determine the number of label ﬂips which would be required to change the consensus label assignments of the clusters .
Revised: Note that each base classiﬁer behaves exactly identically, up to a transpose of the labels: so in practice, we simply count the training samples which associate each of the two cluster centroids with each of the two labels, and determine the number of label ﬂips which would be required to change the consensus label assignments of the clusters .

Original: At the test time, each test image therefore needs to be processed only once.
Revised: At the test time, each test image therefore needs to be processed only once.

Original: The amount of time required for inference is then simply the time needed to calculate the distance from the test sample to each of the two clusters.
Revised: The amount of time required for inference is then simply the time needed to calculate the distance from the test sample to each of the two clusters.

Original: This also means that every image has the same robustness certiﬁcate.
Revised: This also means that every image has the same robustness certiﬁcate.

Original: As stated in the main text, using this method, we are able to achieve clean accuracy of 95 .
Revised: As stated in the main text, using this method, we are able to achieve clean accuracy of 95 .

Original: 5% , with every correctly-classiﬁed image certiﬁably robust up to 5952 label ﬂips (i.e. certiﬁed accuracy is also 95 .
Revised: 5% , with every correctly-classiﬁed image certiﬁably robust up to 5952 label ﬂips (i.e. certiﬁed accuracy is also 95 .

Original: 5% at 5952 label ﬂips.)
Revised: 5% at 5952 label ﬂips.)

Original: This means that the classiﬁer is robust to adversarial label ﬂips on 45.8% of the training data.
Revised: This means that the classiﬁer is robust to adversarial label ﬂips on 45.8% of the training data.

Original: Rosenfeld et al.
Revised: Rosenfeld et al.

Original: (2020) also reports robustness certiﬁcates against label-ﬂipping attacks on binary
Revised: (2020) also reports robustness certiﬁcates against label-ﬂipping attacks on binary

Original: MNIST classiﬁcation with classes 1 and 7.
Revised: MNIST classiﬁcation with classes 1 and 7.

Original: Rosenfeld et al.
Revised: Rosenfeld et al.

Original: (2020) reports clean-accuracy of 94.5% and certiﬁed accuracies for attack magnitudes up to 2000 label ﬂips (out of 13007: 15.4%), with the best certiﬁed accuracy less than 70%.
Revised: (2020) reports clean-accuracy of 94.5% and certiﬁed accuracies for attack magnitudes up to 2000 label ﬂips (out of 13007: 15.4%), with the best certiﬁed accuracy less than 70%.

Original: C R ELATIONSHIP TO R ANDOMIZED A BLATION
Revised: C R ELATIONSHIP TO R ANDOMIZED A BLATION

Original: As mentioned in Section 1, (SS-)DPA is in some sense related to Randomized Ablation (Levine & Feizi, 2020a)
Revised: As mentioned in Section 1, (SS-)DPA is in some sense related to Randomized Ablation (Levine & Feizi, 2020a)

Original: (used in defense against sparse inference-time attacks) for training-time poisoning attacks.
Revised: (used in defense against sparse inference-time attacks) for training-time poisoning attacks.

Original: Randomized Ablation is a certiﬁed defense against L 0 (sparse) inference attacks, in which the ﬁnal classiﬁcation is a consensus among classiﬁcations of copies of the image.
Revised: Randomized Ablation is a certiﬁed defense against L 0 (sparse) inference attacks, in which the ﬁnal classiﬁcation is a consensus among classiﬁcations of copies of the image.

Original: In each copy, a ﬁxed number of pixels are randomly ablated (replaced with a null value).
Revised: In each copy, a ﬁxed number of pixels are randomly ablated (replaced with a null value).

Original: A direct application of Randomized Ablation to poisoning attacks would require each base classiﬁer to be trained on a random subset of the training data, with each base classiﬁer’s training set chosen randomly and independently.
Revised: A direct application of Randomized Ablation to poisoning attacks would require each base classiﬁer to be trained on a random subset of the training data, with each base classiﬁer’s training set chosen randomly and independently.

Original: Due to the randomized nature of this algorithm, estimation error would have to be considered in practice when applying Randomized Ablation using a ﬁnite number of base classiﬁers: this decreases the certiﬁcates that can be reported, while also introducing a failure probability to the certiﬁcates.
Revised: Due to the randomized nature of this algorithm, estimation error would have to be considered in practice when applying Randomized Ablation using a ﬁnite number of base classiﬁers: this decreases the certiﬁcates that can be reported, while also introducing a failure probability to the certiﬁcates.

Original: By contrast, in our algorithms, the partitions are deterministic and use disjoint, rather than independent, samples.
Revised: By contrast, in our algorithms, the partitions are deterministic and use disjoint, rather than independent, samples.

Original: In this section, we argue that our derandomization has little effect on the certiﬁed accuracies compared to randomized ablation, even considering randomized ablation with no estimation error (i.e., with inﬁnite base classiﬁers).
Revised: In this section, we argue that our derandomization has little effect on the certiﬁed accuracies compared to randomized ablation, even considering randomized ablation with no estimation error (i.e., with inﬁnite base classiﬁers).

Original: In the poisoning case speciﬁcally, using additional base classiﬁers is expensive – because they must each be trained – so one would observe a large estimation error when using a realistic number of base classiﬁers.
Revised: In the poisoning case speciﬁcally, using additional base classiﬁers is expensive – because they must each be trained – so one would observe a large estimation error when using a realistic number of base classiﬁers.

Original: Therefore our derandomization can potentially improve the certiﬁcates which can be reported, while also allowing for exact certiﬁcates using a ﬁnite number of base classiﬁers.
Revised: Therefore our derandomization can potentially improve the certiﬁcates which can be reported, while also allowing for exact certiﬁcates using a ﬁnite number of base classiﬁers.

Original: For simplicity, consider the label-ﬂipping case.
Revised: For simplicity, consider the label-ﬂipping case.

Original: In this case, the training set has a ﬁxed size, m .
Revised: In this case, the training set has a ﬁxed size, m .

Original: Thus, Randomized Ablation bounds can be considered directly.
Revised: Thus, Randomized Ablation bounds can be considered directly.

Original: A direct adaptation of Levine &
Revised: A direct adaptation of Levine &

Original: Feizi (2020a) would, for each base classiﬁer, choose s out of m samples to retain labels for, and would ablate the labels for the rest of the training data.
Revised: Feizi (2020a) would, for each base classiﬁer, choose s out of m samples to retain labels for, and would ablate the labels for the rest of the training data.

Original: Suppose an adversary has ﬂipped r labels.
Revised: Suppose an adversary has ﬂipped r labels.

Original: For each base classiﬁer, the probability that a ﬂipped label is used in classiﬁcation (and therefore that the base classiﬁer is ‘poisoned’) is:
Revised: For each base classiﬁer, the probability that a ﬂipped label is used in classiﬁcation (and therefore that the base classiﬁer is ‘poisoned’) is:

Original: [Equation]
Revised: [Equation]

Original: (21) where “RA” stands for Randomized Ablation.
Revised: where “RA” stands for Randomized Ablation.

Original: In this direct adaptation, one must then use a very large ensemble of randomized classiﬁers.
Revised: In this direct adaptation, one must then use a very large ensemble of randomized classiﬁers.

Original: The ensemble must be large enough that we can estimate with high conﬁdence the probabilities (on the distribution of possible choices of training labels to retain) that the base classiﬁer selects each class.
Revised: The ensemble must be large enough that we can estimate with high conﬁdence the probabilities (on the distribution of possible choices of training labels to retain) that the base classiﬁer selects each class.

Original: If the gap between the highest and the next-highest class probabilities can be determined to be greater than 2 Pr( poisoned ) RA , then the consensus classiﬁcation cannot be changed by ﬂipping r labels.
Revised: If the gap between the highest and the next-highest class probabilities can be determined to be greater than 2 Pr( poisoned ) RA , then the consensus classiﬁcation cannot be changed by ﬂipping r labels.

Original: This is because, at worst, every poisoned classiﬁer could switch the highest-class classiﬁcation to the runner-up class, reducing the gap by at most 2 Pr( poisoned ) .
Revised: This is because, at worst, every poisoned classiﬁer could switch the highest-class classiﬁcation to the runner-up class, reducing the gap by at most 2 Pr( poisoned ) .

Original: Note that this estimation relies on each base classiﬁer using a subset of labels selected randomly and independently from the other base classiﬁer.
Revised: Note that this estimation relies on each base classiﬁer using a subset of labels selected randomly and independently from the other base classiﬁer.

Original: In contrast, our SS-DPA method selects each subset disjointly .
Revised: In contrast, our SS-DPA method selects each subset disjointly .

Original: If r labels are ﬂipped, assuming that in the worst case each ﬂipped label is in a different partition, using the union bound, the proportion of base classiﬁers which can be poisoned is
Revised: If r labels are ﬂipped, assuming that in the worst case each ﬂipped label is in a different partition, using the union bound, the proportion of base classiﬁers which can be poisoned is

Original: [Equation]
Revised: [Equation]

Original: where for simplicity we assume that k evenly divides the number of samples m , so s
Revised: where for simplicity we assume that k evenly divides the number of samples m , so s

Original: = m/k labels are kept by each partition.
Revised: = m/k labels are kept by each partition.

Original: Again we need the gap in class probabilities to be at least 2 Pr( poisoned ) SS-DPA to ensure robustness.
Revised: Again we need the gap in class probabilities to be at least 2 Pr( poisoned ) SS-DPA to ensure robustness.

Original: While the use of the union bound might suggest that our deterministic scheme (Equation 22) might lead to a signiﬁcantly looser bound than that of the probabilistic certiﬁcate (Equation 21), this is not the case in practice where rs << m .
Revised: While the use of the union bound might suggest that our deterministic scheme (Equation 22) might lead to a signiﬁcantly looser bound than that of the probabilistic certiﬁcate (Equation 21), this is not the case in practice where rs << m .

Original: For example, in an MNIST-sized dataset ( m = 60000 ), using s = 50 labels per base classiﬁer, to certify for r = 200 label ﬂips, we have Pr( poisoned ) RA = 0 .
Revised: For example, in an MNIST-sized dataset ( m = 60000 ), using s = 50 labels per base classiﬁer, to certify for r = 200 label ﬂips, we have Pr( poisoned ) RA = 0 .

Original: 154 , and Pr( poisoned ) SS-DPA ≤ 0 .
Revised: 154 , and Pr( poisoned ) SS-DPA ≤ 0 .

Original: The derandomization only sightly increases the required gap between the top two class probabilities.
Revised: The derandomization only sightly increases the required gap between the top two class probabilities.

Original: To understand this, note that if the number of poisonings r is small compared to the number of partitions k , then even if the partitions are random and independent, the chance that any two poisonings occur in the same partition is quite small.
Revised: To understand this, note that if the number of poisonings r is small compared to the number of partitions k , then even if the partitions are random and independent, the chance that any two poisonings occur in the same partition is quite small.

Original: In that case, the union bound in Equation 22 is actually quite close to an independence assumption.
Revised: In that case, the union bound in Equation 22 is actually quite close to an independence assumption.

Original: By accepting this small increase in the upper bound of the probability that each base classiﬁcation is poisoned, our method provides all of the beneﬁts of de-randomization, including allowing for exact robustness certiﬁcates using only a ﬁnite number of classiﬁers.
Revised: By accepting this small increase in the upper bound of the probability that each base classiﬁcation is poisoned, our method provides all of the beneﬁts of de-randomization, including allowing for exact robustness certiﬁcates using only a ﬁnite number of classiﬁers.

Original: Additionally, note that in the Randomized Ablation case, the empirical gap in estimated class probabilities must be somewhat larger than Pr( poisoned ) RA in order to certify robustness with high conﬁdence, due to estimation error: the gap required increases more as the number of base classiﬁers decreases.
Revised: Additionally, note that in the Randomized Ablation case, the empirical gap in estimated class probabilities must be somewhat larger than Pr( poisoned ) RA in order to certify robustness with high conﬁdence, due to estimation error: the gap required increases more as the number of base classiﬁers decreases.

Original: This is particularly important in the poisoning case, because training a large number of classiﬁers is substantially more expensive than performing a large number of evaluations, as in randomized smoothing for evasion attacks.
Revised: This is particularly important in the poisoning case, because training a large number of classiﬁers is substantially more expensive than performing a large number of evaluations, as in randomized smoothing for evasion attacks.

Original: We also note that Levine & Feizi (2020b) also used a de-randomized scheme based on Randomized Ablation to certiﬁably defend against evasion patch attacks.
Revised: We also note that Levine & Feizi (2020b) also used a de-randomized scheme based on Randomized Ablation to certiﬁably defend against evasion patch attacks.

Original: However, in that work, the derandomization does not involve a union bound over arbitrary partitions of the vulnerable inputs.
Revised: However, in that work, the derandomization does not involve a union bound over arbitrary partitions of the vulnerable inputs.

Original: Instead, in the case of patch attacks, the attack is geometrically constrained: the image is therefore divided into geometric regions (bands or blocks) such that the attacker will only overlap with a ﬁxed number of these regions.
Revised: Instead, in the case of patch attacks, the attack is geometrically constrained: the image is therefore divided into geometric regions (bands or blocks) such that the attacker will only overlap with a ﬁxed number of these regions.

Original: Each base classiﬁer then uses only a single region to make its classiﬁcation.
Revised: Each base classiﬁer then uses only a single region to make its classiﬁcation.

Original: Levine & Feizi (2020b) do not apply this to derandomization via disjoint subsets/union bound to defend against poison attacks.
Revised: Levine & Feizi (2020b) do not apply this to derandomization via disjoint subsets/union bound to defend against poison attacks.

Original: Also, we note that we borrow from Levine & Feizi (2020b) the deterministic “tie-breaking” technique when evaluating the consensus class in Equation 4, which can increase our robustness certiﬁcate by up to one.
Revised: Also, we note that we borrow from Levine & Feizi (2020b) the deterministic “tie-breaking” technique when evaluating the consensus class in Equation 4, which can increase our robustness certiﬁcate by up to one.

Original: D R ELATIONSHIP TO E XISTING E NSEMBLE M ETHODS
Revised: D R ELATIONSHIP TO E XISTING E NSEMBLE M ETHODS

Original: As mentioned in Section 1, our proposed method is related to classical ensemble approaches in machine learning, namely bootstrap aggregation (“bagging”) and subset aggregation (“subagging”)
Revised: As mentioned in Section 1, our proposed method is related to classical ensemble approaches in machine learning, namely bootstrap aggregation (“bagging”) and subset aggregation (“subagging”)

Original: (Breiman, 1996; Buja & Stuetzle, 2006; B ¨ uhlmann, 2003; Zaman & Hirose, 2009).
Revised: (Breiman, 1996; Buja & Stuetzle, 2006; B¨uhlmann, 2003; Zaman & Hirose, 2009).

Original: In these methods, each base classiﬁer in the ensemble is trained on an independently sampled collection of points from the training set: this means that multiple classiﬁers in the ensemble may be trained on the same sample point.
Revised: In these methods, each base classiﬁer in the ensemble is trained on an independently sampled collection of points from the training set: this means that multiple classiﬁers in the ensemble may be trained on the same sample point.

Original: The purpose of these methods has typically been to improve generalization, and therefore to improve test set accuracy: bagging and subagging decrease the variance component of the classiﬁer’s error.
Revised: The purpose of these methods has typically been to improve generalization, and therefore to improve test set accuracy: bagging and subagging decrease the variance component of the classiﬁer’s error.

Original: In subagging, each training set for a base classiﬁer is an independently sampled subset of the training data: this is in fact an identical formulation to the “direct Randomized Ablation” approach discussed in Appendix C.
Revised: In subagging, each training set for a base classiﬁer is an independently sampled subset of the training data: this is in fact an identical formulation to the “direct Randomized Ablation” approach discussed in Appendix C.

Original: However, in practice, the size of each training subset has typically been quite large: the bias error term increases with decreasing subsample sizes (Buja & Stuetzle, 2006).
Revised: However, in practice, the size of each training subset has typically been quite large: the bias error term increases with decreasing subsample sizes (Buja & Stuetzle, 2006).

Original: Thus, the optimal subsample size for maximum accuracy is large: B¨uhlmann (2003) recommends using s = m/ 2 samples per classiﬁer (“half-subagging”), with theoretical justiﬁcation for optimal generalization.
Revised: Thus, the optimal subsample size for maximum accuracy is large: B ¨ uhlmann (2003) recommends using s = m/ 2 samples per classiﬁer (“half-subagging”), with theoretical justiﬁcation for optimal generalization.

Original: This would not be useful in Randomized Ablation-like certiﬁcation, because any one poisoned element would affect half of the ensemble.
Revised: This would not be useful in Randomized Ablation-like certiﬁcation, because any one poisoned element would affect half of the ensemble.

Original: Indeed, in our certiﬁably robust classiﬁers, we observe a trade-off between accuracy and certiﬁed robustness: our use of many very small partitions is clearly not optimal for the test-set accuracy (Table 1).
Revised: Indeed, in our certiﬁably robust classiﬁers, we observe a trade-off between accuracy and certiﬁed robustness: our use of many very small partitions is clearly not optimal for the test-set accuracy (Table 1).

Original: In bagging, the samples in each base classiﬁer training set are chosen with replacement, so elements may be repeated in the training “set” for a single base classiﬁer.Bagging has been proposed as [Table] CIFAR, SS-DPA
Revised: In bagging, the samples in each base classiﬁer training set are chosen with replacement, so elements may be repeated in the training “set” for a single base classiﬁer.

Original: [Table] an empirical defense against poisoning attacks (Biggio et al., 2011) as well as for evasion attacks (Smutz & Stavrou, 2016).
Revised: Bagging has been proposed as an empirical defense against poisoning attacks (Biggio et al., 2011) as well as for evasion attacks (Smutz & Stavrou, 2016).

Original: However, to our knowledge, these techniques have not yet been used to provide certiﬁed robustness
Revised: However, to our knowledge, these techniques have not yet been used to provide certiﬁed robustness.

Revised: Our approach also bears some similarity to Federated Averaging (McMahan et al., 2017) in that base models are trained on disjoint partitions of the dataset.

Revised: However, in Federated Averaging, the model weights of many distributed base classiﬁers are periodically averaged and re-distributed during learning, in order to allow for efﬁcient massively-parallel learning.

Revised: No theoretical robustness guarantees are provided (as this is not the goal of the algorithm) and it would seem difﬁcult to derive them, given that the relationship between model weights and ﬁnal classiﬁcation is highly non-linear in deep networks.

Revised: By contrast, DPA uses the consensus of ﬁnal outputs after all base classiﬁers are trained independently (or, in the SS-DPA case, independently for labeled data).

Original: E SS-DPA WITH H ASHING
Revised: E SS-DPA WITH H ASHING

Original: It is possible to use hashing, as in DPA, in order to partition data for SS-DPA: as long as the hash function h ( t ) does not use the sample label in assigning a class (as ours indeed does not), it will always assign an image to the same partition regardless of label-ﬂipping, so only one partition will be affected by a label-ﬂip.
Revised: It is possible to use hashing, as in DPA, in order to partition data for SS-DPA: as long as the hash function h ( t ) does not use the sample label in assigning a class (as ours indeed does not), it will always assign an image to the same partition regardless of label-ﬂipping, so only one partition will be affected by a label-ﬂip.

Original: Therefore, the SS-DPA label-ﬂipping certiﬁcate should still be correct.
Revised: Therefore, the SS-DPA label-ﬂipping certiﬁcate should still be correct.

Original: However, as explained in the main text, treating the unlabeled data as trustworthy allows us to partition the samples evenly among partitions using sorting.
Revised: However, as explained in the main text, treating the unlabeled data as trustworthy allows us to partition the samples evenly among partitions using sorting.

Original: This is motivated by the classical understanding in machine learning (e.g. Amari et al.
Revised: This is motivated by the classical un- [Table] derstanding in machine learning (e.g. Amari et al.

Original: that learning curves (the test error versus the number of samples that a classiﬁer is trained on) tend to be convex-like.
Revised: that learning curves (the test error versus the number of samples that a classiﬁer is trained on) tend to be convex-like.

Original: The test error of a base classiﬁer is then approximately a convex function of that base classiﬁer’s partition size.
Revised: The test error of a base classiﬁer is then approximately a convex function of that base classiﬁer’s partition size.

Original: Therefore, if the partition size is a random variable, by Jensen’s inequality, the expected test error of the (random) partition size is greater than the test error of the mean partition size.
Revised: Therefore, if the partition size is a random variable, by Jensen’s inequality, the expected test error of the (random) partition size is greater than the test error of the mean partition size.

Original: Setting all base classiﬁers to use the mean number of samples should then maximize the average base classiﬁer accuracy.
Revised: Setting all base classiﬁers to use the mean number of samples should then maximize the average base classiﬁer accuracy.

Original: To validate this reasoning, we tested SS-DPA with partitions determined by hashing (using the same partitions as we used in DPA), rather than the sorting method described in the main text.
Revised: To validate this reasoning, we tested SS-DPA with partitions determined by hashing (using the same partitions as we used in DPA), rather than the sorting method described in the main text.

Original: See Table 2 for results.
Revised: See Table 2 for results.

Original: As expected, the average base classiﬁer accuracy decreased in most (4/5) experiments when using the DPA hashing, compared to using the sorting method of SS-DPA.
Revised: As expected, the average base classiﬁer accuracy decreased in most (8/9) experiments when using the DPA hashing, compared to using the sorting method of SS-DPA.

Original: However, the effect was minimal in CIFAR-10  experiments: the main advantage of the sorting method was seen on MNIST.
Revised: However, the effect was minimal in CIFAR-10 and GTSRB experiments: the main advantage of the sorting method was seen on MNIST.

Original: This is partly because we used more partitions, and hence fewer average samples per partition, in the MNIST experiments: fewer average samples per partition creates a greater variation in the number of samples per partition in the hashing method.
Revised: This is partly because we used more partitions, and hence fewer average samples per partition, in the MNIST experiments: fewer average samples per partition creates a greater variation in the number of samples per partition in the hashing method.

Original: However, CIFAR-10 with k = 1000 and MNIST with k = 1200 both average 50 samples per partition, but the base classiﬁer accuracy difference still was much more signiﬁcant on MNIST (2.13%) compared to CIFAR-10 (0.48%).
Revised: However, CIFAR-10 with k =and MNIST with k = 1200 both average 50 samples per partition, but the base classiﬁer accuracy difference still was much more signiﬁcant on MNIST (2.13%) compared to CIFAR-10 (0.48%).

Original: On the MNIST experiments, where the base classiﬁer accuracy gap was observed, we also saw that the effect of hashing on the smoothed classiﬁer was mainly to decrease the certiﬁed robustness, and that there was not a signiﬁcant effect on the clean smoothed classiﬁer accuracy.
Revised: On the MNIST experiments, where the base classiﬁer accuracy gap was observed, we also saw that the effect of hashing on the smoothed classiﬁer was mainly to decrease the certiﬁed robustness, and that there was not a signiﬁcant effect on the clean smoothed classiﬁer accuracy.

Original: As discussed in
Revised: As discussed in

Original: Appendix F, this may imply that the outputs of the base classiﬁers using the sorting method are more correlated, in addition to being more accurate.
Revised: Appendix F, this may imply that the outputs of the base classiﬁers using the sorting method are more correlated, in addition to being more accurate.

Original: F E FFECT OF R ANDOM S EED S ELECTION
Revised: F E FFECT OF R ANDOM S EED S ELECTION

Original: In Section 3.2.1, we mention that we “deterministically” choose different random seeds for training each partition, rather than training every partition with the same random seed.
Revised: In Section 3.2.1, we mention that we “deterministically” choose different random seeds for training each partition, rather than training every partition with the same random seed.

Original: To see a comparison between using distinct and the same random seed for each partition, see Table 3.
Revised: To see a comparison between using distinct and the same random seed for each partition, see Table 3.

Original: Note that there is not a large, consistent effect across experiments on either the base classiﬁer accuracy nor the the me- [Table] dian certiﬁed robustness: however, in nine out of 10 experiments, the distinct random seeds resulted in higher smoothed classiﬁer accuracies.
Revised: Note that there is not a large, consistent effect across experiments on either the base classiﬁer accuracy nor the the median certiﬁed robustness: however, in most experiments, the distinct random seeds resulted in higher smoothed classiﬁer accuracies.

Original: This effect was particularly pronounced using SS-DPA on MNIST: using distinct seeds increased smoothed accuracy by at least 1% on each value of k on
Revised: This effect was particularly pronounced using SS-DPA on MNIST: using distinct seeds increased smoothed accuracy by at least 1% on each value of k on

Original: MNIST.
Revised: MNIST.

Original: This implies that shared random seeds make the base classiﬁers more correlated with each other: at the same level of average base classiﬁer accuracy, it is more likely that a plurality of base classiﬁers will all misclassify the same sample (If the base classiﬁers were perfectly uncorrelated, we would see nearly 100% smoothed clean accuracy wherever the base classiﬁer accuracy was over 50%.
Revised: This implies that shared random seeds make the base classiﬁers more correlated with each other: at the same level of average base classiﬁer accuracy, it is more likely that a plurality of base classiﬁers will all misclassify the same sample (If the base classiﬁers were perfectly uncorrelated, we would see nearly 100% smoothed clean accuracy wherever the base classiﬁer accuracy was over 50%.

Original: Also if they were perfectly correlated, the smoothed clean accuracy would equal the base classiﬁer accuracy).
Revised: Also if they were perfectly correlated, the smoothed clean accuracy would equal the base classiﬁer accuracy).

Original: Interestingly, the base classiﬁer accuracy was signiﬁcantly lower for both SSDPA/MNIST experiments when using diverse random seeds; this deﬁes any obvious explanation.
Revised: Interestingly, the base classiﬁer accuracy was signiﬁcantly lower for both SS- [Table] DPA/MNIST experiments when using diverse random seeds; this deﬁes any obvious explanation.

Original: However, this does make the correlation effect even more signiﬁcant: for example, for k = 3000 , the SS-DPA smoothed classiﬁer accuracy is over 2% larger with distinct seeds, despite the fact that the base classiﬁer is over 1% less accurate.
Revised: However, this does make the correlation effect even more signiﬁcant: for example, for k = 3000 , the SS-DPA smoothed classiﬁer accuracy is over 2% larger with distinct seeds, despite the fact that the base classiﬁer is over 1% less accurate.

Original: It is somewhat surprising that the base classiﬁers become correlated when using the same random seed, given that they are trained on entirely distinct data.
Revised: It is somewhat surprising that the base classiﬁers become correlated when using the same random seed, given that they are trained on entirely distinct data.

Original: However, two factors may be at play here.
Revised: However, two factors may be at play here.

Original: First, note that the random seed used in training controls the random cropping of training images: it is possible that, because the training sets of the base classiﬁers are so small, using the same cropping patterns in every classiﬁer would create a systematic bias.
Revised: First, note that the random seed used in training controls the random cropping of training images: it is possible that, because the training sets of the base classiﬁers are so small, using the same cropping patterns in every classiﬁer would create a systematic bias.

Original: Note that the effect was least signiﬁcant for SS-DPA on CIFAR-10, with SimCLR.
Revised: Note that the effect was least signiﬁcant for SS-DPA on CIFAR-10 and GTSRB, with SimCLR.

Original: This may be due to the ﬁnal supervised classiﬁers being linear classiﬁers, rather than deep networks, for these experiments.
Revised: This may be due to the ﬁnal supervised classiﬁers being linear classiﬁers, rather than deep networks, for these experiments.

Original: G SS-DPA WITH R EPEATED U NLABELED D ATA
Revised: G SS-DPA WITH R EPEATED U NLABELED D ATA

Original: The deﬁnition of a training set that we use, T ∈ P ( S L ) , technically allows for repeated samples with differing labels: there could be a pair of distinct samples, t , t (cid:48) ∈ T , such that sample ( t ) =
Revised: The deﬁnition of a training set that we use, T ∈ P ( S L ) , technically allows for repeated samples with differing labels: there could be a pair of distinct samples, t , t (cid:48) ∈ T , such that sample ( t ) =

Original: sample ( t (cid:48) ) , but label ( t ) (cid:54) = label ( t (cid:48) ) .
Revised: sample ( t (cid:48) ) , but label ( t ) (cid:54) = label ( t (cid:48) ) .

Original: This creates difﬁculties with the deﬁnition of the label-ﬂipping attack: for example, the attacker could ﬂip the label of t to become the label of t (cid:48) : this would break the deﬁnition of T as a set.
Revised: This creates difﬁculties with the deﬁnition of the label-ﬂipping attack: for example, the attacker could ﬂip the label of t to become the label of t (cid:48) : this would break the deﬁnition of T as a set.

Original: In most applications, this is not a circumstance that warrants practical consideration: indeed, none of the datasets used in our experiments have such instances (nor can label-ﬂipping attacks create them), and therefore, for performance reasons, our implementation of
Revised: In most applications, this is not a circumstance that warrants practical consideration: indeed, none of the datasets used in our experiments have such instances (nor can label-ﬂipping attacks create them), and therefore, for performance reasons, our implementation of

Original: SS-DPA does not handle these cases.
Revised: SS-DPA does not handle these cases.

Original: Speciﬁcally, to optimize for performance, we verify that there are no repeated sample values, and then sort T itself (rather than samples ( T ) ) lexicographically by image pixel values: this is equivalent to sorting samples ( T ) if no repeated images occur — which we have already veriﬁed — and avoids an unnecessary lookup procedure to ﬁnd the sorted index of the unlabeled sample for each labeled sample.
Revised: Speciﬁcally, to optimize for performance, we verify that there are no repeated sample values, and then sort T itself (rather than samples ( T ) ) lexicographically by image pixel values: this is equivalent to sorting samples ( T ) if no repeated images occur — which we have already veriﬁed — and avoids an unnecessary lookup procedure to ﬁnd the sorted index of the unlabeled sample for each labeled sample.

Original: However, the SS-DPA algorithm as described in
Revised: However, the SS-DPA algorithm as described in

Original: Section 3.3 can be implemented to handle such datasets, under a formalism of label-ﬂipping tailored to represent this edge case.
Revised: Section 3.3 can be implemented to handle such datasets, under a formalism of label-ﬂipping tailored to represent this edge case.

Original: Speciﬁcally, we deﬁne the space of possible labeled data points as S (cid:48)
Revised: Speciﬁcally, we deﬁne the space of possible labeled data points as S (cid:48)

Original: L = S × P ( N ) : each labeled data point consists of a sample along with a set of associated labels.
Revised: L = S × P ( N ) : each labeled data point consists of a sample along with a set of associated labels.

Original: We then restrict our dataset T to be any subset of S (cid:48)
Revised: We then restrict our dataset T to be any subset of S (cid:48)

Original: L such that for all t , t (cid:48) ∈ T , sample ( t ) (cid:54)
Revised: L such that for all t , t (cid:48) ∈ T , sample ( t ) (cid:54)

Original: =
Revised: =

Original: sample ( t (cid:48) ) .
Revised: sample ( t (cid:48) ) .

Original: In other words, we do not allow repeated sample values in T as formally deﬁned: if repeated samples exist, one can simply merge their sets of associated labels (in the formalism).
Revised: In other words, we do not allow repeated sample values in T as formally deﬁned: if repeated samples exist, one can simply merge their sets of associated labels (in the formalism).

Original: Note that using this deﬁnition, the size of T will equal the size of samples ( T ) , and the samples will always remain the same: the adversary can only modify the label sets of samples “in place”.
Revised: Note that using this deﬁnition, the size of T will equal the size of samples ( T ) , and the samples will always remain the same: the adversary can only modify the label sets of samples “in place”.

Original: SS-DPA will always assign an unlabeled sample, along with all of its associated labels , to the same partition, regardless of any label ﬂipping.
Revised: SS-DPA will always assign an unlabeled sample, along with all of its associated labels , to the same partition, regardless of any label ﬂipping.

Original: In practice, this is because, as described in Section 3.3, the partition assignment of a labeled sample depends only on its sample value, not its label: all labeled samples with the same sample value will be put in the same partition.
Revised: In practice, this is because, as described in Section 3.3, the partition assignment of a labeled sample depends only on its sample value, not its label: all labeled samples with the same sample value will be put in the same partition.

Original: Note that this is true even if the implementation represents two identical sample values with different labels as two separate samples: one does not actually have to implement labels as sets.
Revised: Note that this is true even if the implementation represents two identical sample values with different labels as two separate samples: one does not actually have to implement labels as sets.

Original: Therefore, any changes to any labels associated with a sample will only change the output of one base classiﬁer, so all such changes can together be considered a single label-ﬂip in the context of the certiﬁcate.
Revised: Therefore, any changes to any labels associated with a sample will only change the output of one base classiﬁer, so all such changes can together be considered a single label-ﬂip in the context of the certiﬁcate.

Original: In the above formalism, the certiﬁcate represents the number of samples in T whose label sets have been “ﬂipped”: i.e., modiﬁed in any way.
Revised: In the above formalism, the certiﬁcate represents the number of samples in T whose label sets have been “ﬂipped”: i.e., modiﬁed in any way.

Original: H S IM CLR E XPERIMENTAL D ETAILS
Revised: H S IM CLR E XPERIMENTAL D ETAILS

Original: We used the PyTorch implementation of SimCLR provided by Khosla et al.
Revised: We used the PyTorch implementation of SimCLR provided by Khosla et al.

Original: (2020), with modiﬁcations to ensure determinism as described in the main text.
Revised: (2020), with modiﬁcations to ensure determinism as described in the main text.

Original: For training the embeddings, we used a
Revised: For training the embeddings, we used a

Original: ResNet18 model with batch size of 512, initial learning rate of 0.5, cosine annealing, and temperature parameter of 0.5, and trained for 1000 epochs.
Revised: ResNet18 model with batch size of 512 for CIFAR-10 and 256 for GTSRB, initial learning rate of 0.5, cosine annealing, and temperature parameter of 0.5, and trained for 1000 epochs.

Original: For learning the linear ensemble classiﬁers, we used a batch size of 512, initial learning rate of 1.0, and trained for 100 epochs. 
Revised: For learning the linear ensemble classiﬁers, we used a batch size of 512, initial learning rate of 1.0, and trained for 100 epochs. I GTSRB DATASET DETAILS

Revised: The GTSRB dataset consists of images of variable sizes, from 15 × 15 to 250 × 250 .

Revised: We resized all images to 48 × 48 during training and testing (using bilinear interpolation).

Revised: However, we wanted to ensure that our certiﬁcates still applied correctly to the original images.

Revised: In particular, for SSDPA, we sort the dataset using the top-left 64 × 64 pixels of the original image (with black padding for smaller images): this was enough to ensure that repeated images did not occur in the original dataset, as described in Appendix G (even if the resized images could possibly contain repeats).

Revised: For

Revised: DPA, we hash using only the top-left 15 × 15 pixels of the original image.

Revised: Finally, for sorting to ensure determinism in training, we use the images after resizing (ensuring no repeated images is not important for this).

Revised: For both NiN and SimCLR training, we excluded horizontal ﬂips from data augmentation and contrastive learning, because some classes in GTSRB are in fact mirror-images of other classes.

Revised: Finally, due to a technical issue, we excluded one image (00000/00000 00000.ppm) from the original GTSRB training set.