Lethal Dose Conjecture on Data Poisoning
Anonymous Author(s)
AffiliationAddress email
Abstract
Data poisoning considers an adversary that distorts the training set of machinelearning algorithms for malicious purposes.
In this work, we bring to light oneconjecture regarding the fundamentals of data poisoning, which we call the Lethal
Dose Conjecture .
The conjecture states: If n clean training samples are needed foraccurate predictions, then in a size- N training set, only Θ( N/n ) poisoned samplescan be tolerated while ensuring accuracy.
Theoretically, we verify this conjecturein multiple cases.
We also offer a more general perspective of this conjecturethrough distribution discrimination.
Deep Partition Aggregation (DPA) and itsextension, Finite Aggregation (FA) are recent approaches for provable defensesagainst data poisoning, where they predict through the majority vote of many basemodels trained from different subsets of training set using a given learner.
Theconjecture implies that both DPA and FA are (asymptotically) optimal—if wehave the most data-efficient learner, they can turn it into one of the most robustdefenses against data poisoning.
This outlines a practical approach to developingstronger defenses against poisoning via finding data-efficient learners.
Empirically,as a proof of concept, we show that by simply using different data augmentationsfor base learners, we can respectively double and triple the certified robustness of
DPA on CIFAR-10 and GTSRB without sacrificing accuracy.
Introduction
With the increasing popularity of machine learning and especially deep learning, concerns about thereliability of training data have also increased: typically, because of the availability of data, manytraining samples have to be collected from users, internet websites or other potentially malicioussources for satisfying utilities.
This motivates the development of the data poisoning threat model,which focuses on the reliability of models trained from adversarially distorted data [11].
Data poisoning is a class of training-time adversarial attacks: The adversary is given the ability toinsert and remove a bounded number of training samples to manipulate the behavior (e g. predictionsfor some target samples) of models trained using the resulted, poisoned dataset.
Models trained fromthe poisoned dataset are referred to as poisoned models and the allowed number of insertion andremoval is the attack size.
To challenge the reliability of poisoned models, many variants of poisoning attacks have beenproposed, including triggerless attacks [26, 35, 1, 10], which do not modify target samples, andbackdoor attacks [3, 30, 25], which modify them.
In addition, in cases where the adversary canachieve its goal with a non-negligible probability by doing nothing, a theoretical study [21] discoversa provable attack replacing ˜ O ( √ N ) samples in a size- N training set.
Meanwhile, defenses againstdata poisoning are also emerging, including detection-based defenses [28, 8, 23, 29, 31], trying to identify poisoned samples, and training-based defenses [33, 24, 20, 13, 18, 15, 2, 32], aiming atrobustifying models trained from poisoned data.
In this work, we target the fundamentals of data poisoning and ask:
• What amount of poisoned samples will make a specific task impossible?
The answer we propose is the following conjecture, characterizing the lethal dose for a specific task:
Lethal Dose Conjecture (informal)
For a given task, if the most data-efficient learner takes at least n clean samples for an accurate prediction, then when a potentially poisoned, size- N training set is given, any defense can tolerate at most Θ( N/n ) poisoned samples while ensuring accuracy.
Notably, the conjecture considers an adversary who 1. knows the underlying distribution of clean dataand 2.
can both insert and remove samples when conducting data poisoning.
The formal statementof the conjecture is in Section 4.1.
We prove the conjecture in multiple settings including Gaussianclassifications and offer a general perspective through distribution discrimination.
Theoretically, the conjecture relates learning from poisoned data with learning from clean data,reducing robustness against data poisoning to data efficiency and offering us a much more intuitiveway to estimate the upper bound for robustness: To find out how many poisoned samples are tolerable,one can now instead search for data-efficient learners.
In addition, the conjecture implies that Deep Partition Aggregation (DPA) [18] and its extension,
Finite Aggregation (FA) [32] are (asymptotically) optimal: DPA and FA are recent approachesfor provable defenses against data poisoning, predicting through the majority vote of many basemodels trained from different subsets of training set using a given base learner.
If we have the mostdata-efficient learner, DPA and FA can turn it into defenses against data poisoning that approach theupper bound of robustness indicated by Lethal Dose Conjecture within a constant factor.
The optimality of DPA and FA (indicated by the conjecture) outlines a practical approach to developingstronger defenses by finding more data-efficient learners.
As a proof of concept, we show on CIFAR10 and GTSRB that by simply using different data augmentations to improve the data efficiencyof base learners, the certified robustness of DPA can be respectively doubled and tripled withoutsacrificing accuracy, highlighting the potential of this practical approach.
Another implication from the conjecture is that a stronger defense than DPA (which is asymptoticallyoptimal assuming the conjecture) implies the existence of a more data-efficient learner.
As an example,we show how to derive a learner from the nearest neighbors defenses [14], which are more robust than
DPA in their evaluation .
With the derived learner, DPA offers similar robustness to their defenses.
In summary, our contributions are as follows:
• We propose Lethal Dose Conjecture , characterizing the largest amount of poisoned samples anydefense can tolerate for a specific task;
• We prove the conjecture in multiples cases including Gaussian classifications;• We offer a general perspective supporting the conjecture through distribution discrimination;•
We showcase how more data-efficient learners can be optimally (assuming the conjecture)transferred into stronger defenses: By simply using different augmentations to get more dataefficient base learners, we double and triple the certified robustness of DPA respectively onCIFAR-10 and GTSRB without sacrificing accuracy;•
We illustrate how a learner can be derived from the nearest neighbors defenses [14], which, intheir evaluation, are much more robust than DPA (which is asymptotically optimal assuming
theconjecture)—Given the derived learner, DPA transfers it to a defense with comparable robustnessto the nearest neighbors defenses.
Related Work
DPA [18] and FA [32] we discuss in Section 7 are pointwise certified defenses against data poisoning,where the prediction on every sample is guaranteed unchanged within a certain attack size.
[15, 2]offer similar but probabilistic pointwise guarantees.
Meanwhile, [33, 24] use randomized smoothingto provably defend against backdoor attacks [33] and label-flipping attacks [24], [8] provably approximates clean models assuming certain clean data distributions, and [9] studies the PAC learnabilityand certified predictions under poisoning attacks.
Background and Notation
We focus on the classification problems in this work using the following notations.
Classification Problem : A classification problem ( X, Y, Ω , P, F ) consists of: X —the space ofinputs; Y —the space of labels; Ω —the space of all labeled samples
X × Y ; P —the distribution overΩ that is unknown to learners; and F —the set of plausible learners.
Learner :
For a classification problem ( X, Y, Ω , P, F ) , a learner f ∈ F is a (stochastic) functionf : Ω N → C mapping a (finite) training set to a classifier, where C denotes the set of all classifiers.
A classifier is a (stochastic) function from the input space X to the label set Y , where the outputs arecalled predictions.
For a learner f , f D denotes the classifier corresponding to a training set D ∈ Ω
Nand f D ( x ) denotes the prediction of the classifier for input x ∈ X .
Clean Learning : In clean learning, given a classification problem ( X, Y, Ω , P, F ) , a learner f ∈ Fand a training set size n , the learner has access to a clean training set containing n i.i.d.
samples from
P and thus the resulting classifier can be denoted as f D n where D n ∼ P n is the size- n training set.
Poisoned Learning :
In poisoned learning, given a classification problem ( X, Y, Ω , P, F ) , a learnerf ∈ F , a training set size N and a transform T : Ω N → Ω N , the learner has access to a poisonedtraining set obtained by applying T to the clean training set and thus the resulting classifier can bedenoted as f T ( D N ) where D N ∼ P N .
The symmetric distance between the clean training set andthe poisoned training set, i.e. | T ( D N ) − D N | = | ( T ( D N ) \ D N ) ∪ ( D N \ T ( D N )) | , is called theattack size , which corresponds to the minimum total number of insertions and removals needed tochange one training set to the other.
Total Variation Distance : The total variation distance between two distributions U and V overthe sample space
Ω is δ ( U, V ) = max
A ⊆ Ω | U ( A ) − V
( A ) | , which denotes the largest difference ofprobabilities on the same event for U and V .
Lethal Dose Conjecture
4.1 The Conjecture
Below we present a more formal statement of Lethal Dose Conjecture:
Lethal Dose Conjecture
For a classification problem ( X, Y, Ω , P, F ) , let x 0 ∈
X be an input and y
=
arg max y P ( y | x 0 ) be the maximum likelihood prediction.
• Let n be the smallest training set size such that there exists a learner f ∈ F with Pr
[ f D n ( x 0 ) =
y 0 ] ≥ τ for some constant τ ; • For any given training set size N , and any learner f ∈ F , there is a mapping T : Ω N → Ω N with Pr [ f T ( D N ) ( x 0 ) =
y 0 ] ≤ 1 / | Y | while E [ | T ( D N ) − D N | ] ≤ Θ(1 /n ) · N .
Informally, if the most data-efficient learner takes at least n clean samples for an accurate prediction,then when a potentially poisoned, size-N training set is given, any defense can tolerate at mostΘ( N/n ) poisoned samples while ensuring accuracy.
Intuitively, the conjecture makes sense—if one needs n samples to have a basic understanding of thetask, then when Θ( N/n ) samples are poisoned in a size- N training set, sampling n training sampleswill in expectation contain some poisoned samples, preventing one from learning the true distribution.
Theoretically, the conjecture offers an intuitive way to think about or estimate the upper bound forrobustness for a given task: To find out how many poisoned samples are tolerable, one now considersdata efficiency instead.
Practical implications are included in Section 7.
4.2 Proving The Conjecture in Examples
In this section we present two examples of classification problems, where we can precisely prove
Lethal Dose Conjecture.
For coherence, we defer the proofs to Appendix A, B, C and D.
4.2.1 Bijection Uncovering
Definition 1 (Bijection Uncovering) .
For any k , Bijection Uncovering is a k-way classification task( X, Y, Ω , P, F ) defined as follows:
• The input space X and the label set Y are both finite sets of size k
(i e. | X | = | Y | = k );
• The true distribution P corresponds to a bijection g from X to Y (note that g is unknown to
learners), and ∀ x ∈ X, ∀ y ∈ Y, P ( x, y ) =
1 [ g ( x ) =
y ] /k ;
• For y, y ′ ∈ Y , let T y ↔ y
Ω N → Ω N be a transform that exchanges all labels y and y ′ in a the
given training set (i
e. if a sample is originally labeled y , its new label will become y ′ and vice
versa).
The set of plausible learners F contains all learners f such that Pr [ f D ( x 0 ) =
y ]
=
Pr [ f T y ↔ y ′
( D ) ( x 0 ) =
y ′ ] for all y, y ′ ∈ Y and D ∈ Ω N .
This is the setting where there is a one-to-one correspondence between inputs and classes.
This isconsidered the ‘easiest’ classification as it describes the case of solving a k-way classification givena pre-trained, perfect feature extractor that puts samples from the same class close to each othersand samples from different classes away from each others.
In this case, since one knows whethertwo samples belong to the same class or not, samples can be divided into k clusters and the task isessentially uncovering the bijection between clusters and class labels.
Lemma 1 (Clean Learning of Bijection Uncovering) .
In Bijection Uncovering, given a constantτ ∈ (1 / 2 , 1) , for any input x 0 ∈ X and the corresponding maximum likelihood prediction y 0 =arg max y P ( y | x 0 ) , let n be the smallest training set size such that there exists a learner f ∈ F with
Pr [ f D n ( x 0 ) =
y 0 ] ≥ τ .
Then n ≥ Θ( k ) , i.e. the most data-efficient learner takes at least Θ( k )clean samples for an accurate prediction.
Lemma 2 (Poisoned Learning of Bijection Uncovering) .
In Bijection Uncovering, for any inputx 0 ∈ X and the corresponding maximum likelihood prediction y 0 = arg max y P ( y | x 0 ) , for anygiven training set size N , and any learner f ∈ F , there is a mapping T : Ω N → Ω N with
Pr [ f T ( D N ) ( x 0 ) =
y 0 ] ≤ 1 / | Y | while E [ | T ( D N ) − D N | ] ≤ Θ(1 /k ) · N , i.e. poisoning Θ(1 /k )
ofthe entire training set is sufficient to break any defense.
From Lemma 1 and Lemma 2, we see for Bijection Uncovering, the most data-efficient learner takesat least Θ( k ) clean samples to predict accurately and any defense can tolerate at most Θ(1 /k )
ofthe training set being poisoned.
The two quantities are inversely proportional , just as indicated by
Lethal Dose Conjecture.
Definition 2 (Instance Memorization) .
For any k and m , Instance Memorization is a k-way classification task ( X, Y, Ω , P, F ) defined as follows:
• The input space X and the label set Y are both finite with | X | = m and | Y | =
k ;
• The true distribution P corresponds to a mapping g from X to Y (note that g is unknown to learners), and ∀ x ∈ X, ∀ y ∈ Y, P ( x, y ) =
1 [ g ( x ) =
y ] /m ;
• For y, y ′ ∈ Y , let T y ↔ y ′
| x 0 : Ω N → Ω N be a transform that exchanges labels y and y ′ for allsamples with an input x 0 in a given training set (i.e. all ( x 0 , y ) will become ( x 0 , y ′ ) and viceversa).
The set of plausible learners F contains all learners f such that Pr [ f D ( x 0 ) =
y ]
=
Pr [ f T y ↔ y
′| x 0 ( D ( x
) =
y ′ ] for all y, y ′ ∈ Y and D ∈ Ω N .
This is the setting where labels for different inputs are independent and learners can only predictthrough memorization.
This is considered the ‘hardest’ classification problem in a sense that theinputs are completely uncorrelated and no information is shared by labels of different inputs.
Lemma 3 (Clean Learning of Instance Memorization) .
In Instance Memorization, given a constantτ ∈ (1 /k, 1) , for any input x 0 ∈ X and the corresponding maximum likelihood prediction y 0 =arg max y P ( y | x 0 ) , let n be the smallest training set size such that there exists a learner f ∈ F with
Pr [ f D n ( x 0 ) =
y 0 ] ≥ τ .
Then n ≥ Θ( m ) , i.e. the most data-efficient learner takes at least Θ( m )clean samples for an accurate prediction.
Lemma 4 (Poisoned Learning of Instance Memorization) .
In Instance Memorization, for anyinput x 0 ∈ X and the corresponding maximum likelihood prediction y 0 = arg max y P ( y | x 0 ) , forany given training set size N , and any learner f ∈ F , there is a mapping T : Ω N → Ω N with
Pr [ f T ( D N ) ( x 0 ) =
y 0 ] ≤ 1 / | Y | while E [ | T ( D N ) − D N | ] ≤ Θ(1 /m ) · N , i.e. poisoning Θ(1 /m )
ofthe entire training set is sufficient to break any defense.
From Lemma 3 and Lemma 4, we observe that for Instance Memorization, the most data-efficientlearner takes at least Θ( m ) clean samples to predict accurately and any defense can tolerate at mostΘ(1 /m ) of the training set being poisoned.
The two quantities are, again, inversely proportional ,which is consistent with Lethal Dose Conjecture.
This is likely no coincidence and more supports arepresented in the following Sections 5 and 6.
An Alternative View from Distribution Discrimination
In this section, we offer an alternative view through the scope of Distribution Discrimination.
Intuitively, if for two plausible distribution U and V over Ω , the corresponding optimal predictions onsome x 0 are different (i.e. arg max y Pr [ U ( y | x 0 )]
̸ = arg max y Pr [ V ( y | x 0 )] ), then a learner must(implicitly) discriminate these two distributions in order to predict correctly.
With this in mind, wepresent the following theorems.
The proofs are respectively included in Appendix E and F.
Theorem 1.
Given two distributions U and V over Ω , for any function f : Ω N → { 0 , 1 } , we have
[Equation]
where δ ( U, V ) is the total variation distance between U and V .
Thus for a constant τ ,
E D ∼ U n [ f ( D )] − E D ∼ V n [ f ( D )] ≥ τ implies n ≥ τ/δ
( U, V ) =
Θ(1 /δ
( U, V )) .
Theorem 2.
Given two distributions U and V over Ω , for any function f : Ω N → { 0 , 1 } , thereis a stochastic transform T : Ω N → Ω N , such that T ( U N ) is the same distribution as V N (thus
E D ∼ U N [ f ( T ( D ))] − E D ∼ V N [ f ( D )] = 0 ) and E D ∼ U N [ | T ( D ) − D | ]
= δ ( U, V ) · N .
Theorem 1 implies that to discriminate two distributions with certain confidence, the number ofclean samples required is at least Θ(1 /δ
( U, V )) , which is proportional to the inverse of their totalvariation distance; Theorem 2 suggests that for a size- N training set, the adversary can make the twodistribution indistinguishable given the ability to poison δ ( U, V ) · N samples in expectation.
This isconsistent with the scaling rule suggested by the Lethal Dose Conjecture.
We will also see in Section6 how the theorems help the analysis of classification problems.
[Figure]
Verifying Lethal Dose Conjecture in Gaussian Classification
In this section, we analyze the case where data from each class follow a multivariate Gaussiandistribution and we will prove the very same scaling rule stated by Lethal Dose Conjecture.
Definition 3 (Gaussian Classification) .
For any k and d , Gaussian Classification is a k-way classification task ( X, Y, Ω , P, F ) defined as follows:
• The input space
X is R d and the label set Y is a finite set of size k
( i
e. | Y | = k );
• For y ∈ Y , µ y ∈ R
d
denotes the (unknown) center of class y and the true distribution P is:(cid:40)
[Equation]
[Equation]
)( ∀
[Equation]
• The set of plausible learners F contains all unbiased, non-trivial learners, meaning that for anyf ∈ F , given any x 0 , and any plausible P (i
e. the same form as Equation 1 but with potentiallydifferent µ y )
where y
= arg max y P ( y | x 0 ) is unique, we have Pr [ f D n ( x 0 ) =
y 0 ] > 1 k and( ∀ y ̸
= y 0 )
Pr [ f D n ( x 0 ) =
y ] ≤ 1 k for all n ≥ 1 .
First we present a claim regarding the optimal prediction y 0 corresponding to a given input x 0 :
Claim 1.
In Gaussian Classification, for any x 0 , the corresponding maximum [-likelihoodprediction y 
-]likelihood predictiony 0 = arg max y P ( y | x 0 ) is arg min
y || x 0 − µ y || .
This follows directly from Equation 1: y 0 = arg max y P ( y | x 0 ) =
arg max y 1 √ 2 π e −|| x − µ y || 2 / 2 =arg min
y || x − µ y || .
Without loss of generality and with a slight abuse of notation, we assume inthe rest of Section 6 that µ 1 , µ 2 are respectively the closest and the second closest class centers tox
and therefore y
= 1 is the optimal prediction.
Let d 1 = ||
x
0 − µ 1 ||
and d 2 = || x 0 − µ 2 || be thedistances of µ 0 and µ 1 to x 0 .
An illustration is included in Figure 1(a).
First we define a parameter ∆ that will later help us to analyze both clean learning and poisonedlearning of Gaussian Classification: ∆ is the total variation distance between two isotropic Gaussiandistribution with centers separated by a distance of d 2 − d 1 , i.e. ∆ =
δ ( U, V ) for some U = N ( µ, I )and V = N ( µ ′ , I ) where || µ − µ ′ ||
=
d 2 − d 1 .
We introduce a lemma proved by Devroye et al.
Lemma 5 (Total variation distance for Gaussians with the same covariance[7]) .
For two Gaussian distribution U = N ( µ, Σ) and V = N ( µ ′ , Σ) , their total variation distance is δ ( U, V ) =
Pr [ N (0 , 1) ∈ [ − r, r ]] where r = (cid:112) ( µ − µ ′ )
T Σ − 1 ( µ − µ ′ ) / 2 .
Thus ∆ =
Pr [ |N (0 , 1) | ≤ d 2 − d 1 ]
= erf (cid:0) ( d 2 − d 1 ) / √ 2 (cid:1) where erf ( x ) =
(2 / √ π ) · (cid:82) x 0 e − t 2 dt
Lemma 6 (Clean Learning of Gaussian Classification) .
In Gaussian Classification, given a constantτ ∈ (1 / 2 , 1) , for any input x 0 ∈ X and the corresponding maximum likelihood prediction y 0 =arg max y P
( y | x 0 ) , let n be the smallest training set size such that there exists a learner f ∈ Fwith Pr [ f D n ( x 0 ) =
y 0 ] ≥ τ .
Then n ≥ Θ( k/ ∆) , i.e. the most data-efficient learner takes at leastΘ( k/ ∆) clean samples for an accurate prediction.
Lemma 7 (Poisoned Learning of Gaussian Classification) .
In Gaussian Classification, for anyinput x 0 ∈ X and the corresponding maximum likelihood prediction y 0 = arg max y P ( y | x 0 ) , forany given training set size N , and any learner f ∈ F , there is a mapping T : Ω N → Ω N with
Pr [ f T ( D N ) ( x 0 ) =
y 0 ] ≤ 1 / | Y | while E [ | T ( D N ) − D N | ] ≤ Θ(∆ /k )
· N , i.e. poisoning Θ(∆ /k ) ofthe entire training set is sufficient to break any defense.
The proofs of Lemma 6 and Lemma 7 are included respectively in Appendix G and H. We show thatfor Gaussian Classification, the most data-efficient learner takes at least Θ( k/ ∆) clean samples topredict accurately and any defense can be broken by poisoning Θ(∆ /k ) of the training set, onceagain matching the statement of Lethal Dose Conjecture.
Practical Implications
7.1 (Asymptotic) Optimality of DPA and Finite Aggregation
In this section, we highlight an important implication of the conjecture: Deep Partition Aggregation[18] and its extension, Finite Aggregation [32] are (asymptotically) optimal—Using the most dataefficient learner, they construct defenses approaching the upper bound of robustness indicated by
Lethal Dose Conjecture within a constant factor.
Deep Partition Aggregation [18]: DPA predicts through the majority votes of base learners trainedfrom disjoint data.
Given the number of partitions k as a hyperparameter, DPA is a learner constructedusing a deterministic base learner f and a hash function h : X × Y → [ k ] mapping labeled samplesto integers between 0 and k − 1 .
The construction is:
[Equation]
where P i = { ( x, y ) ∈ D | h ( x, y ) = i } is a partition containing all training samples with a hashvalue of i and DPA D ( x 0 , y )
= 1 k (cid:80) k − 1 i =0 1 [ f P i ( x 0 ) =
y ] denotes the average votes count for class y .
Ties are broken by returning the smaller class index in arg max .
Theorem 3 (Certified Robustness of DPA against Data Poisoning [18]) .
Given a training set D andan input x 0 , let y
= DPA D ( x 0 ) , then for any training set D ′ with
[Equation]
we have DPA D ( x 0 ) =
DPA D ′ ( x 0 ) , meaning the prediction remains unchanged with data poisoning.
Let n be the average size of partitions, i.e. n = (cid:80)
k − 1 i =
| P i | /k
= N/k where N = | D | is the size ofthe training set.
Assuming the hash function h : X × Y →
[ k ] uniformly distributes samples intodifferent partitions, we have DPA D ( x 0 , y )
≈
Pr [ f D n ( x 0 )
=
y ] and therefore the right hand side of
Equation 2 approximates
[Equation]
{+y 
+}
Pr [ f D n ( x
[Equation]
= y ]
[Equation] n · N = [Equation] (cid:18)n (cid:19) · N when the base learner f offers a margin ∆ =
Pr [ f D n ( x 0 ) =
y 0 ] − max y ̸
= y
Pr [ f D n ( x 0 ) =
y ] ≥ τwith n samples for some constant τ .
When f is the most data-efficient learner taking n clean sampleswith a margin ∆ ≥ τ for an input x 0 , then given a potentially poisoned, size- N training set is given,
DPA (using f as the base learner) tolerates Θ ( N/n ) poisoned samples, approaching the upper boundof robustness indicated by Lethal Dose Conjecture within a constant factor.
The analysis for Finite Aggregation (FA) is exactly the same by substituting Theorem 3 with thecertified robustness of FA.
For details, please refer to Theorem 2 by Wang et al.
[Figure]
7.2 Better Learners to Stronger Defenses
[Figure]
If assuming the conjecture is true, then we have in our hand the (asymptotically) optimal approachesto convert data-efficient learners to strong defenses against poisoning.
This reduces developingstronger defenses to finding more data-efficient learners.
In this section, as a proof of concept, weshow that simply using different data augmentations can increase the data efficiency of base learnersand vastly improve ( double or even triple ) the certified robustness of DPA, essentially highlightingthe potential of finding data-efficient learners in defending against data poisoning, which has been ablind spot for our community.
Setup.
Following Levine and Feizi [18], we use Network-In-Network[19] architecture for all baselearners and evaluate on CIFAR-10[16] and GTSRB[27].
For the baseline (DPA_baseline), wefollow exactly their learning rates (initially 0 .
1 , decayed by a factor of 1 / 5 at 30% , 60% , and 80%of the training process), batch size ( 128 ) and total epochs ( 200 ).
For our results (DPA_aug0 and
DPA_aug1 on CIFAR-10; DPA_aug on GTSRB), we use predefined AutoAugment [4] policies fordata augmentations, where DPA_aug0 uses the policy for CIFAR-10, DPA_aug1 uses the policy for
Imagenet and DPA_aug uses the policy for SVHN, all included in torchvision[22].
We use an initiallearning rate of 0 .
005 , a batch size of 16 for 600 epochs on CIFAR-10 and 1000 epochs on GTSRB.
Evaluation.
First, we evaluate the test accuracy of base learners with limited data (i.e. using 1 /kof the entire training set of CIFAR-10 and GTSRB where k ranges from 50 to 500).
In Figure 2(a),simply using AutoAugment greatly improves the accuracy of base learners with limited data: On
CIFAR-10, with both augmentation policies, the augmented learners achieve similar accuracy as thebaseline using only 1 / 2 of data (i.e. with k that is twice as large);
On GTSRB, the augmented learnerachieves similar accuracy as the baseline using only 1 / 4 ∼ 1 / 3 of data.
Now we use more data-efficient learners in DPA to construct stronger defenses.
For baselines, weuse DPA with k = 50 , 100 , 250 on CIFAR-10 and k = 50 , 100 on GTSRB, defining essentially 5settings.
When using augmented learners in DPA, we increase k accordingly so that the accuracy of
[Figure]
i ﬁ e d A cc u r a c y
( C
A ( e ))
Bagging
DPA kNN rNN
[Figure]
base learners remains at the same level as the baselines, as indicated by Figure 2(a).
On CIFAR-10, in
Figure 2(b), 2(c) and 2(d), the attack size tolerated by DPA is roughly doubled using both augmentedlearners for similar certified accuracy.
On GTSRB, in Figure 2(e) and 2(f), the attack size toleratedby DPA is more than tripled using the augmented learner for similar certified accuracy.
The improvements are quite significant, corroborating the potential of this practical approach—developing stronger defenses against poisoning through finding more data-efficient learners.
7.3 Better Learners from Stronger Defenses
If assuming the conjecture is true, when some defense against data poisoning is clearly more robustthan DPA (even in restricted cases), there should be a more data-efficient learner.
Jia et al.
[14]propose to use nearest neighbors, i.e. kNN and rNN, to certifiably defend against data poisoning.
Interestingly, in their evaluations, kNN and rNN tolerate much more poisoned samples than DPAwhen the required clean accuracy is low, as shown in Figure 3(a).
Here we showcase how a learner isderived from their defense, which, when being used as base learners, boosts the certified robustnessof DPA to a similar level to the nearest neighbors defenses.
In [14], histogram of oriented gradients (HOG) [5] is used as the predefined feature space to estimatethe distance between samples.
Consequently, we use radius nearest neighbors over HOG featuresas the base learner of DPA.
Given a threshold r , radius nearest neighbors find the majority votes oftraining samples within an ℓ 1 distance of r to the test sample.
When there is no training sample witha distance of r , we let the base learner output a token ⊥ denoting outliers so that it will not affect theaggregation.
The results are included in Figure 3(b), where DPA offers similar robustness curves askNN and rNN in Figure 3(a) by using the aforementioned base learner and hence the same prior.
Conclusion
In this work, we propose Lethal Dose Conjecture , which characterizes the largest amount ofpoisoned samples any defense can tolerate for a specific task.
We prove the conjecture for multiplecases and offer general theoretical insights through distribution discrimination.
The conjectureimplies the (asymptotic) optimality of DPA [18] and FA [32] in a sense that they can transfer themost data-efficient learners to one of the most robust defenses, revealing a practical approach toobtaining stronger defenses via improving data efficiency of (non-robust) learners.
Empirically, asa proof of concepts, we show that simply using different data augmentations can increase the dataefficiency of base learners, and therefore respectively double and triple the certified robustness of
DPA on CIFAR-10 and GTSRB.
This highlights both the importance of Lethal Dose Conjecture andthe potential of the practical approach in searching for stronger defenses.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes]
In Section 1.
(c) Did you discuss any potential negative societal impacts of your work?
[No]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes]
(b) Did you include complete proofs of all theoretical results?
[Yes] Some proofs are
included in Appendix.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
[Table]
mental results (either in the supplemental material or as a URL)?
[Yes]
[Table]
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they
[Table]
were chosen)?
[Yes]
[Table]
(c) Did you report error bars (e.g., with respect to the random seed after running experi-
ments multiple times)?
[No]
[Table] (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[No]
[Table]
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
[Table]
(a) If your work uses existing assets, did you cite the creators?
[Yes]
[Table]
(b) Did you mention the license of the assets?
[No]
(c) Did you include any new assets either in the supplemental material or as a URL?
[No]
[Table]
(d) Did you discuss whether and how consent was obtained from people whose data you’reusing/curating?
[N/A]
[Table]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[No] Both CIFAR-10 and GTSRB are standardbenchmarks. [Table]
If you used crowdsourcing or conducted research with human subjects...
[Table] (a) Did you include the full text of instructions given to participants and screenshots, if [Table]
applicable?
[N/A]
[Table]