Why Do Artificially Generated Data Help Adversarial Robustness?
Abstract
In the adversarial training framework of Carmon et al.
; Gowal et al.
(2021), people use generated/real unlabeled data with pseudolabels to improve adversarial robustness.
We provide statistical insights to explain why the artificially generated data improve adversarial training.
In particular, we study how the attack strength and the quality of the unlabeled data affect adversarial robustness in this framework.
Our results show that with a high-quality unlabeled data generator, adversarial training can benefit greatly from this framework under large attack strength, while a poor generator can still help to some extent.
To make adaptions concerning the quality of generated data, we propose an algorithm that performs online adjustment to the weight between the labeled real data and the generated data, aiming to optimize the adversarial risk.
Numerical studies are conducted to verify our theories and show the effectiveness of the proposed algorithm.
Introduction
Adversarial training is a popular and simple way to improve the adversarial robustness of modern machine learning models.
There are fruitful results in the theoretical justification and methodology development.
Among various research directions, one interesting aspect is to use extra unlabeled data to assist adversarial training.
Recent works successfully demonstrate a great improvement in the adversarial robustness with additional unlabeled data.
For example, Carmon et al.
Xing et al.
(2021b, 2022), show that additional external real data help improve adversarial robustness; Gowal et al.
(2021) uses synthetically generated data to improve the adversarial robustness and achieves the highest 66% adversarial testing accuracy for CIFAR-10 dataset under AutoAttack (AA) in Croce et al.
(2020) 1 in the literature.
The algorithms utilizing real/generated unlabeled data in adversarial training in Carmon et al.
; Gowal et al.
(2021) are summarized in Figure 1.
Note that to unify the these two algorithms, we view the unlabeled real data used in Carmon et al.
as synthetic data generated from an ideal generator .
However, two fundamental questions about utilizing additional unlabeled data remain unclear.
First, as shown by Carmon et al.
(2019), unlabeled data provide more information about the density of the data near the decision boundary, leading to a more robust adversarial estimator (e.g., it increases the robust test accuracy of CIFAR-10 from 53.08% to 59.53% for 8/255 L ∞ attack).
However, the performance improvement of this training strategy is limited when the attack strength is small or zero.
Therefore, it is natural to ask
Q1:
Compared to clean training, why and how can adversarial training significantly benefit from unlabeled data under the framework of Figure 1?
Second besides the ideal generator in Carmon et al.
Gowal et al.
(2021) observe two other interesting phenomena regarding the quality of the generated unlabeled data.
When the unlabeled data are generated from a generative model trained from original labeled data, although they introduce no more information beyond the original labeled data, the adversarial robustness gets enhanced.
On the other hand they also generate synthetic data from a multivariate Gaussian which is much different from the real data.
In this case the generated data still slightly improve the robustness.
Hence, it is important to investigate that
Q2:
How does the quality of the unlabeled data generator affect the adversarial robustness?
This work aims to provide mathematical answers for the above two questions.
Existing literature, (e.g., Alayrac et al., 2019; Uesato et al., 2019; Carmon et al., 2019; Zhai et al., 2019; Sehwag et al., 2021;
Xing et al., 2021b, 2022) provide a justification that adversarial training potentially benefit from unlabeled/generated data and fail to explain that unlabeled data help little when attack strength is small.
In contrast, we explicitly explain why and how adversarial training can be improved via an exact error decomposition rather than an upper bound statement.
For the convenience of our discussion below, we preliminarily introduce some notations, with detailed explanations given later.
Denote (cid:101) θ ( ϵ ) as the robust estimator utilizing additional unlabeled data as in
Figure 1, and (cid:98) θ ( ϵ ) as the estimator via vanilla adversarial training.
Denote the true distribution of X as P 0 , and the distribution of the generated unlabeled data as P a .
The true conditional distribution of Y | X is P y , and Y | X − E
[ Y | X ] has a distribution P ϵ .
As defined in Figure 1, S 1 is the set of original labeled training data with sample size n 1 , and S 2 is the set of generated data with size n 2 .
Denote α = n 1 / ( n 1 + n 2 ) as the proportion of real data.
Let l ϵ be the adversarial loss function w.r.t.
attack strength ϵ , l 0 be the clean loss (i.e., ϵ = 0 ), and θ 0 be the parameter of the underlying true clean model.
Contributions We conduct theoretical analysis under some simple statistical setups.
Through studying the convergence of (cid:101) θ ( ϵ ) and (cid:98) θ
( ϵ ) , we decompose the excess adversarial risk of (cid:101) θ ( ϵ ) into two major components (and their cross product): label cost (related to the quality of the imputed labels of generated unlabeled data) and generator cost (related to the quality of unlabeled data generator) as in
Figure 2.
Using this decomposition, we can answer Q1 and Q2 .
First, unlike (cid:98) θ ( ϵ ) whose risk noticeably affected as ϵ increases, since the adversarial risk is stable to the imputation model, the label cost of (cid:101) θ ( ϵ ) is more insensitive w.r.t.
ϵ .
In consequence, although in clean training, the label cost and the risk of (cid:98)
θ ( ϵ ) are comparable, the former one becomes relatively smaller as ϵ increases.
As in Figure 2, when ϵ > 0 , the label cost of (cid:101) θ ( ϵ ) is smaller than the risk of (cid:98) θ ( ϵ ) .
Using infinite data from the ideal unlabeled data generator, the generator cost becomes zero, and hence the overall cost of (cid:101) θ ( ϵ ) is smaller than (cid:98) θ ( ϵ ) .
Second, we characterize how the quality of the unlabeled data generator affects the final adversarial robustness.
In general, a better data generator is always preferred.
For the ideal generator, it will always improve adversarial robustness effectively , i.e., the generator cost is negligible when n 2 → ∞ .
If the generator is learned from S 1 and captures extra information, it may effectively help as well when n 2 → ∞ .
For a poor generator, it may slightly improve the performance for a large attack when n 2 is small, and the generator cost is always not negligible.
Finally, following Gowal et al.
(2021), we balance the weights of S 1 and S 2 in the adversarial training process in order to achieve the optimal adversarial robustness.
We show that a larger n 2 is always preferred with a proper choice of weight, even if generated from non-ideal generators.
Tuning the best choice of weights via repeated trial-and-error runs can be infeasible due to the slow convergence of adversarial training.
We propose an algorithm that dynamically adapts the weight during the training of neural networks and shows its promising performance in Section 5.
Adversarial Training
To formally introduce adversarial training, let l 0 denote the loss function and f θ ( x ) be the model with parameter θ .
The adversarial loss function l ϵ and the (population) adversarial risk are defined as
[Equation]
where A ϵ is an attack of strength ϵ > 0 and intends to deteriorate the loss in the following way
[Equation]
where B p ( x, r ) is a L p ball centering at x with radius r .
Denote θ ϵ = argmin
θ R ( θ, ϵ ) .
Note that for simplicity, we only consider p = 2 in our theorems and simulations, while in the numerical studies of real-data experiment, we follow the routine setup and set p = ∞ .
Given the data set S 1 consisting of n 1 i.i.d.
labeled samples, the estimator (cid:98) θ ( ϵ ) from the vanilla adversarial training aims to minimize the empirical version of R ( θ, ϵ ) :
[Equation]
To take the advantage of the extra unlabeled data, we first assign pseudo-response variable (cid:98) y
= g ( f (cid:98) θ (0) ( x ) , ε ) to them, where for regression, g ( a, b )
= a + b
and ε follows the true noise distribution; for classification, g ( a, b ) = 1( a ≥ b ) and ε ∼ Unif [0 , 1] .
Then (cid:101) θ ( ϵ ) aims to minimize
[Equation]
To evaluate the performance of an estimator θ , we use the excess adversarial risk E R ( θ, ϵ ) − R ( θ ϵ , ϵ ) .
Besides the pseudolabel (cid:98) y , we define the the imaginary “true” response for the unlabeled data y =
g
( f θ 0 ( x ) , ε ) , where ε is the same realization used in generating pseudolabels.
These true responses are not observable but will be used when analyzing the convergence rate of (cid:101) θ ( ϵ ) .
To evaluate the size of vectors and matrices, we denote ∥ · ∥ as the L 2 norm of vectors, operator norm of matrices, and denote ∥ a ∥ 2 A := a ⊤ Aa for any vector a and positive definite matrix A .
Related Literature
Commonly used techniques (without extra data fed into training process) in adversarial training include adversarial regularization Goodfellow et al.
; Zhang et al.
; Wang et al.
(2019b),
curriculum-based algorithms
Cai et al.
; Zhang et al.
(2020a).
Theoretical investigations, besides the aforementioned works, have been conduct for adversarial training from different perspectives.
For instance, Chen et al.
; Javanmard et al.
Javanmard & Soltanolkotabi (2020); Taheri et al.
; Yin et al.
; Raghunathan et al.
Najafi et al.
; Min et al.
; Hendrycks et al.
; Wu et al.
(2020b);
Xing et al.
(2021a) studied the statistical properties of adversarial training, Sinha et al.
; Wang et al.
(2019a) studied the optimization convergence of adversarial training, Zhang et al.
(2020b); Allen-Zhu & Li (2020);
Wu et al.
(2020a) ; Xiao et al. (2021) studied theoretical issues related to adversarial training with Deep learning models.
Main Result
In this section, we present the main theorem, answer the two key questions, and show that balancing the weights of S 1 and S 2 leads to the better utility of the generated data.
Due to the page limit, we postpone the whole simulation study section to Appendix B. Briefly speaking, our simulation results justify all the theoretical results below.
4.1 General Convergence Result
To present our theoretical investigations in (cid:101) θ ( ϵ ) , we first introduce some assumptions:
Assumption 1.
The loss function satisfies the following conditions:
A1.
The clean loss function l 0 is the square loss or logistic loss.
The attributes X ∈ R d follow a sub-Gaussian distribution.
The true clean model θ 0 is defined as E [ Y | X = x
] = x ⊤ θfor regression and P ( Y = 1 | X = x ) = 1 / (1 + exp
x ⊤ θ 0 )) for classification.
In addition, it satisfies that ∥ θ 0 ∥
≤ b 0 for some constant b 0 > 0 and the noise in regression has finite variance.
The distribution P a is sub-Gaussian as well.
A2.
The distribution P a and n 2 satisfy ∥
[Equation]
where P a ⊗ P y denotes the joint distribution induced by a marginal distribution P a and a conditional distribution P y .
Assumption A1 is for the simplicity of derivation.
When doing Taylor expansion as in (4), the remainder term o (if exists) does not explode under A1.
Under Assumption A2, if P a is of poor quality, i.e., ∥ E P a ⊗P y ∂∂θ
ϵ
l ϵ ( X, Y, θ ϵ ) ∥ is not vanishing, then one can only take a relatively small ncompared to n 1 .
Vice versa, to allow a large n 2 , the generator P a need to be almost “unbiased”.
It is possible to relax Assumptions A1 and A2 as discussed in Section E in the appendix.
However, the relaxations are technical and tailored to the derivations of Theorem 1.
The following theorem is a general result to decompose E R ( (cid:101) θ ( ϵ ) , ϵ ) − R
( θ ϵ , ϵ ) :
Theorem 1.
Under Assumption A1 to A2, assuming the unlabeled data generator is independent to
S 1 , when n 1 → ∞ , the excess adversarial risk using (cid:101) θ ( ϵ ) is
[Equation]
[Equation]
where θ ϵ − (cid:101) θ ( ϵ ) is dominated by n 2 Σ − 1 ϵ
[Equation]
and
[Equation]
The term “ o ” represents the remainder terms which are not dominant.
Therefore, taking the decomposition of (cid:101) θ (0) into E R ( (cid:101) θ ( ϵ ) , ϵ ) , the excess adversarial risk becomes
[Equation]
where E (cid:98) θ (0)
∥
E 1 ∥
2Σ ϵ / 2 quantifies how the quality of pseudolabel affects the excess risk (label cost), and E S 1 ,S
∥ E 2 ∥
2Σ ϵ / 2 quantifies how the quality of P a affects the excess risk (generator cost).
Meanwhile, the vanilla adversarial training estimate (cid:98) θ ( ϵ ) satisfies
[Equation]
A proof sketch can be found in Section 6, while the detailed proof is postponed to the appendix.
For simplicity of the derivation, P a in Theorem 1 is independent to S 1 .
A case-by-case study is essential if P a is trained from S 1 .
In Example 2 later, we provide a case where P a is trained from S 1 .
Based on Theorem 1, as shown in Figure 2 in the introduction, the error of (cid:101) θ ( ϵ ) consists of two parts and their cross terms:
Label cost, E (cid:98) θ (0) ∥
E 1 ∥
2Σ ϵ / 2 : It is mainly due to the discrepancy between the pseudolabel (cid:98) y and “true” label y of the generated data.
The label cost depends on both clean training error (cid:98) θ (0) − θ 0 and the matrix Σ − 1 ϵ (cid:101) Σ ϵ , and is maximized when n 2 → ∞ .
It measures the quality of the pseudolabel.
Generator cost, E S 1 ,S 2 ∥ E 2 ∥
2Σ ϵ / 2 :
By the definition of θ ϵ , i.e. the global minima of the adversarial risk, the expectation of ∂∂θ ϵ l ϵ ( x, y, θ ϵ ) under P 0 ⊗ P y is 0 , thus
(cid:80) S 1 ∂∂θ ϵ l ϵ ( x, y, θ ϵ ) =
O p ( √ n 1 ) .
Consequently, the generator cost mostly rely on the term (cid:80) S 2 ∂∂θ ϵ l ϵ ( x, y, θ ϵ ) , which depends on the quality of the unlabeled data generator P a .
For ideal P a , i.e., P a = P 0 , E P a ⊗P y ∂∂θ
ϵ
l ϵ ( X, Y, θ ϵ ) = 0 , and the generator cost goes to zero when n 2 → ∞ .
For non-ideal P a , ∥ E P a ⊗P y ∂∂θ
ϵ l ϵ ( X, Y, θ ϵ ) ∥ may be vanishing, but not exact zero.
Besides Figure 2, we also provide graphical illustration on how the label cost and generator cost change along n 2 /n 1 in Section F to ease the understanding.
4.2 Discrepancy between Clean and Adversarial Training with Unlabeled Data
We provide some intuitions to explain the discrepancy in the algorithm performance between ϵ =and ϵ > 0 from two aspects, the loss function aspect and information (minimax lower bound) aspect.
The following discussion assumes that the ideal data generator is used.
Loss function and label cost We rewrite the adversarial loss as R ϵ
( θ, θ 0 )
=
E l ϵ ( X, g
( f θ 0 ( X ) , ε ) , θ ) to explicitly reveal its dependency on the true parameter θ 0 .
A crucial difference between clean training and adversarial training is the roles of the θ and θ 0 in the loss functions.
For clean loss, e.g. linear regression, the loss function can be rewritten as ( x ⊤ θ − x ⊤ θ 0 − ε ) 2 , where θ and θ 0 play symmetric roles.
In contrast, for adversarial loss, the loss becomes (( x + A ϵ ( f θ , x, y ))
⊤ θ − x ⊤ θ 0 − ε ) 2 , thus θ and θ 0 have asymmetric roles.
The adversarial loss is more sensitive to θ than θ 0 .
Consequently, when (cid:98) θ (0) is treated as true parameter and used to impute labels for S 2 , the impact of the error in (cid:98) θ (0) is much less influential for adversarial training than clean training.
In Theorem 1, the matrix (cid:101) Σ
ϵ reflects the sensitivity mentioned in the above illustration.
Assume ϵ
=
0 , for both
A smaller-than-one ratio implies better performance of (cid:101) θ ( ϵ ) .
Detailed model information is in linear regression and logistic regression, we have − (cid:101) Σ
= Σ 0 , so when n 2 → ∞ ,
[Equation]
which means that clean training cannot be improved using S 2 with pseudolabel.
On the other hand, for adversarial training, the matrix Σ − 1 ϵ (cid:101) Σ
ϵ is no longer − I d , and the singular values get changed.
As a consequence, the label cost is cheaper than the overall risk of (cid:98) θ ( ϵ ) even if the label cost is maximized as n 2 → ∞ .
A toy simulation in Figure 3 justifies this.
In Figure 3, we calculate the theoretical values of the label cost of (cid:101) θ ( ϵ ) and the risk of (cid:98) θ ( ϵ ) to compare them.
When ϵ gets larger, the label cost in (cid:101) θ ( ϵ ) becomes much smaller compared to the overall risk of (cid:98) θ ( ϵ ) .
Together with an ideal P a that yields zero generator cost ( n 2 → ∞ ), we obtain that (cid:101) θ ( ϵ ) is much better than (cid:98) θ ( ϵ ) .
The simulation results in the left panel of Figure 4 verifies this as well.
All these theoretical insights and numerical observations under ϵ > 0 , are summarized in Figure 2.
Figures 2 and 3 jointly are the main answer towards Q1 .
Note that we assert adversarial training benefits much more from unlabeled data than clean training, rather than that clean training cannot benefit from unlabeled data.
See the remark below:
Remark 1.
It is possible to improve clean training using unlabeled data with pseudolabels.
In Lee et al.
; Sun et al.
(2017), in each training iteration, pseudolabels are updated based on the model of the current step.
This is equivalent to a SGD with the gradient of (cid:80) S 1 l ( f θ ( x ) , y ) /n
+ L ( θ, θ ( t ) ) for some regularization term L that stabilizes the training process, where θ ( t ) as the current model.
Minimax lower bound Besides the exact decomposition of (cid:101) θ ( ϵ ) to answer Q1 , we also provide minimiax lower bound result to reveal another difference between clean and adversarial training.
Minimax rate refers to the best possible convergence rate that can be achieved by any estimator in the worst case given finite samples.
As mentioned in Dan et al.
(2020), it represents the informationtheoretical limit given the data set.
In general, for the clean estimate without model misspecification, this information limit regards mostly the uncertainty about the conditional distribution of Y given X = x , which completely determines θ 0 .
The additional information about the marginal distribution of X only affects the multiplicative constant of the bound but not the rate of convergence.
However, the true robust model θ ϵ can depend on both Y | X and X for the adversarial estimate.
The following example demonstrates the difference between clean and adversarial training:
Example 1.
Consider linear regression under Gaussian model Y =
θ ⊤ 0 X + ε where X ∼ N ( 0 , Σ) and ε ∼ N (0 , σ 2 ) .
Then θ 0 is the best clean model, and θ ϵ
= ( I + λ ϵ Σ − 1 ) − 1 θ 0 for some λ ϵ > 0 .
The derivation of Example 1 can be found in Appendix H. Based on Example 1, it is intuitive that the minimax lower bound for the adversarially robust estimate is contributed by two sources, one for the uncertainty of θ 0 , and the other one for the distribution of X , where the additional unlabeled data can improve the latter term.
The following proposition reveals this difference theoretically:
Proposition 1.
Under the setup of Example 1, assume ∥ θ 0 ∥ ≤ b 0 and there are n 2 unlabeled real data.
Assume R ( θ ϵ , ϵ ) is bounded from above and below, and ϵ ∈ [0 , ϵ ∗ ] with some constant ϵ ∗ for
L 2 attack.
Assume Σ is unknown.
Then for some positive constant c , for any estimator (cid:98) f , inf
[Equation]
4.3 Quality of Unlabeled Data Matters
We study how the quality of a non-ideal unlabeled data generator affects adversarial training.
Unsurprisingly, we always prefer a high-quality/ideal P a .
In addition, when there is some extra information of the marginal distribution of X (such as parametric modeling assumptions) which cannot be directly utilized in the vanilla adversarial training process, training P a from S 1 can potentially give a high-quality adversarially robust model.
On the other hand, given a poor P a , via bias-variance trade-off, it is still possible to reduce the error of (cid:101) θ ( ϵ ) .
Note that the discussion in the section assumes that the unlabeled data generator ( P a ) is independent to the training data S 1 , as assumed in Theorem 1, unless stated otherwise.
A high-quality P a is preferred The term E 2 in Theorem 1 satisfies
[Equation]
Variance .
For the variance term in (5), when n 2 → ∞ , it is always negligible for reasonable P a .
For the square bias in (5), it is in general nonzero if P a ̸ = P 0 , and is similar to the maximum mean discrepancy (MMD, Gretton et al., 2012) that measures the dissimilarity between P a and P 0 .
There exists some function class F such that ∂l ϵ /∂θ ϵ ∈ F and
[Equation]
Based on the above decomposition, a higher quality of the unlabeled data generator is preferred because it leads to small generator cost.
And as shown in Figure 2, with the ideal P a , the generator cost can be zero when n 2 → ∞ .
Using an ideal/good P a , we can have E ∥ (cid:101) θ ( ϵ ) − θ ϵ ∥ 2 Σ ϵ < E ∥ (cid:98) θ ( ϵ ) − θ ϵ ∥ 2 Σ ϵ .
A graphical illustration can be found in the right panel of Figure 4.
When ϵ is large, we know that the ideal P a will efficiently improve the adversarial robustness.
When P a gets worse, there is less improvement in the adversarial robustness.
This is the main answer towards Q2 , and we illustrate some specific cases in the following discussions.
Training data generator from S
For the vanilla adversarial training, there is no trivial way to incorporate the extra information of the marginal distribution of X into the optimization of (2).
In contrast, (cid:101) θ ( ϵ ) can use the extra information to train a good unlabeled data generator from S 1 .
Example 2 (Sparse Covariance Matrix Estimate) .
Assume X ∼ N ( 0 , Σ) and Σ is unknown.
We follow Cai et al.
(2010) to consider a family of sparse covariance matrix as follows:
[Equation]
If we use the sparse estimator proposed in Cai et al. (2010), then E ∥ (cid:98) Σ sparse
[Equation]
Σ ∥
[Equation]
Besides, when (cid:98) Σ is either the sample covariance matrix or (cid:98) Σ sparse , with probability tending to 1 over the generation of S 1 , the decomposition of (cid:101) θ ( ϵ ) in Theorem 1 still holds, and the bias in (5) satisfies
[Equation]
If Σ ∈ F β and d ≫ n 1 / (2 β +1) , when n 2 → ∞ , (cid:101) θ ( ϵ ) with (cid:98) Σ sparse leads to a rate of o ( d/n 1 ) for the generator cost, which is negligible compared to the label cost.
Compared to Theorem 1, a difference in the assumption is that in Example 2, the generator is not independent to S 1 .
Therefore, the proof of (6), presented in Appendix G, is more involved.
Bias-variance trade-off in poor P a
For a poor unlabeled data generator, when n 2 → ∞ , it leads to a poor (cid:101) θ ( ϵ ) as shown in Figure 2.
However, based on the bias-variance trade-off of E 2 , for a relatively n 2 , it is still possible to obtain a (cid:101) θ ( ϵ ) better than (cid:98) θ ( ϵ ) .
Proposition 2 (Bias-variance trade-off) .
Assume the conditions of Theorem 1 hold, n 2 ≪ n 1 and the unlabeled data generator is independent to S 1 , then
[Equation]
As shown by Proposition 2, there is a trade-off between bias and variance w.r.t.
the size of S 2 .
This implies a U-shaped curve of the generator cost w.r.t.
S 2 , and there exists some small n 2 that strikes the balance of this trade-off and can improve the adversarial robustness.
4.4 Balancing the Weights between S 1 and S
Following Gowal et al.
(2021) and Carmon et al.
(2019), we try to improve the performance of (cid:101) θ ( ϵ ) via balancing the weights of S 1 and S 2 in (3), and consider the weighted minimization,
[Equation]
When using the ideal P a and assuming (cid:98) θ (0) ≡ θ 0 (i.e., we can generate extra independent labeled real data), trivially the optimal w minimizing the risk of (cid:101) θ ( ϵ ) is 1.
When using non-ideal P a or (cid:98) θ (0)
̸
= θ 0 , intuitively, one need to downweight the generated data and the best w < 1 .
We demonstrate this observation in the simulation in Figure B.3 as well.
In addition, as stated in the following proposition, through taking the optimal choice of w , for any
P a , a larger n 2 always leads to better (cid:101) θ when ϵ > 0 .
Proposition 3.
We assume the assumptions of Theorem 1 hold, and consider a fixed ( ϵ > 0 , n 1 ) .
Denote (cid:101) θ ( w, n 2 ) as the minimizer of (7), and w ∗ ( n 2 ) minimizes the excess adversarial risk of (cid:101) θ ( w, n 2 ) w.r.t.
w .
Then for n ′ 2 > n 2 , taking w ′ such that w ′ n ′ 2 =
w ∗ ( n 2 ) n 2 , the risk of (cid:101) θ ( w ′ , n ′ 2 ) is smaller than (cid:101) θ ( w ∗ ( n 2 ) , n 2 ) , and the risk of (cid:101) θ ( w ∗ ( n ′ 2 ) , n ′ 2 ) is further smaller than (cid:101) θ ( w ′ , n ′ 2 ) .
As a result, a larger n 2 always gives a better (cid:101) θ .
4.5 Summary of Numerical Experiments
Due to the space limit, we postpone simulations and most real experiments to Appendix B, C.
Below is a summary of simulation and empirical studies.
In Appendix B, we aim to use simple models to numerically verify: (1) given the ideal data generator, the performance of (cid:101) θ ( ϵ ) is better than (cid:98) θ ( ϵ ) when ϵ deviates from zero; (2) the better quality of the data generator implies the better performance of (cid:101) θ ( ϵ ) ; and (3) balancing the weight between S 1 and
S 2 improves the performance.
We observe all (1) to (3) in the simulations.
In Appendix C, we aim to verify that the label cost and the generator cost are important factors in deep learning.
We aim to show (1) adding more unlabeled samples from the ideal generator will improve adversarial robustness, and (2) adding unlabeled samples from a poor generator with a small n 2 will slightly improve the performance.
We perform an experiment on the CIFAR-10 data set.
We take a part of the samples as labeled training data and remove the label of the other samples.
Therefore, if we train a classifier to classify airplanes and cars, then (i) the unlabeled airplane and car pictures can be viewed as ideal unlabeled samples, and (ii) the unlabeled samples from other classes are viewed as from a poor generator.
Our empirical results verify (1) and (2).
Besides, we also find that the unlabeled data with the pseudo label can improve the adversarial training as much as labeled data of the same sample size (the middle panel of Figure C.1).
This observation further verifies the effectiveness of unlabeled data and implies that the label cost is small in adversarial training.
How to Decide a Proper w
To determine a proper w , it is possible to conduct cross validation for simple models.
However, due to the heavy computation of adversarial training under DNN models, this is infeasible for real applications.
Our proposed algorithm in below aims to simplify the tuning process of w .
To design a proper metric to tune w , denote b as the average adversarial loss difference in S 1 and S 2 , v 1 and v 2 as the variance of the adversarial loss for S 1 and S 2 respectively.
Since in real practice one can often obtains a high-quality clean classifier (i.e., low label cost), we tune w based on the generator cost only.
Inspired by the variance-bias trade-off in (5), we define a surrogate of the generator loss as
[Equation]
Due to over-parameterization, the mean and variance of the gradient are volatile in neural networks.
Hence, instead of using the gradient of loss (i.e., ∂l ϵ /∂θ ϵ ) as suggested by (5), we use the adversarial loss to construct κ and introduce w bias to balance b and ( v 1 , v 2 ) .
To control over-fitting problem, we tune w during the first 20% iterations in the experiment.
The algorithm is shown in Algorithm 1.
Algorithm 1 Select α during Training
Input: Training dataset S 1 and S 2 with size n 1 and n 2 , optimizer OPT, total number of iterations
T , number of iteration per weight τ , initial weight w 0 , bias weight w bias , decay factor δ , number of epochs K to train weight.
Initialize the model parameters θ , take w = w 0 .
for k = 1 , . . .
, T/τ do b
= v 1 = v 2 = 0 .
for t = 1 , . . .
, τ do
Calculate the mean and variance of adversarial loss for the samples from S 1 and S 2 in this batch as b t 1 , b t 2 , v t 1 , v t 2 .
Update b =
b ∗ δ + b t 2 − b t 1 , v 1 = v 1 ∗ δ + v t 1 , v 2 = v 2 ∗ δ + v t 2 .
Update θ using OPT.
end for
Update b = b/ (1 − δ ) , v 1 = v 1 / (1 − δ ) , v 2 = v 2 / (1 − δ ) .
if k ≤ K then
Update w to reduce κ .
end if end for
Output: θ .
In the experiment, we consider binary classification for the CIFAR-10 dataset to classify airplane and car.
The implementation for all real-data experiments is modified from Rice et al.
We take 500 samples from each class as labeled data, i.e., n 1 = 1 , 000 .
To form S 2 , we sample n 2 / 2 data from the other 9,000 airplane and car pictures, and n 2 / 2 data from other classes, i.e., the unlabeled data generator generates both ideal and poor samples.
The experiment setups are postponed to Appendix
C. We repeat the experiment for 10 times to get the average robust testing accuracy and its standard error.
The details of how to tune w bias and how to update w using κ are in Appendix C.
The results are summarized in Table 1.
One can see that adjusting w using Algorithm 1 can always lead to a better performance than the unweighted training and is as good as the best fixed w obtained by grid search.
We found that 75% of the total epochs are needed for each fixed w to reveal the performance difference among w ’s (Figure C.3), leading to long computing time for grid search of w .
In contrast, our proposed method only needs several epochs to tune w bias .
The latter two methods reaches similar performance.
[Table]
We also evaluate the performance of Algorithm 1 using the full CIFAR-10, CIFAR-100, and SVHN with the additional data generated by Gowal et al.
The results are postponed to Appendix C.
Proof Sketch of Theorem
We assume (cid:101) θ ( ϵ ) is consistent to θ ϵ and start from the first-order optimality condition of (cid:101)
θ ( ϵ ) and obtain that
[Equation]
[Equation]
We observe that, when (cid:101) θ ( ϵ ) is consistent, i.e., ∥ (cid:101) θ ( ϵ ) − θ ϵ ∥ P −→ 0 , with probability tending to 1,
A
[Equation]
As a result, rearranging the terms in (8), we can prove Theorem 1.
One the other hand, one can also show that ∥ (cid:101) θ ( ϵ ) − θ ϵ
∥
P −→ 0 because (cid:101) R ( θ, ϵ )
→ R ( θ, ϵ ) for any reasonable θ .
Conclusion
This paper studies how adversarial training benefits from unlabeled (generated) data.
We show that (i) the label cost of (cid:101) θ
( ϵ ) is small in adversarial training; (ii) with a high-quality data generator, a large
S 2 leads to a negligible generator cost.
These two facts together indicate that adversarial training benefits a lot from unlabeled (generated) data.
Motivated by these observations and our theory, we balance the weights between S 1 and S 2 to improve the performance and propose an algorithm to determine the weight automatically during training.
Acknowledgements
This project is partially supported by NSF-SCALE MoDL (2134209) and ONR N00014-22-1-2680.
A Limitations of this Work
In this study, we aim to provide theoretical justification on why adversarial training can benefit more from generated data than clean training, which potentially improves the adversarial robustness of machine learning methods.
We mainly focus on the intuitions, theories in simple statistical models, and numerical verification.
Major limitations of the scope of this work include: (1) We focus on justifying how the quality of the data generator affects the final adversarial robustness, rather than how to design better generators, while designing better generators is an interesting and important practical problem; (2) Our theoretical investigation only focuses on simple models.
For deep neural networks that are used in real data sciences, we only provide empirical justifications, and an exact theory of how neural networks balances the label cost and generator cost is not currently available.
Regarding social impacts, our paper mainly focuses on the theories and uses public data sets in our real-data experiments.
We are not aware of any direct negative social impact by our results.
However, we still suggest people be cautious when applying any machine learning algorithm to real applications, e.g., auto-drive, in which safety is essential.
Based on our knowledge, there is only limited literature interpreting how machine learning algorithms work.
B Simulation Study
In the simulation, we aim to numerically verify: (1) given the ideal data generator, the performance of (cid:101) θ ( ϵ ) is better than (cid:98) θ ( ϵ ) when ϵ deviates from zero; (2) the better quality of the data generator implies the better performance of (cid:101) θ ( ϵ ) ; and (3) balancing the weight between S 1 and S 2 improves the performance.
We first present the results and analysis under L 2 attack, and then present the corresponding L ∞ attack version in a later section.
B.1 Simulations under L 2 Attack
The simulation model is as follows.
Assume X ∼ N (0 , I 10 ) , θ 0 =
[1 / √ 10 , . .
. , 1 / √ 10] ⊤ , and the response is generated via Y =
θ ⊤ 0 X + ε where the noise ε follows N (0 , 0 . 1) .
The unlabeled data are generated from N (0 , Σ) , where Σ can be I 10 , 3 I 10 , (cid:98) Σ sparse in Example 2 or sample covariance matrix (cid:98) Σ .
We take L 2 attack, n 1 = 50 , and repeat the experiment for 100 times to obtain the MSE of (cid:101) θ ( ϵ ) .
We first verify (1) and (2) when the data generator is not related to S 1 .
The results are shown in
Figure B.1.
In the left panel, we use the ideal data generator and change the value of ϵ .
With a larger attack, it is more likely that adding more data will help reduce the MSE of (cid:101) θ ( ϵ ) .
In the right panel, when using a poor generator N (0 , 3 I d ) to generate S 2 , the reduction of MSE of (cid:101) θ ( ϵ ) is limited.
In addition, when ϵ = 0 .
5 , the best performance is obtained when n 2 is a finite value.
This verifies the observation in Proposition 2.
For the case where the data generator is learned from S 1 , the simulation results are summarized in
Figure B.2.
As discussed in Example 2, the performance of (cid:98) Σ sparse should be theoretically better than
(cid:98) Σ .
Figure B.2 and Figure B.1 together show that the ideal generator leads the MSE largest reduction, the generator with (cid:98) Σ sparse has a slightly worse performance, and the generator with (cid:98) Σ cannot work well.
These observations match our theoretical investigations.
In terms of (3) adjusting the weight, we fix ϵ = 0 .
5 and compare the performance while varying w .
The results are summarized in Figure B.3.
When the data generator is of high quality (the left panel), the best w is around 1.
When the data generator is not good (the right panel), the best w has larger deviation from 1.
In addition, from both cases in Figure B.3, the minimal MSE ratio is smaller thanand keeps decreasing when enlarging n 2 , which verifies Proposition 3.
B.2 Simulations under L ∞ Attack
We repeat the experiments in Section B.1 using L ∞ attack.
Since L ∞ attack is stronger, we take slightly smaller ϵ .
From Figures B.4, B.5, B.6, all the observations are the same as in Section B.1.
C Deep Learning Experiments C.1 General Configurations
We use the CIFAR-10 dataset for this experiment.
We pick two classes, airplanes and cars, to do a binary classification.
Each class in CIFAR-10 contains 5,000 training samples.
We randomly pick 500 from each class as labeled training data.
For unlabeled data, we (i) sample from remaining 4,500*2 samples of airplanes and cars as an ideal data generator or (ii) sample from the images of the other 8 classes as a poor generator.
We repeat this experiment 10 times to obtain a boxplot for each setup.
We use a single NVIDIA Tesla V100 GPU to do the training.
C.2 Quality of Data Generator Matters
In Section 4.1, we decompose the error of (cid:101) θ ( ϵ ) into a label cost and a generator cost.
We want to examine whether these two errors are the key factors in deep learning.
We vary the value of n 2 and expect that (1) adding more unlabeled samples from the idea generator will help improve adversarial robustness; (2) when adding unlabeled sample from the poor generator, a small n 2 will lead to slight improvement.
For both adversarial training and clean training, the learning rate is divided by ten at the 50% and 75% epochs.
For each ( n 1 , n 2 ) , we take the total number of iterations as T = 100( n 1 + n 2 ) / 128 .
Although small n 2 leads to fewer training iterations, some simple trials indicate that this choice of T is sufficient to get the best performance.
The results are shown in Figure C.1.
For (1), from the left figure of Figure C.1, when n 2 gets larger, i.e. α gets smaller, for adversarial training with ϵ = 4 / 255 , 8 / 255 , the adversarial robustness of the model gets improved.
Clean training also benefits from S 2 , but not as much as adversarial training.
In addition, the middle figure in Figure C.1 compares the performance of using n 1 labeled samples and n 2 unlabeled samples, against the performance of using n 1 + n 2 labeled samples (we still take α = n 1 / ( n 1 + n 2 ) to match the x-axis).
Using extra unlabeled data improves the performance significantly as n 2 increases.
Knowing the true labels of the unlabeled data can further improve the performance, but this additional improvement is only marginal.
These observations also verify the hypothesis in Gowal et al.
(2021) that it is critical to complement the original training data set with additional (unlabeled) data in adversarial training.
In terms of (2), from the right figure of Figure C.1, for clean training and 4 / 255 L ∞ adversarial training, the testing performance does not improve.
Only when ϵ = 8 / 255 , the adversarial robustness slightly gets better when α ∈ (0 .
Based on our theory, adversarial training is more likely to benefit from the poor generator when ϵ is large and n 2 is small, and our experiment observations match this result.
C.3 Weighted Sampling
We examine the performance of weighted sampling in adversarial training when n 1 = 500 .
For S 2 , we randomly select n 2 / 2 from the remaining samples whose label is airplane/car, and n 2 / 2 from those with other labels.
How to implement weighted sampling Instead of performing a weighted optimization (7), equivalently, we use stratified sampling to sample a batch of S 1 and S 2 , such that the proportion of S 1 samples in the batch is n 1 / ( n 1 + wn 2 ) , and perform SGD for the unweighted objective over the random batch.
This helps reduce the variance of the updating gradient.
We take n 2 ∈ { 200 , 2000 , 4000 , 18000 } and try different w for each n 2 .
The results are shown in Figure C.2.
When changing w , S 1 is larger, leading to more batches in each epoch.
We keep the total number of iterations unchanged as T = 100( n 1 + n 2 ) / 128 .
Performance Based on the left panel of Figure C.2, through tuning the weight w , one can obtain a better adversarial test accuracy.
Configurations in Algorithm 1 We initialize w as w 0 = n 1 /n 2 and take δ = 0 .
9 in the experiments.
We check κ after every epoch and update 1 /w with a magnitude of 1 / (4 w 0 ) , i.e., in the next epoch, we take 1 /w next as 1 /w , 1 /w + 1 / (4 w 0 ) , or max(0 , 1 /w − 1 / (4 w 0 )) depending on which one gives the smallest κ .
Table 2 is the w bias we take in experiments.
To choose w bias , we try five to six different candidates to pick the one that is most stable in the first 20% iterations.
[Table]
C.4 Full Real Datasets
For the generated data, we download from Gowal et al.
The implementations of the training, attack, and neural network architectures are the same as Rice et al.
Detailed configurations can be found in Table 4.
The results are summarized in Table 3.
For all the data sets and neural network architectures, the proposed algorithm leads to better performance than the vanilla adversarial training and w = 1 .
On the other hand, for CIFAR-10, compared to the > 10% improvement for binary classification, there is only 3% to 6% improvement in the robust test accuracy.
There are two possible explanations.
First, the model capacity of the neural network architectures may not be sufficient to learn the ten classes.
In our experiment as well as Gowal et al. (2021); Xie & Yuille (2019), we all observe that wider neural networks lead to better robustness.
Second, we are using ideal (real) data in the binary classification but generated data in the ten-class classification task.
The quality of generated data deteriorates the performance.
[Table] [Table]
D Results on Empirical Process Theory
In this section
[Table]
we present some results in Koltchinskii et al.
(2006) which will be used in our proof.
Let σ 2 = sup f ∈F E f 2 , and | f ( x ) | ≤ F ( x ) ≤ U .
Define P n as the empirical measure and ∥ P n −
P ∥ F = sup f ∈F | P n f − Pf | .
If for some A > 0 , V > 0 , ∀ ϵ > 0 ,
N
[Equation]
(cid:18)
[Equation]
ϵ
[Equation]
, then (2.2) in Koltchinskii et al.
(2006) states that with some universal constant C > 0 ,
E ∥
P n − P ∥ F ≤
[Equation]
Furthermore, a refined version of Talagrand’s concentration inequality states that for any countable class of measureble functions F with elements mapping into some [ − M, M ] .
Combining Bousquet’s bound and Klein’s bound together,
[Equation]
E How Does Assumptions A1 and A2 Affect the Theory
Assumptions A1 and A2 are imposed to regulate the behavior of the loss function, gradient, and
Hessian.
The following discussion provides some details on how to possibly relax the conditions and what are the potential observations if some conditions are violated.
E.1 Assumption A
Assumption A1 is a sufficient condition for Theorem 1, Lemma 1, 2, 3, and Example 2.
It can be relaxed to general distributions if the following conditions are satisfied.
For some ϵ 0 > 0 , r > 0 , for any ϵ ∈ [0 , ϵ 0 ] and θ ∈ B 2 (0 , r ) ,
• With probability one, the loss function l ϵ is smooth, and the gradient and Hessian exist.
• The loss function l ϵ is convex.
• The loss function l ϵ has a finite expectation and finite variance.
• The gradient ∂l ϵ ( X, Y, θ ) /∂θ has finite expectation and finite variance.
• The eigenvalues of the expected Hessian E P 0 ⊗P y ∂
2 l ϵ ( X, Y, θ ) /∂θ 2 are finite and bounded away from zero.
• Considering the expectation over Y as a function of θ 0 , E P 0 ⊗P
y l ϵ ( X, Y, θ ) is smooth in θ 0 , the gradient ∂ E P 0 ⊗P
y l ϵ ( X, Y, θ ) /∂θ 0 is finite, and ∂ 2 E P 0 ⊗P y l ϵ ( X, Y, θ ) /∂θ
∂θ has non-zero singular values.
E.2 Assumption A
Assumption A2 is essential because the proof in Theorem 1 is based on the fact that (cid:101) θ ( ϵ ) is consistent to θ ϵ .
E.3 If the Assumptions Do Not Hold
Multi-class classification, or other generalized linear models.
In general, as long as the relationship between Y and X is well specified, and the loss function is convex and smooth enough,
Theorem 1 is valid.
In addition, the intuitions of label cost and generator cost in Sections 4.2 and 4.
still apply.
The distribution P 0 has a heavy tail.
It depends on the design of the loss function.
If l ϵ and its gradient and Hessian are still well-behaved, we still have the convergence.
Otherwise, (cid:98) θ (0) may not be consistent, i.e., it is hard even for the clean model estimation.
The parameterization between Y and X is misspecified.
In this case, θ 0 is a biased model to describe P ( Y | X = x ) .
If l ϵ is smooth enough, the results in Theorem 1 will still hold.
However, it is more important to deal with the bias in this case.
Assumption A2 is violated.
In this case, Theorem 1 may fail because (cid:101) θ ( ϵ ) may not be consistent to θ ϵ .
F How n 2 /n 1 Affects the Performance
In Figure 2, we explain how the quality of the generator affects the generator cost when n 2 → ∞ .
In this section, we discuss how n 2 /n 1 affects the label cost and generator cost.
Based on the formula of E 1 and E 2 , when n 2 increases, E E 21 is a monotone increasing function, and E
E 22 could even decrease or increase depending on the quality of the generator.
Figures F.1 and F. show how the costs are changed with an ideal generator or a poor generator.
G Proofs G.1 Theorem
Since (cid:101) θ ( ϵ ) minimizes (cid:101) R ( θ, ϵ ) as in (3), we have
[Equation]
using which we can get the following decomposition,
[Equation]
+ n
[Equation]
n
[Equation]
∂ ∂θ
[Equation]
(cid:32)(cid:88)
Sl ϵ ( x, y, (cid:101) θ ( ϵ )) (cid:33) − n 1 + n∂ ∂θ ϵ
(cid:32)(cid:88)
Sl ϵ ( x, y, θ ϵ ) (cid:33) (cid:124) (cid:123)(cid:122) (cid:125) :
= A+ n 1 + n∂ ∂θ ϵ (cid:32)(cid:88)
Sl ϵ ( x, y, θ ϵ ) + (cid:88)
Sl ϵ ( x, y, θ ϵ ) (cid:33) .
In the following, we present Lemma 1 to show the consistency of (cid:101) θ ( ϵ ) , then show Lemma 2 and 3 to bound some terms in A 1 to A 3 , and finally present the main proof for Theorem 1.
Lemma 1 (Consistency of (cid:101) θ ( ϵ ) ) .
Under the same assumptions as Theorem 1, (cid:101) θ ( ϵ ) converges to θ ϵ in probability.
Proof of Lemma 1.
Denote ¯ θ ϵ as the minimizer of n 1 E P 0 ⊗P y l ϵ ( X, Y,
[Equation]
) + n 2 E P a ⊗P y l ϵ ( X, Y, θ ) .
Denote P as the weighted average of P 0 and P a .
Under Assumption A2, we know that ∥ ¯ θ ϵ − θ ϵ ∥ → 0 .
For both linear regression and logistic regression, one can check that l ϵ is strongly convex in θ .
Therefore, we have for some C > 0 , such that
[Equation] [Equation]
Furthermore, when ∥ θ ∥ ≤ b 0 for some large constant B , following the same arguments in Lemma 2, there exists some C 0 > 0 ,
[Equation] [Equation]
and
P
[Equation] [Equation]
From the definition of (cid:101) θ ( ϵ )
[Equation]
we know that
[Equation]
then combining with (14), (15) and the fact that E P a ⊗P
[Equation]
) ( (cid:98) θ (0) P −→ θ 0 ), we have
[Equation]
[Equation]
Combining (16) and (13), with probability tending to 1,
[Equation]
On the other hand, from the definition of ¯ θ ϵ , we have
[Equation]
thus one can conclude that E P⊗P y l ϵ ( X, Y, (cid:101) θ ( ϵ ))
− E P⊗P y l ϵ ( X, Y, ¯ θ ϵ )
→ 0 .
Further using (12), we get ∥ (cid:101) θ ( ϵ ) − ¯ θ ϵ ∥ → 0 as well, and thus (cid:101) θ ( ϵ ) converges to θ ϵ .
Lemma 2.
Denoting
[Equation]
under the same assumptions as in Theorem 1, there exists some ξ =
o (1) such that
[Equation]
Proof of Lemma 2.
Since ∂l ϵ /∂θ is square integrable, based on Theorem 2.6.7 of Van Der Vaart &
Wellner (1996), taking F 1 as the class of function of δ 1 for θ ∈ B
( θ θ , ξ ) , we have for some C 11 > 0 ,
V > 0 ,
[Equation]
Based on Lemma 1, we know that there exists some ξ → 0 such that ∥ (cid:101) θ ( ϵ ) − θ ϵ ∥ ≤ ξ in probability.
As a result, with probability tending to 1, sup ∥ θ − θ ϵ ∥≤ ξ E ∥
∂l ϵ /∂θ ∥
= o ( ξ 2 ) .
Following (9) and (10), we know that there exists some positive constant C 12 such that
[Equation]
Lemma 3.
Define r 0 such that r
= o (1) and ∥ (cid:98) θ (0) − θ
∥ ≤ r 0 with probability tending to 1.
Denote δ
[Equation]
then under the same assumptions as Theorem [Equation] there exists some positive constants C 21 , C 22 such that
[Equation]
Proof of Lemma 3.
The proof of Lemma 3 is the same as Lemma 2.
Note that we do not assume Sand (cid:98) θ (0) are independent.
Instead, we consider changing both θ and (cid:98) θ (0) in δ 2 to form the class of functions.
Proof of Theorem 1.
We start from studying A 1 to A 3 .
For A 1 ,
[Equation]
ϵ ))
(cid:33) − n 1 + n∂ ∂θ ϵ
(cid:32)(cid:88)
Sl ϵ ( x, y, θ ϵ ) (cid:33) =
n 1 + n∂ ∂θ ϵ
(cid:32)(cid:88)
Sl ϵ ( x, y, (cid:101) θ ( ϵ )) (cid:33) − n 1 + n∂ ∂θ ϵ
(cid:32)(cid:88)
Sl ϵ ( x, y, θ ϵ ) (cid:33) (cid:124) (cid:123)(cid:122) (cid:125) :
= B− E (cid:34) n 1 + n∂ ∂θ ϵ (cid:32)(cid:88)
Sl ϵ ( x, y, (cid:101) θ ( ϵ )) (cid:33) − n 1 + n∂ ∂θ ϵ
(cid:32)(cid:88)
Sl ϵ ( x, y, θ ϵ ) (cid:33)(cid:35) (cid:124) (cid:123)(cid:122) (cid:125) :
= B+ E (cid:34) n 1 + n∂ ∂θ ϵ (cid:32)(cid:88)
Sl ϵ ( x, y, (cid:101) θ ( ϵ )) (cid:33) − n 1 + n∂ ∂θ ϵ
(cid:32)(cid:88)
Sl ϵ ( x, y, θ ϵ ) (cid:33)(cid:35) .
From the definition of Σ ϵ , we obtain that
[Equation]
and based on Lemma 2, (taking t = 2 log n 1 in (17)) with probability tending to 1, ∥ B 11 − B 12 ∥ =
o ( ∥ B 12 ∥ ) .
In terms of A 2 , we have
[Equation]
Similar to A 1 , from the definition of (cid:101) Σ ϵ ,
[Equation]
and from Lemma 3, with probability tending to 1, ∥ B 21 − B 22 ∥ =
o ( ∥ B 22 ∥ ) .
For A 3 , similar to A 1 , based on Lemma 2,
[Equation]
Under Assumption A2, although E P a ⊗P y ∂ 2 l ϵ ( X, Y, θ ϵ ) /∂θ 2 is different from Σ ϵ , we still have
[Equation]
Finally, taking A 1 to A 3 back to the decomposition of (cid:101) θ ( ϵ ) − θ ϵ , we conclude that with probability tending to 1,
[Equation]
Remark 2.
A converge-in-probability result and a Bahadur representation does not directly imply
L 2 convergence, i.e., (cid:101) θ ( ϵ ) P −→ θ ϵ does not imply E ∥ (cid:101) θ ( ϵ ) − θ ϵ ∥ 2 → 0 .
However, based on assumption
A1 and A2, due to the choice of P 0 and the loss functions we take, one can obtain E ∥ (cid:101) θ ( ϵ ) − θ ϵ ∥ 2 → 0 .
As a result, we can use the terms in the Bahadur representation to discuss the label cost and the generator cost.
One can replace A1 and A2 with more general conditions in other models with proper conditions.
G.2 Proposition
The proof mainly follows Xing et al.
(2021b).
The definition of adversarial risk in Xing et al.
(2021b) is different to our scenario.
Using our definition, in linear regression setup, we know that as long as σ 2 ( V ar ( Y | X ) ) and ϵ are nonzero, then θ ϵ will deviate from θ 0 .
And one can obtain that,
[Equation]
and λ =
Θ( ϵ ) when ϵ ∈ [0 , ϵ ∗ ] .
One can take this rate of λ into Lemma 5 of Xing et al.
(2021b) to obtain the rate Ω( ϵ 2 dR 2 / ( n 1 + n 2 )) .
The other term in the lower bound is obtained from Lemmaof Xing et al.
(2021b) without any changes.
G.3 Proposition
From Theorem 1, θ ϵ − (cid:101) θ ( ϵ ) is decomposed into
[Equation]
where
[Equation]
and E 2 = Σ − 1 ϵ
[Equation] [Equation]
x
[Equation]
θ ϵ )
[Equation]
In terms of the variance of θ ϵ − (cid:101) θ ( ϵ ) , we know that tr ( V ar ( E 2 ))
=
O ( d/ ( n 1 + n 2 )) , and tr ( V ar ( E 1 ))
=
O ( dn 22 / ( n 1 + n 2 ) 2 n 1 ) .
Therefore, the dominant part is tr ( V ar ( E 2 )) .
In terms of the expectation of θ ϵ − (cid:101) θ ( ϵ ) , since θ ϵ is the minimizer of E P
⊗P y l ϵ
( X, Y, θ ) , we have
[Equation]
As a result, the bias in E 2 are all contributed by the samples in S 2 .
When ∥ E P a ⊗P y ∂∂θ ϵ
l ϵ ( X, Y, θ ϵ ) ∥ =
O ( √ d ) , we have ∥ E E 2 ∥ =
O ( √ dn 2 / ( n 1 + n 2 )) .
Meanwhile, since (cid:98) θ (0) is unbiased to θ 0 , E E 1 is at most in O ( √ dn 2 / ( n 1 + n 2 )) .
G.4 Proposition
The statement of Proposition 3 itself can be viewed as a simple sketch of the proof.
We provide some more details below:
Adding w into Theorem 1, we have
[Equation]
Denote (cid:101) θ ( w, n 2 ) as the minimizer of (7), and w ∗ ( n 2 ) minimizes the excess adversarial risk of (cid:101) θ ( w, n 2 ) w.r.t.
w .
For n ′ 2 > n 2 , take w ′ such that w ′ n ′
2 =
w ∗ ( n 2 ) n 2 , then
• In (18), A ( w ∗ ( n 2 ) , n 2 )
=
A ( w ′ , n ′ 2 ) and B ( w ∗ ( n 2 ) , n 2 ) =
B ( w ′ , n ′ 2 ) .
• Since n ′ 2 > n 2 , we have w ′ <
w ∗ ( n 2 ) , and E ∥ C ( w ∗ ( n 2 ) , n 2 ) ∥
2Σ ϵ > E ∥ C ( w ′ , n ′ 2 )
∥ 2Σ ϵ .
• Since (cid:98) θ (0) is consistent, and data in S 1 are real data, we have E A ( w, n 2 ) + B ( w, n 2 ) is negligible, and thus E D ( w, n 2 ) is negligible.
As a result, the risk of (cid:101) θ ( w ′ , n ′ 2 ) is smaller than (cid:101) θ ( w ∗ ( n 2 ) , n 2 ) .
On the other hand, by the definition of w ∗ ( · ) , the risk of (cid:101) θ ( w ∗ ( n ′ 2 ) , n ′ 2 ) is further smaller than (cid:101) θ ( w ′ , n ′ 2 ) .
Eventually, we obtain that a larger n 2 always gives a better (cid:101) θ .
G.5 Derivation of Example
In Theorem 1, we assume the data generator and S 1 are independent, which is different this scenario where (cid:98) Σ is learnt from S 1 .
To show that the estimate in this example still converges, there are several steps in the proof.
We firstly bound the difference between E P a ⊗P y ∂∂θ
ϵ l ϵ ( x, y, θ ϵ ) and E P 0 ⊗P y ∂∂θ
ϵ l ϵ ( x, y, θ ϵ ) .
Denote f A as the density of ( x, y ) given a covariance matrix A , then given (cid:98) Σ ,
[Equation]
[Equation]
Since (cid:98) Σ is consistent, E P a ⊗P y ∂
∂θ ϵ l ϵ ( X, Y, θ ϵ ) converges to E P 0 ⊗P
y ∂
∂θ ϵ l ϵ ( X, Y, θ ϵ ) as well, where the latter one is a just zero vector.
Second, to show that (cid:101) θ ( ϵ ) is consistent as in Lemma 1, for some constant c 0 , with probability tending to 1 over the generation of S 1 ,
[Equation]
Since (cid:98) θ (0) P −→ θ 0 , following the same arguments as in Lemma 1, we have
[Equation]
Furthermore,
[Table]
[Equation]
[Table] [Equation] [Table] [Equation]
[Table]
[Equation]
[Table] [Equation] [Table] [Equation]
[Table]
[Equation] [Table] [Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
Thus following the same arguments as in Lemma 1, we get (cid:101) θ ( ϵ )
→ θ ϵ in probability.
In terms of the Bahadur representation of (cid:101) θ ( ϵ ) , from the decomposition in (11), we always deal with
S 1 and S 2 separately.
The clean estimate (cid:98) θ (0) is related to P a , but this does not affect Lemma 2 and 3 because their bounds are valid uniformly for proper θ .
As a result, we obtain the same Bahadur representation for (cid:101) θ ( ϵ ) − θ ϵ .
H Linear Regression with Gaussian Design
In this section, we present the derivation of label cost in (cid:101) θ ( ϵ ) and excess adversarial risk of (cid:98) θ ( ϵ ) in linear regression with Gaussian design.
For Figure 3, we consider Y =
X ⊤ θ 0 + ε , θ
= 1 d / √ d , X ∼ N ( 0 , I d ) , ε ∼ N (0 , 0 .
The following is a list of contents in this section:
• Paragraph “formula of θ ϵ ” describes how θ ϵ is related to Σ and θ 0 .
• Paragraph “label cost” and “vanilla adversarial training” present the formula of these two costs under Gaussian design, using which we obtain the theoretical curves in Figure 3.
• Paragraph “simulation” presents
the simulation setups to get the dashed curves in Figure 3.
Formula of R ( θ, ϵ ) Assume X ∼ N ( 0 , Σ) and V ar ( ε ) =
σ 2 , then in the adversarial loss:
[Equation]
We know that X ⊤ θ − X ⊤ θ 0 − ε is also a Gaussian distribution with zero mean and variance as ∥
θ − θ
∥ 2Σ + σ 2 , so the expectation of its absolute value is c 0 (cid:112) ∥ θ − θ
∥ 2Σ + σ 2 where c 0 = (cid:112) 2 /π .
As a result, the population adversarial risk is
[Equation]
and its gradient becomes
[Equation]
One can solve θ ϵ numerically via solving ∂R/∂θ = 0 or minimizing R ( θ, ϵ ) .
Formula of θ ϵ
Under first-order optimality condition, we have
[Equation]
which implies that
[Equation]
Remark 3.
As in Javanmard et al.
Xing et al.
(2021b),
when Σ = I d , λ ϵ is only a function of ( ∥ θ 0 ∥ , σ 2 , ϵ ) , regardless of d .
However, in terms of the excess adversarial risk, it is related to d .
Label cost For E 1 ,
[Equation]
∂θ∂θ
[Equation]
For the term (cid:98) θ (0) − θ 0 in E 1 , we know that,
[Equation]
When θ = θ 0 ,
[Equation]
As a result
[Equation]
the label cost becomes
[Equation]
The matrix Σ ϵ has an analytical form as
[Equation]
Vanilla adversarial training For the excess adversarial risk of (cid:98) θ ( ϵ ) , we have
[Equation]
where
[Equation]
Since E ∂l ϵ /∂θ =
[Equation]
we have
[Equation]
where
[Equation]
And some cross terms can be calculated as
[Equation]
For the other cross terms, ξ 1 ξ ⊤ 3 , ξ 1 ξ ⊤ 4 , ξ 2 ξ ⊤ 3 , there is no closed-form formula for their expectation, and we use simulation to approximate them.
Simulation To obtain an (cid:101) θ ( ϵ ) which does not involve generator cost, the procedure is slightly different to other simulation studies in this paper.
Rewrite R ( θ, ϵ )
=
R ϵ
( θ, θ 0 ) .
We take n 1 = 1 ,to obtain (cid:98) θ (0) , and directly numerically minimize R ϵ
( θ, (cid:98) θ (0))
to get (cid:101) θ ( ϵ ) .
We repeat this procedure for 1,000 times to get a stable estimate of the excess risk ratio.