T EST -T IME A DAPTATION AND A DVERSARIAL R OBUSTNESS
Anonymous authors
Paper under double-blind review
A BSTRACT
This paper studies test-time adaptation in the context of adversarial robustness.
We formulate an adversarial threat model for test-time adaptation, where the defender may have a unique advantage as the adversarial game becomes a maximin game, instead of a minimax game as in the classic adversarial robustness threat model.
We then study whether the maximin threat model admits more “good solutions” than the minimax threat model, and is thus strictly weaker .
For this purpose, we ﬁrst present a provable separation between the two threat models in a natural Gaussian data model. For deep learning, while we do not have a proof, we propose a candidate, Domain Adversarial Neural Networks ( DANN ), an algorithm designed for unsupervised domain adaptation, by showing that it provides nontrivial robustness in the test-time maximin threat model against strong transfer attacks and adaptive attacks.
This is somewhat surprising since DANN is not designed speciﬁcally for adversarial robustness (e.g., against norm-based attacks), and provides no robustness in the minimax model.
Complementing these results, we show that recent data-oblivious test-time adaptations can be easily attacked even with simple transfer attacks.
We conclude the paper with various future directions of studying adversarially robust test-time adaptation.
I NTRODUCTION
There is a surge of interest to study test-time adaptation to help generalization to unseen domains (e.g., recent work by Sun et al.
; Wang et al.
Nado et al.
At the high level, a generic test-time adaptation can be modeled as an algorithm Γ which accepts an (optional) labeled training dataset D , an (optional) model F trained on D (usually used as a starting point), and an unlabeled test feature set U , outputs a model (cid:101) F = Γ( F, D, U ) , in order to achieve high test accuracy on U .
For large test set U , test-time adaptation can be viewed as a form of transductive learning (Joachims (1999); Vapnik (1998))
(i.e., using D, U to train a model to predict speciﬁc instances in
U ), which is argued to be easier than more traditional inductive learning.
This paper studies test-time adaptation in the context of adversarial robustness (i.e., there is an active agent who tries to fool the test-time adaptation by perturbing the input so that (cid:101) F gives wrong predictions).
There are several motivations in pursuing this direction.
First, this question is of practical interest: Many practical ML pipelines run in a batch mode 1 , where they ﬁrst collect a set of unlabelled data points, and then send them to a model (e.g. Nado et al.
In such cases, data in the batch may have been adversarially perturbed, and it is a natural question whether we can leverage the large batch size and test-time adaptation to enhance adversarial robustness.
Second, from a purely theoretical point of view, since test-time adaptation is a form of transductive learning, it is intriguing to ask whether transductive adversarial learning can be easier , given that traditional adversarial robustness is formulated in the inductive learning setting (e.g. Madry et al.
To this end, a recent work by Goldwasser et al.
(2020) shows that, with transductive learning, one can achieve nontrivial guarantees for classes of bounded VC dimension with arbitrary train and test distributions.
The current work complements their paper in the setting of deep learning.
To study these questions, we formalize a threat model, which we call (test-time) maximin threat model , for the adversarial robustness of test-time adaptation.
Recall that the classic adversarial robustness game is a minimax game min F E V [max (cid:101) V L ( F, (cid:101) V )] , where V is random sampled data, (cid:101) V is the perturbed data generated from V by the adversary, and L ( F, (cid:101) V ) is the loss of the model
F on (cid:101) V .
By contrast, in the maximin threat model, we allow V to be sampled from a different domain, and the game is maximin:
E V [max U min (cid:101) F L ( (cid:101) F, (cid:101) V )]
(where U is the perturbed features of
V , subject to the attack type, and (cid:101) V is the labeled perturbed data, see Deﬁnition 2).
By the maximin inequality, it follows that this threat model is no harder than the minimax model (to allow source and target domains to be different, we need to generalize the classic minimax model, see Deﬁnition 3).
We then move on to the focus of this work: Whether the maximin threat model is “strictly weaker” than the minimax threat model.
We note that any good defender solution (a robust model) in the minimax game induces a good defender solution in the maximin game (an adaptation algorithm that outputs that robust model), thus intuitively, the good defender solutions of the minimax model is a subset of the good defender solutions of the maximin threat model.
We ask whether such a containment is proper : That is, whether there exists a defender solution that is good in the maximin threat model, but is bad in the minimax threat model.
The existence of such a defender will demonstrate that the maximin threat model admits more good solutions.
Besides theoretical interest, this question is also of practical importance since these “new” solutions may possess desirable properties that good solutions in the minimax threat model may lack .
For example, one such property is that the defender solution is attack agnostic (Goodfellow (2018) (pp.30)): That is, the solution is not to directly optimize the performance measure for a particular type of perturbation 2 . To this end, we ﬁrst present a provable separation between the maximin and minimax threat models in a natural Gaussian data model. We then move to deep learning.
While we do not have provable guarantees, we empirically examine Domain Adverarial Neural Networks ( DANN ) (Ganin et al.
(2017)), an algorithm designed for unsupervised domain adaptation (UDA), as a candidate for the separation.
Speciﬁcally, we demonstrate that DANN provides nontrivial test-time adversarial robustness against both transfer attacks and adaptive attacks , in both homogeneous and inhomogeneous cases.
This is somewhat surprising as DANN is also attack agnostic as we mentioned above, and has not been considered for adversarial robustness.
Not surprisingly, the accuracy becomes very low when we evaluate (cid:101) F in the minimax model.
These thus provide positive evidence for our hypothesis that DANN provides a separation.
Complementing the above result, we explore the maximin robustness of the recent data-oblivious adaptation algorithms (namely, the adaptation algorithms do not use D , but just the pretrained model F and unlabeled test set U ).
Speciﬁcally, we consider Test-Time Training ( TTT ) by Sun et al.
We show that TTT can be easily attacked using simple transfer attacks.
While this is not surprising as authors of Sun et al.
(2020) have cautioned that TTT is not designed for adversarial robustness, the situation is in sharp contrast to our results with DANN .
The rest of the paper is organized as follows: Section 2 presents the setup.
In Section 3 we deﬁne threat models.
In Section 4 we present theoretical results about separation, and examine DANN as a candidate separation in the deep learning.
Finally, Section 5 explores the maximin robustness of oblivious test-time adaptation, and concludes the paper with future directions.
P RELIMINARIES
Let F be a model, for a data point (
x, y ) ∈
X × Y , a loss function (cid:96) ( F ; x, y ) give the loss of F on x given the true label y .
Let V be a set of labeled data points.
We use the notation L ( F, V ) =|
V | (cid:80) ( x,y ) ∈ V (cid:96) ( F ; x, y ) to denote the empirical loss of F on V .
For example, if we use binary loss (cid:96) 0 , 1 ( F ; x, y ) = 1 [ F ( x ) (cid:54) =
y ] , this gives the test error of F on V .
We use the notation V | X to denote the projection of V to its features, that is { ( x i , y i ) } mi =1 (cid:55)→ { x 1 , . . .
, x m } .
Threat model for classic adversarial robustness.
To formulate the threat model for test-time adaptation, we ﬁrst present a threat model for the classic adversarial robustness.
Although the classic adversarial robustness can be written down succinctly as a minimax objective, namely min F E ( x,y ) ∼ ( X,Y ) (cid:2) max x (
cid:48) ∈ N ( x ) [ (cid:96) ( F ; x (cid:48) , y )] (cid:3) ( N ( x ) is a neighborhood function of x , determined by the attack type), a threat model formulation will help us develop more nuanced models.
Deﬁnition 1 ( Threat model for classic adversarial robustness).
Attacker and defender agree on a particular attack type.
Attacker is an algorithm A , and defender is a supervised learning algorithm T .
[Table]
• A (labeled) training set D is sampled i.i.d.
from from ( X, Y ) .
[Table] time • ( Defender ) Train a model F on D as F = T ( D ) .
Test time • A (labeled) natural test set V is sampled i.i.d.
from ( X, Y ) .
• ( Attacker ) On input F , D , and V , A perturbs each point ( x
, y ) ∈ V to ( x (cid:48) , y ) (subject to the agreed attack type), giving (cid:101) V
=
A ( F, D, V ) .
[Table]
Attacker’s
[Table]
maximize the test loss
[Table]
the defender’s goal is to minimize the test loss.
We stress that the i.i.d sampling of V is important (which is also present in the expectation in the minimax objective): Otherwise an attacker can pick any point that fools F and repeat it arbitrarily many times.
(we refer readers to Goodfellow (2019) for more discussions along this line).
Notations for models and attacks.
In this paper we mainly use the PGD attacks (Projected Gradient
Descent attacks) with norm-based perturbations Madry et al.
For example, given a model
F , we use the notation PGD ( F, V ) to denote PGD attacks against F , on data V (the attack type is speciﬁed in the context).
We adopt the following notations: [Table]
T A target model trained on the labeled target data V .
AdvT
An adversarially trained target model using the labeled target data V .
S
A source model trained on the labeled source data D .
AdvS
An adversarially trained source model using the labeled source data D .
PGD ( · , · ) PGD Attacks on a model and data.
For example, PGD ( AdvT , V ) means to use PGD attacks on the model AdvT and data V .
Test-time defenses and BPDA.
Various previous work have investigated test-time defenses where a pretrained model is ﬁxed and there is a “preprocessing procedure” to preprocess an input before sending it to the model.
Several such defenses were described and attacked in Athalye et al.
(2018), by the BPDA technique (Backward Pass Differentiable Approximation).
While syntactically one can ﬁt test-time defenses into our framework, the test-time adaptation concept discussed in this paper is signiﬁcantly different, which usually allows one to completely modify a model.
As we will see later in the paper, we use an unsupervised domain adaptation algorithm, such as DANN , for test-time adaptation.
In these cases, it is unclear how to apply BPDA.
T HREAT MODELS 3. T EST - TIME MAXIMIN THREAT MODEL
The intuition behind the test-time maximin threat model is as follows: After we receive the adversarially perturbed data U to classify (at test time), the defender trains a model based on U , and we evaluate the test accuracy only for U .
(i.e., for different test set U we may have different models and different test accuracy.)
This perspective of training a model using labeled data D and unlabeled data U , and only test on U , is essentially the transductive learning by Vapnik (1998) (however, we consider it in an adversarial setting).
Formally, we have the following deﬁnition:
Deﬁnition 2 ( Maximin threat model for test-time adaptation).
Fix an adversarial perturbation type.
We have a source domain ( X s , Y s ) , and a target domain ( X t , Y t ) .
Attacker is an algorithm A , and defender is a pair of algorithms ( T , Γ) , where T is a supervised learning algorithm, and Γ is a test-time adaptation algorithm.
Before [Table]
• A (labeled) training set D is sampled i
i d. from ( X s , Y s ) .
Training time • (Defender, optional) Train a model F trained on D as F = T ( D ) .
Test time • A (labeled) natural test set V is sampled i.i.d from ( X t , Y t ) , V is sent to the attacker.
• (Attacker) Produce an unlabeled dataset U as follows: 1.
On input Γ , F , D , and V , A perturbs each point ( x
, y ) ∈ V to ( x (cid:48) , y ) (subject to the agreed attack type), giving (cid:101) V
=
A (Γ , F, D, V ) .
Send U =
(cid:101) V | X (that is, the feature vectors of (cid:101) V ) to the defender.
• (Defender) Produce an adapted model as (cid:101) F = Γ( F, D, U ) .
Evaluation: Evaluate the test loss of (cid:101) F on (cid:101) V , L ( (cid:101) F, (cid:101) V ) .
On the adversary: While the adversary knows the adaptation algorithm, Γ may use some private ran- domness, such as random initialization.
Since defender proceeds after the attacker to apply Γ , these private randomness is assumed not to be known by the adversary.
On notations: If there is no need for a pretrained model, we use the notation Γ( ⊥ , · , · ) .
Modeling differences.
The ﬁrst obvious distinction is that source domain may differ from the target domain.
This is called a test-time threat model since (cid:101) F is trained based on U =
(cid:101) V | X , and the test error is only evaluated on (cid:101) V (the perturbed test set).
This is in sharp contrast with the classic minimax threat model, where the model F is trained on D , which is independent from V and so (cid:101) V .
This threat model enables us to study whether a large test set (i.e. large | U | ) can beneﬁt a defender for adversarial robustness .
This is in sharp contrast with the classic minimax threat model, where the attacker is granted much power, and can pick points in a one-by-one, or “online”, fashion (i.e., sample a point from the nature, perturb it, and send to the defender).
This is called a maximin threat model since the adversary must move ﬁrst to submit the challenge set U , and the defender then moves based on it.
In fact, this can be written as a maximin game
E V (cid:2) max U min (cid:101) F test err( (cid:101) F, (cid:101) V ) (cid:3) , different from the minimax game in the classic threat model.
Same security goal.
However, in the case where these two domains coincide, the maximin threat model is still nontrivial.
In fact, from the security perspective, both threat models are about the same end-to-end security goal: Correct predictions on adversarially perturbed data.
Attacking a model vs. Attacking an algorithm.
In the classic threat model, a model F is ﬁxed after training, and at the test time the adversary attacks the model.
However, in the maximin threat model, the adversary must attack an adaptation algorithm Γ .
Restrictions of the threat model.
Astute readers may realize that we have intentionally left the deﬁnition of the test-time adaptation Γ to have a lot of freedom.
For example, Γ can leverage the labeled training data D , which may be practically not available or computationally infeasible for test-time computations.
We consider the following possibilities of restrictions:
Homogeneity .
If the source and target domains equal, we call it the homogeneous maximin model .
This setting is directly related to the classic threat model (Deﬁnition 1), where we have a single domain ( X, Y ) , but an adversary can perturb the test instances.
From a security perspective, maximin threat model gives a relaxed threat model, but the same end-to-end security guarantees .
(i.e., correct predictions against adversarially perturbed input)
Data-obliviousness .
In this case, the adaptation algorithm Γ cannot leverage the labeled source data
D .
In other words, the agent can only leverage the pretrained model F and the unlabeled data set U .
3. A DVERSARIAL SEMI - SUPERVISED MINIMAX THREAT MODEL
An unsatisfying aspect of the theory thus far is that the classic minimax threat model is only deﬁned in the homogeneous case, but the maximin threat model can be inhomogeneous, which makes the direct comparison difﬁcult.
To bridge the gap, we provide a generalization of the classic minimax threat model that lies between the classic minimax threat model (Deﬁnition 1) and the test-time maximin threat model (Deﬁnition 2).
Deﬁnition 3 ( Adversarial semi-supervised minimax threat model).
Fix an adversarial perturbation type.
We have a source domain ( X s , Y s ) , and a target domain ( X t , Y t ) .
The attacker is a pair of algorithms ( A 0 , A 1 ) , and the defender is a pair of algorithms ( T , Γ) , where T is a supervised learning algorithm, and Γ is a semi-supervised learning algorithm.
[Table]
• A training set D is sampled i.i
d. from ( X s , Y s ) .
• A semi-supervision set V is sampled i.i.d from ( X t , Y t ) , V is sent to the attacker.
Training time • (Defender, optional) Train a model F trained on D as F = T ( D ) .
• (Attacker) Produce an unlabeled dataset U as follows: 1.
On input Γ , F , D , and V , A 0 perturbs each point ( x, y ) ∈ V to ( x (cid:48) , y ) (subject to the agreed attack type), giving (cid:101) V =
A 0 (Γ , F, D, V ) .
Send U =
(cid:101) V | X (that is, the feature vectors of (cid:101) V ) to the defender.
• (Defender) Produce a ﬁnal model as Γ : (cid:101) F = Γ( F, D, U ) .
Test time • A (labeled) natural test set V (cid:48) is sampled i.i.d.
from ( X t , Y t ) .
• (Attacker)
On input (cid:101)
[Table]
Modeling differences.
(1) This threat model is semi-supervised , because the defender (learner) receives an unlabeled data set U from the target domain.
Note that the procedure to produce (cid:101) F at the training time of this threat model is exactly the same as the procedure to produce (cid:101) F in the test-time maximin threat model.
The key difference is that, once (cid:101) F is trained, we evaluate it on (cid:102) V (cid:48) , which is the adversarially perturbed data on independently sampled V (cid:48) .
This is thus inductive learning, and a minimax game.
This threat model is adversarial , because the attacker can adversarially perturb the clean semi-supervision set V to produce U .
Classic minimax model as a special case.
The classic minimax threat model is a special case of this threat model, by putting source and target domains equal, and choosing a trivial Γ : Γ( F, D, U ) =
F = T ( D ) .
We list several other specializations of this threat model in Appendix B.3.
Therefore, without loss of generality, for the rest of the paper, by minimax model we mean the adversarial semi-supervised minimax threat model .
Γ : One algorithm, two interpretations.
We note that for an algorithm Γ : ( F, D, U ) (cid:55)→ (cid:101) F , one can have now two interpretations: In the maximin threat model, we interpret it as a adaptation algorithm, because we are only going to evaluate (cid:101) F on (cid:101) V ; in the minimax threat model, we interpret it as a semi-supervised learning algorithm, because we are going to evaluate (cid:101) F on unseen points (cid:102) V (cid:48) .
S EPARATING M AXIMIN AND M INIMAX
We now move on to the focus of this work:
Is the maximin threat model “strictly weaker” than the minimax threat model?
4. V ALUATION OF THE GAMES
Proposition 1 ( Homogeneous maximin vs. Classic minimax threat model).
Let k ≥ 1 be a natural number, and F be the hypothesis class.
For a given V , the domain of (cid:101) V is a well-deﬁned function of V (e.g., (cid:96) ∞ ball around V ).
We have that: E V ∼ ( X,Y ) k (cid:104) [Equation] U
[Equation]
L
[Equation] [Equation] [Equation] The proof (A.1) does not rely on the homogeneity condition, and holds verbatim to the more general semi-supervised threat model.
We also note that, in fact, if the concept class has unbounded VC dimension, then good models always exist that can ﬁt both D and V perfectly. So the valuation of the maximin game is actually always 0 : Proposition 2 ( Good models exist with large capacity ) . Consider binary classiﬁcation tasks and that the hypothesis class F has unbounded VC dimension.
Then the valuation of the maximin game [Equation] is
[Equation]
That is, perfect models always exist to ﬁt U . This thus gives a ﬁrst evidence that that transductive advesarial learning is strictly easier. We remark that transductive learning here is essential (differnet models are allowed for different U ).
We conclude this section by noting the following:
Proposition 3 ( Good minimax solution is also a good maximin solution).
Suppose T ∗ is a supervised learning algorithm which trains a model F ∗ =
T ∗ ( D ) , where its adversarial gain in the adversarial semi-supervised minimax model is bounded by κ (i.e. E V (cid:48)
[max (cid:102) V (cid:48)
L ( F ∗ , (cid:101) V )] ≤ κ. )
Then in the maximin threat model, the adversarial gain of the strategy ( T ∗ , Γ ∗ ) , where Γ ∗ ( F ∗ , D, U ) =
F ∗ =
T ∗ ( D ) , is also upper bounded by κ .
However, clearly, the valuation of the game does not answer a key question whether there is a “real” adaptation algorithm, which can only leverage unlabeled data U , that separates between the two theat models. In other words:
Is there a Γ such that:
• As a test-time adaptation algorithm in the maximin model, it provides robustness, but • As a learning algorithm in the minimax model, the model it produces has no robustness.
The existence of such a Γ would provide a separation between the minimax and the maximin threat models, and indicates that the maximin threat model admits more good solutions.
Despite theoretical interest, this question is of practical importance since these “new” solutions may possess desirable properties that good solutions in the minimax threat model may lack .
For the rest of this section, we consider one such desirable property that the defender solution is attack agnostic (Goodfellow (2018) (pp.30)): That is, The defender strategy is not to directly optimize the performance measure.
(e.g., we know the attack is (cid:96) ∞ -norm attacks, and we directly train for it).
In this section we consider a simple Gaussian data model, and the task of learning a hyperplane to classify data points from these two Gaussians. In this data model, we prove that test-time maximin threat model is strictly easier than the semi-supervised minimax threat model. At the high level, our data model and the lower bound part (for the minimax threat model) are based on the work of Schmidt et al.
; Carmon et al.
(2019b).
On the other hand, we provide a novel upper bound for transductive learning, even when the unlabelled data set only contains one point .
Gaussian model and the learning task.
We consider the homogeneous case (the source and target are the same distribution) and the (cid:96) ∞ attack (however, our transductive learning algorithm does not explicitly levearge the attack type and is thus attack agnostic).
We consider a binary classiﬁcation task where X = R d and Y =
{ +1 , − 1 } , y uniform on Y and x | y ∼ N ( yµ, σ 2 I ) for a vector µ ∈ R d and coordinate noise variance σ 2 > 0 .
In words, this is a mixture of two Gaussians, one with label +1 , and one with label − 1 .
We will consider the following parameters.
First, ﬁx an integer n 0 > 0 , and an (cid:15) ∈ (0 , 1 / 2) , then set the following parameter values:
[Equation]
For both threat models, the datasets D = { ( x i , y i ) } ni =1 and V = { ( x, y ) } .
In particular, V only has one data point.
In the maximin threat model, we let x (cid:48) denote the perturbed input obtained from x by the l ∞ attack with bounded norm (cid:15) > 0 , i.e., x (cid:48)
=
x + ν with (cid:107) ν (cid:107) ∞ ≤ (cid:15) .
Put (cid:101) V
= { ( x (cid:48) , y ) } and
U =
{ x (cid:48) } .
We prove the following:
Theorem 1 ( Separation of Maximin and Minimax ) .
Let K be a (sufﬁciently large) constant (which is used for control classiﬁcation error).
Consider the Gaussian data model above with
[Equation]
, and
[Equation]
(the ﬁrst holds for all sufﬁciently large d , which then determines all n that falls into the range).
Then the following statements are true.
(1) In the semi-supervised minimax threat model, the learned model (cid:101) F = Γ( T ( D ) , D, U ) by any algorithms T and Γ must have a large error: E (cid:110) L ( (cid:101) F, (cid:102) V (cid:48) )
(cid:111) ≥ 12 (1 − d − 1 ) , where the expectation is over the randomness of D, V and possible algorithm randomness.
In the maximin threat model, there exist attack-agnostic algorithms T and Γ such that for some absolute constant c > 0 , the adapted model (cid:101) F = Γ( T ( D ) , D, U ) has a small error:
E
[Equation]
L ( (cid:101) F, (cid:101) V ) (cid:111) ≤ e − cK .
S EPARATION
We now consider deep learning. While we are not able to prove the existence of such a defender solution in this setting, we present a somewhat surprising connection between transductive adversarial robustness and unsupervised domain adaptation. In particular, we propose Domain Adversarial Neural Networks ( DANN ) as a candidate algorithm for the separation, and provide empirical evidence that it provides the desired separation.
The experiment design is as follows:
(1) We use the DANN algorithm with random initialization ( RI ), that is, DANN ( RI , · , · ) , as a test-time adaptation.
There are several motivations of choosing DANN :
(A) DANN ﬁts our framework as it is designed for unsupervised domain adaptation.
(B) DANN is however not designed for adversarial robustness.
Thus is it a very interesting question whether DANN can provide test-time adversarial robustness against attacks (e g. norm-based attacks) that it is not speciﬁcally designed for .
(C) DANN algorithm leverages source data D , which could beneﬁt the maximin robustness.
(2) We generate adversarially perturbed (cid:101) V , and check whether DANN can provide robustness.
(3) Note that DANN ( RI , · , · ) can be viewed as a semi-supervised learning algorithm in the adversarial semi-supervised minimax threat model .
Therefore, we check whether the adapted model (cid:101) F = DANN ( RI , D, U ) is robust in the minimax model.
If (2) and (3) show that DANN ( RI , · , · ) is signiﬁcantly more robust in the test-time maximin threat model than in the minimax model, then the experiment goal is achieved.
Attacks.
We use (cid:96) ∞ attacks in this section (note that DANN is not designed for (cid:96) ∞ attacks).
We report more results about (cid:96) 2 attacks in Appendix E.3.
We consider two categories of attacks:
Transfer attacks .
Transfer attacks are a common class of attacks, where we transfer attacks on one model to the target model (in our context, produced by a maximin defender strategy).
In this paper we will mainly apply PGD attacks on adversarially trained models to generate transfer attacks.
Adaptive attacks .
We also consider adaptive attacks 4 , where an adversary can leverage the knowledge of the adaptation algorithm.
To this end, we notice that test-time adaptation is typically an optimization objective L tta ( (cid:101) F, F, D, U ) , which gives rise to the following bilevel optimization objective for the attacker:
[Equation]
To solve this bilevel optimization, we generalize the work of Lorraine & Duvenaud (2018) to an algorithmic framework, called Fixed Point Alternating Method (Algorithm 1 and 2).
We consider two instantiations in this section: (1) FPAM L L DANN , which is a standard instantiation, where the outer objective is L ( (cid:101) F, (cid:101) V ), and (2) J - FPAM L DANN L DANN , where the outer objective is the exact same DANN objective L DANN .
This is also a natural instantiation where the adversary sets out to fail the DANN optimization.
Note that in this case one can naturally apply the more traditional alternating optimization J - FPAM , because the objective becomes a more traditional minimax objective.
Datasets.
For the homogeneous case, we consider MNIST and CIFAR10 (i.e., for both source and target domains).
The homogeneous case represents the security problem considered in the classic threat model.
For the inhomogeneous case, we consider MNIST → MNIST-M (Ganin et al., 2017), and CIFAR10 → CIFAR10c-fog (Hendrycks & Dietterich, 2019).
For CIFAR10c-fog, all 5 levels combined and perform and 80 - 20 train test split, to obtain the training set of size 40000 and test dataset of size 10000 (for space reasons, we combine all corruption levels for experiments in this section, results where we separately study different levels are reported in Section E.2).
Models.
For MNIST, we use the original DANN architecture from Ganin et al.
(2017) with slight modiﬁcations (e.g. adding batch normalization and dropout layers).
For CIFAR10, we use preactivation ResNets from He et al.
(2016) for the prediction branch of DANN , and for the domain prediction branch
we use the architecture from Sun et al.
(2020) (convolution layer).
This architecture is slightly different from the the typical DANN architecture used for MNIST, which assumes vectorized feature and fully connected domain prediction branch.
[Table] [Table]
[Figure]
We plot accuracy as we train more epochs with DANN
[Table]
Experiment results.
(A)
As a sanity check ﬁrst, we evaluate the accuracy of the adapted DANN models in the minimax threat model.
Not surprisingly, the accuracy becomes very low (close to 0% ), which shows that DANN provides no robustness in the minimax threat model.
(B) Then in the maximin threat model, Table 1 summarizes the results under transfer attacks.
In the homogeneous case, the adversarial accuracy DANN provided in the maximin model is comparable to the adversarial accuracy an adversarially trained model provided in the minimax model.
And the adversarial accuracy becomes signiﬁcantly higher in the inhomogeneous case (compared to using an adversarially trained source model).
(C)
In the maximin threat model, Table 2 summarizes the results under FPAM L L DANN attacks.
Similar to the transfer attack case, DANN provides noticeable robustness.
Note that since the defender moves after the attacker, he or she always applies adaptation “one more time”.
(D)
In the maximin threat model, Table 3 summarizes the results under J - FPAM L L DANN attacks.
This is by far our most effective attacks against DANN , which decreases the robustness to ∼ 40% in the homogeneous case on CIFAR10, and to ∼ 30% in the inhomogenous case on CIFAR10 → CIFAR10c-fog.
Nevertheless, DANN still provides nontrivial robustness, and thus provides positive evidence to our hypothesis that DANN separates maximin and minimax threat models.
(E) Finally, Figure 1 gives the robustness results under different target size.
As we can see, the robustness of DANN degrades as the target size decreases, conﬁrming our intuition that large target size beneﬁts test-time robustness.
R OBUSTNESS OF OBLIVIOUS ADAPTATION AND FUTURE DIRECTIONS
We brieﬂy explore the adversarial robustness of the recent data-oblivious test-time adaptation algorithms.
Speciﬁcally, we focus on the TTT algorithm by Sun et al.
Recall that a test-time adaptation algorithm is data-oblivious, if it does not use the labeled training data D at the test time.
TTT algorithm uses a specially trained (with self training) pretrained model, which we denote as
PrTTT .
Similar to our DANN experiment, we conduct experiments on CIFAR10 → CIFAR10c-fog: (1) We found that transfer attacks against PrTTT can already break TTT algorithm.
While this is not surprising as authors of Sun et al.
(2020) have cautioned that TTT is not designed for adversarial robustness, this is in sharp contrast to our results with DANN .
(2) We then use an adversarially trained model as the pretrained model for TTT .
We found that this indeed increases the maximinrobustness of TTT .
However, the robustness is roughly the same (about 1% difference) as directly using the pretrained model.
This indicates that the robustness mainly comes from the adversarially trained model, and TTT algorithm provides no robustness.
This is again in sharp contrast to DANN .
We list several implications as future directions:
• Is transductive adversarial deep learning easier?
Both the current work and the work of Goldwasser et al.
(2020) have given indication that adversarial robustness may be easier in the transductive learning setting.
To this end, we ask: Can we devise more effective algorithms against DANN , or we can prove its robustness in the maximin threat model?
More generally, can we provide more theoretical evidence for transductive adversarial robustness in the setting of deep learning?
• The surprising power of distribution matching for robustness.
It is somewhat surprising that a distribution matching algorithm, such as DANN , can provide adversarial robustness (albeit in a weaker threat model), even when the target domain is already a corrupted domain such as CIFAR10c-fog.
This means that for practical applications where we run an ML model in a batch mode on some collected unlabeled data, one can consider ﬁrst apply UDA to produce an adapted model and then classify the batch and this may provide adversarial robustness.
To this end, can we certify such robustness?
Or can we ﬁnd better attacks?
• Does robust test-time adaptation always need to access source data?
We note that while DANN provides maximin robustness, it needs to access the labeled training data D at test time, and while TTT does not need D , it provides no maximin robustness.
Can we achieve the best of both of worlds and get adversarially robust oblivious test-time adaptation algorithms?
A P ROOFS A. P ROOF OF P ROPOSITION
Let F be the family of models (cid:101) F we can choose from.
From the maximin inequality we have that
[Equation]
Note that for the minimax the max over (cid:101) V is also constrained to perturb features (as we want to ﬁnd adversarial examples).
If we take expectation over V , we have then
[Equation]
Note that
[Equation]
which completes the proof
[Equation]
A. P ROOF OF P ROPOSITION
In the maximin threat model [Equation] let us take the strategy ( T ∗ , Γ ∗ ) as deﬁned.
Therefore the maximin adversarial gain is
[Equation]
which, by assumption [Equation] is bounded by κ .
B M ORE ON THREAT MODELS B. O N THE MODELING
Why is the adversary constrained to attack each point in Deﬁnition 2 and Deﬁnition 3?
This is set up as the rule of the threat model, because if not, an adversary can pick any point that fails the model and repeats it arbitrary number of times, which trivializes the model.
In fact, such one-by-one attack is implicit in the original minimax objective, and is also the common practice of evaluating adversarial robustness (i [Equation] e., typical evaluation is to take a test set, attack the points one by one, and then evaluate adversarial accuracy).
Our threat model formulation makes this hidden assumption explicit
We refer discussion
.d assumption.
Why in the minimax modeling, we do not assume A in Deﬁnition 1, or A 1 in Deﬁnition 3 to know the training algorithms?
For example, T is not in the input of A in Deﬁnition 1.
This is the case because in the inner most expression L ( F, (cid:101) V ) only depends on F , and F is ﬁxed after the training.
Therefore, bringing in the training algorithm gives no additional information (note that this is also the typical practice of evaluating adversarial robustness in the classic setting, where we just attack the model).
By contrast, this is not the case for the maximin threat model because attacker moves ﬁrst and the model is not ﬁxed yet, so in that case, a strong white-box attacker may need to take into consideration the adaptation algorithm, not just the pretrained model.
Why do we need to have two algorithms A 0 , A 1 in Deﬁnition 3?
A 0 attacks the semi-supervised learning algorithm, and thus has Γ as input.
On the other hand, A 1 attacks a model, so neither Γ nor
T is in the input.
Therefore, these two algorithms are different and are separated in the deﬁnition.
Note that the joint goal of A 0 , A 1 is to fail the ﬁnal model on (cid:102) V (cid:48) , so A 0 can be totally different from
A 1 in order to fail the training.
In this respect, A 0 is like data poisoning the unlabeled data set in the semi-supervised learning.
Unlike data poisoning though, we have the constraint that it must attack each point in V .
There are two algorithms A 0 , A 1 in Deﬁnition 3, wouldn’t A 0 help the defender because it leaks information about the target distribution.
We have shown that the defender can choose to ignore the unlabeled data set, and this degenerates to the classic threat model in the homogeneous case.
Otherwise, the semi-supervised learning setting may indeed help the defender, because the information about the target domain (or more unlabeled data if we are in the homogeneous case) is leaked to the defender.
This is, however, exactly why in these cases that this threat model lies between the classic threat model and the maximin threat model.
In the maximin threat model, the attacker leaks U to the defender, and the defender is only required to predict U .
In the semisupervised threat model, things becomes more challenging, because ﬁnally we want to evaluate the model on independent samples.
In the experiments, we essentially instantiated the semi-supervised threat model, for example, as ( A 0 = FPA , A 1 = PGD ) .
Is that too weak?
A strong adversary of course may want to use a strong attack A 0 to fail the training, and thus fail the ﬁnal model, and choosing FPA may indeed be a weak choice.
However, we note that our goal in Step (3) of the experiment design is to show that we can break the adapted model, and ( FPA , PGD ) already full-ﬁlls this purpose.
As another note, in the maximin threat model, currently FPA is the most natural adaptive attack we can think of.
Exploring stronger attacks is an intriguing future direction.
B. M ORE EXTENSIONS AND RESTRICTIONS FOR MAXIMIN THREAT MODEL
Singleton .
In the standard model, we allow U to be a batch which consists of many point.
In the singleton restriction, U is restricted to have only one point (for example, the test-time training algorithm by Sun et al.
(2020) works in this setting).
Deﬁnition 4 ( Online version).
In the online maximin model, we assume that we will sequentially receive batches of test samples, one after another, and we allow Γ to leverage the previous samples received.
We assume, for simplicity of this work however, that the batches are sampled independently from each other.
For the online maximin model, data obliviousness means that the agent cannot directly leverage the previous unlabeled samples he or she received.
However, the agent can save the previous model parameters and use that.
B. S PECIALIZATION OF THE ADVERSARIAL SEMI - SUPERVISED MINIMAX THREAT MODEL
Adversarial training considered in Miyato et al.
Shu et al.
The setting becomes a special case of our general minimax threat model by:
• Instantiating A 0 as a trivial adversary that does not do any perturbation.
That is the defender receives a set of clean set U of unlabeled features.
• The defender adaptation strategy does adversarial training on the unlabeled data, given by
[Equation]
[Equation] where r ∗ = argmax r : (cid:107) r
(cid:107)≤ ε d ( p ( y | x ) , p ( y | x + r, θ ))
Unsupervised Adversarial Training Uesato et al.
This is similar to the above except that the source and target domains are equal.
C R ELATED W ORK
Adversarial robustness.
Adversarial robustness of deep learning has received signiﬁcant attention in recent year.
By far a standard approach is to do adversarial training directly for the attack type Madry et al.
; Sinha et al.
Several recent works have studied adversarial training in a semi-supervised setting Carmon et al. (2019a)
; Uesato et al.
As we have discussed, these work can be viewed as instantiations in the adversarial semi-supervised minimax threat model.
Our work can be viewed as one step further to study adversarial robustness in the transductive setting for deep learning.
Test-time adaptation and transductive learning.
Transductive learning is a long line of research (for example, see the classic work of Transductive Support Vector Machine Joachims (1999)), where one attempts to leverage unlabeled data U to predict on speciﬁc instances of U Vapnik (1998).
On the other hand, transductive learning has not received much attention in the deep learning setting.
The recent proposals of test-time adaptation Sun et al.
; Wang et al.
Nado et al.
(2020) seems to be an reincarnation of this idea, with the additional twist of leveraging a pretrained model.
Our work considers test-time adaptation in an adversarial deep learning setting, which to the best of our knowledge, is new.
Unsupervised Domain Adaptation.
A classic approach for analyzing domain adaption is based on H -divergence Kifer et al.
; Blitzer et al.
; Ben-David et al. (2010).
That theoretical framework is the basis for a line of methods that uses adversarial training with neural networks to learn representations that are indistinguishable between source and target domain, in particular domain adversarial neural network (DANN)
Ajakan et al.
; Ganin et al.
and
related techniques Pei et al.
; Zhao et al.
Some other approach used different divergence notions, such as MMD Long et al.
(2014; 2015), Wasserstein distance Courty et al.
; Shen et al.
(2018), and R´enyi divergence Mansour et al.
D A DAPTIVE ATTACKS AND BILEVEL OPTIMIZATION
In this section we present details of our considerations of adaptive attacks in the test-time adaptation setting.
To start with, in the maximin threat model, for a ﬁxed D (labeled training set) and V (labeled test set), the game is max U min (cid:101) F L ( (cid:101) F, (cid:101) V ) (the adversary can only modify features, namely
U , without modifying the labels).
We focus on norm-bounded attacks, even though the test-time defense strategy can be agnostic to the attack type (i.e., the adaptation algorithm does not explicit leverage the attack type information).
More speciﬁcally, the constraint of the adversary is that he can only perturb the feature of each point in V using norm-based perturbation: Namely for each ( x, y ) the adversary can generate ( x (cid:48) , y ) where (cid:107) x − x (cid:48) (cid:107) ≤ ε .
we use the notation N ( V ) to denote the neighborhood of V which includes all such legitimate perturbed datasets.
In our settings, the adaptation algorithm Γ is typically an optimization objective (which revises the model, we will demonstrate an instantiation using DANN below).
We assume that this objective is
L tta ( (cid:101) F, F, D, U ) , where (cid:101) F is the we want to solve by optimization, F is a (ﬁxed) pretrained model (which can be ⊥ if a pretrained model is not needed, see the DANN instantiation below), D is the labeled data set from source domain, and U = (cid:101) V |
X is the test features (of (cid:101) V ) from the target domain.
Under these conditions, the maximin game can then be written as a bilevel optimization as follows:
[Equation]
To incorporate more objectives, such as that the adversary can attack the inner minimization as a special case, we generalize the loss function of the outer optimization to the form of L ( (cid:101) F, F, D, V ) .
Note that compared to L tta we allow labeled data V for the attacker).
This gives rise to the following objective (we have speciﬁcally generalized the outer maximization objective):
[Equation]
In words, the outer optimization is the adversarial game where the adversary tries to ﬁnd (cid:101) V to fool the defense, while the inner minimization solves the test-time adaptation objective L tta , leveraging the information of U = (cid:101) V | X , in order to derive a revised model (cid:101) F ∗ for predictions.
Example: An objective of maximin game with DANN as an defender .
If the inner test-time defense DANN, then we know that (cid:101) F can be written as f ◦ φ , and we can write the following bilevel optimization objective, where the inner minimization is the DANN objective Ganin et al.
[Equation]
where d is a distance function, which is realized as a domain discriminator (network).
We refer readers to Ganin et al.
(2017) for more details.
We remark that this objective with DANN gives evidence that: (1) Defenses using test-time adaptation are signiﬁcantly different from test-time defenses discussed in Athalye et al.
The game is harder for the adversary because he or she needs to solve a harder optimization problem.
To this end, we note that the test-time defenses discussed in Athalye et al.
(2018) do not amount to bilevel optimizations.
This is because that those defenses are merely about sanitizing input x given a ﬁxed pretrained model, which is known to the adversary at the attack time and can thus be differentiated.
On the other hand, the use of DANN as a test-time adaptation algorithm makes bilevel optimization essential to the adversarial game. Algorithm
Require:
A
dataset D a natural test set V optional pretrained
F
test-time adaptation
an integer parameter k
pretrained
V
ﬁx-
solves
V
V
adversarially perturbed
do
[Table]
[Table]
Solve
[Table]
Solving the bilevel optimization (4).
To solve the bilevel optimization, we propose two algorithm frameworks.
Fixed Point
Alternating Method ( FPAM , Algorithm 1), and Joint Fixed Point
Alternating Method ( J - FPAM , Algorithm 2).
These two algorithms generalize the work of Lorraine &
Duvenaud (2018) (speciﬁcally, their Algorithm 2, “optimization of hypernetwork, then hyperparameters”), to solve (4).
Speciﬁcally, the joint optimization can be effective in the case where the outer and inner objectives are similar to each other, in which case the objective degnerates to a more traditional minimax objective for optimization.
Instantiations of FPAM LL tta with DANN .
We now instantiate this framework by considering L tta as the DANN objective (with random initialization).
In this case, there is no pretrained model F , so the inner test-time adaptation objective simpliﬁes to L DANN ( (cid:101) F, D, U ) .
We consider two instantiations of the outer maximization:
FPAM L L DANN :
Outer objective is L ( (cid:101) F, (cid:101) V ) .
This is the most standard instantiation, where for the outer maximization, the adversary directly search for (cid:101) F to maximize the test loss of (cid:101) F on (cid:101) V .
J - FPAM L DANN L DANN :
Outer objective is L DANN , the same DANN objective .
This is a natural and standard instantiation where the adversary uses the same DANN objective for both inner and outer objectives.
In this case, J - FPAM can be naturally applied as an alternating optimization based solver.
E M ORE EXPERIMENTS E. B ASELINES : A CCURACY OF ADVERSARIALLY TRAINED MODELS .
[Table]
See Table 4 for
E. T RANSFER ATTACKS ON DIFFERENT CORRUPTION LEVELS
[Table]
See Table 5 for the results.
E. T RANSFER ATTACKS WITH (cid:96) 2 ATTACKS
Homogeneous
[Table]
= [Table] See Table 6 for the results.
Plots of ﬁxed point attacks See Figure 2 for the results.
[Figure]
E. D ETAILS ON E XPERIMENT
Models.
For MNIST, we use the original DANN architecture from Ganin et al.
(2017) with slight modiﬁcations (e.g. adding batch normalization and dropout layers).
For CIFAR10, we use preactivation ResNets from He et al.
(2016) for the prediction branch of DANN , and for the domain prediction branch
we use the architecture from Sun et al.
(2020) (convolution layer).
This architecture is slightly different from the the typical DANN architecture used for MNIST, which assumes vectorized feature and fully connected domain prediction branch.
Training Details.
For our CIFAR10 and CIFAR10c experiment, we obtain both the baseline standard pretrained model and the adversarial pretrained model via the following optimization schemes: SGD with 150 epochs, multiple step learning rate [0 . 1 , 0 . 01 , 0 . 001] with milestone at epoch [75 , 125] , momentum 0 .
9 , and weight decay 0 . 0005 , and batch size 128.
The baseline adversarial pretrained model is trained with 7 -step PGD.
In evaluating the test time robustness for transfer attacks, we generate the adversarial samples with 20 -step PGD.
In evaluating the test time robustness for adaptive attacks, we evaluate with 7 -step PGD.
For our MNIST and MNIST-M experiment, we obtain both the baseline standard pretrained model and the adversarial pretrained model via the following optimization schemes: ADAM withepochs, learning rate 3 × 10 − 4 and batch size 128.
The baseline adversarial pretrained model is trained with 40 -step PGD.
In evaluating the test time robustness for transfer attacks, we generate the adversarial samples with 100 -step PGD.
In evaluating the test time robustness for adaptive attacks, we evaluate with 40 -step PGD.
The optimization scheme for DANN during test time adaptation (both transfer and adaptive attack experiments) follows as: ADAM with learning rate 0 .
004 , batch size 128 .
F P ROOF OF T HEOREM
We prove this theorem by a series of lemmas.
Lemma 1 ( Part (1) ) .
In the semi-supervised minimax threat model, the learned model (cid:101) F = Γ( T ( D ) , D, U ) , by any algorithms T and Γ , must have a large error:
[Equation]
(5) where the expectation is over the randomness of D, V and possible algorithm randomness.
Proof.
This follows from Theorem 1 in Carmon et al.
(2019b) (or equivalently Theoremin Schmidt et al.
The only difference of our setting from their setting is that we additionally have unlabeled data U for the algorithm.
Since the attacker can provide x (cid:48)
= x , the problem reduces to a problem with at most n + 1 data points in their setting, and thus the statement (1) follows.
Transductive learning algorithms ( T , Γ) :
To prove the statement (2), give conrete algorithms learning algorithms that achieves small test error on x (cid:48) .
High-level structure of the learning algorithms .
At the high level, the learning algorithms work as follows:
At the training time we use part of the training data (denoted as D 2 to train a pretrained model ¯ θ ), and part of the training data (denoted as D 1 , is reserved to test-time adaptation).
Then, at the test time, upon receiving U , we use U to tune ¯ θ , and get two large-margin classiﬁers, ¯ θ + and ¯ θ − , which classify x (cid:48) as +1 and − 1 , respectively.
Finally, we check these two large margin classiﬁers on D 1 (that’s where D 1 is used), and the one that generates smaller error wins and we classify x (cid:48) into the winner class.
Detailed description.
More speciﬁcally, the learning algorithms ( T , Γ) work as follows:
Before game starts .
Let m (cid:48)
=
Kn 0 , m = 10 n 0 .
We split the training set D into two subsets: D 1 :=
{ ( x i , y i ) } m (cid:48)
i =1 and D 2 :=
{ ( x m (cid:48)
+ i , y m (cid:48)
+ i ) } mi =1 .
D 2 will be used to train a pretrained model at the training time
and D 1 will be used at the test time for adaptation.
Training time .
T uses the second part D 2 to compute a pretrained model, that is, a parameter vector:
[Equation]
Test time .
On input U, Γ uses D 1 and U to perform adaptation.
At the high level, it adapts the pre-trained ¯ θ along the direction of x (cid:48) , such that it also has a large margin on x (cid:48) , and also it makes correct predictions on D 1 with large margins.
More speciﬁcally:
(a) First, Γ constructs two classiﬁers, θ + and θ − , such that θ + classiﬁes x (cid:48) to be +1 with a large margin, and θ − classiﬁes x (cid:48) to be − 1 with a large margin.
Speciﬁcally:
[Equation]
where θ + and θ − are viewed as the parameter vectors for linear classiﬁers.
Note that θ + is constructed such that θ (cid:62) + x (cid:48)
=
γ and θ − is such that θ (cid:62)− x (cid:48) =
− γ .
(b) Finally, Γ checks their large margin errors on D 1 .
Formally, let
[Equation]
where c 1 is the constant in Lemma 4.
If (cid:100) err t (¯ θ + ) ≤ (cid:100) err t (¯ θ − ) , then Γ sets (cid:101) F ( x ) :=
sgn(¯ θ (cid:62) + x ) and classiﬁes x (cid:48) to +1 ; otherwise, it sets (cid:101) F ( x ) :
= sgn(¯ θ (cid:62)− x ) and classiﬁes x (cid:48) to − 1 .
Lemma 2 ( Part (2) ) .
In the maximin threat model, there is an absolute constant c > 0 , such that for the T and Γ described above, the adapted model (cid:101) F = Γ( T ( D ) , D, U ) has a small error:
[Equation]
Proof.
Now, we have speciﬁed the algorithms and are ready to prove that w.h.p.
(cid:101) F ( x (cid:48) ) is the correct label y .
Without loss of generality, assume y
= +1 .
By Lemma 3 w.h.p, (cid:100) err t
(¯ θ + ) ≤ (cid:100) err t (¯ θ − ) .
Then by the Hoeffding’s inequality, D 1 is sufﬁciently large to ensure (cid:100) err t (¯ θ + ) ≤ (cid:100) err t (¯ θ − ) with probability ≥ 1 − 2 e − c 2 K/ 2 .
This proves the statement (2).
Lemma 3.
Suppose y = +1 .
There exists absolute constants c such that with probability ≥ 1 − 2 e − K − 2 e − c 2 ( d/n 0 ) 1 / 4 ,
[Equation]
F. P ROOF OF L EMMA
Note that
[Equation]
where
[Equation]
First, consider ¯ θ .
[Equation]
By Lemma 4, we have with probability ≥
[Equation]
−
[Equation]
which gives
[Equation]
Since m = 10 n 0 and d (cid:29) n
[Equation]
we have
[Equation]
for a small constant c .
Next
[Equation]
we have
[Equation]
We now check the sizes of s + and s − .
[Equation]
Then by deﬁnition and bounds in Claim 1,
[Equation]
Since | s | is bounded by a constant
[Equation]
we know | s + | is also bounded by a constant.
Similarly, | s − − s | and thus | s − | are also bounded by some constants.
Furthermore,
[Equation]
[Equation]
[Equation]
{+By Claim 2, we have (cid:107) θ − (cid:107) 
+}
=
(cid:107) θ
+ (cid:107) 2 .
So
[Equation]
Now we are ready to bound the error difference:
[Equation]
for some absolute constant c .
F. T OOLS
Claim 1.
There exists a absolute constant c and K , such that with probability ≥ 1 − 2 e − K −
[Equation]
Proof.
First, since x (cid:48) =
µ + σζ + ν for ζ ∼ N (0 , I ) , with probability ≥ 1 − e − cd
[Equation]
we have:
[Equation]
[Equation]
By Lemma 4, with probability ≥ 1 − e − c 2 ( d/n 0 ) 1 / 4 min { m, ( d/n 0 ) 1 / 4 } ,
[Equation]
Also, with probability 1 − e − K , | ¯
[Equation]
Finally, |
[Equation]
Then
[Equation]
and
[Equation]
We have with probability ≥
[Equation]
By deﬁnition:
[Equation]
Similarly,
[Equation]
Claim 2.
[Equation]
Proof.
We have by deﬁnition:
[Equation]
Then
[Equation]
This completes the proof
[Equation]
Lemma 4 (Paraphrase of Lemma 1 in Carmon et al. (2019b)) .
Let ˆ θ n = 1 n (cid:80) ni
{+=
+}
y i x i .
There exist absolute constants c 0 , c 1 , c 2 such that under parameter setting (1) and d/n 0 > c 0 ,
[Equation]
with probability ≥ 1 − e − c 2 ( d/n 0 ) 1 / 4 min { n, ( d/n 0 ) 1 / 4 } .