T OWARDS U NDERSTANDING GD WITH H ARD AND C ONJUGATE P SEUDO - LABELS FOR T EST -T IME A DAPTATION
A BSTRACT
We consider a setting that a model needs to adapt to a new domain under distribution shifts, given that only unlabeled test samples from the new domain are accessible at test time.
A common idea in most of the related works is constructing pseudolabels for the unlabeled test samples and applying gradient descent (GD) to a loss function with the pseudo-labels.
Recently, Goyal et al.
(2022) propose conjugate labels, which is a new kind of pseudo-labels for self-training at test time.
They empirically show that the conjugate label outperforms other ways of pseudolabeling on many domain adaptation benchmarks.
However, provably showing that
GD with conjugate labels learns a good classiﬁer for test-time adaptation remains open.
In this work, we aim at theoretically understanding GD with hard and conjugate labels for a binary classiﬁcation problem.
We show that for square loss,
GD with conjugate labels converges to an (cid:15) -optimal predictor under a Gaussian model for any arbitrarily small (cid:15) , while GD with hard pseudo-labels fails in this task.
We also analyze them under different loss functions for the update.
Our results shed lights on understanding when and why GD with hard labels or conjugate labels works in test-time adaptation.
I NTRODUCTION
Fully test-time adaptation is the task of adapting a model from a source domain so that it ﬁts to a new domain at test time, without accessing the true labels of samples from the new domain nor the data from the source domain (Goyal et al., 2022;
Wang et al., 2021a; Li et al., 2020;
Rusak et al., 2021; Zhang et al., 2021a; S & Fleuret, 2021; Mummadi et al., 2021; Iwasawa & Matsuo, 2021;
Liang et al., 2020; Niu et al., 2022; Thopalli et al., 2022; Wang et al., 2022b; Kurmi et al., 2021).
Its setting is different from many works in domain adaptation or test-time training, where the source data or statistics of the source data are available, e.g., Xie et al.
Liu et al.
(2021a)
; Prabhu et al.
; Sun et al.
; Chen et al.
; Hoffman et al.
; Eastwood et al.
Kundu et al.
; Liu et al.
(2021b)
; Schneider et al.
; Gandelsman et al.
; Zhang et al.
(2021b)
; Morerio et al.
Su et al.
Test-time adaptation has drawn growing interest recently, thanks to its potential in real-world applications where annotating test data from a new domain is costly and distribution shifts arise at test time due to some natural factors, e.g., sensor degradation (Wang et al., 2021a), evolving road conditions (Gong et al., 2022; Kumar et al., 2020), weather conditions (Bobu et al., 2018), or change in demographics, users, and time periods (Koh et al., 2021).
The central idea in many related works is the construction of the pseudo-labels or the proposal of the self-training loss functions for the unlabeled samples, see e.g., Wang et al.
(2021a)
; Goyal et al.
More precisely, at each test time t , one receives some unlabeled samples from a new domain, and then one constructs some pseudo-labels and applies a GD step to the corresponding self-training loss function, as summarized in Algorithm 1.
Recently, Goyal et al.
(2022) propose a new type of pseudo-labels called conjugate labels, which is based on an observation that certain loss functions can be naturally connected to conjugate functions, and the pseudo-labels are obtained by exploiting a property of conjugate functions (to be elaborated soon).
They provide a modular approach of constructing conjugate labels for some loss functions, e.g., square loss, cross-entropy loss, exponential loss.
An interesting ﬁnding of Goyal et al.
(2022) is that a recently proposed self-training loss for test-time adaptation of Wang et al.
(2021a) can be recovered from their conjugate-label
[Table] [Table]
[Table]
framework.
They also show that GD with conjugate labels empirically outperforms that of other pseudo-labels like hard labels and robust pseudo-labels (Rusak et al., 2021) across many benchmarks, e.g., ImageNet-C (Hendrycks & Dietterich, 2019),
ImageNet-R (Hendrycks et al., 2021), VISDA-C
(Peng et al., 2017), MNISTM (Ganin & Lempitsky, 2015).
However, certain questions are left open in their work.
For example, why does GD with conjugate labels work?
Why can it dominate GD with other pseudo-labels?
To our knowledge, while pseudo-labels are quite indispensable for self-training in the literature (Li et al., 2019; Zou et al., 2019), works that theoretically understand the dynamic of
GD with pseudo-labels are very sparse, and the only work that we are aware is of Chen et al.
Chen et al.
(2020) show that when data have spurious features, if projected GD is initialized with sufﬁciently high accuracy in a new domain, then by minimizing the exponential loss with hard labels, projected GD converges to an approximately Bayes-optimal solution under certain conditions.
In this work, we study vanilla GD (without projection) for minimizing the self-training loss derived from square loss, logistic loss, and exponential loss under hard labels and conjugate labels.
We prove a performance gap between GD with conjugate labels and GD with hard labels under a simple Gaussian model (Schmidt et al., 2018; Carmon et al., 2019).
Speciﬁcally, we show that GD with hard labels for minimizing square loss can not converge to an (cid:15) -optimal predictor (see (8) for the deﬁnition) for any arbitrarily small (cid:15) , while GD with conjugate labels converge to an (cid:15) -optimal predictor exponentially fast.
Our theoretical result champions the work of conjugate labels of Goyal et al.
We then analyze GD with hard and conjugate labels under logistic loss and exponential loss, and we show that under these scenarios, they converge to an optimal solution at a log( t ) rate, where t is the number of test-time iterations.
Our results suggest that the performance of GD in test-time adaptation depends crucially on the choice of pseudo-labels and loss functions.
Interestingly, the problems of minimizing the associated self-training losses of conjugate labels in this work are non-convex optimization problems.
Hence, our theoretical results ﬁnd an application in non-convex optimization where GD can enjoy some provable guarantees.
P RELIMINARIES
We now give an overview of hard labels and conjugate labels.
But we note that there are other proposals of pseudo-labels in the literature.
We refer the reader to Li et al.
; Zou et al.
Rusak et al.
(2021) and the references therein for details.
Hard labels: Suppose that a model w outputs h w ( x ) ∈ R K and that each element of h w ( x ) could be viewed as the predicted score of each class for a multi-class classiﬁcation problem with K classes.
A hard pseudo-label y hard w ( x ) is a one-hot vector which is 1 on dimension k (and 0 elsewhere) if k = arg max k h w ( x )[ k ] , i.e., class k has the largest predicted score by the model w for a sample x
(Goyal et al., 2022).
On the other hand, for a binary classiﬁcation problem by a linear predictor, i.e., h w ( x ) = w (cid:62) x , a hard pseudo-label is simply deﬁned as:
[Equation]
see, e.g., Kumar et al.
(2020), Chen et al.
GD with hard labels is the case when Algorithmuses a hard label to construct a gradient ∇ w (cid:96) self ( w t ; x t ) and update the model w .
Conjugate labels (Goyal et al., 2022): The approach of using conjugate labels as pseudo-labels crucially relies on the assumption that the original loss function is of the following form:
[Equation]
where f ( · ) : R K → R is a scalar-value function, and y ∈ R K is the label of x , which could be a one-hot encoding vector in multi-class classiﬁcation.
Since the true label y of a sample x is not
[Table]
[Equation]
[Table]
[Equation]
available in test-time adaptation, it is natural to construct a pseudo-label y pseudo w ( x ) and consequently a self-training loss function by replacing y with y pseudo w ( x ) in (2),
[Equation]
One can then compute the gradient ∇ (cid:96) conj ( w ; ( y, x )) and use GD to adapt the model w at test time.
Deﬁne h ∗ ∈ R K as h ∗ ← arg min h ∈ R K f ( h ) − y (cid:62) h, where − f ∗ ( y ) =
min h ∈ R K f ( h ) − y (cid:62) h is the conjugate function, see e.g, Chapter 3.3 in Boyd et al.
It turns out that h ∗ satisﬁes y = ∇ f ( h ∗ ) .
From the similarity, Goyal et al.
(2022) propose conjugate labels:
[Equation]
where y conj w ( x ) is possibly a real-value vector instead of a one-hot encoding vector.
Let y pseudo w ( x ) ← y conj w ( x ) in (3).
Then, we get the self-training loss function using the conjugate label:
[Equation]
We note that GD with conjugate labels is an instance of Algorithm 1 when we let ∇ w (cid:96) self ( w t ; x t )
← ∇ w (cid:96) conj ( w t ; x t ) at each test time t .
Table 1 summarizes conjugate labels and hard labels as well as their self-training loss functions using square loss, logistic loss, and exponential loss.
We provide the derivation of the case using square loss below, while the rest of them are available in Appendix A.
Square
loss
) Example of a conjugate label
y conj
w
x ) and
its
self-training function
(cid:96) conj ( w ; x ) :
Observe that square loss (cid:96) ( w ; ( x, y )) :
= 12 ( y − w (cid:62) x ) 2 is in the form of (2) up to a constant, where f ( · ) = 12 ( · ) 2 : R → R + .
Substituting f ( · ) = 12 ( · ) 2 and h ( w )
= w (cid:62) x in (4) and (5), we get y conj
[Equation]
=
[Equation]
and (cid:96)
[Equation]
T HEORETICAL FRAMEWORK : G AUSSIAN MODEL
Our theoretical analysis considers a binary classiﬁcation setting in which samples from the new domain are generated as x ∼ N ( yµ, σ 2 I d ) ∈ R d , where µ ∈ R d is the mean and σ 2 > 0 is the magnitude of the covariance.
The label y is assumed to be uniform on {− 1 , 1 } .
Therefore, we have
P ( X | Y = y ) =
N ( yµ, σI d ) and P ( y = − 1) =
P ( y = 1) =
12 under Gaussian model (Schmidt et al., 2018;
Carmon et al., 2019; Kumar et al., 2020).
Given a test sample x , a linear predictor w ∈ R d makes a prediction of the label ˆ y w ( x ) as ˆ y w ( x ) = sign( w (cid:62) x ) .
While a model could be self-trained under various loss functions, the natural metric to evaluate a model for classiﬁcation is the expected 0 - 1 loss.
Under Gaussian model, the expected 0loss enjoys a simple closed-form expression:
[Equation] [Equation]
where Φ( u ) := 1 √ 2 π (cid:82) ∞ u exp( − z 2 / 2) dz is the Gaussian error function.
From (7), one can see that the predictors that minimize the 0 − 1 loss are those that align with µ in direction and the minimum error is Φ (cid:16) (cid:107) µ (cid:107) σ (cid:17) .
In other words, an optimal linear predictors w ∗ ∈
R
d has to satisfy cos (cid:16)
w ∗ (cid:107) w ∗ (cid:107) , µ (cid:107) µ (cid:107) (cid:17) = 1 .
In our theoretical analysis, we let µ =
[ (cid:107) µ (cid:107) , 0 , . . .
, 0] (cid:62) ∈ R d ; namely, the ﬁrst element is the only non-zero entry.
Our treatment is without loss of generality, since we can rotate and change a coordinate system if necessary.
For any vector w ∈ R d , its orthogonal component to µ is (cid:16) I d − µ | µ |
[Equation]
|
[Equation]
w .
Thanks to the assumption of µ , the orthogonal space (to µ ) is the subspace of dimension 2 to d .
Indeed, for any vector w , its orthogonal component (to µ ) (cid:16) I d − µ | µ |
[Table]
| µ | (cid:17) w is always 0 in its ﬁrst entry.
Therefore, we can represent an orthogonal component of w as [ w [2] , . . .
, w [ d ]] ∈ R d − 1 .
We call a model w ∈ R d an (cid:15) -optimal predictor under Gaussian model if it satisﬁes two conditions:
Condition 1:
[Equation]
= w [1] > 0 and Condition 2:
[Equation]
≥
[Equation]
− (cid:15).
Using (7), the expected 0 − 1 loss of an (cid:15) -optimal predictor is (cid:96) 0 − 1 ( w ) =
Φ (cid:16) (cid:107) µ (cid:107) σ √ 1 − (cid:15) (cid:17) .
To get an (cid:15) -optimal predictor, we need to satisfy (cid:104) w, µ (cid:105) > 0 and also need that the ratio of the projection onto µ to the size of the orthogonal component to µ is as large as possible, i.e.,
[Equation]
[ i ] is large, which can be seen from the following equalities: cos
[Equation]
The projection of w onto µ has to be positive and large when the size of the orthogonal component is non-zero to get an (cid:15) -optimal predictor, i.e., w [1] (cid:29) 0 .
Finally, in our analysis we will assume that the initial point satisﬁes Condition 1 on (8), which means that the initial point forms an acute angle with µ .
This is a mild assumption, as it means that the source model is better than the random guessing in the new domain.
Related works of Gaussian model: In recent years, there are some works that adopt the framework of Gaussian model to show some provable guarantees under various topics.
For example, Schmidt et al.
and
Carmon et al.
(2019) studying it for adversarial robustness.
For another example,
Kumar et al.
(2020) recently show that self-training with hard labels can learn a good classiﬁer when inﬁnite unlabeled data are available and that the distributions shifts are mild.
Their theoretical result perhaps is the most relevant one to ours in the literature, in addition to Chen et al.
(2020) that we have discussed in the introduction.
Kumar et al.
(2020) consider the setting of gradual distribution shifts so that the data distribution in each iteration t is different and that the update in each t is a minimizer of a constrained optimization:
[Equation]
On (9), L ( · ) :
R → R + is a continuous decreasing function, D t represents the data distribution at t , and y hard w ( x ) :
= sign( w (cid:62) x ) is the hard label for an unlabeled sample x .
The main message of their result is that even though the data distribution of the target domain could be very different from that of the source domain, by using data from the intermediate distributions that change gradually, a good classiﬁer for the target domain can be obtained in the end.
On the other hand, we consider analyzing
GD with pseudo-labels at test-time iterations, and we do not assume that there are intermediate distributions.
Our goal is to provably show that GD with pseudo-labels can learn an optimal classiﬁer in a new domain when only unlabeled samples are available at test time, which is different from the setup of Kumar et al.
(2020) that simply assumes the access to a minimizer of a certain objective.
(A NEGATIVE EXAMPLE ) GD WITH HARD LABELS UNDER SQUARE LOSS
One of the common loss function is square loss.
Recent works have shown that even for the task of classiﬁcation, a model trained under square loss can achieve competitive performance for classiﬁcation as compared to that of a model trained under certain classiﬁcation losses like cross-entropy loss (Demirkaya et al., 2020; Han et al., 2022; Hui & Belkin, 2020).
In this section, we analyze test-time adaptation by GD with hard pseudo-labels under square loss.
Recall the deﬁnition of square loss: (cid:96) ( w ; ( x, y ))
=
12 ( y − w (cid:62) x ) 2 .
By using hard labels as (1), the self-training loss function becomes
[Equation]
It is noted that the derivative of sign( · ) is 0 everywhere except at the origin.
Furthermore, sign( · ) is not differentiable at the origin.
Deﬁne sign(0)
= 0 .
Then, sign( w (cid:62) x ) − w (cid:62) x = 0 when w (cid:62) x = 0 , which allows us to avoid the issue of the non-differentiability.
Speciﬁcally, we can write the gradient as ∇ (cid:96) hard ( w ; x ) = − (cid:0) sign( w (cid:62) x ) − w
[Equation]
x (cid:1) x .
Using the gradient expression, we obtain the dynamic of GD with hard labels under square loss,
[Equation]
What we show in the following proposition is that the update w t of (11) does not converge to the class mean µ in direction.
However, it should be noted that a perfect classiﬁer (i.e., one that has the zero 0 - 1 loss) does not necessarily need to align with the class mean µ depending on the setup.
Proposition 1.
GD with hard labels using square loss fails to converge to an (cid:15) -optimal predictor for any arbitrarily small (cid:15) > 0 even under the noiseless setting of Gaussian model ( σ = 0) .
More precisely, we have cos (cid:16) w t (cid:107) w t (cid:107) , µ (cid:107) µ (cid:107) (cid:17) ≤ 1 − ¯ (cid:15) , for some ¯ (cid:15) > 0 as t → ∞ if w ∞ exists.
Proof.
In this proof, we denote ¯ a t :=
w t [1] =
(cid:68) w t , µ (cid:107) µ (cid:107) (cid:69) .
From (11), we have
[Equation]
Let us consider the simple noiseless setting of Gaussian model, i.e., σ = 0 , as we aim at giving a non-convergence example.
Then, we have x t =
y t µ and the dynamic (12) becomes
[Equation]
where we used y 2 t = 1 and y t sign( y t · ) = sign( · ) because y t = {− 1 , +1 } .
Case: η ≤ 1 (cid:107) µ (cid:107) 2 :
Given the initial condition ¯ a 1 > 0 , we have ¯ a t > 0 , ∀ t from (13), and sign(¯ a t (cid:107) µ (cid:107) )
= 1 , ∀ t .
Then, we can recursively expand (13) from time t + 1 back to time 1 and obtain ¯ a
[Equation]
[Equation]
From (14), we know that ¯ a t → 1 (cid:107) µ (cid:107) , as t → ∞ , where we used that (cid:80) ∞ s =
(1 − η (cid:107) µ (cid:107) 2 ) s = 1 η
[Equation]
µ (cid:107) 2 .
On the other hand, the dynamic of the orthogonal component i (cid:54) = 1 ∈ [ d ] is
[Equation]
sign
[Equation]
(15) where in the last equality we used that x t = y t µ and µ =
[ (cid:107) µ (cid:107) , 0 , . . .
, 0] (cid:62)
∈ R d so that x [ i ] = 0 , ∀ i (cid:54)
= 1 .
By (14) and (15), we get (cid:80) di (cid:54) =1 w ∞ [ i ] 2 w ∞ [
[Equation] [Equation]
That is, the ratio converges to a non-zero value, which implies that GD with hard labels fails to converge to an (cid:15) -optimal predictor for any arbitrarily small (cid:15) , i.e., cos (cid:16) w ∞ (cid:107) w ∞ (cid:107) , µ (cid:107) µ (cid:107) (cid:17) ≤ 1 − ¯ (cid:15) for some ¯ (cid:15) > 0 .
Case: η > 1 (cid:107) µ (cid:107) 2 : Suppose ¯ a t > 0 .
Then, the condition that ¯ a t +1 ≥ ¯ a t is 1 (cid:107) µ (cid:107) ≥ ¯ a t from (13), which means that the projection to µ is bounded and hence the model w t cannot be an (cid:15) -optimal classiﬁer for any arbitrarily small (cid:15) .
On the other hand, if ¯ a t > 1 (cid:107) µ (cid:107) , then ¯ a t +1 <
¯ a t , and ¯ a t +1 could even be negative when ¯ a t >
[Equation]
Moreover, if η >
[Equation]
then the magnitude | ¯ a t | is increasing and the sign of ¯ a t is oscillating; more precisely, we will have | ¯ a t +1 | ≥ | ¯ a t | and sign(¯ a t +1 )
= − sign(¯ a t ) .
Consequently, the model w t is not better than the random guessing at every other iteration (recall (7)), which is not desirable for test-time adaptation.
In the next section, we will provably show that GD with conjugate labels under square loss can learn an (cid:15) -optimal predictor for any arbitrary (cid:15) , which is the ﬁrst theoretical result in the literature that shows the advantage of conjugate labels over hard labels, to the best of our knowledge.
[Figure]
C ONVERGENCE RESULTS OF GD WITH PSEUDO - LABELS
Recall that we have (cid:96) self ( w ; x ) = ψ ( w (cid:62) x ) for some scalar function ψ ( · ) :
R → R under the scenario of linear predictors.
If ψ ( · ) is an even function, i.e., ψ ( u ) = ψ ( − u ) for all u ∈ R , then
[Equation]
[Equation]
where the second equality uses x = y ( µ + σξ ) under Gaussian model, and the last equality uses the assumption that ψ ( · ) is an even function.
We emphasize that the underlying algorithm itself does not have the knowledge of µ , σ , or ξ , and the last expression simply arises from our analysis.
From (16), we know that the gradient is
[Equation] [Equation]
Hence, the dynamic of GD with pseudo-labels is
[Equation]
Now let us analyze the population dynamics, which means that we observe inﬁnitely many unlabeled samples, so we can take expectation on the r.h.s.
of (18).
We get
[Equation]
where the second to last equality uses Stein’s identity (Stein, 1981): for any function ψ : R d → R and ξ ∼ N (0 , I d ) , it holds
that E ξ [ ξψ ( ξ )] =
E ξ [ ∇ ξ ψ ( ξ )] .
Denote a t := (cid:104) w t , µ (cid:105) the dynamic of the component of w t along µ .
Given the dynamic (20), it is clear that the component along µ evolves as:
[Equation] [Equation]
On the other hand
[Equation]
denote b t := (cid:107) [ w t [2] , . . . ,
w t
[ d ]] (cid:62) (cid:107) the size of the component orthogonal to µ .
Then
[Equation]
its population dynamic evolves as:
[Equation]
We further deﬁne the ratio r t := a t b t .
By (21) and (22)
[Equation]
we have
[Equation] [Equation]
It turns out that cos (cid:16) w t
[Equation]
(cid:17) is an increasing function of r t , Indeed, cos
[Equation]
[Figure]
where we used (cid:107) w t (cid:107) =
(cid:112) ( w (cid:62) t µ/ (cid:107) µ (cid:107) )
2 + b 2 t .
A successful recovery ( cos → 1 ) means that we would like r t → ∞ .
In the rest of this paper, we will use the notations ♦ + ♥ or GD + ♦ + ♥ , where ♦ = { conj , hard } and ♥ = { square , logistic , exp } for brevity.
For example, hard + exp represents the self-training loss based on hard labels under exponential loss, i.e., (cid:96) hard ( w ; x ) =
exp( −| w (cid:62) x | ) ,while GD + conj + square stands for GD with conjugate labels under square loss in test-time adaptation.
5. (E XPONENTIAL - RATE CONVERGENCE ) GD + conj + square
Proposition 2.
(GD + conj + square )
The ratio of the projection onto µ to the size of the orthogonal component grows as
[Equation]
Furthermore, GD learns an (cid:15) -optimal predictor after t ≥
[Equation]
iterations.
Proof.
For GD + conj + square , the self-training loss is (cid:96) conj ( w ; x ) = − 12 ( w (cid:62) x ) 2 from (6).
Hence, ψ ( · ) = − 12 ( · ) 2 in (16); moreover, ψ (cid:48) ( · ) = − ( · ) and ψ (cid:48)(cid:48) ( · ) = − 1 in (23).
Therefore, we have
E ξ (cid:2) − ψ (cid:48) (cid:0) w (cid:62) t ( µ + σξ ) (cid:1)(cid:3) =
E ξ (cid:2) w (cid:62) t ( µ + σξ ) (cid:3) =
w (cid:62) t µ since E ξ [ w (cid:62) t ξ ] = 0 , and we also have
E ξ (cid:2) ψ (cid:48)(cid:48) (cid:0) w (cid:62) t ( µ + σξ ) (cid:1)(cid:3) =
E ξ [ − 1] = − 1 in (23).
Consequently, the dynamic of the ratio is
[Equation]
From (24) and (25), the cosine between w t and µ is positive and increasing, given the initial condition a 1 > 0 (or equivalently, r 1 > 0 ).
Hence, Condition 1 on (8) holds for all t .
By using (24), we see that to get an (cid:15) -optimal predictor at test time t , we need to satisfy (cid:107) µ (cid:107) 2 /
[Equation]
Simple calculation shows that t ≥
[Equation]
[Table]
Proposition 1 and 2 together provably show a performance gap between GD + conj + square and
GD + hard + square .
Using conjugate labels, GD converges to the class mean µ in direction exponentially fast, while GD with hard labels fails in this task.
5. log( t ) - RATE CONVERGENCE OF GD
In this subsection, we consider self-training loss functions, (cid:96) self ( w ; x ) = ψ ( w (cid:62) x ) , that satisfy the following set of properties ♣ with parameter ( L, a min ) : (i) Even: ψ ( − a )
=
ψ ( a ) for all a ∈ R .
(ii)
There exists 0 < L < ∞ such that − ψ (cid:48) ( a ) ≥ e − La for all a ≥ a min .
Lemma 1.
The following self-training loss functions (cid:96) self ( w ; x ) = ψ ( w (cid:62) x ) satisfy ♣ .
More precisely, we have:
hard + exp : ψ ( u ) =
exp( −| u | ) satisﬁes ♣ with ( L = 1 , a min = 0) .
hard + logistic : ψ ( u ) =
log (cosh ( u )) − | u | satisﬁes ♣ with ( L = 2 , a min = 0) .
conj + exp : ψ ( u ) =
sech( u ) satisﬁes ♣ with ( L = 1 , a min = 0 .
conj + logistic : ψ ( u ) = log (cosh ( u ))
− tanh ( u ) u satisﬁes ♣ with ( L = 2 , a min = 0 . 5) .
The proof of Lemma 1 is available in Appendix C. Figure 2 plots the self-training losses listed in
Lemma 1.
From the ﬁgure, one might ﬁnd that Property ♣ is evident for these self-training losses.
We will also need the following supporting lemma to get a convergence rate.
Lemma 2.
Consider the dynamic: r t +1 ≥ r t + ce − Lr t , for some L > 0 and c ≥ 0 .
Suppose that initially r 1 > 0 .
Then, r t − τ ∗ ≥
L log c ( t − 1) , for all t > τ ∗ , where τ ∗ = 0 if ν ≤ e Lν , ∀ ν ≥ 0 ; otherwise, τ ∗ =
ν 2 ∗ ( L ) /c ,
where ν ∗ ( L ) is the unique ﬁxed point of ν ∗
=
e Lν ∗ if it exits.
Proof.
From the dynamic, it is clear that r t +1 ≥ r t since c ≥ 0 .
Then,
[Equation] [Equation]
where the last step follows from unrolling the recursion t times.
We ﬁrst analyze the case that r t ≤ e Lr t .
Since r t ≤ e Lr t , we have e 2 Lr t ≥ c ( t − 1) from (26).
Hence, r t ≥ 12 L log c ( t − 1) .
Now let us switch to the case that r t ≥ e Lr t .
Let ν ∗ ( L ) the unique point of ν ∗ such that ν ∗ =
e Lν ∗ .
If r t ≤ ν ∗ ( L ) , then r t ≥ e Lr t .
Hence, we have r 2 t ≥ r t e Lr t (26) ≥ c ( t − 1) .
So r t ≥ (cid:112) c ( t − 1) .
Note this possibility cannot happen more than τ ∗ :
= ν 2 ∗ ( L ) /c times, since we need r t ≤ r ∗ to stay in this regime.
So eventually we get out of this regime after a constant number τ ∗ iterations.
Now we are ready to state another main result in this paper.
Proposition 3 below shows a log( t ) - convergence rate of GD with pseudo-labels in the noiseless setting σ 2 =
if the underlying selftraining loss function satisﬁes ♣ .
The gap between the exponential rate of GD with conjugate labels using square loss shown in Proposition 2 and the logarithmic rate in Proposition 3 suggests that the performance of GD in test-time adaptation also crucially depends on the choice of loss functions, in addition to the choice of pseudo-labels.
Proposition 3.
(Noiseless setting)
Apply GD to minimizing (cid:96) self ( w ; x ) = ψ ( w (cid:62) x ) , where ψ ( · ) satisﬁes ♣ .
If the initial point satisﬁes a 1 > a min , then the ratio of w (cid:48)
t s component along µ to the size of its orthogonal component to µ at test time t , i.e., r t in (23), satisﬁes r t − τ ∗ =
Ω (cid:18)
Lb 1 log (cid:18) η (cid:107) µ (cid:107)bt (cid:19)(cid:19) , for all t > τ ∗ , where τ ∗ is a constant deﬁned in Lemma 2.
Proof.
From (19) or (22), we know that the size of the orthogonal component does not change throughout the iterations when σ 2 = 0 , i.e., b t +1 =
b t , ∀ t .
On the other hand, the component along µ in the noiseless setting has the dynamic,
[Equation] [Equation]
where we recall a t := (cid:104) w t , µ (cid:105) and the inequality uses the property regarding − ψ (cid:48) ( · ) as stated in ♣ .
It is noted that (27) implies that a t is non-decreasing, and hence the condition about the initial point, i.e., a 1 ≥ a min , guarantees a t ≥ a min for all test time t .
By using the above results, we deduce that the dynamic of the ratio r t := a t b t satisﬁes r t +1 ≥
[Equation]
, where we used that b t +1 =
b t = b 1 , ∀ t .
Invoking Lemmaleads to the result.
Proposition 3 implies that GD for minimizing a self-training loss with a smaller constant L can result in a faster growth of the ratio r and consequently a faster convergence rate.
Recall the deﬁnition of
L in Property ♣ : a smaller constant L means that the (minus) derivative − ψ (cid:48) ( · ) of the self-training loss has a heavier tail.
We therefore compare the tails of the self-training loss functions by plotting
L ( z
[Equation] [Equation]
of each
on Figure 3, which shows that there exists a threshold z min such that for all z ≥ z min the number L ( z ) that corresponds to the loss function with the conjugate label is smaller than that of the hard label.
This implies that the self-training loss derived from conjugate labels can have a smaller constant L (for a ﬁnite z ) compared to that of hard labels, which in turn might hint at a faster convergence of GD + conj compared to GD + hard for exponential loss and logistic loss.
Figure 4 shows the experimental results under Gaussian model, where GD uses a received mini-batch of samples to conduct the update at each test time.
The detailed setup is available in Appendix B. We ﬁnd that GD with conjugate labels dominates GD with hard labels empirically, which is aligned with our theoretical result.
It is noted that for the case of exponential loss, Goyal et al.
(2022) report a similar experimental result under Gaussian model — GD + conj+exp outperforms GD + hard+exp .
L IMITATIONS AND OUTLOOKS
In this paper, we analyze GD with hard and conjugate pseudo-labels for test-time adaptation under different loss functions.
We study the performance of each of them under a binary classiﬁcation framework, identify a scenario when GD with hard labels cannot converge to an (cid:15) -optimal predictor for any small (cid:15) while GD with conjugate labels does, and obtain some convergence results of GD with pseudo-labels.
However, there are still many directions worth exploring.
First of all, while our current analysis in the binary classiﬁcation setting might be viewed as a ﬁrst step towards systematically studying GD with pseudo-labels, analyzing GD with pseudo-labels in multi-class classiﬁcation is left open in this work and could be a potential direction.
Second, while analyzing the population dynamics has already given us some insights about GD with pseudo labels, it might be useful to study their ﬁnite-sample dynamics.
Third, theoretically understanding GD with other pseudo-labels or combined with other domain adaptation techniques like ensembling (e.g., Wortsman et al.
or others (e.g., Li et al.
; Schneider et al.
; Eastwood et al.
(2022)) might be promising.
Finally, analyzing momentum methods (e.g., Nesterov (2013); Wibisono et al.
; Wang &
Abernethy (2018); Wang et al. (2022a; 2021b;c)) with pseudo-labels is another interesting direction, and one of the open questions is whether they enjoy provable guarantees of faster test-time adaptation compared to GD.
Overall, we believe that the connection between optimization, domain adaptation, and machine learning under distribution shifts can be strengthened.
A CKNOWLEDGMENTS
The authors appreciate Shikhar Jaiswal spotting out a minor error in our previous version of the proof of Proposition 1, which has been corrected in this version.
The authors thank the constructive feedback from the reviewers and comments from Sachin Goyal, which helps improve the quality of this paper.
The authors also thank Chi-Heng Lin for valuable discussions
A D ERIVATIONS OF CONJUGATE LABELS AND THE ASSOCIATED SELF - TRAINING LOSSES ON T ABLE
(Square loss): Square loss (cid:96) ( w ; ( x, y )) :
= 12 ( y − w (cid:62) x ) 2 is in the form of (2), where f ( · ) = 12 ( · ) 2 :
R → R + .
Substituting f ( · ) = 12 ( · ) 2 and h ( w )
= w (cid:62) x into (4) and (5), we get y conj w ( x ) =
w (cid:62) x, and (cid:96) conj ( w ; x ) = − 1 2( w (cid:62) x ) 2 .
On the other hand let y ← sign( w (cid:62) x ) .
we have y hard w ( x ) = sign( w (cid:62) x ) , and (cid:96) hard ( w ; x ) = 1 2 (cid:0) sign (cid:0) w (cid:62) x
[Equation]
[Equation]
(Logistic loss): Recall that logistic regression predicts P (ˆ y = 1) = exp
[Equation]
1+exp(
[Equation]
x ) and P (ˆ y = 0)
= 1 − P (ˆ y = 1) , and the loss function is:
[Equation]
where ˆ y = { 0 , 1 } .
Let y = 2ˆ
y − 1 ∈ {− 1 , 1 } .
Then, substituting ˆ y =
+ y 2 back into (30) and using the equation cosh( z ) = exp( z )+exp( − z )for any z ∈ R , we obtain an equivalent objective:
[Equation]
Now by renaming w 2 ← w , we get
[Equation]
where the last term is a constant and can be dropped without affecting the training.
Observe that (32) is in the form of (2), where f ( · ) = log (cosh ( · )) and h w ( x ) =
w (cid:62) x .
Using (4) and (5), we get
[Equation]
and
[Equation]
On the other hand, let y ← sign( w (cid:62) x ) in (32).
we have y hard w
[Equation]
and
[Equation] [Equation]
(Exponential loss): Recall that exponential loss is (cid:96) exp
[Equation]
x
[Equation]
y )) :
= exp( − yh w ( x )) =
exp( − yw (cid:62) x ) , where y =
+1 , − 1 } , which can be rewritten as
[Equation]
=
[Equation]
The above function is in an expanded conjugate form (Goyal et al., 2022):
[Equation]
where f ( · ) = cosh( · ) , g ( · )
= sinh( · ) , and h w ( x ) = w (cid:62) x .
Let h ∗ ← arg min h f ( h ) − yg ( h ) .
Then, h ∗ satisﬁes ∇ f ( h ∗ ) =
∇ g
( h ∗ ) y. Goyal et al.
(2022) deﬁne the conjugate label y conj w ( x ) via the equality
[Equation]
for this case.
Therefore, we have y conj w ( x ) =
tanh( w (cid:62) x ) .
By substituting y ← y conj w ( x ) in (35), we get the self-training loss function using the conjugate label: (cid:96) conj ( w ) =
sech( w (cid:62) x ) .
To conclude, we have:
[Equation]
and
[Equation]
On the other hand, let y ← sign( w (cid:62) x ) in (cid:96) exp ( w ; ( x, y )) :
= exp( − yh w ( x )) , we have y hard w ( x ) =
sign( w (cid:62) x ) , and (cid:96) hard ( w
[Equation]
x
[Equation]
= exp
[Equation]
x
[Equation]
B S ETUP OF THE SIMULATION IN F IGURE 1 AND F IGURE
Below we describe how to reproduce Figure 1 and Figure 4.
We ﬁrst specify the mean and covariance µ S , µ T , Σ S =
σ S I d , Σ T =
σ T I d as follows, where the subscript S stands for the source domain, and the subscript T is the target domain.
We set µ S =
e 1 and then set set µ T [1] = 0 .
6567 , and the remaining elements of µ T is set randomly from a normal distribution and were normalized to ensure that µ T is a unit norm vector.
Then, we set σ T = 0 .
This way we have µ (cid:62)T µ S σ T (cid:107) µ S (cid:107) = 0 .
8416 so that Φ (cid:16) µ (cid:62)T µ S σ T (cid:107) µ S (cid:107) (cid:17) = Φ(0 . 8416)
= 0 .
2 , i.e., the initial model w 1 = w S has 20% expected 0 − 1 loss in the new domain T .
Also, the best minimal error in the new domain T is Φ (cid:16) (cid:107) µ T (cid:107) σ T (cid:17) =
Φ (cid:16) 1 0 .
6567 / 0 .(cid:17)
= 0 .
In the simulation result depicted in Figure 1, a sample of ( x = µ ) arrives when the test time t is an odd number and a sample of ( x = − µ ) arrives when the test time t is an even number.
Note that the algorithms do not know the labels.
In the simulation result depicted in Figure 4, we consider the setting of noisy data, i.e., x t ∈ R d is sampled as x t ∼ N ( µ T , σ 2 T I d ) instead of x t = yµ T .
We search the step size η over the grid { 10 − 3 , 5 × 10 − 3 , 10 − 2 , 5 × 10 − 2 , 10 − 1 , 5 × 10 − 1 , 10 0 , 5 × 10 0 , 10 1 , 5 × 10 1 , 10 2 } for each
GD + hard + exp , GD + conj + exp , GD + hard + logistic , or GD + conj + logistic , and report the best result of each one.
C P ROOF OF L EMMA
Lemma 1:
The following self-training loss functions (cid:96) self ( w ; x ) = ψ ( w (cid:62) x ) satisfy the set of properties ♣ .
More precisely, we have
hard + exp : ψ ( u ) =
exp( −| u | ) satisﬁes ♣ with ( L = 1 , a min = 0) .
hard + logistic : ψ ( u ) =
log (cosh ( u )) − | u | satisﬁes ♣ with ( L = 2 , a min = 0) .
conj + exp : ψ ( u ) =
sech( u ) satisﬁes ♣ with ( L = 1 , a min = 0 .
conj + logistic : ψ ( u ) = log (cosh ( u ))
− tanh ( u ) u satisﬁes ♣ with ( L = 2 , a min = 0 . 5) .
Proof.
• For hard + exp , we have ψ ( u ) =
exp( −| u | ) , ψ (cid:48) ( u ) =
− sign( u ) exp( −| u | ) , and ψ (cid:48)(cid:48) ( u ) = exp( −| u | )
+ δ 0 ( u ) .
It is evident that ψ ( u ) = exp( −| u | ) is an even function and that it is differentiable
every
- where except at the origin.
We also have | − ψ (cid:48)
( u ) | ≤ 1 and − ψ (cid:48)
( u ) ≥ exp( − u ) for all u ≥ 0 .
We conclude that ψ ( u ) =
exp
( −| u | ) satisﬁes ♣ with parameter ( L = 1 , a min = 0) .
• For hard + logistic , we have ψ ( u ) = log (cosh ( u )) − | u
| , ψ (cid:48) ( u ) = tanh( u ) − sign( u ) , and ψ (cid:48)(cid:48) ( u ) = sech 2 ( u ) − δ 0 ( u ) .
It is evident that ψ ( u ) = log (cosh ( u )) − | u | is an even function and that it is differentiable everywhere except at the origin.
We also have | − ψ (cid:48)
( u ) | ≤ 1 .
Furthermore,
[Equation]
Hence, for u > 0 , − φ (cid:48) ( u ) = 1 − tanh( u ) =
[Equation]
u
[Equation]
exp( u
[Equation]
+
[Equation]
) ≥ exp( − 2 u ) , since
[Equation]
, and the later is evident for u ≥ 0 .
We conclude that ψ ( u ) = log (cosh ( u ))
−| u | satisﬁes ♣ with parameter ( L = 2 , a min = 0) .
• For conj + exp , we have ψ ( u ) =
sech( u ) , ψ (cid:48) ( u ) =
− tanh( u )sech( u ) , and ψ (cid:48)(cid:48) ( u ) =
− sech( u ) 3 + tanh 2 ( u )sech( u ) .
It is evident that ψ ( u ) = sech( u ) is an even function and that it is differentiable everywhere.
We also have | − ψ (cid:48)
( u
| ≤ 1 , as |
tanh
( u
| ≤ 1 and sech( u )
≤
Note that − ψ (cid:48) ( u ) = tanh( u )sech( u )
=
2(exp( u ) − exp( − u )) (exp( u )+exp( − u )) 2 .
Moreover,
[Equation]
which holds when u ≥ 0 .
That is, − ψ (cid:48)
( u ) ≥ exp( − u ) for all u ≥ 0 .
We conclude that ψ (
u )
=
sech
u
) satisﬁes ♣ with parameter ( L = 1 , a min = 0 .
• For conj + logistic , we have ψ ( u ) = log (cosh ( u ))
− tanh ( u ) u , ψ (cid:48) ( u ) =
− sech 2 ( u ) u , and ψ (cid:48)(cid:48) ( u ) = − sech( u )
+ 2 u tanh( u )sech 2 ( u ) .
It is evident that ψ ( u ) = log (cosh ( u )) − tanh ( u ) u is an even function and that it is differentiable everywhere.
We also have |−
[Equation]
Note that − ψ (cid:48) ( u ) = sech 2 ( u ) u =
[Equation]
Moreover,
[Equation]
which holds when u ≥ 0 .
That is, − ψ (cid:48)
( u ) ≥ exp( − 2 u ) for all u ≥ 0 .
We conclude that ψ ( u ) = log (cosh ( u )) − tanh ( u ) u satisﬁes ♣ with parameter ( L = 2 , a min = 0 . 5) .