Test-Time Adaptation via Conjugate Pseudolabels
Anonymous Author(s)
AffiliationAddress email
Abstract
Test-time adaptation (TTA) refers to adapting neural networks to distribution shifts,with access to only the unlabeled test samples from the new domain at test-time.
Prior TTA methods optimize over unsupervised objectives such as the entropy ofmodel predictions in TENT [42], but it is unclear what exactly makes a good TTAloss.
In this paper, we start by presenting a surprising phenomenon: if we attemptto meta-learn the “best” possible TTA loss over a wide class of functions, then werecover a function that is remarkably similar to (a temperature-scaled version of) thesoftmax-entropy employed by TENT.
This only holds, however, if the classifier weare adapting is trained via cross-entropy loss; if the classifier is trained via squaredloss, a different “best” TTA loss emerges.
To explain this phenomenon, we analyzetest-time adaptation through the lens of the training losses’s convex conjugate .
Weshow that under natural conditions, this (unsupervised) conjugate function can beviewed as a good local approximation to the original supervised loss and indeed, itrecovers the “best” losses found by meta-learning.
This leads to a generic recipethat can be used to find a good TTA loss for any given supervised training lossfunction of a general class.
Empirically, our approach consistently dominatesother TTA alternatives over a wide range of domain adaptation benchmarks.
Ourapproach is particularly of interest when applied to classifiers trained with novelloss functions, e.g., the recently-proposed PolyLoss [19] function, where it differssubstantially from (and outperforms) an entropy-based loss.
Further, we show thatour conjugate based approach can also be interpreted as a kind of self-trainingusing a very specific soft label, which we refer to as the conjugate pseudolabel .
Overall, our method provides a broad framework for better understanding andimproving test-time adaptation.
Introduction
Modern deep networks perform exceeding well on new test inputs that are close to the trainingdistribution.
However, this performance dramatically decreases on test inputs drawn from a differentdistribution.
While there is a large body of work on improving the robustness of models, most robusttraining methods are highly specialized to the setting they cater to.
For e.g., they assume pre-specifiedperturbations, subpopulations, and spurious correlations, or access to unlabeled data from the targetdistribution, and most methods offer close to no improvement on general distribution shifts beyondwhat they were trained for [9, 16].
In practice, it is often cumbersome (or even impossible) to precisely characterize all possible distribution shifts a model could encounter and then train accordingly.
Instead, a model already trained onsome source data must be able to adapt to the new test data from a different target domain, whilemaking predictions i.e. test-time adaptation.
This setting of test-time adaptation (TTA) has gainedinterest in recent years [5, 39, 42, 46].
TTA is typically accomplished by updating the source modelparameters via a few steps of optimization on an unsupervised objective involving the new test samplefrom the target distribution.
The choice of this unsupervised objective, which we call TTA loss, governs how successful the adaptation process is.
[39] uses a self-supervised objective on the testsample, [42] uses the entropy of model predictions, and several follow-ups have proposed variants oralternatives [31, 46].
However, it remains unclear as to how to choose or guide the selection of this
TTA loss, and thus far the choice of these losses has remained largely heuristic in nature.
In this work, we begin by presenting a set of intriguing experiments where we attempt to learn the“best” TTA loss for a given source network and distribution shift.
We parameterize the TTA loss byanother neural network whose parameters are learnt via meta-learning [3, 7] where we differentiatethrough the adaptation process to find the TTA loss that achieves the best adaptation on distributionshifts.
Surprisingly, we ultimately learn a TTA loss that looks remarkably similar to (a temperaturescaled version of) the softmax-entropy loss, which was already proposed by [42, 46].
Why did werecover the commonly used softmax-entropy loss despite the fact that the procedure is capable oflearning a very general class of losses and the meta-learning process could potentially specialize tothe distribution shift of interest?
Furthermore, we find that this pattern only holds when the loss usedto train the source classifier is the cross-entropy loss; when a different loss such as squared loss isused instead, the meta-learning procedure recovers a TTA loss that itself looks more like a negativesquared error, and is very different from the softmax-entropy loss.
In order to explain this phenomenon, we propose to consider TTA through the lens of the convexconjugate function.
Specifically, given a hypothesis function h ( x ) and label y , several common losses(cross-entropy and the squared loss amongst them, but not limited to these) can be written in theform L ( h ( x ) , y ) =
f ( h ( x ))
− y T h ( x ) for some function f .
In these cases, we show that “natural”
TTA loss for such classifiers is precisely the (negation of) the convex conjugate evaluated at thegradient of h , L TTA ( x ) = − f ∗ ( ∇ f ( h ( x )) , where f ∗ is the convex conjugate of f .
This frameworknot only explains the results of our meta-learning experiments, but also justifies why some specificchoices of TTA loss in the previous literature work well
(e.g., this framework recovers TENT’schoice of softmax-entropy for cross-entropy-trained classifier).
Moreover, it also provides a broadframework for what the TTA loss should be when the source model is trained using various differentloss functions (for example the recently-proposed PolyLoss [19, 23]) as is becoming increasinglycommon in machine learning.
Further, we show that our proposed conjugate adaptation loss is infact a kind of self-training with pseudo-labels [35], a classic approach in machine learning.
Variousformulations of the pseudo-label have been proposed in the literature, and our conjugate analysisprovides a general recipe for the “correct” choice of soft pseudo-labels given by ˆ y ( x ) = ∇ f ( h ( x )) .
We thus refer to these as conjugate pseudo-labels (Conjugate PL’s), and believe our work provides abroad framework for understanding adaptation with unlabeled data in general.
Finally, we empirically verify the effectiveness of our proposed conjugate adaptation loss acrossseveral datasets and training losses, such as cross-entropy and squared loss, along with the recentlyproposed PolyLoss [19] (which itself has shown higher standard test accuracy on a wide rangeof vision tasks).
Over all models, datasets and training losses, we find our proposed conjugatepseudo-labeling consistently outperforms prior TTA losses and improves TTA performance over thecurrent state of the art.
Background and preliminaries.
Test-time adaptation.
We are interested in mapping an input x ∈ R d to a label y ∈ Y .
We learn amodel h θ : R d (cid:55)→ R |Y| parameterized by θ that maps an input x to predictions h θ ( x ) .
We assumeaccess to a trained source model and adapt at test-time over the test input, before making the finalprediction.
This is the standard test-time adaptation (TTA) setting [39, 42].
During TTA, we updatethe model parameters on an unsupervised objective L ( x, h θ ) .
For example, in TENT [42], this loss isthe entropy of the softmax-normalized predictions of the model.
At each time step of adaptation, weobserve a batch of test inputs and we take a gradient step towards optimizing the TTA loss on this testbatch.
As is standard, we measure the average online performance of models across all steps (numberof test batch inputs seen) in the adaptation process.
Meta learning the loss function.
In order to explore the existence of different TTA losses, weemploy the meta-learning procedure where we attempt to learn the TTA loss.
We use a similarprocedure as prior work on meta-learning loss functions [3, 29] and parameterize the loss functionvia a neural network m ϕ : R |Y| (cid:55)→ R that takes in the model predictions/logits and outputs a lossvalue.
We want to learn parameter ϕ such that when we update θ via the loss function m ϕ , our finalperformance is optimal.
In order to do so, let x be the unlabeled test samples to adapt to, and y be the corresponding labels.
We update θ and ϕ alternatively as follows.
[Equation]
where L is some supervised surrogate loss function such as cross-entropy.
Please refer to Appendix
A3 for further details regarding meta-learning setup.
Note that the meta-learning process aboveassumes access to labels y of test inputs.
In this paper, we do not propose meta-learning the TTA lossas an approach.
Rather, we use meta-learning to explore what the “best” TTA losses look like.
Wediscuss our findings from this exploration in the next section.
Test-time Adaptation via Meta-Learnt Losses
The objective used in TENT is the softmax-entropy of the model predictions which essentially makesthe classifier more confident in its current predictions.
The same can be achieved by various other lossformulations such as those mentioned in [31].
With so many possible choices for the loss function,what should we use for TTA?
In this section, we attempt to answer this empirically and present someintriguing observations.
[Figure]
106 Experiment 1.
We learn the TTA loss parameterized by a neural network via meta-learning asdescribed in Section 2.
Our source classifier is a ResNet-26 trained on CIFAR-10 and we adapt todistribution shifts in CIFAR-10-C.
We use the 4 labeled validation noises in CIFAR-10-C to learn themeta-loss network parameters and we denote the resulting learnt loss function by meta-TTA loss.
Wethen adapt the source classifier to the test set of 15 corruptions by optimizing the meta-TTA loss.
Observations.
First, we find that TTA using meta-TTA loss performs better than TENT ( 12 . 35% vs13 .
14% ), suggesting that there are better TTA losses than previous losses based on softmax-entropy.
However, on examining this meta-TTA loss, we find a surprising observation.
Figure 11b (bluecurve) visualizes the learnt meta-loss over model predictions as we vary a single class predictionwith the rest fixed.
Qualitatively, the learnt meta-loss looks very similar to softmax-entropy in onedimension.
In fact, we can fit it closely with a scaled softmax-entropy function (dashed red curve):α · H ( softmax ( h θ ( x ) /T )) , where α is a magnitude parameter and T is a temperature scaler.
Wewant to test if the meta-loss is basically learning the softmax-entropy function.
Hence, we performtest-time adaptation with the fitted softmax-entropy function instead (dashed red curve) and achievean error of 12 .
32% , essentially recovering the performance of meta-TTA.
Despite the ability to represent many different loss functions and potentially specialize to the CIFAR10-C setting, the meta-loss procedure gave back the standard entropy objective.
Do we always recovera loss that looks like softmax-entropy?
Experiment 2.
In an attempt to isolate when we get back the entropy objective, we vary severalthings.
We tried different architectures for the source classifier, different losses L during the metalearning process (1) and different training losses for the source classifier.
Results.
We observed that we consistently recovered the temperature scaled softmax-entropyfunction in all cases except when we varied the training loss for the source classifier.
On usingthe squared loss function [15], a strikingly different meta-TTA loss emerges.
Figure 11a (bluecurve) shows the learnt meta-loss (13.48 % error) for this network.
Here again, the meta-TTA lossoutperforms entropy ( 14 . 57% ) but it is not simply due to a scaling factor.
The loss now looks like thenegative squared error (red curve).
Like previously, we tried fitting a quadratic loss directly to themeta loss in Figure 11a, and this time we even slightly outperformed the meta-TTA loss.
To summarize, we used a meta-learning procedure to search for the “best” TTA loss, where the lossitself was parameterized by a neural network that could potentially represent arbitrarily complex lossfunctions.
However, we ended up with loss functions displaying remarkable structure: across differentarchitectures and different variants of meta-learning, for a classifier trained with cross-entropy, themeta-TTA loss was temperature scaled softmax-entropy and for a classifier trained with squaredloss, the meta-TTA loss was a negative squared loss.
This is interesting from both a practical andconceptual standpoint where the “best” TTA loss depends on the loss used to train the source classifierin a clean fashion.
We attempt to understand and explain this phenomenon in the next section.
Conjugate Pseudo Labels
Results in the previous section raise an obvious question: why does softmax-entropy as used in TENTseem to be the “best” possible test time adaptation loss for classifiers trained via cross-entropy (atleast, best in the sense that meta-learning consistently recovers something which essentially mimicssoftmax-entropy, even though meta-loss is parameterized by a neural network and hence could learnmuch more complex functions specific to the model and the particular shift)?
And why, alternatively,does a quadratic TTA loss seem to perform best when the classifier is trained via squared loss?
In this section, we offer an explanation of this phenomenon via the construct of the convex conjugatefunction [1].
As we will see, our method recovers softmax-entropy and quadratic loss as the “natural”objectives for classifiers trained via cross-entropy and squared loss respectively.
Furthermore, forclassifiers trained via other loss functions, as is becoming increasingly common in deep learning, ourapproach naturally suggests corresponding test-time adaptation losses, which we show in the nextsection to comparatively outperform alternatives.
Thus, we argue that our framework overall providesa compelling recipe for specifying the “correct” method for TTA for a large class of possible losses.
4.1 Losses and the convex conjugate
We begin by formally considering loss functions between a hypothesis output h θ ( x ) (e.g., the logitoutputs of a classifier, or the direct prediction of a regressor) and target y that take the following form
[Equation]
for some function f ; when there is no risk of confusion, we will use h in place of h θ ( x ) for simplicityof notation.
While not every loss can be expressed in such a form, this captures a wide variety ofcommon losses (possibly scaled by a constant value).
For example, cross-entropy loss correspondsto the choice f ( h ) =
log (cid:80) i exp( h i ) and where y denotes a one-hot encoding of the class label;similarly, squared loss corresponds to the choice f ( h )
=
1 2 ∥ h ∥ 22 .
When training an over-parameterized classifier, we can roughly view the training process as (approximately) attaining the minimum over hypotheses h for each training example
[Equation]
where t is the number of training samples.
However, in the case of losses in the form (2), theminimization over h in this form represents a very specific and well-known optimization problem: itis known as the convex conjugate [1] of the function f
[Equation]
where f ⋆ denotes the convex conjugate of f .
f ⋆ is a convex function in y (and indeed, is convexregardless of whether or not f is convex).
Furthermore, for the case that f is convex differentiable,the optimality condition of this minimization problem is given by ∇ f ( h opt ) =
y , so we also have that
[Equation]
where h opt refers to the optimal classifier (used interchangeably with h θ opt ).
Putting this all together,we can state (admittedly, in a rather informal manner) that under the assumption that θ opt is chosen soas to approximately minimize the empirical loss on the source data in the over-parameterized setting,we have that for t inputs
[Equation]
) i.e., the empirical loss can be approximated by the (negative) conjugate applied to the gradient ofthe f , at least in a region close to the optimal θ opt that minimizes the empirical loss.
But the laterexpression has the notable benefit that it does not require any label
y i in order to compute the loss,and thus can be used as a basis for TTA on target domain of the hypothesis function h θ opt .
Definition 1 (conjugate adaptation loss) Consider a loss function that takes the form given in2, used for training a hypothesis h θ in the over-parameterized regime.
We define the conjugateadaptation loss L conj ( h θ ( x )) : R |Y| (cid:55)→ R as follows. [Equation]
[Equation]
4.2 Recovery of existing test-time adaptation strategies
Cross-entropy The interesting aspect to this formalism is that when applied to classifiers trainedwith cross-entropy, it recovers exactly the TENT approach to TTA : minimizing the softmax-entropyof h θ ( x ) .
And indeed, this loss was also recovered when using meta-learning to learn the “optimal”test-time adaptation loss.
To see this, note that for cross-entropy, we have that f ( h )
= log (cid:80)
i exp( h i ) ,giving the optimality condition y =
[Equation]
exp
[Equation]
exp
[Equation]
) and the conjugate function
[Equation]
In other words,
[Equation]
i.e. softmax-entropy of the model prediction, which is exactly the TTA loss that TENT uses.
Squared loss For the squared loss, we have that f ( h ) = 12 ∥ h ∥ 22 , leading to the optimality conditiony
=
h and conjugate function f
⋆ ( y ) = 12 ∥ y ∥ 22 .
Hence, the adaptation loss in this case would besimply given by L conj ( h θ ( x ))
= − f ⋆ ( ∇ f ( h θ ( x )))
= − 12 ∥ h ∥ 22 which is also what we observed inthe meta-learning experiments discussed in section 3.
4.3 Conjugate pseudo-labels
We now emphasize that by the nature of our approximations, there is an additional simple interpretation of the conjugate loss: it is also equal to the original loss (2) applied to the “psuedo-labels”y CPL θ ( x )
= ∇ f
( h θ ( x )) , where CPL refers to conjugate pseudo-labels, i.e.,
[Equation]
This property is known as the Fenchel-Young inequality, that is f ( x ) + f ⋆ ( u ) ≥ x T u holding withequality when u = ∇ f ( x )
In other words, our conjugate adaptation loss is precisely equivalentto self-training under the specific soft pseudo-labels given by y = ∇ f ( h θ ( x )) .
And indeed, formany cases, this may be a more convenient form to compute than explicitly computing the conjugatefunction at all.
For this reason, we refer to our method as that of conjugate pseudo-labels .
In the case of cross-entropy loss, this approach then corresponds exactly to self-training using labelsgiven by the softmax applied to the current hypothesis.
We must emphasize, however, that whileour conjugate formulation indeed has this “simple” form for the case of cross-entropy loss, the realadvantage comes in that it provides the “correct” pseudo-label for use with other losses, which mayresult in pseudo-labels different from the “common” softmax operation.
Example: conjugate pseudo-labels for PolyLoss.
PolyLoss [19] is a recently-proposed simplealternative to cross-entropy loss than has been shown to improve performance across a wide varietyof compute tasks.
This loss is given by the form
[Equation]
We note that this can be put exactly into our conjugate form (equation 2) by writing the loss in aslightly more involved fashion, which we refer to as the expanded conjugate form [Equation]
[Equation]
where f is the log-sum-exp function as before, and g ( h )
=
h − ϵ (1 − softmax( h )) .
In order toformally put this into the form of the previous loss function (equation 2), we can simply define analternative hypothesis as the function h ′ θ ( x )
=
g ( h θ ( x )) , and then define PolyLoss in the conjugateform as
[Equation]
Typically, however, it is easier to simply operate on the expanded conjugate form, which yields theoptimality condition for the pseudo-label ∇ f ( h opt ) =
D g
( h opt )
y CPL θ ( x ) , where D is the Jacobianoperator.
For the case of PolyLoss, this leads to the conjugate pseudo-label of the following form:
[Equation]
Test-time adaptation.
Finally, we note that the above discussion doesn’t actually address any topicsrelated to test-time adaptation to OOD data, but merely provides a generic characterization of a selftraining procedure for generic loss functions of the form (2).
However, the application to TTA on OODdata is fairly straightforward: as long as the learnt source parameters θ is a reasonable approximationto the true optimal θ opt on the shifted domain, self-training with the conjugate pseudo-labels providesa reasonable proxy for fine-tuning the network on the true OOD loss.
We emphasize that, commonto most approaches for TTA , there are still some amount of design decisions that must be putin place; these are detailed in Section 5.1.
In practice, we observe OOD generalization typicallybenefits (across all baselines) from an additional “temperature” scaling, i.e., applying the TTA loss toh θ ( x ) /T for some fixed temperature T , although it requires a held-out validation dataset for tuning T .
However, we should emphasize that truly unsupervised TTA would require making an informed guessfor the value of these hyper-parameters.
The full procedure for test time adaptation via conjugatepseudo-labels is shown in Algorithm 1.
Algorithm 1 Conjugate pseudo-labeling (Conjugate PL
Input: Source classifier θ 0 trained using loss L ( h θ ( x ) , y ) =
f ( h θ ( x )) − h θ ( x ) ⊤ y .
N batches of test data D test =
[ x 1 , x 2 , . . .
, x N ]
Hyperparams: learning rate η and temperature T .
Let ¯ h θ ( x ) def = h θ ( x ) /T be the temperature scaled predictor.
Let y CPL θ ( x ) denote the conjugate pseudo-label function y CPL θ ( x ) = ∇ ( f (¯ h θ ( x ))) .
for n = 0 , 1 , . .
N − 1 do {+θ n +
+}
=
θ n − η ∇L (cid:16) ¯ h θ ( x n ) , y CPL θ ( x n ) (cid:17) [Self-training with conjugate pseudo-labels]
Experiments
In this section, we empirically evaluate the effectiveness and generality of the proposed conjugatepseudo-labeling procedure (Algorithm 1) for test-time adaptation on a variety of datasets.
5.1 Setup
Datasets We evaluate on the three common corruption benchmarks: adapting a classifier trainedon CIFAR-10 to CIFAR-10-C, CIFAR-100 to CIFAR-100-C and ImageNet to ImageNet-C [11].
Following the previous works [39, 42], we report the error averaged across corruptions at the highestseverity for CIFAR-10/100-C and averaged across corruptions and severity level for ImageNet-C. Wealso evaluate on three domain adaptation datasets: adapting a classifier trained on SVHN to MNIST,an ImageNet classifier to ImageNet-R [12] and adapting from synthetic to real data in VISDA-C [30].
Models and Training Losses Following previous works on TTA[39, 42], we use {+ResNet-
 [10]as the+} source classifier architecture for CIFAR-10/100 experiments, ResNet-18 for SVHN to MNISTand a ResNet-50 for ImageNet and source synthetic data on VisDA-C.
We consider source classifierstrained via the following loss functions: the de-facto cross-entropy, recently proposed polyloss
[19]and squared loss [15].
Baselines.
Our proposed conjugate pseudo-label is the classic approach of self-training with aspecific form of pseudo-labels.
In self-training, we replace the label y with a pseudo-label ˜
y ( x ) andadapt by optimizing the loss function
L ( h θ ( x ) , ˜ y
( x )) .
Note that we could either instantaneouslyupdate the pseudo-labels using the current classifier, or generate pseudo-labels once with just thesource classifier.
Instantaneous updates have been shown to work better for domain adaptation [6, 31],and we perform instantaneous updates for all methods.
While we propose using ˜ y
( x ) = ∇ f ( h θ ( x ))
(See Section 4.3), we compare to the standard pseudolabels used in the literature:
• (i) the “hard” pseudo-label (hard PL) where
˜ y ( x ) = arg max i (cid:0) h θ ( x ) (cid:1) i is the most likely class aspredicted by h θ .
As is common in the self-training literature
we perform confidence thresholding.
• (ii) The “soft” pseudo-label (soft PL) where ˜ y ( x ) is obtained by applying a softmax function to themodel predictions h
θ ( x ) .
We also compare with the following recently proposed test-time adaptation methods.
• Entropy Minimization (ENT) [42] minimizes the entropy of model predictions.• Robust Pseudo-Label
[31] where we minimize a robust classification loss, L rpl =
q − 1 (1 − p ( i | x ) q )where i = argmax j p ( j | x ) and q ∈ [0 , 1] .
• MEMO [46] minimizes entropy of a model’s outputs across different augmentations of a test input.
We implement
a
batch version , where we see multiple test points at
once
, for fair comparisons
TTA methodology Following [42] and [32], we fine-tune the learnable scale and shift parametersof the batch normalization layers across all adaptation losses.
For each batch, batch normalizationstatistics is also updated, as suggested in [34].
We report performance at the end of one round oftest-time adaptation over the entire test set.
We tune the learning rate (LR) and temperature (T) on the validation noises in the corruptionbenchmark by grid-search.
LR is selected from { 1 e − 1 , 1 e − 2 , . . .
1 e − 4 } and T from { 1 , 2 . . .
Allthe experiments have been performed on A6000 GPU’s.
On domain adaptation benchmarks, wherethere is no held-out target domain, we set T to be 1 and use the LR suggested by [5, 42].
We usethe same hyperparameter tuning protocol across all methods.
We single out temperature as a veryimportant hyperparameter, as we discuss in the results below.
[Table]
Prior reported gains of other methods over softmax-entropy minimization disappear when we use temperature scaling.
For additional context, the source classifier errors without adaptation are:
CIFAR-10-C ( 29 . 54% ), CIFAR-100-C ( 62 .
26% ), ImageNet-C ( 61 . 89% )
280 5.2 Results on classifiers trained with cross-entropy
We study the effectiveness of our proposed conjugate pseudo-labels when the source classifier istrained via cross-entropy loss.
In this case, baselines Softmax PL and ENT are the same as Conjugate
PL.
Thus we omit them in our results.
Table 1, reports the performance of various TTA methods.
When the source classifier is trained via cross-entropy, our conjugate pseudo-label algorithm exactlycorresponds to entropy minimization with an additional temperature scaling.
Entropy minimization as proposed in prior work [42] does not tune the temperature parameter, and some newer objectives suchas robust PL or MEMO outperform vanilla entropy minimization.
For example, on CIFAR-100-C,vanilla ENT obtaines 41 .
15% average error, while robust PL improves this to 39 .
80% and MEMOto 38 .
However, with the right temperature scaling, entropy minimization obtains 36 .
10%error which outperforms the newer objectives (with and without temperature scaling).
A similarobservation holds for CIFAR-10-C and ImageNet-C as well.
Essentially, the gains over vanilla entropyminimization vanish when we do temperature scaling, and entropy minimization (i.e. conjugatepseudo-labeling corresponding to cross-entropy) turns out to be the best objective after all.
5.3 Results on classifiers trained with polyloss and squared loss
In the case of cross-entropy, conjugate pseudo-labeling reduces to the familiar notion of entropyminimization.
We now explore the performance of our method on different loss functions where theconjugate pseudo-labels differ substantially from entropy minimization (section 4.3, section 4.2).
Table 2 presents the results on the corruption benchmarks and Table 3 presents the results on theother domain adaptation datasets for source classifiers trained with PolyLoss.
[Table]
[Table]
First, we note that, across all datasets in Table 2 and Table 3, our conjugate PL approach outperformsall other TTA losses.
With polyloss classifiers, entropy minimization is no longer the best method—on CIFAR-100-C, entropy minimization achieves 38 .
23% error while our conjugate PL achieves 36 . 83% .
We see similar consistent gains on CIFAR-10-C, ImageNet-C, ImageNet-R and VisDA-C. On digitadaptation tasks from SVHN to MNIST/USPS/MNISTM, where there is a larger shift between sourceand target, the gains are especially pronounced.
Figure 2 compares how the task loss (polyloss ϵ = 6 )on the test data decreases as we adapt the model through conjugate PL and other baselines.
We use CIFAR-10-C as an example.
Observe that our proposed conjugate PL indeed reduces the task loss themost among other baselines.
Furthermore, on CIFAR-10-C and ImageNet-C, we find that adapting polyloss classifiers via conjugate
PL improves the performance over all methods applied to cross-entropy trained source classifiers.
For e.g., on ImageNet-C, the performance improves from 45 .
34% to 44 .
However, this is onlytrue when using the proposed conjugate PL.
If we just did softmax-entropy minimization (even withtemperature scaling), the final adapted performance of a polyloss classifier ( 45 . 5% ) is in fact worsethan that of a cross-entropy classifier ( 45 .
Our results suggest that as we develop new training losses that improve the source classifiers, it is important to adapt via conjugate pseudo-labeling toreap the maximum gains.
Similarly, we experiment with the case when the source classifier is trained using squared loss onthe CIFAR-10 and CIFAR-100 datasets, and observe consistent gains using the proposed conjugatepseudo-labels over the baselines.
For example, on CIFAR-10-C, TTA using conjugate PL gives anderror of 12 .
87% , outperforming baselines like ENT ( 13 .
24% ) and Softmax PL ( 31 . 81% ).
Table 5 in
Appendix A.7 shows the detailed results.
Comparing Table 1 and Table 2, we see that the relative ordering between the various baselines differs.
This is further evidence that the adaptation loss has to depend on the training loss, and we believe ourconjugate pseudo-label approach captures this appropriately by offering consistent gains across thevarious settings we experimented with.
Related Works
Source-Free Domain Adaptation These approaches [18, 20, 22] in general consider an aprioriaccess to unlabeled data from the target domain.
SHOT [22] proposes information maximizationon the unlabeled data while [45] promotes feature clustering.
[18, 20] use generative modeling toestimate the underlying source distributions for enforcing feature invariance.
These approachesrequire offline optimization on unlabeled target dataset with carefully designed objectives and customtraining pipelines.
Test-time adaptation (TTA) adapts the model at test-time in an online mannerbased on its own predictions.
Although TTA hinges on the optimization setup [31, 34, 42], previousworks are mostly focused on proposing new unsupervised objectives.
[39] optimizes over the taskloss from a self-supervised learning problem, but it alters the training of source classifier.
Closest toour setting, is TENT ([42]), which proposes to minimize the softmax-entropy of model predictions attest time.
Several follow ups like [5, 28, 31, 36, 46] propose alternative TTA objectives, e.g. robustpseudo-labelling [31], augmentations [28, 46] and contrastive learning [5, 41].
However, most ofthese are based on heuristically chosen TTA objectives.
In this paper, we provide a principled way ofdesigning unsupervised objectives for TTA .
Pseudo-Labels and Self Training [6, 17, 43] provide theoretical insights into how self-trainingwith pseudo-labels can help under distribution shift.
Our work provides a general framework for thechoice of soft pseudo-labels based on the conjugate analysis of the source training objective.
Someprior works like [6, 13, 21, 25, 47, 48] have documented the improvement in performance when usinginstantaneous pseudo-labels over pre-computed pseudo-labels, and thus lend further support to thebenefits of our proposed conjugate pseudo-labeling approach.
Distributionally Robust Training For completeness, we briefly discuss different robust trainingmethods.
These approaches towards adaptation alter the source training under various set of assumptions in order to learn more robust and transferable representations.
[14, 33, 44] consider riskminimization under various formulations of worst-case loss.
[2] proposes a new objective for learninginvariant correlations.
[4, 24] propose reweighing the training samples.
A host of approaches like[26, 27, 37, 40] try to achieve feature alignment by matching distribution statistics, notably maximummean discrepancy.
Similarly, [8] tries to align feature representations in a common space usingdiscriminator based approach whereas [38] tries to achieve the same using self-supervised learning.
However, retraining with source data might not be feasible due to data privacy and efficiency issues.
Conclusion and Limitations
In this paper, we proposed a general method of conjugate pseudo-labeling that derives an appropriate TTA loss for a given source classifier.
Across a variety of different training losses and distributionshifts, our method offered consistent gains over alternatives.
This conjugate formalism was inspiredby an intriguing set of meta-learning experiments that suggested that these conjugate pseudo-labelsare somehow the “best” adaptation loss.
However, the role of several heuristics like updating onlythe batch normalization layers and temperature scaling which seems to help all the baselines too,is not well understood.
An immediate extension of our work includes broadening the class of lossfunctions considered beyond equation 2.
Similarly, the meta-learning setup can be extended toinclude intermediate representations and consider learning functions over a batch of input whileaccounting for their interactions.
The experimental investigation in this work focused on adapting inthe presence of distribution shift, but the notion of conjugate pseudo-labels is more general, and isworth investigating even in the standard semi-supervised learning setting.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] We add the limitations of our
work in the end.
See Section
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] SeeSection 7.
We do not foresee any specific negative societal impact of our work, apart
from general guidelines valid for deep learning in general.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main ex-
perimental results (either in the supplemental material or as a URL)?
[Yes]
We will
be submitting the code along with the supplementary material.
We will be adding adetailed readme for replicating the experiments.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] All the implementation details like the gridsearch and validation
data have been mentioned in the paper, see section 5.1.
Further, we will be adding
exact hyper-parameters used for all the datasets in the appendix
(c) Did you report error bars (e.g., with respect to the random seed after running experi-
ments multiple times)?
[Yes] All the results in the paper have the standard deviations
reported, obtained by running the experiment 5 times with random seeds.
(d) Did you include the total amount of compute and the type of resources used (e.g., type
of GPUs, internal cluster, or cloud provider)?
[Yes] See Section 5.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]
We will be adding the code to replicate experiments in this paper
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A] We use all the publicly available datasets
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]