Improving self-training under distribution shifts via anchored confidence with theoretical guarantees
Authors: Taejong Joo
Diego Klabjan
Venue: NeurIPS 2024 poster (2024)
Keywords: self-training
distribution shifts
source-free domain adaptation
test-time adaption
Primary area: learning_theory

Abstract: Self-training often falls short under distribution shifts due to an increased discrepancy between prediction confidence and actual accuracy. This typically necessitates computationally demanding methods such as neighborhood or ensemble-based label corrections. Drawing inspiration from insights on early learning regularization, we develop a principled method to improve self-training under distribution shifts based on temporal consistency. Specifically, we build an uncertainty-aware temporal ensemble with a simple relative thresholding. Then, this ensemble smooths noisy pseudo labels to promote selective temporal consistency. We show that our temporal ensemble is asymptotically correct and our label smoothing technique can reduce the optimality gap of self-training. Our extensive experiments validate that our approach consistently improves self-training performances by 8% to 16% across diverse distribution shift scenarios without a computational overhead. Besides, our method exhibits attractive properties, such as improved calibration performance and robustness to different hyperparameter choices.

Improving self-training under distribution shifts via anchored confidence with theoretical guarantees
Taejong Joo & Diego Klabjan
Department of Industrial Engineering & Management Sciences
Northwestern University
Evanston, IL, USA
{taejong- joo, boooddkiin (norrajjnnnnoteeeeeeeee
Abstract
Self-training often falls short under distribution shifts due to an increased discrep- ancy between prediction confidence and actual accuracy. This typically necessitates computationally demanding methods such neighborhood ensemble-based la- bel corrections. Drawing inspiration from insights on early learning regularization, we develop a principled method improve self- self-training under distribution shifts based on temporal consistency. Specifically, we build an uncertainty aware tem- poral ensemble with a simple relative thresholding. Then, this ensemble smooths noisy pseudo labels to promote selective temporal consistency. We show that our temporal ensemble is asymptotically correct and our label smoothing technique can reduce the optimality gap of self-training. Our extensive experiments vali- date that our approach consistently improves self-training performances by 8% to 16% across diverse distribution shift scenarios without computational overhead. Besides, our method exhibits attractive properties, such as improved calibration performance and robustness to different hyperparameter choices.
Introduction
In this work, we address the challenge of adapting pre-trained neural networks at test time under distribution shifts, a problem known as test-time adaptation (TTA) or source-free domain adaptation model trained on one distribution is then tested different
(SFDA). Distribution shifts-wil one-are ubiquitous in many practical scenarios due to demographic subpopulation shift []] and changes in data collection environments 24 Despite the robust performance of neural networks under independent and identically distributed (i.i.d.) settings, they often suffer from substantial perfor- mance degradation under such shifts (5) 6. Recently, TTA and SFDA have proven their effectiveness in resolving these critical issues by effectively leveraging information about the distribution shifts contained in unlabeled samples given the test time.
Self-training is the basis of many state-of-the-art methods in TTA and SFDA 77 utilizing pseudo labels generated from the model's own predictions train model unlabeled samples [10). Since pseudo labels are regarded as true labels in self-trrining, the success of self-training methods highly depends how to filter incorrect pseudo labels prevent self-con celf-confirmation bias (TI. This issue has been effectively handled via simple confidence- based thresholding in i.i.d. settings [22 14]. However, the distribution shifts make hard to filter incorrect pseudo labels due to high noise rates even under high threshold 15]. Thus, sophisticated methods filter incorrect pseudo labels based on a neighborhood structure of the data [16 and consistency of multiple predictions under different models [9 or augmentations (00 Section 5), which are computationally intensive by nature.
Recent insights in suggest an alternative strategy can be also effective: promoting temporal consistency can enhance self-training performance in SFDA without the computational burden of
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

previous methods. The temporal consistency regularizer, so called early learning regularization (ELR) 21, was originally developed to address neural networks' tendency to learn clean information first and then gradually memorize noisy labels [22 23 this setting is naturally connected to self- self-training scenarios when we regard the pseudo labels as random noisy labels. However, the impacts of ELR on self-training have not been fully understood. Also, since ELR does not consider unique characteristics of distribution shifts, we aim to answer the following question: Is there any principled way to improve the way of memorizing all past predictions tailored for self lelf-training under distribution shifts?
In this work, we show that the answer is affirmative by proposing Anchored Confidence (AnCon) that uses confident predictions to support a generalized notion of temporal consistency. Specifically, we construct generalized temporal ensemble, which weighs predictions based on predictive uncertainty. and then use the ensemble as a smoothing vector in label smoothing (24.. Then, through rigorous theoretical analyses, we show that our simple heuristic for the generalized temporal ensemble is asymptotically correct and that the label smoothing formulation can reduce the optimality gap. As a result, AnCon can correct wrong pseudo labels without expensive computations and can be easily applied to self-training methods by replacing one-hot pseudo labels with smooth pseudo labels, unlike neighborhood-baae basee or centroid methods. Through extensive experiments, we show that AnCon improves self-training under diverse distribution shift scenarios and posses many attractive properties.
Our contribution can be summarized as follows: 1) We develop AnCon, which is the first algorithm that attempts to improve self- ooffttrainig under distribution shifts by generalizing a notion of temporal consistency with theoretical guarantees; 2) Without any additional forward passes or neighborhood search, AnCon improves self-training performances 8% and 16% under domain shifts and image corruptions, respectively; 3) Remarkably, we also show that AnCon significantly improves calibration performance and is robust with respect to model selection methods and hyperparameter choices.
2 Background
Notation and setup For an input space X' C Rdd and a label space y' [K] == 21,2.... ,.), we de- fine X and Y to be random variables of input and output with probability densities px and py, respec- tively. We also define a neural network f(:0): X + A parameterized by a parameter 0 E I' where A K- is the probability simplex with K elements. Our goal is to minimize the cross-entropy loss minges> 1(6) == Exy (xxxH(((X;)) pr]x)] where (f
P). (x := - -EkE(K] p(Y = k| k|zz) log fu (z; 0). In the SFDA setting, we are given an initial parameter Og that is trained on a different data generating distribution (X', Y'), e.g., Bu E arg mingeR= Ex [H( py )]- We can think about this setting either (1) (X', Y') being pre-training data and On the foundation model with the task of fine-tuning the model on unlabeled X or (2) a transfer learning problem with (X', Y") being the source domain data and X the target domain. Here, we assume only covariate shift without concept shift; that is, Px * px but pYix = D pY Even in this case, we note that suboptimality under the distribution shift, i.e., mingeRe 1(6) I(fo), can be large.
Self- Self-training In this work, we tackle the distribution shifts by using the self-training method that re- places the true label Y (x) by the pseudo label, i.c., ming..... i(0mew: 0) Ex log fy (w: Onew)] where (2; 8) == arg maX fa 9) is pscudo label under 0. Specifically, the algorithmic framework of self-training is as follows: given Ou. we iteratively find model parameter 0m for m = 0. with 0m+1 arg min, i(Brews 8m)) For later use, we also define prediction confidence c( 6) := max fa (x; 8y
and f0.m := (6o. ... 0m)).
Early learning regularization In the learning from noisy labels (LFFN) scenario, 21 identified an "early-learning phenomenon" " where neural networks initially learn information contained in clean labels before gradually memorizing noisy labels, leading to a performance deterioration as training progresses. To mitigate this issue, ELR penalizes predictions that deviate from earlier predictions by defining target network from the past predictions: FELR (2; 0o.m) ;= > (211-))- 7"- fx; 02). Then, adding an auxiliary loss of LELLE((; 0o.m) = Ex (log(1 f(x; 8)T JELn( (X; 00.m) to ī(0: 9m) can prevent memorization of noisy labels while preserving correct patterns.
Notably, this insight has recently been confirmed to be applicable in the SFDA setting by 8 This observation is appealing because ELR can be efficiently implemented by reusing past predictions without additional forward passes or neighborhood searching, unlike dominant methods in SFDA

---

16 20]. Nevertheless, given that ELR stems from general property of the neural network training the i..d. setting, herein we aim step towards a more principled approach to encourage temporal consistency tailored for distribution shift scenarios.
3 Anchored confidence
In this section, we introduce AnCon, which promotes the temporal consistency on selectively chosen predictions via label smoothing. In Section 3.1. we first explain the idea of promoting selective temporal consistency based on confident predictions via label smoothing [24] with a temporal ensemble. Then, in Section [.2] we explain how to effectively construct temporal ensemble for improving self-training under distribution shifts. Finally, we theoretically analyze the efficacy of
AnCon by drawing connection between our method and knowledge distillation in Section 3.3
3.1 Selective temporal consistency via label smoothing
In this work, we utilize label smoothing [24] to promote the selective temporal consistency instead of using an auxiliary loss function like ELR. Specifically, given a generalized temporal ensemble [2; 00:m, Wo:m) with Wo:m == (wo. Wm) which will be specified in Section 3.2, we construct regularized pscudo label (0:00mm Worm) by using f(;; 0g:m. Wo:m) smoothing vector for the pseudo label Y (x; 0m). That is, we perform self-training by
min Fx (X( (X; foom, Wo:m))] Y(X; 0p:m, Wi:m) (1 (Y;;)
(1) P0:m, W0:m)
where À E [0, 1] is a coefficient, E-(-) is the one-hot encoding, and f(x) = (fi(z), ... fx(x))
the K K-dimensional output of the generalized temporal ensemble (cf. 2
Thus, AnCon can control the usage of potentially noisy information in (x;; 0m) based on its consis- tency with f(a; form; Wo.m). Not only can this approach preserve the early learning phenomenon as in ELR (cf. Section 2) but the label smoothing formulation also significantly stabilizes the self- training performance under different hyperparameter choices due to the fact that the optimal values of hyperparameters are less problem dependent compared to its equivalent auxiliary regularization (25). Beyond removing the burden of hyperparameter search, this robustness is a particularly intriguing property under distribution shift scenarios where the model selection becomes challenging task.
Further, encouraging the temporal consistency through label smoothing enables us to connect our method with knowledge distillation (KD) (26). which can provide a wide range of principled tech- niques and theoretical results developed KD. a concrete example, we will show when and how AnCon can reduce the optimality gap of self-training, i.e., case with X = 0, in Section 3.3
3.2 Constructing an effective generalized temporal ensemble with prediction confidences
Next, we construct the generalized temporal ensemble that makes the selective temporal consistency in I work effectively in self-training under distribution shifts. Specifically, given f..m and weights
Wo:m (I) E A for each x E X, the prediction by the generalized temporal ensemble is
Fol 0orm, Woom)
Σ wx(x) p(y k|x, 0.), E [ t=0
(2)
where p(yla, 02) is the prediction made by fa; 0z), which can be either soft (p(y = j|z 0.) = [2; 6,)) hard (p(y 5|z, 1 if j arg maX fa (x; 8x) and p(y = م|z 0,) = 0 otherwise). In this work, we use hard prediction because soft prediction puts more weights on recent predictions since self- lif-training tends to keep increasing the prediction confidence during training.
Surprisingly, we will show that the following simple relative thresholding for determining Woom gives the asymptotic optimal weights achieving the minimum worst-case optimality gap of self-training:
m
Wm (x) x [c((x; 0m) ((8)
3) (Bx()))0 (c(x;).))
where 8100) is an exponential moving average (EMA) of prediction confidence with hyperparameter B and B[e(X;; 0.)] is a Monte-Carlo approximation of [[]({x00]] with mini-batch samples.
3

---

Intuitively, our weighting mechanism aggregates only relatively confident predictions with uniform weight. Given the observation that relative ordering of confidence is highly correlated with accuracy even under distribution shifts [27 28 our thresholding rule would tend to put non-zero weights on correct predictions. Besides, by employing the relative criterion, the thresholding does not suffer from the problems that neglect predictions obtained in the early stage of training.
We also remark that AnCon has almost the same computational cost as ELR. Specifically, (2) can be implemented by (x; 00:m. Wo:m) = f& (2; 00.m-11 W0:m -1) + Wmm (2 Jply Ar,rr 0m) that requires to store the weighted sum of previous predictions without additional forward passes or storing previous parameters, which is the same as storing the previous logit vector in ELR. Similarly, (3) can
be efficiently implemented by 5% 36 + B [c(X; 0m)] that requires constant additional computational costs compared to vanilla self-training with the constant being small. Therefore, AnCon shares the same computational benefits ELR compared to other state-of-the-an methods in SFDA.
On optimality of the relative thresholding in 3 From the optimization perspective, the optimal weights wi... correspond to the weights under which self-training with Y(X; 0o:m Woo)) can minimize the expected loss:
W Wam
E arg min ((
o Woom
arg min Ex [x (X;; fom. Wo:m)]]
arg
(4)
Wi:m
Unfortunately, ). or its empirical counterpart, is not available in self-training due to the absence of labels. Further, even if labels are given, solving 4 is intractable due to non-smoothness
of [(Oha... with respect Woon and the cost of finding @ wa...
To circumvent this issue, we show in Section 3.3 that (4) can be relaxed to the problem of finding
ensemble weights that give maximum likelihood estimation (MLE) solution under certain conditions. As result, instead of solving the intractable optimization in (4), we find the optimal weights by
w 0:m E arg max Exy [log fv (X; 0o:m; Woon).
Worm
In the following theorem which is proven in Appendix B.2
we show that the simple relative thresholding in (3) can make f(;; fyrm. Woo)) asymptotically correct for samples where the neural network tends to be relatively confident during self-training and thus our simple weighting mechanism in (3 to be the solution of (5) in the asymptotic region.
Theorem 3.1. Let A((c) = fa E Xlc(x; 0.) > c), Co:m) == o e Alce)) and pla; Corm) = Q(::oum
D (1 = (xx (I)) E Ar())) for x such that
Q(1; C0:m) >0. Let us assume that random events (Y = (x; 0:)) and (x) = 8(; 0,)) are conditionally independent given X E Aclc) for j (0..... , -1-1) x E X, e [0, 1). If E X
such that p(x; Co:m) > 1/2, then for the generalized temporal ensemble in 33. it holds that
p(arg max fz; Po:ms Woo) + Y(x)) I exp (exppp000 ((x:00mm) kE[K]
E(p(x; co:m))
(6)
2
where E(z) :- 2z 1 log(2z) is a positive increasing function in E [0.5, 1].
The result states that as long as the average accuracy for relatively confident predictions over iter- ations exceeds 50%, the error rate of the generalized temporal ensemble monotonically decreases as Q(2; Cq.m) increases. Furthermore, [(;; 0n.m, Wo.m) is asymptotically correct on z such that Q(1; C Ca.)) + 0o as m + xo. AnCon aims to achieve these desirable properties through the uncertainty-ave aware temporal consistency that helps to satisfy the condition p(x; Co.m) > 0.3. Specifi- generalized temporal ensemble's 's accuracy tends to
cally, as shown in Figure 4a in Appendix, significantly increase as the number of confident samples increases, being consistent with our theory. We note that this monotonic improvement would not be the case for the temporal ensemble without uncertainty eicertainty-wwarnness awareness and vanilla self eeff-training (cf. Figure 4a).
Finally, we emphasize that the assumption p(æ; Co:m) > 0.5 applies only to relatively confident predictions which are averuged over iterations. This is significantly weaker than requiring lower bound of an expected accuracy of each sample for every iteration, which is the case when LFN methods are directly applied to the self-training scenario. Also, due to its dependency on the choice of the confidence thresholds Com. the assumption can hold by controlling Cor at the expense

---

of loosening the upper bound in 66) (e.g., selecting 90th-quantile as in Figure 45 in Appendix). Specifically, increasing the thresholds can improve {(p(a; Co:m) and enhance the chance of satisfying pla; C0:m) > 1/2 but reducing (x:; Coam). While this trade-off necessitates a proper choice of Co.m.
our extensive experiments show that setting the threshold Cm by the EMA of prediction confidence, i.e. dm. in 3 works effectively.
3.3 Theoretical insights from knowledge distillation
In this section, we present a novel connection between AnCon and KD for addressing intractability of (.. KD is framework for training a small student network e.g. ResNet-50 [29], with an additional supervision from a large teacher network f()) e.g., ResNet-152. Specifically, a cross-entropy under Ex [I(f();) 0), (1 ARD) (Y (X)) + A (X)) with AKD E [0, 1].
KD is (0) which bears a significant similarity with AnCon (cf. (). Indeed, AnCon can be understood as special case of KD called self-distillation when ʃ and ((() have the same architecture, where the generalized temporal ensemble corresponds to the teacher network f(()) with notable difference that the pseudo label Y is used instead of the true label Y. Based on this connection, we perform convergence analysis of AnCon by modifying the partial variance reduction theory [30), as given below. We note that the usage of Y and f results in inherently biased gradient estimator, which requires special treatments for the convergence analysis unlike the typical self-distillation setting.
Setup Following B0), we assume a linear model (x; 0) = exp(Of.)) x Σie(k] exp(O) x)) with O Rdd for E [K]. 9 := Concat((), ①x) E æ and Concat(-) is the concatenation operation. Also, we assume bounded support for X; that is, || || C' for all where px px(x) > 0. Under this setting, we repeat the following steps starting from given 8g (i.c., m = 0):
1. Outer temporal ensemble update: Update Wo:m by (3) to obtain [(z; 0om Wo (cf. (
2. Inner parameter update: With 0m,0 := 6m and (fo.m: Wo.)). solve (1) with stochastic
gradient descent mm,t+1 8m, 796 (m..) for t {0, 10 T 1} and set 0m 0m,r-
3. Iteration number update: Set m m + and terminate if m = f.
Here, 7 is the learning rate, & contains b random samples from px. and the stochastic gradi- ent under AnCon is defined as 9 (m,.() == Vem Ex.eg ([ff(x;;mm.. (X;;; 00mm Woom)). For the linear model, we note that OE where (0(mm))) = Ex,ee H(f(X,; (x); and ge (0 wom) = Concat(@1,4) (91.5 (f0.m Wo.).. ... 9 (fum:: Woom))
(m,.)) = Vie(Om,r) - Aāe (0o:m: Woo)
with
Ix.g Wo.m)
Ex.ee (X,; 00:m: Wo.m) - (X (x;) for 1 E [K].
In Theorem 3.2 which is proven in Appendix B.3] we analyze the convergence of the inner parameter update step under AnCon and vanilla self- telf-traiming.
Theorem 3.2. Let us assume I(0) satisfies L-smoothness, L-expected smoothness, and μ P-llyyk- Lojasiewicz (PL) condition (cf. Assumptions B.1 .B..B... B.2 and m.3 in Appendix B.I) For y S TC-T.
carefully chosen À (cf. X == X)n Ecllse(Oom Wo mlll + 2 (..0u.m.a...... in Lemma B.6 in Appendix
Ee Vg[[Vle (6" (.0g(0 (0pmm. WD:m 17
B.5 and any realization of Om. it holds that
El((0m,,) - 1(0000)))00)) 10ml
(1 -1 --) (1(0m))))))))))))))))))))))))))) 1(0))) +
80 9 (0m)
+ N (Ahms 0orm, Worm)
Optimality gop
Improvement over the initial model
Blas of the porudo labe!
Neighborhood size
(7)
where
1(8*) with H* E arg mingee 1(0), ( = E[I(Y(X;0) # Y(X),.
N N(x; 00:m: Wo:m) = X2 || g(Oum, Worm) 1|2 + E II Vig(O") Age (Bum Worm) ) IP]. and (N), 00:m, Worm) S (0) where N(O) == NO; 0o.m, ) for any (0o.m. Wo:m).
Further, when Woom is such that Be (e(Viç((") (fo.m, Worm)) 2 0, i.e. the teacher has suffi- ciently good performance, it holds that
N(A. :0 wo.m). N(O)
S min 1, min (1, C2a x (8o.m Worm))
20 9 (.B0.m Worser
(8)
(00 womm)
Loya (0a. Wilso)
5

---

where o. == Ee || Vie(0") 11?, 80o... wo..) == Eg |1 Ee 0g(Ou.m, Wo:m) 11. g¹ ((0:m Wo:m) Ex|D (eDDk([(l (f(;; :0:00 Wo:m)) with ( q) is the KL-divergence between p and q. and g (00.m: Wo:m) =I| 1x (f(X; 00:m, Woom) Ex (x; (x;0m) 1|?
In Theorem 3.2, characterizes the optimality gap in the inner loop optimization. Specifically, the first term is about reducing the initial optimality gap over iterations and motivates why we need adaptation, e.g., by self-training, if the performance deteriorates under severe distribution shifts. The second term is about the bias of the pseudo label and motivates the challenges of self-training under poorly performing pseudo labels in the case of severe distribution shifts. Crucially, these two terms can be fully characterized by the quality of initial model fn and do not depend on Wo.m- Thus, we concentrate on the impacts Worm designed in 3 on N (Ai,, 0prm; Woom) to show that AnCon's effectiveness on improving self-training performance under distribution shifts.
First, Theorem 3.2 shows the effectiveness of our label smoothing formulation in (I) under properly chosen A/n Specifically, compared to vanilla teafnntaannng, AnCon results in the smaller neighborhood size of the stochastic gradient descent; .N fa.m. We:n) Σ (0). That is, the result suggests that AnCon is at least better than vanilla self-training under the mild regularity conditions.
Further, under the additional assumption of a sufficiently good performance temporal ensemble, relaxed solution of the intractable optimization problem in (4). Specifically, if the marginal distribution of the pseudo labels does not change quickly outer iterations (which is the case especially for the later training stages shown in Figure 5 in Appendix), changing Wo:m would have only a marginal impact on g (00.m Wo.m). Thus, the weighting mechanism that minimizes g (00:m Wo:n) would minimize the worst-case optimality gap, which justifies our approach of circumventing intractability of (4) with (5). In this regard, AnCon's weighting mechanism in (3) could be thought of as relaxed solution of (4) it minimizes gx k (fu.m Wu:m) in the asymptotic region (cf. Theorem 3.1
(8) motivates AnCon's 's weighting mechanism
We conclude this section by analyzing the three iterative steps where the pseudo labels and the temporal ensembles keep updating.
Corollary 3.2.1. Let us assume 1(0) satisfies L-smoothness, C-expected smoothness, and JtPLL condition. For y S 4C-T, X carefully adjusted at each outer temporal ensemble update (cf. x= X1 in Lemma 6.6 each outer iteration j E {0,.... ,T 1}), and any initial parameter Og. it holds that [)(((,)) - 1r) 5 (y) (1- - 1*) + Ci g-r 1802 of (0,) + NAN( Po.g wo:))l (9)
,0,...,--1 where pl p(T(T) ]) x (1 ( for (0.... 11 and G7 = D - nT)
(1-
Corollary 3.2.1 is proved in Appendix B.4 and gives whole picture of the optimality gap under AnCon. We first remark the trade-off associated with î on the suboptimality Ei((0,) 1"] which characterize the early-learning phenomenon observed in the biased gradient settings (e.g.. self trainiga
31 and LFN )3 Specifically, in (9), increasing 7 reduces the first term (1-17) (---)) (1(0)) -()))))))
but increases the coefficient of the second term G7- Therefore, longer training with a large T may not enhance the self-training performance especially under a large second term due to inaccurate pseudo labels or the temporal ensemble.
Nevertheless, for each outer loop iteration, o8 (0,) would be smaller under AnCon than its value under vanilla self-training E[((0,) [*] has tighter upper bound under AnCon due to Theorem 3.2 Therefore, with the guarantee N (2);: foog: Wo:)) S (0). AnCon would achieve a tighter upper bound of (9) than the vanilla self-training method, enabling longer training with smaller value of the second term as observed Figure Tb] Finally, we remark that this theoretical superiority of AnCon can be extended to the self- telf-training methods with other weighting mechanisms in the asymptotic region
when Eg [ (Vi (0*), 9e (0am, Wo.m))] 2 0 (cf. Theorem 3.2
4 Experiments
Goal and baselines Part of the experiments shows that AnCon surpasses the vanilla self-training method that uses (x;; 0) as a pseudo label, which serves as a strong baseline, under different types

---

Table 1: SFDA benchmark results. The numbers indicate the mean test accuracy across three repetitions. We present half of the domain pairs for Office-31 and OfficeHome in the main body and the rest of the pairs are presented in Appendix (cf. Tables
and 60.
Office-31
OfficeHomme
VisDa
Method
A2D
A2W
D2A
Avgiall
Ar
Ar2Pr
Ar3C1
Rw2Ar
Rw2FF
Rw2C1
Avg (all)
VisDa
Self-Training
79.32
80.88
62.07
80.47
75.19
69.20
43.07
66.13
78.10
46.64
61.35
67.77
ELR (A MELA
79.02
81.19
63,70
80.84
75.97
70.47
45.80
67.00
78.64
50.01
63.03
71.89
AnCon
82.23
79.94
63.99
81.37
76.13
70.56
48.06
67.57
79.27
51.89
63.91
71.11
GCE
86.85
86.16
64.03
85.16
74,05
69.89
43,46
08.87
79.25
49.18
63.00
65.20
+ ELR (A MELA)
86.85
86.54
64.80
83.20
74.89
71.62
43.45
69.63
79.98
49.68
63.77
66.90
AnCon
86.75
87.11
65.78
83.50
76.47
71.80
44.83
70.21
80.15
51.74
64.81
68.40
NRC
92.67
88. 11
72,83
87.42
73,86
75.04
47.90
61.15
76.39
51.75
63.92
74,30
ELR (A MELA
92.77
87.92
72.77
87.46
76.82
76.01
52.19
66.01
80.40
55.69
66.89
82.80
AnCon
94.28
91.45
74.49
89.07
79.64
77.11
53.56
69.47
80.04
$7.30
67.96
83.70
of distribution shifts (Sections 4.2 4.2). We also show that AnCon achieves better performance than ELR with NELR E {1,3, 7, 12, 25 which is the coefficient multiplied to the auxiliary loss, to show effectiveness of uncertainty aware temporal ensemble. Finally, we assess the integration of AnCon with generalized cross-entropy (GCE) 3 3 and neighborhood reciprocity clustering (NRC)
[17,, which are frequently cited state-of-the ofrthe--rttttttt methods (20 34 31. We note that AnCon can be seamlessly applied to GCE and NRC by replacing one-hot pseudo labels by AnCon's 's regularized
pscudo labels Y (X; 9o:m: Woom) ). For descriptions of the training configurations which we adopt
from literature, see Appendix D
Evaluation our self-training settings (SFDA and TTA), we should determine the best checkpoint
without labeled samples, i.e. to select model op. 01. ..... O2 which is used for evaluation on test. We use information maximization (35), (M) = [=(Ex[[(X; ()) - Ex [H O)] when H is the entropy, which is proven to be effective in unsupervised domain adaptation (UDA)
model selection literature [36 [7]- Specifically, we evaluate (0m) on the hold-out unlabeled samples at the end of each epoch m for / number of training epochs and select the checkpoint with 8 E arg ma 1M (0mx) as the best one. This 0 is used for final evaluation on the test data.
Implementation details of AnCon For each (outer) iteration m, we set wz(x) 1c(x;;,) > ((())
1(c(x:0))>6)))
for j E (0,...,m) m,. instead of (x)(x) Σr (c(z;;0)));))) Note that the former with À can be thought of as the latter with instance-dependent coefficient X Q(x; Co:m). which assigns more weight on the generalized temporal ensemble for x with large Qxx; Co:m). In addition, we consider the online update scenario of pseudo labels, i.e., T = 1. Finally, given the challenging nature of hyperparameter selection in self-training under distribution shifts, we perform all experiments by using a single configuration of hyperparameters of AnCon (A = 0.3 and 8 = 0.9). These hyperparameters result in the maximum value of max,me(") IM (0m) on the hold-out unlabeled samples on Office-31.
4.1 Self-training under domain shifts
We first consider SFDA that adapts a model trained in one domain by performing self-training in the distribution shifted domain. For the network architecture, use the modified ResNet used in [6], which includes batch normalization (38 after the bottleneck layer and weight normalization [3 in
the last linear layer, which is used in 3 for stabilizing the learning process in SFDA.
Datasets We evaluate AnCon on the following datasets: Office-31 440
with 4,000 images of 31
categories from three domains (amazon, dslr, webcam); OfficeHome categories from four domains (art, clipart, product, real-world); VisDa 42 31 categories from two domains (synthetic, real).
41 with 15,000 images of 65
with 280,000 images of
Results AnCon consistently improves self-training in diverse domain pairs across the three datasets (Table [T. Specifically, it reduces the average self-training test error by 5% in Office-31, 6% in OfficeHome, and 13% in VisDa. In addition, compared ELR with its dataset-dependent optimal hyperparameter value, AnCon shows comparable performance to ELR (2%, 3%, and -3% perfor- mance differences in Office-31, OfficeHome, and VisDa, respectively). Despite its slightly inferior performance in VisDa, we note that AnCon achieves significantly better accuracy for low low-performing classes: for "Skateboard" and "Truck" classes, AnCon achieves 50% and 16% of accuracies, while ELR achieves 39% and 0..3% in these classes. Thus, AnCon can be as effective as ELR for improving the self-training performances under distribution shifts without needing to adjust hyperparameter values for each datasct unlike ELR.
7

---

M
40
Selftraining
ELR
'ezz Accuracy
20
AnCon
belf training
20
AnCon
10
Selftraiiing FLK (A 3.0)
5
Lon
Levi3 intensity
2014
Lnd5
Corruation intersity
(a)
(b)
(c)
Figure 1: Section 4.2:
(a) Test accuracy for each intensity level in ImageNet-C. (b) Performance
degeneration in the defocus blur corruption with intensity 4. Section 4.3.1; (c) Maximum performance
changes under different model selection methods. We present performances for individual corruptions in Appendix. For all boxplots used in the paper, the box represents interquantile range with whiskers as + 1.5 interquantile range and the horizontal line inside the box represents the median.
Notably, AnCon significantly improves performances of both GCE and NRC via fundamentally different mechanisms for handling noisy pseudo labels (e.g.. reducing test accuracy of 3% and 8% on average in OfficeHome, respectively). Specifically, NRC filters incorrect predictions based on local consistency, while AnCon uses temporal consistency. Combining NRC and AnCon leverages pseudo labels that are both locally and temporally consistent, resulting in significant performance improvements over NRC or Self-Training AnCon (ef. Table T In addition, GCE reduces the impact of wrong pseudo labels rather than finding them. Applying GCE to AnCon minimizes the effects of potentially wrong but temporally consistent pseudo labels, which can be implied by the performance of GCE AnCon compared to GCE or Self-Training AnCon (ef. Table D). Thus, the impressive performance gains from AnCon, which would be orthogonal to the gains from state-of-the-ar methods in SFDA, show its significant practical implications.
4.2 Self-training under synthetic corruption operations
While we have considered the domain shift, e.g., adaptation of model trained synthetic images to real images. in Section 4.1 this section examines the self-training' s ability to adapt to distribution shifts by synthetic image corruptions. This setting has been used to measure the robustness of neural networks with respect to a general oot-of-distribution setting. To this end, we consider ImageNet-C (6), which consists of 50,000 images drawn from validation of ImageNet 43] where each image corrupted by 15 types of synthetic corruptions related to noise, blur, weather and digital.
Result Consistent with the findings under the domain shift, AnCon outperforms the average perfor- mances self-training and ELR under varying levels corruption intensities (cf. Figure Iaa and Tables 7 Ti in Appendix), improving the self-training method's accuracy by 16% on average. Further, the gains from AnCon significant when the distribution shifts are intense (e.g.. improving accuracies by 20% and 52% on average intensities of 4 and 5) where the initial model trained the source domain significantly deteriorates. Specifically, for Shot, Impulse, and Gaussian corruptions with the most extreme shift intensity of 5, where the initial model achieves accuracies of (3.04%, 1.76%, 2. 12%%) AnCon achieves (22.56%, 26.56%, 25.85% (cf. Table 11). This striking improvement com- pared vanilla self-training with performances (0.26%, 1.72%, 1.04%) and ELR with performances (8.00%, 14. 12%,, 16.00%), underscores the importance the AnCon's 's uncertainty-awart temporal consistency scheme, as shown in Corollary 3.2. We note that this impressive result also explained by AnCon's 's ability to prevent the gradual performance degradation during the course of training with the extremely noisy pseudo labels (cf. Figure [b].. Combined with previous results in domain shift scenarios, we expect that AnCon would work effectively in various -ittr--distribution settings.
4.3 Versatility of AnCon
In previous sections, we have shown the universality of AnCon by evaluating it on diverse distribution shift scenarios. In this section, we show versatility of AnCon by analyzing attractive properties in robustness and uncertainty representation.
4.3.1 Robustness to model selection
There is no universally agreed model selection criterion, such as cross-validation in the i.i.d. setting, in self-training under distribution shifts. This is partly due to the variety of distribution shift scenarios, where an effective criterion in one may be ineffective inapplicable in another; for instance,

---

principled criteria called importance-weighted cross validation 44 in UDA cannot be applied to SFDA. this regard, it would be an important characteristic of self-training method under distribution shift to be robust with respect to different choices of model selection criteria. Therefore, we evaluate robustness with respect to the following different model selection criteria: InfoMax 35
Corr-C [36]. and Ent [5] (see Appendix D.3 for the description).
Figure Ic shows that AnCon's maximum performance change
balative parto parto mance chan change (K)
due to different model selection methods is much lower than that of other methods, especially under severe distribution shifts. This valuable advantage can be contributed to the property of AnCon that can prevent performance degeneration (cf. Figure [bb.
Given that, in practice, we barely know when the model collapse happens and which model selection criteria are the best, the results highlight significant practical value of AnCon.
0.5
0.99 hal
Smoothing confficiont (A)
4.3.2 Robustness to the choice of hyperparameters
(a)
Throughout this paper. we have shown that our single config-
Relative perfarmance 196)
3
uration of parameters (A = 0.3, 3 = 0.9) work well across wide range of benchmark problems. In this section, we aim to show our findings can be preserved when the hyperparam- eter values deviate from the default setting by performing a sensitivity analysis for values À E {0.1, 0.3, 0.5, 0.7, 0.9} and E {0.1, 0.3, 0.5, 0.7, 0.9]. We also test two frequently used annealing schedules that Am m// and Am min 2m/ I).
2-
called full and half, respectively. Figure [igure]]soows that AnCon is sta- ble even under extreme values of hyperparameters. Specifically, for both hyperparameters, the maximum average performance change is less than 1 %, and / barely impacts the performance of AnCon. Indeed, our analysis suggests to increase from our default setting; that is, put a higher weight the general temporal ensemble's 's prediction. Here, we note that our subop- timal choices of hyperparameters are due our rigorous and practical hyperparameter choice. Given the challenging nature of hyperparameter optimization under distribution shifts, the sta- ble performances of AnCon under arbitrary choices of hyperparameters would enable AnCon to be seamlessly applied to diverse practical settings.
0.1 0.3
0.5
0.7
0.9
Confidence EMA coefficient (9)
(b)
Figure 2: Sensitivity analysis with respect to X and & on four domain pairs (Ar-Pr, Pr-CI, Rw- CI. Rw-Pr) in OfficeHome. Here, green triangles are means.
4.3.3 Improved calibration performance
120
Sofftraning
We have shown that all self-training methods significantly improve the performance of the baseline method after the adaptation period. However, it is widely known that these noticeable improvements come with the price of sacrificing an uncertainty representation ability which is critical real-world decision- mmaking scenarios 46
CLA
Specifically, the calibration performance, which is the gap between the prediction confidence and accuracy, usually monotonically increases self-training keeps reducing the uncertainty for all predictions during the course of training. In this regard, we analyze the calibration performance with respect to the expected calibration error (ECE; see Appendix for definition). Here, lower ECE means lower gap between confidence and accuracy.
Leve Computtion Intonsity
sancaang
Jecon
Accin
As shown in Figure 3a and Table in Appendix, AnCon gives much lower ECE compared to other methods. Con- sidering ELR and GCE both have regularization effects, we conjecture that this phenomenon is due to selective regularization in AnCon that increases prediction confi- dences of samples only if the past confident predictions consistent with the current prediction. Especially, in
(b)
Figure 3: (a) ECEs under five levels of intensities in ImageNet-C; (b) Accuracy and ECE changes during the course of training in VisDa.

---

Figure 3b which confirms the accuracy- catitaraciciiiatiionn dilemma in VisDa, AnCon is shown to limit the ECE increases during training compared to all other methods. That is, AnCon helps to significantly reduce the price of the calibration performance we need to pay for improving accuracy, which are both important measures in practice.
4.4 Algorithmic design choices
Recall that we define [2; 0p:m, Wo:n) Exo ()(ar) p(ylar, 0,) with our simple design choices:
the relative thresholding for weighting scheme wx(x)) x 1 (c(x;; 0,) > (((7) and hard prediction for p(ylæ, Oz). In Appendix C.3 we found that our simple design choices are more appropriate for the
distribution shift settings than several more sophisticated alternatives, which can be summarized as follows.
More sophisticated weighting schemes (c.g., Entropy (wi(r) xx (xp(-He 0.) 0;)]
reduce the self-training performance, despite being more accurate measure of prediction uncertainty. We conjecture that the poor calibration performance of the neural network in self-training under distribution shifts prevents the sophisticated weighting schemes from accurately reflecting the goodness of the prediction.
Various soft prediction schemes, which can give more information about the non-leading entry values, leads to performance reductions. We conjecture that the continuously increasing confidence in the later stage of self-training would make soft prediction ignore early-stage predictions which may be valuable to memorize.
5 Related work
Filtering incorrect pscudo labels Popular confidence-based thresholding methods 13 14| 9 fall short under distribution shifts since even high confident predictions can be highly incorrect. Therefore, recent advances in SFDA and TTA utilize higher order information to filter incorrect pseudo labels. For instance, based on the intuition that true labels of adjacent samples would be same, centroids for each predicted class can be maintained in the feature space and then the pscudo label for each input is corrected by the adjacent centroid [6]. The idea of using per-class centroids has been extended to incorporate more general clustering structures [18] 17]. However, the neighborhood structure- bbsed methods are computationally demanding due storage of memory banks in the feature space and nearest neighbors search. Such computational complexity persists in other approaches, which are based on the consistency of multiple predictions from different augmentations [20) and models trained with different loss functions [19). Compared to these solutions, AnCon can efficiently estimate correct labels with only limited extra memory overhead of storing past predictions.
Learning from noisy labels Treating pseudo labels as inherently noisy, techniques from the LFN literature have been integrated to self-training. For instance, the LFN literature has proposed robust loss functions that reduce impacts of random noisy labels [48] 33 and a recent large-scale experimental study shows the applicability of the generalized cross-entropy in the SFDA setting [T] The effectiveness of ELR on SFDA B bears similar idea because ELR was developed to regularize the neural networks' tendencies to memorize incorrect labels 21. Despite their effectiveness, by nature, these approaches do not consider important characteristics of the unbounded and instance- dependent noise rates inherent in self-training under distribution shifts, which results in significant suboptimality in both theory and practice. However, by considering the unique characteristics of self-training under distribution shift, AnCon relaxes the conditions required to achieve optimality as well as boosts the self telf-training performance in diverse scenarios.
6 Conclusion
This paper introduces AnCon, which effectively improves self-training performances under diverse distribution shift scenarios by promoting selective temporal consistency based on confident predic- tions. As result, AnCon effectively mitigates the detrimental effects of noisy pseudo labels without much computational overhead, unlike the previous methods. We show that AnCon not only advances our theoretical understanding of generalized notion of temporal consistency in self-training but also can be a practical asset as a simple and effective self-training method with attractive properties. In Appendix C.4] we present limitations and future directions, such as adaptive determination of X, combining local and temporal consistency, and extending the selective temporal consistency in the sequential decision making problems.
10

---

Acknowledgments and Disclosure of Funding
We would like thank Jihyeon Hyeong, Yuchen Lou, Jiezhong Wu, and anonymous reviewers for their valuable discussions and constructive suggestions during the preparation of this manuscript. We declare that there was no funding received for this work.
References
[1] Joy Buolamwini and Timnit Gebru. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on Fairness, Accountability and Transparency, 2018.
[2] Benjamin Recht. Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do Imagenet classifiers generalize to Imagenet? In International Conference on Machine Learning, 2019.
[3] Neal Jean, Marshall Burke, Michael Xie, W Matthew Davis, David B Lobell, and Stefano Ermon. Combining satellite imagery and machine learning to predict poverty. Science, 353 (6301):790-794, 2016.
[4] Sara Beery, Grant Van Horn, and Pietro Perona. Recognition in terra incognita. In European Conference on Computer Vision, 2018.
[5] Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. Measuring robustness to natural distribution shifts in image classification. In Advances in Neural Information Processing Systems, 2020.
[6] Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. In International Conference on Learning Representations, 2019.
[7] Hong Liu, Jianmin Wang, and Mingsheng Long. Cycle self- self-training for domain adaptation. In Advances in Neural Information Processing Systems, 2021.
[8] Li Yi, Gezheng Xu, Pengcheng Xu, Jiaqi Li, Ruizhi Pu, Charles Ling, Ian McLeod, and Boyu Wang. When source-free free domain adaptation meets learning with noisy labels. In International Conference Learning Representations, 2023.
[9] David Berthelot, Rebecca Roelofs, Kihyuk Sohn, Nicholas Carlini, and Alex Kurakin. Adamatch: unified approach to semi-supervised learning and domain adaptation. arXiv preprint MiXiv:2106.0475550 2021.
[10] Dong-Hyun Lee. Pseudo-label:
The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on Challenges Representation Learning. ICML., 2013.
[11] Eric Arazo, Diego Ortego, Paul Albert, Noel O'Connor, and Kevin McGuinness. Pseudo- labeling and confirmation bias in deep semi-supervised learning. In International Joint Confer- ence on Neural Networks, 2020.
[12] Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A Raf- fel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li. Fixmatch: Simplifying semi- supervised learning with consistency and confidence. In Advances in Neural Information Processing Systems, 2020.
[13] Bowen Zhang, Yidong Wang, Wenxin Hou, Hao Wu, Jindong Wang, Manabu Okumura, and Takahiro Shinozaki. Flexmatch: Boosting semi-supervised learning with curriculum pseudo labeling. In Advances in Neural Information Processing Systems, 2021.
[14] Yidong Wang, Hao Chen, Qiang Heng, Wenxin Hou, Yue Fan, Zhen Wu, Jindong Wang, Marios Savvides, Takahiro Shinozaki, Bhiksha Raj. et al. Freematch: Self-adaptive thresholding for semi-supervised learning. arXiv preprint arXiv:2205.07244 2022.
[15] Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan, and Jasper Snoek. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. In Advances in Neural Information Processing Systems, 2019.
11

---

[16] Jian Liang, Dapeng Hu, and Jiashi Feng, Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation. In International Conference on Machine Learning, 2020.
[17] Shiqi Yang. Joost van de Weijer, Luis Herranz, Shangling Jui, et al. Exploiting the intrinsic neighborhood structure for source-free domain adaptation. In Advances in Neural Information Processing Systems, 2021.
[18) Shiqi Yang, Yaxing Wang. Joost Van De Weijer, Luis Herranz, and Shangling Jui. Generalized freeee-free domain adaptation. In IEEE/CVF International Conference Computer Vision, 2021.
[19] Hao- Wei Yeh, Thomas Westfechtel, Jia-Bin Huang, and Tatsuya Harada. Boosting source-free domain adaptation via confidence-basee based subsets feature alignment. In International Conference on Pattern Recognition, 2022.
[20] Nazmul Karim, Niluthpol Chowdhury Mithun, Abhinav Rajvanshi, Han-pang Chiu, Supun Samarasekera, and Nazanin Rahnavard. C-sfda: A curriculum learning aided self-training framework for efficient source free domain adaptation. IEEE/CVF Conference on Computer Vision and Puttern Recognition, 2023.
[21] Sheng Liu, Jonathan Niles- Weed, Narges Razavian, and Carlos Fermanezz-Granda. Early- learning regularization prevents memorization of noisy labels. In Advances in Neural Informa- tion Processing Systems, 2020.
[22] Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning (still) requires rethinking generalization. Communications of the ACM, 64(3): 107-115, 2021.
[23] Devansh Arpit, Stanislaw Jastrzębski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, al. A closer look at memorization in deep networks. In International Conference on Machine Learning, 2017.
[24] Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jon Shlens, and Zbigniew Wojna. Rethink- ing the inception architecture for computer vision. In IEEE Conference on Computer Vision and Pattern Recognition, 2016.
[25] Gabriel Pereyra, George Tucker, Jan Chorowski, Lukasz Kaiser, and Geoffrey Hinton. Reg- ularizing neural networks by penalizing confident output distributions. arXiv preprint arXiv:1701.05548 2017.
[26] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv: 1xXiv:5503.025 02531, 2015.
[27] Yaodong Yu, Stephen Bates, Yi Ma, and Michael Jordan. Robust calibration with multi-domain temperature scaling. In Advances in Neural Information Processing Systems, 2022.
[28] Taejong Joo and Diego Klabjan. IW-GAE: Importance weighted group accuracy estimation for improved calibration and model selection in unsupervised domain adaptation. In International Conference on Machine Learning, 2024.
[29] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition, 2016.
[30] Mher Safaryan, Alexandra Peste, and Dan Alistarh. Knowledge distillation performs partial variance reduction. In Advances in Neural Information Processing Systems. 2023.
[31] Evgenia Rusak, Steffen Schneider, George Pachitariu, Luisa Eck, Peter Vincent Gehler, Oliver Bringmann, Wieland Brendel, and Matthias Bethge. If your data distribution shifts, use self- learning. Transactions on Machine Learning Research, 2022.
[32] Mingchen Li, Mahdi Soltanolkotabi, and Samet Oymak. Gradient descent with early stopping provably robust to label noise for overparameterized neural networks. International Conference Artificial Intelligence and Statistics, 2020.
12

---

[33] Aritra Ghosh, Himanshu Kumar, and P Shanti Sastry. Robust loss functions under label noise for deep neural networks. In AAAI Conference on Artificial Intelligence, 2017.
[34] Ori Press, Steffen Schneider, Matthias Kümmerer, and Matthias Bethge. Rdumb: simple approach that questions our progress in continual test-time adaptation. In Advances in Neural Information Processing Systems, 2023.
[35] Yuan Shi and Fei Sha. information-theoretie learning of discriminative clusters for unsuper- vised domain adaptation. In International Coference on International Conference on Machine Learning, 2012.
[36] Weijie Tu, Weijian Deng, Tom Gedeon, and Liang Zheng. Assessing model out-of-distribution generalization with softmax prediction probability baselines and correlation method, 2023. URL htt.ttt:://pperreieeeeeee// http:///www....eeeeoooorrdddddaa
[37] Dapeng Hu, Jian Liang, Jun Hao Liew, Chuhui Xue, Song Bai, and Xinchao Wang. Mixed samples as probes for unsupervised model selection in domain adaptation. In Advances in Neural Information Processing Systems, 2023.
[38] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International Conference on Machine Learning. 2015.
[39] Tim Salimans and Durk P Kingma. Weight normalization: A simple reparameterization to accelerate training of deep neural networks. In Advances in Neural Information Processing Systems, 2016
[40] Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell. Adapting visual category models to new domains. In European Conference on Computer Vision, 2010.
[41] Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan. Deep hashing network for unsupervised domain adaptation. In IEEE Conference on Computer Vision and Pattern Recognition, 2017.
[42] Xingehao Peng, Ben Usman, Neela Kaushik, Judy Hoffman, Dequan Wang, and Kate Saenko. Visda: The visual domain adaptation challenge. arXiv preprint 17X1v:1710.06924, 2017.
[43] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In JEEE Conference on Computer Vision and Pattern Recognition, 2009.
[44] Masashi Sugiyama, Matthias Krauledat, and Elaus--Rbber Müller. Covariate shift adaptation by importance weighted cross validation. Journal of Machine Learning Research, 8(5), 2007.
[45] Pietro Morerio, Jacopo Cavazza, and Vittorio Murino. Minimal-entroop cwwpeeetttooooooollllloltcttotoo correlation alignment for unsupervised deep domain adaptation. arXiv preprint arXiv: 2017.
[46] Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. Concrete problems in Al safety. arXiv preprint arXiv:1606.06653 2016.
[47] John D Lee and Katrina A See. Trust in automation: Designing for appropriate reliance. Human Factors, 46(1) (661))500 5050800 2004.
[48] Yang Liu and Hongyi Guo. Peer loss functions:
Learning from noisy labels without knowing noise rates. International Conference on Machine Learning, 2020.
[49] Boris Teodorovich Polyak. Gradient methods for minimizing functionals. Zhurnal Vychisli- tel' noi Matematiki Matematicheskoi Fiziki, 3(4):643 653, 1963.
[50] Zachary Charles and Dimitris Papailiopoulos. Stability and generalization of learning algorithms that converge to global optima. In International Conference on Machine Learning, 2018.
[51] Arthur Jacot, Franck Gabriel, and Clément Hongler. Neural tangent kernel: Convergence and generalization in neural networks. In Advances in Neural Information Processing Systems, 2018.
13

---

[52] Herman Chernoff. A measure of asymptotic efficiency for tests of hypothesis based on the sum of observations. The Annals of Mathematical Statistics, pages 493-507, 1952.
[53] Taejong Joo, Uijung Chung, and Min-Gwan Seo. Being Bayesian about categorical probability, In International Conference Machine Learning, 2020.
[54] Yarin Gal and Zoubin Ghahramani. Dropout Bayesian approximation: Representing model uncertainty in deep learning. In International Conference on Machine Learning, 2016.
[55] Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael Jordan. Conditional adversarial domain adaptation. Advances in Neural Information Processing Systems, 2018.
[56] Yuchen Zhang, Tianle Liu, Mingsheng Long, and Michael Jordan. Bridging theory and algorithm for domain adaptation. In International Conference on Machine Learning. 2019.
14

---

A
List of acronyms
TTA
Test time adaptation
SFDA
Source-free domain adaptation
i..d.
independent and identically distributed
ELR
Early learning regularization
LFN
Learning from noisy labels
KD
Knowledge distillation
EMA
Exponential moving average
GCE
Generalized cross-entropy
NRC
Neighborhood reciprocity clustering
PL
Dolyak-Lojasiewiez
B
Proof of claims
B.1 Assumptions
Let us recall that ((0) = Ex (Xy(H((((;)))) PYJx)] and (c(0)=
O), where
& contains number of random samples from px. We also denote 8" arg mingeR I(0). In this work, assume the following three regularity conditions on ((0) which is differentiable with respect to 0, which are mild but essential for most theoretical studies with convergence analyses.
Assumption B.1 (L-smoothness). I(-) is L-smooth for some constant L
0: that is,
((0') - ((0) + (VI(O), 6' 0) + 2 0 - 0 11
VO R and 8' E RP.
(10)
Assumption B.2 (C-expected smoothness [30]. ((.) is L-smooth in expectation with respect to C ~ D; that is,
E E (l) Vlz(0*) 17 (0((()))(())))))))))))))) ((()))), 4A e R..
(11)
Assumption B.3 (y-Poolya--la Solyykk-Ljasiewiczz (PL) condition
49 ) I(-) satisfies the μ-PL condition for
some constant μ 0; that is,
VI(0) 12> (T(( - 1(0))), 40 R.P
(12)
We remark that Assumptions B.I and B.2 can trivially hold under bounded parameter values that can guaranteed by optimizing neural networks with finite iterations under gradient or weight clipping. Assumption B.3 holds for infinite-width neural networks, i.e., the neural tangent kernel (NTK) regime [50]. Given that the gradient descent training dynamies of neural networks can be well approximated by NTK [51], the PL condition can be generally regarded as mild assumption.
B.2 Proof of Theorem 3.1
Proof. For x such that Qx:; Cp:m) > 0, let us consider subsequence of index such that x E Ar(ci): that is, (Ji,' -f)(z)aa.)) where jI min(? 2 j E Ay(c;)) and jo 0. Let SQ(eioum) ((::cm)) ((Y(x) = (x; ())). Then, we have the following inequality
1=
SP (S(msco...) S Q(2; Co:m)
arg max (a; 8orm: Wool # ((x))
2
Q(x; co:m) (2p(1; Coom) log(2p((; co:m)))
S <exp
(13)
2
where the first inequality holds because arg max, fe fo.m Wi:m) = Yxx) if So(mooom
Q(x; Cu:m)/2 and the second inequality holds due to Lemma B.4 given later in Appendix B.5 with p. (xY(x) = {x; Q(æ; Com)) and q 1/2.
15

---

B.3 Proof of Theorem 3.2
Proof. By Lemma B.5 any realization of 6n satisfies
802
[l((0m,))) - -1(0))((0000))))))))))) [0mm) (11 (1mm))(((0m=-(11)) +
o (0m)
1
Eg [1| Vie - >ge (00m Woom) 1P)) (14)
+ X² (000m Woom)
112 +
2
Then, from the definition ((); uu:m. Worm
g(Ou:m: Wo:m)
1|a
+ L Ee (0*) Agg (0a.,, Wo:m) [2]
we get
To show N N(Nn Oorm, Woo) ((). we
apply Lemma as
(fo.m, Wo:m))))
N (A O0m, Woom)
N(0)
2 [Bg ll (00mm Woom) 11 + 2z @00.m Woom) |1 (Eg
Vig(O+) (1?)
(15)
To show
8 let us assume Ee (V (e Woom))l 2 0. Then, we get the following inequal-
ity:
[e((Vie((') 95 (00:m, Woom)) })]
(16)
(Be (0*) 112)1/2 (E& DE (00:mm Worm) 112)1/2
Ee (0²) 11a +B ll 96 (00r,, Woom) 112 -Eg |I Vie(0*) DC (0am;; Woom) 1|
(17)
2(BB Vlr(O)) 112)2/2. , (Bg ll (00mm Worm) 112)2/2
Eg ll Vlr( - (Po:m, Woom) 1| 1. 2(E5 || (ig 1/2)1/2 (EEg || (00:m. Worm) 1/2)1/2
2 1 - min
(18)
C² DKL(f(L(Xf 0*) F(X; Buam. Worm)) })
2 1 min
1.
(19)
(Be || vie(0-) 1132//2 (Be Ôe (0o.m Woom) |/2)//2
where 18) holds due to a² + 12 2 2ab; 19 holds due to the assumption x SS C. || and the Pinsker's inequality applied
||25|| 111
Bg || Vig(0") de (ou.m. Woom)
1p C
1
Σ (1 J(X,; Ourns Woom)
X.CE
1 S C J X,E9
Σ [1 J(X,; 0pam, Worm) lli
< Dr DrI 00.m, Woom)). (20)
Thus, we get our desired result by combining the following inequality with T5
Eg B (VV(((())) 00 wo.m))))
N (D Pom Woom).
N(0)
[Eg (00.ms W Wo:m) ) 1|2 2 |l (00.m Worm) 112)
[ || vig(O*) [1
(21)
Eel(V [e((V((( 9e (fom, Wo.m )}])
-
(22)
(Ee || (l+( 112)1/2. (Eg 0e (0o.,, Worm) 112]1/2
+
2
10 Wo
||
Eg
100 (o m. Wa
16

---

CFExDKI(((X;0)) I1 f(X; 00:m, Woom) )) 1 (1 min 1, ("
-1
31
(Eg ll Vlg(0*) ||3/1/2 (EEg |I ge (e(00mm Wor)) 112)1/2
1 +
[1o(Bamm Waom 111
ing l1 (#a.m Wa 111
(23)
(f || f(x;00mm Waon) )
2
@(fomm Woom) ) 11?
< 22min
(B& Vle(0*) 11/22/2 (LEg 0e (00.m, Worm) 1|2)a/2
Ly 18 || ge (60:m. Worm) ||2
(24)
C=ExDR((((X;0) I1 f(X; 0orm; Woom)) )) $ 2min (I
(25)
(IDE || (0+ 112)2/2 (EEg |l de (00m Wor) 112)2/2
20
Ex Op.ms Worm)] Ex ( (X; 0m)) 1|
(26)
Ly
Eg de (60:m: Wo.m) 1|
where (24 holds due to 1 (1-w)?
- 2u +
30
14v
B.4 Proof of Corollary 3.2.1
Proof. For any realization of 9m for m E 10, - -1,, applying Theorem 3.2 with X = At
f-
(cf. Lemma B.6) gives
[[((0m+1) [01mm] = Bl((Om,TT) - 1mm
$ (1 ny)² (1(0m) - 17) ge (0m)) + 2 N Borm. Wo.).. (27)
8C2
p
Thus, by recursively applying 27 to 0o, we get our desired result as
B((()))-r] s
-100) -1-
(1 p7) ((F--1)))
8C2 (SC² M
g (0,)
+
2 NN(2): fo:. wox))
P
1=0
(28)
B.5 Supporting lemmas
Lemma B.4. Let A; Bern(pi) with p: [0, and S. = EiEjo Ai, where As and A₃ are independent for i # j. Then for any < := _ Eielol Ps. we have the tail probability bound
p(S. S q 0) S exp (o(1-D-9los))
1-999 g0g 9 ) )
(29)
Proof. For any 1, the moment generating function of Sm is given by
(1) = Egg (exp((S.)] = II
EA, [exp((A.)] = exp
Σ KEfO
log(LE,, (exp(tA,)))
=E|o]
pa (exp(1) -1) (01(())-))) = =exp(op(exp()) -1))
Σ FE
(ex(p() - + +1)
Σ iE|o
-exp
s exp
(30)
where the first equality comes from the independence and the inequality is from log x -
Then, for t 0 and q p. the Chermoff bound (52 gives
p(S. < g o) < ([1)exp(-1) q o) = exp(op(exp()) 1) - L L.).0).
(31)
Finally, setting f = log (g/p) p) < 0 gives our desired result.
17

---

Lemma B.5 (Modification from (30). Let us assume that 1(0) satisfies the L-smoothness, the L-expected smoothness, and the p-PL condition. Also, we assume a linear model fa( 8) (exp((" 2) x) LiE(K] (exp((" (f (0)) with 0 Ba Then, for the stochastic gradient descent
0 m 79g (m.)) with 9
(m.t)
m,,)) V 00:m Wom)), 0m. 6m.
and a constant learning rate "y I aC-L' any realization of 8m satisfies
p
802
B[((Om.t) - [(m0))[mm] 5 (1(0m) - (((")) +
9 (0m) (0m)
1
+ (2x² 11 @(0um Wo:m) 112 +LyBS [1 vig(0") >re (00(00m Wo.m) IP]). (32)
Proof. For any realization of m,,e. we have the following bound:
Eg ((0")))m))))mm (((0m,)))((0))) (V((((mmmm)) (mit) In²
Eg
[|
9 ||]
(m,,))
(33)
= =((0m,))- - - (V(0)) V=(0m,;)) bb(m.)) + Ln²
EE [1 Vle (0m(0m.))) >xe (0o.m; Worm) ||
2
(34)
(1(0m,))) 1 1(8)) 7 (l(em..) 1| +V((((mm)))) b(m,)))
In²
Bg [1| (ig(Om.s) Age(Oomm Wo:m) [13
+ 2
(35)
$ (1 - (10((((mm..)) - 1(8))) ))- - YP (2(0m),)) - [(0")) + ((m,e) |
2
I^2 Bg [1 Vig(Om. Age(0o:m Wa.m) 113
+ 2
(36)
< (1- 71)(((0m,.)) - ((0)))
7H
(1(0m.t) - (((0)) + ((m))) 1|2
2
+ + (ig()m,)) (ir( [1-]
+ LyPBB [1 vig(0t) -A0g(0omm Woo)) [13]
(37)
< <(1 (-7))(((0mm.)) 1(0')) 7/
1
4LC
(10m,,)-1(00 + ((m,0) 1|2
2
1
+ +L7 (i' ^ge (0o.m: Worm) [13]
(38)
< (1 7))((((mm.)) 1(8")) )) + > ((m))) 1|2 L Vig(Ot) (f0:m Wum)) [12] (39)
$ (1 74)((((mm.)) - 1(8')) )) + 2 || Vi(0 vi(Om.e) 1|2
+ 27/22 ((0... Wo.)) 1|2 + || vig(0") >ge (00.m. Worm) 11?
(40)
< (1 (1(0m)))))))) - 1(0*)) ) + 8y of (0m)
+ 2 (00.m, Wo.)) ) 21a + Ee
|| vie(0") Xge (0o.m, Worm) 112
(41)
(m.)) (l. (mm.1) (0m where 33) holds due to L-smoothness; 34 holds due to g
Xge ( (fo:,, Wo.m). ): (36) holds due to (a, b)
2| + & 11: 37 holds due to the inequality
|| I + 1
+2 y 1| 38 holds due to the expected smoothness assumption; (39) holds
due to the condition 7 < zc
holds due to the law of total expectation with the random event
1 (X) ((;; and then applying the bounded support assumption.
18

---

By recursively applying the above bound to any realization of mm we get our desired result as follows:
8C2
m()(0m,)) - -1mm))|mm] = (() - +
(0
P
(2>>² 9(00mm Worm) 112 + [1 Vig(0") 10c(00.m Woom) [Pe)
+ 7/0
8C2
-1 -(1-7))(((0m)) -1(0") + 1
(0mm)
+ 2x² || g (000m Wu:m) LLyB || Vig(0")
Wor)) [P])
(42)
Lemma B.6 (Lemma in 301). Let ((); Oom Worm)
X
9 g(00:m: Worm)
11
+ E [II (ig
Agg (60m. Woom) [P]
Ee ((V (0") de (Bomm Wo D1 Then, for Nt. Eglldg (10 WD. m) ]1l 2 11 Wo
have
Eel (e((Vig((*)) (lum. wom)))))
N N(Mis Opmm, Woom) N(0)
Egg
ll De (fo:m Woom) 112 +
2 b5 1000m Wool 11 (Es Vig(o-) 112)
(43)
Additional results and discussions
C.1 Empirical evidence for Theorem 3.1
1.
10
Ganensteed be reporal anserbble
Genertieed to oponaal ensemblee
Tempordl ersambiee
Tempond ensamble
0.6
Pherudo label
Parudo label
0.5
c.4-
0.4
0.4
Moan (16fw12))
61
0.1
Prousto
0.2
0.2
0.2
Quantie
Quantie
0.3
200
5.0
75
35.0
25.0
The Sumber OF contident Serrccess
1he number contident serrecies
The number contident scmoles
(b)
Figure 4: (a) The accuracy of the generalized temporal ensemble along with the number of confident samples under different degrees distribution shifts. Here, the temporal ensemble constructed by averaging all predictions over iterations. (b) On-average accuracies per the number of confident samples over iterations under different thresholding rules.
C.2 Marginal distribution of pseudo labels over epochs
C.3 Algorithmic design choices
Here, we examine effectiveness of our design choices-the -th relative thresholding for weighting scheme (z) and hard prediction for p(y|æ, x 0,). To this end, we replace our design choices with several alternatives, keeping other features of AnCon the same. Then, we analyze the changes in the performance compared to the default setting AnCon in four domain pairs of OfficeHome.
Sophisticated weighting scheme for wx(x) We compare our relative thresholding with sophisti- cated weighting schemes called Entropy (wx(x) x exp [-Hr 0,)]}) and Maxprob (w,(x) x maxxe[K] [s(r;; 0.)). These weighting schemes aim to more precisely weight the predictions based on the uncertainty of the prediction. However, Figure 6al the simple relative thresholding works better than these sophisticated weighting schemes. We conjecture that the poor calibration performance of
19

---

1000
5000
0.077
800
0.065
0.060
19ta) Vanator
Count
3000
0.055
0.050
2000
200
0.035
20 40
70
100
Epochs
(a)
(b)
Figure 5: (a) Counting the number pseudo labels for each class with 5,000 training samples in ImageNet-C over 100 training epochs, which shows that the marginal distribution of pseudo labels barely changes during training. (b) Changes in the total variation distance of the marginal distributions of the pseudo labels for each consecutive epochs.
Halanow womormmaaaa Change (5)
Roteavve per "ereooommrcoo Chtnng (7)
Ant.on
Entropy
Naxprob
Drnchlet
MC MK (p-U2)
Ant.on (Herd) 1=05
1w0.15
1-1.0
1w1.25
1m1.
Mechods
Methidss
(a)
Figure 6: Ablation study of (a) weighting and (b) prediction schemes.
the neural network in self-training under distribution shifts prevents the estimated uncertainty of a prediction from accurately reflecting the goodness of the prediction. Further, the ineffectiveness of these sophisticated weighting schemes persists even when we improve the calibration performance with last-layer Dirichlet (53) and Monte-Carlo Dropout (54) (see Appendix D.3 for detail). Therefore, our relative thresholding is simple yet effective in self-training under distribution shift.
Soft prediction for p(ylæ, 0,) We compare hard prediction soft prediction with softmax tempera- ture parameter T E {0.3, 0.75, 1.0, 1.25, 1.5) (cf. Appendix D.3 for the definition). By design, soft prediction is more informative, giving values in non-leading entries. However, in Figure 6b] soft prediction turns out to underperform hard prediction for various values of We conjecture that the continuously increasing confidence in the later stage of self-training would make soft prediction ignore early-stage predictions which may be valuable to memorize. Thus, we believe that hard prediction would be more appropriate for AnCon.
C.4 Limitations and future directions
Albeit AnCon proving its effectiveness with fixed value of Theorem 3.] suggests that its optimal value depends on the temporal ensemble performance. Thus, future works could aim for adaptively determining X. In addition, we observe that all self-training based methods fall short without sophisticated model designs tailored for distribution shifts [16 31 (cf. Appendix E). In this regard, would valuable to rigorously understand properties of function classes that enable successful self-training under distribution shifts. Also, while AnCon significantly relaxes the conditions required to achieve optimality compared to L.NN techniques, the on-average correct prediction condition still can be violated in challenging self-training scenarios. Given the compatibility of AnCon and NRC
20

---

(cf. Table TV, efficiently combining local and temporal consistencies could be step toward further relaxing the optimality condition. Finally, we remark that the idea of selective temporal consistency in sequential decision-making scenarios could be a notable future direction of research, Extending AnCon to sequential decision-making scenarios presents significant challenges as they involve the fundamentally different mechanisms. In sequential decision-making leveraging observed rewards and balancing exploitation and exploration are core aspects (e.g., constructing the upper confidence bound of the reward in the bandit problem), unlike in self-training. Therefore, the main challenges for applying AnCon to this setting would be defining rewards and incorporating exploration strategies into pseudo label generation.
D Additional details
D.1 SFDA training configurations
Our training configuration is based on [6]. Specifically,, we train ResNet-50 [2 a source classifier, which is used as an initial point for SFDA, for 50 epochs, with minibatch stochastic gradient descent with Nesterov momentum by using the initial learning rate of 0.01, the momentum parameter of 0.9, and the minibatch size of 64 in the Office Home dataset. The learning rate is decayed by (1 + 10 current iteration number maximum number of iterations) (..0)-007.. 00.75 and for the pre-trained layers we use 10 times smaller learning rate. For Office-31, 1, we change the number of epochs by 100 and other configurations are the same as those used in the setting in Office Home. Similarly, we only change the number of epochs to 10, the learning rate to 0.001, and the architecture to ResNet- 101 for VisDa. We split dataset into 90% of the training set and 10% of the validation set, and the best model is chosen based on the lowest error rate on the validation set. For adapting the pre-trained model in the target domain, we use the exactly same configurations, except reducing the number of epochs to 30 for Office-31 and OfficeHome and to 15 for VisDa.
D.2 ImageNet-C training configuration
For applying self-training, follow the training configuration used in 31: specifically, given ResNet-50 pretrained on ImageNet, we perform self-training without threshold by running the stochastic gradient descent optimizer with the learning rate of 0.001 and the minibatch size of 100 for 20 epochs. We remark that other techniques such as momentum and learning rate scheduling are not used.
D.3 Further experimental details in analyses
Definition of ECE
ECE(0,,) = Σ
5. Acc(gg)) - C
(44)
D
9=1
where G = E S <((=;0) the average confidence in Gy
< & ]. A is the average accuracy in Sg. and C
Configuration of the sophisticated uncertainty representation methods We used the regular- ization coefficient of 0.001 for the belief matching loss (53) and 20 forward passes to compute the posterior predictive distribution for MC-dropout with dropout probabilities 0.2 and 0.5 [54].
Definitions of model selection criteria
InfoMax (higher is better)
TM(0) H[X - -x Ex(HH((X;
(45)
where Hp)) ELEjKY ps pt for p E AK-1, Note that we abuse the notation H(-) with the definition of the cross-entropy; that is, H(g.)) = -- -E Ps log q; and H(p) =- -EE(RR] P. log Pt.
21

---

Table Accuracy across different domain pairs for each UDA method OfficeHome.
Method
Ar2C1
Rw2C1
Pr2Ar
C12Pr
Avg.
CDAN
77.48
71.77
54.23
74.37
83.60
61.01
61.39
79.21
53.95
62.26
71.08
69.88
68.35
Self-traiming
76.82
71.68
51.57
73.51
83.10
58.63
60.36
78.68
51.91
61.97
69.24
68.37
67.15
+ELR
77.44
71.91
47.01
73.96
83.53
59.98
61.68
78.88
51.39
62.88
70.81
70.33
67.48
+AnCon
78.22
71.53
53.59
74.08
84.03
60.23
61.23
78.79
53.81
62.92
72.04
70.44
68.41
MDD
78.54
75.74
56.43
73.71
84.30
60.73
63,70
80,26
52.99
63.62
73,10
72.38
69.24
Self-training
76.93
71.37
52.62
73.47
83.24
59.38
60.57
78.40
52.49
62.01
69.43
68.17
67.34
+ELR
77.30
71.93
47.12
74,00
83.53
62.05
78.75
50.81
62.30
70.53
70.29
67.37
AAnCon
77.67
71.84
53.24
74.00
83,96
60.18
60.86
79.07
53.38
62.75
71.63
70,56
68.26
Corr-C (lower is better)
Corr-C(0)
Cy Σ ((xnn:))f
(46)
where C E R and - || is the Frobenius norm.
Ent (lower is better)
Ent(O) = H (R
(47)
Softmax with temperature parameter
p(y = j|z 0, (xx
(48)
where (x; 0) are the logits of the neural network such that fi(:; 0) E [K] and ((x) is the softmax function.
expls
for
EnE(n exp(Sx (x:0))
D.4
Computational resources for experiments
In this work, use multiple servers which consist of multiple GPUs including TITAN XP (12GB),
RTX 8000 (50GB), and A100 (40GB). AnCon takes 1.16 seconds on average for each iteration.
E Result in UDA
While SFDA does not assume any information, unlabeled target domain samples domain shift information can be available during the source domain training process in some practical scenarios, e.g., when labeling target domain samples is costly. In this case, UDA methods are used to minimize the potential impact of distribution shifts by matching input marginal probability distributions of source and target domains. Given the prevalence of UDA scenarios in practice, we aim to show whether AnCon and other self-training methods can be effective for the UDA methods (conditional domain adversarial network (CDAN) (55) and maximum mean discrepancy (MDD) (56)) in the adaptation stage. However, shown in Table 2 all methods decrease the performance of both CDAN and MDD for most cases, albeit the reduction rates in AnCon are smaller than self-training and ELR. We conjecture that this is because the base methods (CDAN and MDD) do not contain the sophisticated tricks, such as adding weight normalization to the final linear layer, which are used in the SFDA literature [16].
22

---

Additional tables
Table Benchmark results in Office-31. The numbers indicate the mean test accuracy across three repetitions with boldface for the best score.
Method
A-D
A-W
D-A
D-W
W-A
W-D
Avg
Self-Training
79.32
80.88
62.07
97.67
62.99
99.90
80.47
+ELR
79.02
81.19
63.70
97.61
63.63
99.90
80.84
+AnCon
82.23
79.94
63.99
97.80
64.29
100.00
81.37
GCE
86.85
86.16
64.63
98.62
62.83
99.90
83.16
+ELR
86.85
86.54
64.80
98.62
62.48
99.90
83.20
+AnCon
86.75
87.11
65.78
98.49
62.99
99.90
83.50
NRC
92.67
88.11
72.83
98.62
72.40
99.90
87.42
+ELR
92.77
87.92
72.77
98.68
72.68
99.90
87.46
+AnCon
94.28
91.45
74.49
98.62
75.56
100.00
89.07
Table 4: Benchmark results in OfficeHome. The numbers indicate the mean test accuracy across three repetitions with boldface for the best
Method
Ar2Rw
Ar2Pr
Ar2C1
Rw2Ar
Rw2Pr
Rw2C1
Pr2Ar
Pr2Rw
Pr2C1
C12Ar
C12Rw
C12Pr
Average
Self-training
75.19
69.20
43.07
66.13
78.10
46.64
35.25
73.84
42.98
55.46
66.51
66.28
61.55
+ELR
75.97
70.47
45.80
67.00
78.64
50.01
55.38
75.42
44.56
56.61
69.15
67.38
63.03
AnCCon
76.13
70.56
48.06
67.57
79.27
51.89
55.62
76.22
58.34
70.23
68.21
GCE
74.65
69.69
43.46
68.87
79.25
49.18
60.01
75.14
43.78
57.44
68.85
65.77
63.00
EER
74.89
71.62
43.45
69.63
79.98
49.68
61.27
76.57
44.44
58.69
68.67
66,29
63.77
+AnCon
76.47
71.80
44.83
70.21
80.15
51.74
61.93
76.91
46.69
59.21
70.09
67.74
64.81
73.86
75.04
47.90
61.15
76.59
51.75
57.56
71.88
46.12
59.74
70.44
70.47
63.92
+EIR
76.82
76.01
52.19
66.01
80.40
55.69
60.82
74.87
49.62
63.21
73.56
72.65
66.89
+AnCon
79.64
77.11
53.56
69.47
80.04
57,30
59.99
77.37
50.42
63.21
75.95
74.97
67.96
Table 5: ECE benchmark results in OfficeHome. The numbers indicate the mean test ECE across three repetitions with boldface for the best score (lower is better).
Method
Ar2Rw
Ar2Pr
Ar2C1
Rw2Ar
Rw2Pr
Rw2C1
Pr2Ar
Pr2Rw
Pr2C1
C12Ar
C12Rw
C12Pr
Avg
Self-urining
13.52
25.10
42.78
14.97
15.84
34.16
18.12
13.98
35.43
24.67
18.10
21.97
23.22
F.RR
12.61
23.12
44.89
13.92
15.51
30.12
19.06
14.30
36.86
21.25
17.00
22.69
22.61
AnCon
22.48
13.40
23.05
11.54
35.10
19.50
15.19
20.07
19.68
GCF
10.51
20.90
45.25
12.29
13,90
35.37
14.02
10.99
39.68
19.10
15.91
23.57
21.79
HEL
13.19
20.74
46.50
13.84
15.83
36.35
15.08
12.66
40.32
21.09
20.82
26.27
23.56
+AnCon
Table 6: Benchmark results in VisDa. The numbers indicate the mean test accuracy across three repetitions with boldface for the best score.
Method
Acroplane
Bicycle
Bus
Car
Horse
Knife
Motorcycle
Plant
Skateboard
Train
Truck
Avg
Source-ond only
52.74
10.50
36.16
48.15
48.24
5.83
$2.80
15.03
52.43
16.57
92.75
7.41
38.71
Self training
89.69
53.15
87.08
57.71
88.
99.23
$8.30
77.15
73.38
16.13
79.86
0.00
67.77
+ELR
92.21
53.53
85.59
60.22
89.85
98.70
90.80
77.65
80.68
39.11
84,66
0.43
71.89
+AuCon
91.25
52.32
81.68
58.60
88.72
85.93
90.60
77.78
50.64
84.16
16.35
71.11
GCE
93.88
0.00
89.62
72.55
91.69
99.08
90.11
79.80
84.68
0.04
81.37
0.00
65.20
+ELR
93.50
18.82
84.50
68.19
92.05
98.22
89.80
82.07
84.04
6.44
$4.89
0.00
66.90
+AnCon
95.15
24.00
86.20
73,26
93.03
98.36
91.60
82.72
88.46
0.83
87.04
0.00
68.40
NRC
37
89.50
83.45
65.
95.16
96. 05
80.73
89.67
91.10
86.76
30.68
74.30
+ELR
75.40
83.97
78.78
67.62
94,14
97.30
88.87
82.40
92.
88.29
87.39
57
82.80
+AnCon
95.83
87.05
79.66
61.67
94.39
95.57
86.06
81.18
90.83
89.30
85.84
57.41
83,70
23

---

Table ImageNet-C experiments with corruption intensity 1. The numbers indicate the mean test accuracy across three repetitions with boldface for the best score. We only present the first five letter for each corruption type. The full names are available [6]
Mothod
Shot
Tmpul
Gans
Glass
Motic
7oum
Snow
Frost
Brigh
Contr
Flast
Pixel
Jpes
Speck
Satur
Spsit
Avg
Self-training
69.04
68.78
71.46
70.83
74.24
ELR
69.05
54
71.19
70.31
71.20
74.21
70.54
Table 8: ImageNet-C experiments with corruption intensity 2. The numbers indicate the mean test
accuracy across three repetitions with boldface for the best score.
Method
Sbot
Impul
Gaus
Glass
Motic
Zoom
Snow
Frost
Fog
Brigh
Contr
Elast
Piael
Jpes
Speck
Satur
Sputt
Arg
Self-tainin
63.82
60
64.85
62.31
68.08
63.33
59.00
57.62
60.84
74,09
71.51
57.85
72.16
69,05
67.97
72.79
70.19
66.15
ELR
64.06
60.57
64.99
63.27
68.27
63.42
59.26
58.12
70.00
74,02
71.56
58.34
72.24
69.13
68.21
72.71
70.32
66.38
AnCon
60.38
64.90
63.53
68.31
63.48
71.71
58.42
72.84
Table ImageNet-C experiments with corruption intensity 3. The numbers indicate the mean test accuracy across three repetitions with boldface for the best
Method
Shot
Insul
Gauss.
Glass
Motic
Zoom
Snow
Frost
Fog
Brigh
Cont
Elast
Pixel
1peg
Spaak
Sotur
Spart
Avg
Self trainin
56.90
55.37
56.76
26.37
57.11
58.38
59.22
3.92
67.71
72.93
66.88
70.65
69.17
67.21
60.11
74.26
66.13
58.17
ELK
$7.33
55.88
$7.21
45.78
63.00
00.35
59.61
43
67.86
72,92
69.20
70.65
69.35
67.25
60.33
74.22
66,33
62.39
Anton
56.95
55.67
57.01
48.85
62.64
60.64
39.33
49.11
66.41
73,05
69.37
70.10
67.94
67.46
60.13
74.39
64.50
62,60
Table 10: ImageNet-C experiments with corruption intensity 4. The numbers indicate the mean test accuracy across three repetitions with boldface for the best score.
Metholl
Impul
Gauss
Glass
Motio
Zoum
Snow
Fog
Brigh
Contr
Elast
Ptxel
Jpeg
Speck
Satur
Spuil
Avg
Self-training
14,05
41.54
26.46
0.61
54 14
39.06
52.56
2.83
66.73
71.21
41.71
67.08
64.52
62.36
53.28
20.06
61.77
46.41
ELR
36.10
43.3
45.07
32.12
54.45
56.46
52,98
41 11.60
65.90
71.23
58.27
67.91
64.78
62.47
54.45
70.26
62.41
55.28
AaCoa
40.36
43.71
45.03
40.96
$4.16
56.81
52.46
46.66
64.20
71.43
99.42
66.67
62.37
02.55
53.65
69.34
62.34
$6.01
Table 11: ImageNet-C experiments with corruption intensity The numbers indicate the mean test accuracy across three repetitions with boldface for the best score.
Impul
Gars
Fog
Brigh
Jpeg
Speck:
Spott
Avg
Self-training
0.26
1.72
1.04
16.1
60.23
ELR
8.00
14.12
16.00
4.32
39.59
50.28
51.14
33.90
60.54
69.22
0.37
56.52
61.71
55.17
45.75
64.98
54.47
40.35
Anfon
22.56
26.56
25.85
19.21
45.91
52.00
$1.30
39.02
58.06
68.24
9.24
57.00
61.48
54.96
44.76
62.68
$4.29
44.35
24

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? Answer: [Yes]
Justification: In Section 1. we clearly state this work's scope and contributions, which are
shown in Theorem
3.1 Theorem 3.2 and extensive experiments in Section 4
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors? Answer: [Yes]
Justification: We include "Limitations and future directions" in Section o and discuss implications of the assumptions for theoretical results.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: We have added assumptions statements Theorems 3.1 and 3.2 with proofs in Appendix.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Answer: [Yes]
Justification: We include training configurations of all experiments in Appendix D
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We release our code at
6. Experimental Setting/Detail!
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: include training configurations of all experiments in Appendix D
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We have included error bars with box plots for experiments that have high variance across different random seeds.
8. Experiments Compute Resources
Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification: We provide computational resources used in the research in the Appendix.
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the http://ubbbbbieeeeccccccccceeeee
NeurlPS Code of Ethics
Answer: [Yes]
Justification: We thoroughly read the NeurlPS Code Ethics and have confirmed that our research does not violate any of them.
Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
25

---

Justification: This work considers general principle of promoting temporal consistency for improving self- self-training under distribution shifts. Therefore, it would not have direct societal impacts, which are contributed particularly by our work. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA]
Question:
Justification: Our experiments concern only standard models with publicly available bench- marks.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We cite papers related to code, data, and models used in our experiments.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper does not include the new assets.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This paper does not include crowdsourcing or research with human subjects.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human
Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: This paper does not include crowdsourcing or research with human subjects.
26


Reviews:
[{"review_id": "Xn2RONflZ8", "reviewer": "Reviewer_fUqB", "summary": "the authors propose a novel approach to enhance self-training for test-time adaptation (TTA) or source-free domain adaptation (SFDA) in neural networks facing distribution shifts. The core idea revolves around a method called Anchored Confidence (AnCon), which uses temporal ensembles and label smoothing to improve the accuracy and robustness of pseudo labels generated during self-training. This method aims to address the challenge of filtering incorrect pseudo labels, a common issue under distribution shifts, without incurring significant computational overhead. Theoretical guarantees and extensive experiments validate the efficacy of AnCon in diverse distribution shift scenarios.", "questions": "* The paper claims that the generalized temporal ensemble with prediction confidences is asymptotically correct. Can you provide a detailed proof or further elaboration on the conditions under which this asymptotic correctness holds? Specifically, what assumptions about the data distribution or model convergence are necessary to guarantee this property?\n\n* The theoretical analysis suggests that label smoothing can reduce the optimality gap in self-training. Could you explain the underlying mechanism of how label smoothing achieves this reduction? Additionally, what are the theoretical bounds on the optimality gap with and without label smoothing, and how do these bounds depend on the hyperparameters of the smoothing technique?\n\n* How does AnCon perform on highly imbalanced datasets or with extreme distribution shifts?\n* What are the specific hyperparameter settings used in the experiments, and how sensitive is the method to these settings?\n* Can the proposed method be integrated with other advanced TTA or SFDA techniques, and what would be the potential benefits or drawbacks?\n* How does the method handle concept shifts, where p(Y|X) changes between the training and test distributions?\n* are there any specific types of neural network architectures or tasks (e.g., vision, NLP) where AnCon performs exceptionally well or poorly?", "limitations": "Please see the questions and the weaknesses.  \nMy current decision on this paper is borderline (reject/accept). I look forward to the authors' rebuttal to address the following questions and concerns, which will help in making a final decision.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* the paper provides rigorous theoretical analyses to support the proposed method. It shows that the generalized temporal ensemble with prediction confidences is asymptotically correct and that label smoothing can reduce the optimality gap.\n* The method does not require additional forward passes or neighborhood searches, making it computationally efficient compared to existing techniques.\n* experimental results demonstrate improvements in self-training performance across distribution shift scenarios.", "weaknesses": "* The novelty of the paper is limited! (*The integration of temporal consistency and ensembles for self-training under distribution shifts)\n* Although the method shows good empirical results on small datasets, there may be scenarios or datasets where the performance gains are less pronounced, which are not extensively discussed.\n* The success of the method is somewhat dependent on the quality of the initial model parameter $\\theta_0$. If the initial model is significantly biased or underperforming, the improvements might be limited.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank Reviewer fUqB for the valuable suggestions and comments. We are glad that the reviewer acknowledges our rigorous theoretical analyses, efficient algorithm development, and effective performances of AnCon under different distribution shifts scenarios. We believe that addressing the valuable comments further improves our paper, which we hope can effectively address any concerns of Reviewer fUqB. \n* [G#] is the reference for the global responses. \n(Unfortunately, due to the number of comments from Reviewer fUqB, our rebuttal addresses parts of responses. We will post the remaining responses in the official comment and apologize for any inconvenience.)\n\n\n**W1:** Thank you for highlighting the novelty aspect of our paper. As Reviewer fUqB noted, promoting temporal consistency is a well-established framework in self-training algorithms. However, our contribution lies in demonstrating that \"uncertainty-awareness\" enhances the effectiveness of temporal consistency under distribution shifts. Specifically, Theorem 3.1 shows that uncertainty awareness improves the quality of temporal ensembles proportional to the number of temporally confident predictions. Additionally, Theorem 3.2 and Corollary 3.2.1 illustrate that \"AnCon\" reduces the optimality gap of self-training, which can be further improved by a high-quality, uncertainty-aware teacher. Crucially, such high-quality teachers are driven by our simple thresholding rule (Eq. (3)) with theoretical guarantees (Theorem 3.1). The significant theoretical contribution has not been studied and would not be possible by just incorporating the temporal consistency through ensemble as shown in Figure R6. In order to help readers to conceptualize the core theoretical contributions of the paper, we have the following discussion to Line 199.\n\n‘’’\nFurthermore, we highlight the significance of AnCon’s uncertainty awareness for improving effectiveness of temporal consistency in self-training. Specifically, from the result of Theorem 3.1, $g^{KL}(w_{0:m})$, thereby the optimality gap of the self-training, can be further reduced by collecting more confident samples, while guaranteeing the condition $\\bar{p}(c_{0:m}) \\geq 0.5$. Therefore, it is important to collect high-quality predictions for matching the condition, which would be hard to satisfy under other temporal ensemble mechanisms (cf. Figure R3). \n‘’’ \n\n**W2:** Kindly see [G3 in the global reference] that explains the unique and significant performance gains from AnCon in challenging distribution shifts scenarios. \n\n**W3:** Thank you for pointing out concerns regarding the dependency on the initial model parameter. As Reviewer fUqB conjectured, we have rigorously shown the impact of initial model performance on the sub-optimality of AnCon and the vanilla self-training method (cf. $l(\\theta_0) - l^*$ and $Err(\\hat{Y}; \\theta_0)$ in Corollary 3.2.1). Crucially, in Corollary 3.2.1, both AnCon and the self-training method can improve by reducing the initial optimality gap  $l(\\theta_0) - l^*$ as the number of inner and outer iterations $T \\cdot (m+1)$ increases, which is why we need “adaptation” if the performance deteriorates under severe distribution shifts. Finally, we kindly refer to [G3] for the discussion of empirical results supporting this argument. \n\n**Q1:** We gently remind Reviewer fUqB that the asymptotic convergence on inputs on which $ f $ produces infinitely many relatively confident predictions is rigorously proven in Theorem 3.1 with explicitly stated assumptions. We also kindly refer to [G1] for the discussion about validity of assumptions and [G2] for the empirical evidence. \n\n**Q2:** This is actually rigorously proven in Theorem 3.2, a core technical contribution of our work, as discussed in Lines 187-199. Specifically, “the underlying mechanism of how label smoothing achieves this reduction” is through connecting AnCon with the self-distillation. Crucially, this connection enables the adoption of the partial variance reduction theory, which can show that the generalized temporal ensemble reduces the neighborhood size of SGD. \nOur convergence analysis provides the optimality gap in Eq. (7), with the maximum gap occurring in self-training without label smoothing (cf. Line 187-188). This demonstrates that AnCon is at least better than vanilla self-training (cf. Line 189). Additionally, AnCon further improves the optimality gap by reducing $g^{KL}(w_{0:m})$, which explains the effectiveness of selective temporal consistency. \n\n**Q3:** The impacts of severe distribution shifts typically result in a poorly performing initial model, which can significantly reduce the effectiveness of self-training methods like AnCon. However, as discussed in our response to W3, AnCon successfully improves poorly performing initial model parameters under severe distribution shifts, unlike other self-training methods. Further, following Reviewer fUqB’s suggestion, we performed additional experiments in the highly class imbalance scenario. This setting is significant as effective methods like oversampling are not applicable in self-training, yet they largely impact self-training performance. In the new experiments, we compare AnCon with ELR and vanilla self-training under a heavy-tailed class imbalance with an imbalance ratio of up to 65 times. As shown in Figure R2, AnCon's effectiveness is well-preserved under severe class imbalance scenarios. In summary, we believe AnCon would show promising performances in multiple challenging self-training scenarios. \n\n**Q4:** Kindly see [G4] for the detailed explanation of our rigorous hyperparameter selection procedure and its significant practical implications. We also kindly refer to “Section 4.3.2. Robustness to the choice of hyperparameters” which shows the stable performances of AnCon even under arbitrary choices of hyperparameters.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HTVk9Mmmwc", "reviewer": "Reviewer_qtUp", "summary": "The paper aims to improve self-training, a common technique used for learning on unlabeled data (with iteratively generated pseudo labels), when it is applied to scenarios with distribution shifts. The proposed approach, Anchored Confidence (AnCon), essentially applies label smoothing on the pseudo labels with the ensemble prediction of the models obtained from previous iterations (temporal ensemble) to achieve better temporal consistency and improve the quality of the pseudo labels. Theoretical analysis is provided on the benefits of using temporal ensemble and the connection to ensemble knowledge distillation in the linear case. Empirically, when AnCon is shown both **effective** and **stable** when combined with a few orthogonal baselines such as Self-training and GCE under settings such as unsupervised domain adaptation (Sec. 4.1), robustness against label corruption (Sec. 4.2), calibration, etc. (Sec. 4.3).", "questions": "1. The pseudo label $\\tilde{Y}$ is actually also part of the smoothing vector from $\\bar{f}$. I wonder how necessary it is to separate $\\tilde{Y}$ out as its purpose is to increase the weight of the most recent model, or whether there is a more unified notion to describe the proposed approach with just $\\bar{f}$. \n2. In Theorem 3.1, the random events of model predictions from different iterations $i, j$ are assumed to be independent. How realistic is the assumption given that the model is trained based on the labels given by the previous iterations?\n3. Throughout the paper, $m$ is already defined as the number of models, but the proof of Theorem 3.1 assigns $m = Q(x;c_{0;m})/2$, which is the number of sufficiently confident predictions for $x$ made by the $m$ models. I don’t think it’s appropriate to assign $m$ something else here as it is already defined. If you fix the notation of $m$ (of $q \\cdot m$) in Eq. (29) and Eq. (31), Lemma A.4, the bound may look a bit more complex, because the $m$ in Eq. (30) cannot be assigned to a different value. Please clarify. \n4. Moreover, in line 152, the paper claims $\\bar{f}$ is asymptotically correct, but it is unclear to me if the probability on the L.H.S. of Eq. (6) still goes to 0 when $m \\rightarrow \\infty$ after fixing the aforementioned problem with $m$. Please clarify. \n5. Theorem 3.1 studies the asymptotic correctness of the temporal ensemble with large $m$. The experiments mostly show the AnCon under 20 epochs. How does the performance change when more epochs are added? Is it still consistent with the theoretical implications?", "limitations": "The authors have discussed the limitations in Section 6.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The main paper is well-written and pleasant to read. \n2. The paper effectively modifies and adapts Early Learning Regularization (ELR) to the setting of self-training under distribution shifts with both theoretical justification and empirical evidence. The theoretical derivations are generally rigorous with assumptions clearly stated. \n3. The connection to self-distillation helps to better understand the approach. \n4. The empirical and ablation studies are comprehensive and clearly discuss a few critical criteria such as model selection, hyperparameter, and calibration.", "weaknesses": "1. Empirically, the approach improves over various baselines when being added, but what about the existing SoTA baselines for unsupervised domain adaptations, which seem relevant to this work. Besides, it seems rather odd to only report the results in Sec. 4 with non-optimal $\\lambda=0.3$, as the sensitivity analysis in Figure 2(a) clearly prefers much larger $\\lambda$. \n2. A critical assumption in the theorems is that the teacher (temporal ensemble) has a sufficiently good performance. It will be helpful to justify this with more empirical evidence. \n3. Non-trivial typos: In line 149, $x$ is already used in the main theorem, so may want to change $x$ to $z$ to avoid confusion, and this issue also occurred on 643 with $m$, which might have already caused *major* problems in proof (see below in Question 3)\n4. Minor typos: Line 667, Proof of Lemma A.4, Eq. (31), $k$ should be written as $mq$; Some references from NeurIPS 2024 should be 2023. \n5. There might be some mistakes in the theorem proofs. See below in Questions 3 and 4.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank Reviewer qtUp for the insightful suggestions and constructive comments, which have significantly improved the clarity of our paper. We are glad that the reviewer enjoyed reading our paper and acknowledged our principled improvements over the existing temporal consistency methods with theoretical guarantees and attractive properties. In our response below, we have addressed the questions and comments. \n_Note: [G#] is the reference for the global responses._\n\n**W1-1:** To clarify, AnCon aims to solve the source-free domain adaptation (SFDA) and test-time adaptation (TTA) problems, not the unsupervised domain adaptation (UDA) problem wherein the labeled source domain dataset is available during the adaptation. The unavailability of any reliable labeled samples in SFDA and TTA makes more challenging learning scenarios, which involve early learning phenomenon and model collapse under severe distribution shifts (cf. Figure 1 (a)). Therefore, the UDA methods would not be directly comparable to AnCon. Also, we note that the chosen baselines show state-of-the-art performances in general settings (cf. [G5 in the global response]).\n\n\n**W1-2:** Our suboptimal choices of hyperparameters are due to our rigorous and practical hyperparameter choice setting, and we kindly refer to [G4] for detailed explanation on the hyperparameter selection procedure and its strong practical implication. \n\n\n**W2:** Thanks for pointing out the significance of the assumption made in Theorem 3.2, which asserts that \"AnCon is at least better than vanilla self-training.\" However, the assumption is required only for deriving the final bound in Eq (7) to gain more insights about the impact of the generalized temporal ensemble's quality on the suboptimality of self-training. Rather, the core message of Theorem 3.2 that “AnCon is at least better performance than the vanilla self-training” holds _without_ this assumption. Specifically, in Eq. (13) on page 15, $ N(\\lambda^\\dagger) / N(0) $ corresponds to the ratio between the last term of Eq. (12) under AnCon and the vanilla self-training. Crucially, this ratio is less than or equal to 1, supporting the central message without the assumption.\n\nThanks to the Reviewer qtUp’s insightful question, we have separated the statement of Theorem 3.2 into two parts and modified the discussion in Lines 187-190 (Kindly see the comment below for the final form) to avoid the initial confusion and emphasize that AnCon guarantees at least better performance than the vanilla self-training can hold without such assumption. \n\n\n\n\n**W3:** Thanks for pointing out the notation mistake. Since the argument of the function $ \\xi $ is an arbitrary scalar, we have changed $ x $ to $ z $ in Line 149. We have changed the index notation of $ m $ in the Lemma A.4. to $ o $. These notational changes do not invalidate the proof. Regarding the confusion from the index notation, kindly see the response to Q3.  \n\n\n**W4:** Thanks for pointing out the minor typos! We have changed $ k $ by $ m \\cdot q $ and fixed typos for the references. \n\n\n**W5:** Kindly see the responses to Q3 and Q4.  \n\n\n**Q1:** Thanks for suggesting to unify the notation of the generalized temporal ensemble $ \\bar{f} $ and the pseudo label $ \\hat{Y} $ that might be clearer than the current notation. Despite careful thoughts, unfortunately, we do not see any feasible way to unify the notation without sacrificing the insights and clarity that our current notation can provide. Specifically, our separate notation enables clear explanations of the selective temporal consistency (cf. Eq (2)) that is only involved in $ \\bar{f} $. Also, the notation can intuitively explain the role of $\\bar{f}$ that regularizes the pseudo label $ \\hat{Y} $ through label smoothing, which makes the discussions in Lines 105-115 and the connection between AnCon and knowledge distillation clearer. Finally, there are points that require discussion solely on either of them (e.g., when referring to the accuracy of pseudo labels in Line 145 and Line 184; when discussing the quality of the generalized temporal ensemble in Theorem 3.1). Given the above reasons, we hope that Reviewer qtUp could agree with the advantage of separating notation for the pseudo label and the generalized temporal ensemble.\n\n\n**Q2:** Thanks for highlighting the validity of the assumption in Theorem 3.1. Here, we emphasize the important aspects of the assumption: the assumption is based on the “conditional” independence between temporal correctness given that the current prediction on a sample is relatively confident. This is grounded in the strong correlation between accuracy and confidence across a wide range of scenarios. Another way to understand this assumption is that the previous history of predictions is summarized in the random event whether the prediction is confident or not. We further remark that the validity of this assumption could be reinforced by strong empirical support for Eq. (6), as shown in Figure R3. \n\n\n**Q3:** Thanks for pointing out the unclear notation for the supporting lemma. The number of random Bernoulli samples $m$ in the Lemma A.4 (which we change to $o$) is not used in the other parts. Therefore, all other parts remain the same, except Line 624 which refers to this lemma is changed to “... due to Lemma A.4 with $ p_i = P(Y(x) = \\hat{Y}(x; \\theta_i)) $, $ o = Q(x; c_{0:m}) $, and $ q= 1/2 $.”  \n\n\n**Q4:** We kindly refer to [G2] for the clarification of the notion of asymptotic convergence. Also, we note that [G2] presents the important implication of Eq. (6) in the non-asymptotic region, i.e., monotonic improvement of quality of the generalized temporal ensemble, with strong empirical evidence. \n\n\n**Q5:** As explained in [G2], we kindly refer to Figure R3 that shows monotonic improvements over 100 epochs of quality of the generalized temporal ensemble under diverse distribution shifts scenarios, being consistent with Theorem 3.1.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2a8Ej9C8SN", "reviewer": "Reviewer_hP9c", "summary": "This paper presents Anchored confidence (AnCon), a novel self-training algorithm to improve test-time accuracy under distribution shifts. AnCon modifies the standard self-training algorithm by adding a temporal ensemble regularization. This regularizer is constructed by the consistency of temporal ensembles weighting with their predictive confidences. This intuitively helps the model avoid the \"early-learning phenomenon\", i.e., the model exploiting noisy labels after initially learning clean labels. Contributions include:\n1. A new algorithm for memorizing past ensemble predictions to avoid the \"early-learning phenomenon\", enhancing self-training performance under sequential distribution shifts.\n\n2. In terms of theory, the authors provide a rigorous theoretical analysis for the upper bound of the test-time error of the proposed temporal ensembles by concentration inequality in Theorem 3.1. Inspired by Knowledge Distillation (KD), they also design an upper bound of the expected cross-entropy loss between the temporal ensembles and the optimal value in Theorem 3.2.\n\n3. Extensive experiments confirm the hypothesis that the proposed algorithm can improve self-training performance regarding test-time accuracy and calibration under distribution shifts across different hyperparameter choices.", "questions": "1. The connection of AnCon with KD is not clear to me. The pseudo label $\\hat{Y}(x;\\theta)$ in self-training with AnCon and the true label  $Y(X)$ in L-163 ($Y(x)$ in L-74) are completely different because $Y(X)$ is unobservable while $\\hat{Y}(x;\\theta)$ is model’s prediction. So in Section 3.3, the statement in L-165-166 to connect AnCon with KD is still wrapping my head. In particular, why AnCon's pseudo-label can be considered as $Y(x)$ (perhaps the true label $Y(X)$ in L-163)? Also, I am curious about why Eq. (40) can lead to Eq. (41), i.e., how we can bound the norm of gradient loss $||\\nabla l(\\theta_{m,t}) - \\nabla \\tilde{l}(\\theta_m)||^2$ by the $Err(\\hat{Y};\\theta_{m})$ while this upper bound related the true label $Y(X)$?\n2. Some questions for clarification: is $\\delta$ in L-83 $\\in [0,1]$? $\\sum_0^m$ or $\\sum_0^{m-1}$ in Eq. 3? $\\theta_k$ or $\\theta_m$ in $Err_{XY}(\\cdot)$ in L-184? $\\mathbf{w}\\_{1:m}$ or $\\mathbf{w}_{0:m}$ in RHS Eq. in L-186?\n3. Compared with ELR which computes the temporal ensembles by the logit vector $f(x;\\theta_i)$ only in L-83, the proposed algorithm requires computing $\\mathbb{E}\\_{x\\in X}\\max_{k\\in [K]}f_k(x;\\theta_i)$ in Eq. 3, does this causes a higher computational demand?\n4. Can you generate a reliability diagram [2] to show the overconfidence and underconfidence of methods? I think this could be easier to assess how your method can improve the model's uncertainty quality.\n6. In the testing of different weighting schemes in Fig. 4 at Section 4.4, have you considered the setting of a uniform weighting for $\\mathbf{w}$, i.e., set Eq. 2 to $\\bar{f_k}(x;\\theta_{0:m},\\mathbb{w}_{0:m}):= \\sum_i^m p(y=k|x;\\theta_i)$? This is a simple setting but no longer temporal ensembles by discount factors, and also not standard (vanilla) self-training. I am quite curious about this result.\n5. Do the authors think your proposed method can work in other sequential decision-making, e.g., bandits, RL, bayesian optimization, active learning, etc.? If yes, could you please give some comments? Otherwise, could you please raise some challenges?\n\nReferences:\n\n[1] Fang et al., Source-Free Unsupervised Domain Adaptation: A Survey, arXiv, 2023.\n\n[2] Guo et al., On Calibration of Modern Neural Networks, ICML, 2017.", "limitations": "Please see my comments on the Weaknesses.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- This paper is very well-written and clear to understand the important aspects of the algorithm.\n- Although the temporal ensemble weighing technique reminds me of similarities with Early learning regularization (ERL) in discount factor from past predictions, I still like the AnCon in general because of a **novel** of high-quality uncertainty estimation behavior in neural network self-training.\n- Similar to ERL, AnCon is also more computationally efficient than non-parametric techniques and standard sampling-based Deep Ensembles.\n- The theoretical contribution of this paper is good, yielding a solid AnCon algorithm. Specifically, under mild assumptions, Theorem 3.1 and 3.2 formally explain why AnCon works and how temporal ensemble regularizers can contribute to improving self-training under covariate shifts (details in Contribution 2).\n- Experiments show a significant improvement of the standard self-training algorithm across different distribution shifts like artificial corruptions and real-world domain shifts.\n- The robustness of the proposed model across different settings is also confirmed across different hyperparameter choices of coefficient regularizer $\\lambda$ in Eq. 1 and discount factor $\\beta$ in Eq. 3, different proportional weighting schemes for $w_m(x)$ in Eq. 3, and different soft-hard predictions for $p(y|x,\\theta_i)$ in Eq. 2.", "weaknesses": "- Regarding the proposed algorithm: compared with the standard self-training, AnCon requires pre-defining the regularizer $\\lambda$ and the discount factor for the temporal ensembles $\\beta$. \n - Regarding the theory: Theorem 3.1 requires $\\bar{p}(x;\\mathbf{c}_{0:m}) \\geq 1/2$, i.e., at least 50% accuracy on average for the temporal ensembles. This is a quite strong assumption.\n - Regarding the experiments: (1) The quantitative results in all tables are reported without the standard deviation and significant tests; (2) The improvement in calibration is shown without any theoretical evidence. Fig. 3 (a) only shows ECE without the reliability diagram, making it hard to assess the correlation between the accuracy and uncertainty of softmax’s probability; (3) Lacks of comparison with other baselines, e.g., many baselines [1] in UDA could be added to compare in Appendix C.\n  - Others that are mentioned by the authors: the challenge of self-training methods under distribution shifts and the violation of average correct predictions in self-training.\n- Miscellaneous: (1) Although assumptions A.1, A.2, and A.3 are mild and have been cited, I believe it is still worth discussing how they are realistic with the paper's setting; (2) Some mathematical notations are vague and perhaps need to be polished (e.g., $f(x;\\theta)$ or $f_k(x;\\theta)$ in L-78, how $\\bar{f}(\\cdot)$ in Eq. 1 connect to the Def. $\\bar{f_k}(\\cdot)$ in Eq. 2, $\\hat{\\mathbb{E}}$ in Eq. 3 seems not to be defined, the same notation $\\sigma$ for different $\\sigma_*^2$ in Theorem 3.2 and perhaps the softmax $\\sigma_k(x;\\theta_i)$ in L-339, more in Question 2); (3) I find it quite hard to follow acronyms without links, I would recommend using \\usepackage[acronym]{glossaries} so readers can smoothly follow the paper; (4) Some Figure’s x-y-titles are tiny, e.g., Fig. 3 (b).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate Reviewer hP9c's insightful suggestions and thoughtful comments. We are pleased that Reviewer hP9c acknowledges our core technical contributions to showing the effectiveness of high-quality uncertainty estimation in the temporal ensemble for improving self-training with solid theoretical and empirical results. Below, we have carefully addressed Reviewer hP9c’s valuable comments, which we believe significantly improve our work.\n(Unfortunately, due to the number of comments from Reviewer hP9c, our rebuttal addresses only Weaknesses. We will post the remaining responses to Questions in the official comment and apologize for any inconvenience.)\n\n_Note: [G#] is the reference for the global responses._\n\n**W1:** Yes, but we hold a positive view on using the provided default hyperparameter values in diverse distribution shift scenarios. Specifically, we note that the single configurations ($\\lambda=0.3$ and $\\beta=0.9$) work well for 105 numbers of different distribution shifts with varying difficulties. Also, the performances are shown to be not drastically affected by changes in the hyperparameter values. Finally, we kindly refer to [G4 in the global response] for more detailed explanations of our rigorous hyperparameter selection procedure and its strong practical implications. \n\n\n\n\n**W2:** We kindly refer to [G1] for the validity of the assumption of $ \\bar{p}(x; c_{0:m}) $ made in Theorem 3.1. Also, we note that AnCon’s strong empirical performance can be preserved even if the condition is violated as discussed in [G3]. \n\n\n**W3-1:** Thank you for pointing out the important missing details in the error bars. In the original version, we opted to omit the variance terms because the variance is not particularly large for any method (e.g., on average 0.96 for OfficeHome), which gives statistical significance at a P-value < 5%. However, in order to prevent concerns about the statistical significance, we have added P-values for each table. \n\n\n**W3-2:** Thanks for the great suggestion that can help readers to understand the uncertainty estimation behavior of AnCon. Following the Reviewer hP9c’s insightful suggestion, we have analyzed the reliability diagram. As widely known in the literature, all self-training methods turn out to be overconfident. However, as shown in Figure R5, AnCon relaxes the overconfident behavior, which can be inferred from its better ECEs in the manuscript (e.g., Table 5). \n\n**W3-3:** To clarify, AnCon aims to solve the source-free domain adaptation (SFDA) and test-time adaptation (TTA) problems, not the unsupervised domain adaptation (UDA) problem wherein the labeled source domain dataset is available during the adaptation. The unavailability of any reliable labeled samples in SFDA and TTA makes more challenging learning scenarios, which involve early learning phenomenon and model collapse under severe distribution shifts (cf. Figure 1 (a)). Therefore, the UDA methods would not be directly comparable to AnCon. Also, we note that the chosen baselines show state-of-the-art performances in general settings (cf. [G5 in the global response]).\n\n**W4:** Kindly see [G3] for the unique and significant performance gains from AnCon in challenging distribution shift scenarios, wherein the average correct predictions assumption is violated. \n\n**W5-1:** As Reviewer hP9c commented, regularity conditions in Assumptions A.1, A.2, and A.3 are mild but essential for most theoretical studies with convergence analyses. Assumptions A.1 and A.2 can trivially hold under bounded parameter values that can be guaranteed by optimizing neural networks with finite iterations under a gradient or weight clipping. Assumption A.3 holds for infinite-width neural networks, i.e., the neural tangent kernel (NTK) regime (Charles & Papailiopoulos, 2018). Given that the gradient descent training dynamics of neural networks can be well approximated by NTK (Jacot et al., 2018), the PL condition can be generally regarded as a mild assumption. We fully agree with Reviewer hP9c’s suggestion and have added these discussions to “Appendix A.1 Assumptions.”\n\nCharles, Z., & Papailiopoulos, D. (2018). Stability and generalization of learning algorithms that converge to global optima. In ICML.\nJacot, A., Gabriel, F., & Hongler, C. (2018). Neural tangent kernel: Convergence and generalization in neural networks. In NerIPS.\n\n\n**W5-2**: Thanks for the valuable clarifying questions! $ f_k(x; \\theta) $ is right; $ \\bar{f}(x) := (\\bar{f}_1(x), \\bar{f}_2(x), \\cdots, \\bar{f}_K(x)) $; $ \\hat{E} $ is the Monte-Carlo estimator with mini-batch samples; we have introduced $ \\sigma^{2}(\\bar{\\theta}) $ for the variance of stochastic gradient of $\\bar{f}$; finally, we introduce a new notation for the softmax as $\\phi$ instead of $\\sigma$ to avoid the confusion. \n\n\n**W5-3/4**: Thanks for the great suggestions. We have added links for all acronyms and increased the font size for all small figures.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
**G1:** On the validity of the on-average assumption 
In Theorem 3.1, we made an assumption that $ \bar{p}(x; c_{0:m}) > 0.5 $, which we argue is not strong because of its dependence on the confidence thresholds $ c_{0:m} $. Specifically, $ \bar{p} $ is measured only for relatively confident predictions (cf. Line 144-145). This means that the assumption can hold by controlling the confidence threshold (cf. $ c_{0:m} $ in Theorem 3.1) at the expense of loosening the upper bound in Eq. (6) as mentioned in Line 156-157. For a concrete example, we have added Figure R1 that explains both (1) the average accuracy can be higher than 50% in the challenging setting where the pseudo label accuracy is below 50% and (2) $ \bar{p} $ can be further increased by increasing the confidence threshold (e.g., select 90th-quantile).

**G2:** Further elaborations of Theorem 3.1
In Theorem 3.1, the convergence holds for the asymptotic region of the number of confident predictions over iterations $ Q(x; c_{0:m}) $, not the number of iterations $ m $; that is, the inputs on which the neural network produce relatively confident predictions infinitely many times as the number of iterations goes to infinity. 

Further, we remark that the bound monotonically decreases as $ Q(x; c_{0:m}) $ increases. This means that the generalized temporal ensemble can provide high-quality learning signals for self-training in non-asymptotic regions through the “uncertainty-aware” temporal consistency that helps to satisfy the condition $ \bar{p}(x; c_{0:m}) > 0.5 $. Specifically, as shown in our new figures (cf. Figure R3), our generalized temporal ensemble’s accuracy tends to increase by a margin significantly as the number of confident samples increases, being consistent with our theory. We note that this monotonic improvement would not be the case for the temporal ensemble without uncertainty-awareness and vanilla self-training, which highlights the importance of ‘selectivity’ for constructing the temporal ensemble. 

To clarify the notion of asymptotic convergence and monotonic improvement, we have changed Line 151-152 as follows: 

“””
The result says that as long as the expected accuracy for relatively confident predictions is at least 50% on average over iterations, $ \bar{f}(x; \theta_{0:m}, w_{0:m}) $ is asymptotically correct on $ x $ such that $ Q(x; c_{0:m}) \rightarrow \infty $ as $ m \rightarrow \infty $. Besides, we remark that the error rate of the generalized temporal ensemble monotonically decreases as $ Q(x; c_{0:m}) $ increases if the iteration on-average accuracy condition holds. 
“””

Also, in Line 143, we have added “for samples where the neural network tends to be relatively confident during the self-training” after “asymptotically correct.”


**G3:** On performance of AnCon under severe distribution shifts 
Performance gains over a poorly performing initial parameter through self-training are shown in our extensive experiments. Specifically, in challenging scenarios where the initial model trained on the source domain significantly deteriorates, AnCon significantly improves performance, unlike vanilla self-training. This aligns with our theoretical results in Theorems 3.1 and 3.2. For example, in VisDa-2017 (Table 6), while the initial model accuracy is 38.71%, AnCon and self-training improve it to 71.11% and 67.77%, respectively. Additionally, for Shot, Impulse, and Gaussian corruptions with the most extreme shift intensity of 5, where the initial model achieves accuracies of (3.04%, 1.76%, 2.12%), AnCon achieves (22.56%, 26.56%, 25.85%) (cf. Table 11). This striking improvement, compared to vanilla self-training and ELR, underscores the importance of AnCon's uncertainty-aware temporal consistency scheme.

**G4:** On rigorous hyperparameter selection procedure and its practical implication
We perform hyperparameter selection without labels instead of looking at the test performance and selecting the best value which is data snooping and can result in overly optimistic performances that are hard to reproduce. Therefore, even though we found (and were not surprised about) that the found hyperparameters were sub-optimal during the sensitivity analysis, we opted to maintain our realistic hyperparameter selection setting. 

Another point we should mention is our choice of fixing single hyperparameter values across diverse settings that include 105 numbers of distribution shift scenarios. Of course, even under our aforementioned realistic hyperparameter selection setting, tuning hyperparameters for each scenario could result in better performances. However, we opted to fix the hyperparameter values for all scenarios, considering that in practice the environments (e.g,. Data, distributions, features) change frequently and thus tuning each time is very expensive. By doing so, we believe that we do not report overly optimistic performances of AnCon obtained under settings that are abstracted to the practice. We hope this can resolve Reviewer qtUp’s question on the sub-optimal choice of the hyperparameter value.  


**G5:** On performances of baseline methods
In this work, we apply AnCon to two baseline methods, NRC and GCE. While their ideas are simple, these methods are frequently cited as achieving state-of-the-art performances in recent literature (cf. Karim et al., 2023 CVPR; Press et al., 2023 NeurIPS; Rusak et al., 2022 TMLR). The basic intuitions behind these baselines—promoting local consistency and reducing the impact of wrong pseudo labels—are dominant principles in SFDA and TTA literature. To the best of our knowledge, no significantly better methods than these baselines exist without including computationally heavy methods such as ensembles. Therefore, we believe AnCon's compatibility with these baselines demonstrates its compatibility with state-of-the-art methods, and including other similar methods would provide only marginal additional insights.

Meta-review: The paper introduces a novel self-training method (AnCon) to improve test-time adaptation (TTA) and source-free domain adaptation (SFDA) under distribution shifts. It proposes using uncertainty-aware temporal ensembles and label smoothing to enhance pseudo-label quality during self-training. Theoretical guarantees are provided for the method's asymptotic correctness and reduced optimality gap. Extensive experiments demonstrate AnCon's effectiveness in various distribution shift scenarios.

The paper provides a well-founded and efficient approach to improving self-training under distribution shifts, with solid theoretical support and strong empirical results. Although its novelty is incremental, the method's practicality and demonstrated effectiveness across diverse settings make it a valuable contribution to the field. The thorough rebuttals addressed reviewers' concerns. As such, the AC recommends acceptance to NeurIPS.
Decision: Accept (poster)