Abstract:
We propose a framework for learning calibrated uncertainties under domain shifts. We consider the case where the source (training) distribution differs from the target (test) distribution. We detect such domain shifts through the use of binary domain classifier and integrate it with the task network and train them jointly endto-end. The binary domain classifier yields a density ratio that reflects the closeness of a target (test) sample to the source (training) distribution. We employ it to adjust the uncertainty of prediction in the task network. This idea of using the density ratio is based on the distributionally robust learning (DRL) framework, which accounts for the domain shift through adversarial risk minimization. We demonstrate that our method generates calibrated uncertainties that benefit many downstream tasks, such as unsupervised domain adaptation (UDA) and semi-supervised learning (SSL). In these tasks, methods like self-training and FixMatch use uncertainties to select confident pseudo-labels for re-training. Our experiments show that the introduction of DRL leads to significant improvements in cross-domain performance. We also demonstrate that the estimated density ratios show agreement with the human selection frequencies, suggesting a positive correlation with a proxy of human perceived uncertainties.

text: We propose a framework for learning calibrated uncertainties under domain shifts. We consider the case where the source (training) distribution differs from the target (test) distribution. We detect such domain shifts through the use of binary domain classifier and integrate it with the task network and train them jointly endto-end. The binary domain classifier yields a density ratio that reflects the closeness of a target (test) sample to the source (training) distribution. We employ it to adjust the uncertainty of prediction in the task network. This idea of using the density ratio is based on the distributionally robust learning (DRL) framework, which accounts for the domain shift through adversarial risk minimization. We demonstrate that our method generates calibrated uncertainties that benefit many downstream tasks, such as unsupervised domain adaptation (UDA) and semi-supervised learning (SSL). In these tasks, methods like self-training and FixMatch use uncertainties to select confident pseudo-labels for re-training. Our experiments show that the introduction of DRL leads to significant improvements in cross-domain performance. We also demonstrate that the estimated density ratios show agreement with the human selection frequencies, suggesting a positive correlation with a proxy of human perceived uncertainties.

heading: 1 INTRODUCTION
text: Uncertainty estimation is an important machine learning problem that is central to trustworthy AI (Tomsett et al., 2020; Antifakos et al., 2005). In addition, many important downstream applications rely on the correct estimation of uncertainties. This includes unsupervised domain adaptation (UDA) (Zou et al., 2019) and semi-supervised learning (SSL) (Sohn et al., 2020) where they are used to solicit confident pseudo-labels for re-training. In these applications, reliable pseudo-labels help avoid error propagation and catastrophic failures in early iterations (Kumar et al., 2020).
Obtaining reliable uncertainty estimation is challenging. In contrast to human annotation of labels, obtaining the ground-truth uncertainties from real-world data can be costly or even infeasible. It is also known that commonly used uncertainty proxies in deep neural networks, such as the softmax output, tend to give overconfident estimates (Guo et al., 2017a). This overconfidence is further amplified under domain shifts, where the target (test) domain and the source training domain differ significantly. Such distributional shifts tend to aggravate the existing issues in uncertainty estimation, leading to wrong but overconfident predictions on unfamiliar samples (Li & Hoiem, 2020).
Many methods have been proposed to calibrate the confidence of deep learning models so that the uncertainty level of a model prediction reflects the likelihood of the true event (Guo et al., 2017a). Label smoothing is a popular approach to reduce overconfidence and to promote more uniform outputs (Szegedy et al., 2016). Temperature scaling is another method where the logit scores are rescaled by a calibrated temperature (Platt et al., 1999). Approaches such as Monte-Carlo sampling (Gal & Ghahramani, 2016) and Bayesian inference (Blundell et al., 2015; Riquelme et al., 2018) model uncertainties from a Bayesian perspective but are computationally more expensive. Even though these methods lead to more calibrated uncertainties, recent studies show that their results cannot be fully trusted under domain shift (Snoek et al., 2019).
Our approach. To handle domain shifts, we characterize the “overlap” between the source training data and the test data. Intuitively, if the test sample is highly unlikely in the training distribution,
then the resulting confidence levels should be lowered. We incorporate this idea through the density ratio between the two domains and employ it for confidence calibration.
To make our idea more concrete, recall that the probability output of a standard neural network for classification can be expressed as P (y|x) ∝ exp (θy · φ(x)), where x is the predictor input, φ(x) is the data feature, and θy is the model parameter of the y-th class. Instead, we propose the following predictive form for our neural network:
P (y|x) ∝ exp ( Ps(x)
Pt(x) θy · φ(x)
) , (1)
where Ps(x) and Pt(x) are the densities of a data sample under the source and target distributions, respectively. When a target sample is close to the source domain (large Ps(x)/Pt(x)), the prediction is confident. However, when a target sample x is far away from the source distribution (small Ps(x)/Pt(x)), the confidence is lowered and the prediction is closer to a uniform distribution. This intuition is analogous to incorporating a sample-wise temperature to adjust the confidence according to the closeness of a test sample to the training distribution.
Eq. (1) is based on the distributionally robust learning (DRL) framework. DRL is an adversarial risk minimization framework that involves a two-player minimax game between a predictor and an adversary (Grünwald et al., 2004). While many previous DRL methods (Liu et al., 2020; Nakka et al., 2020) operate in low dimensional spaces using kernel density estimators for the density ratio estimation, we develop a DRL method to scale to real-world computer vision tasks, which is able to produce calibrated uncertainties under domain shift. Our contributions are:
1) We propose a DRL method for uncertainty estimation under domain shift. We introduce a binary domain classifier network which learns to predict the density ratios between source and target domains (See Fig. 1(a)). The domain classifier and the target classifier are trained simultaneously in an end-to-end fashion. We also introduce a regularized DRL framework to further promote smoothed model prediction and improve the calibration.
2) We show that the estimated density ratio reflects the distance of a test sample from both training and test distributions (Fig. 1(b)). Our experiments further empirically show that these estimates are also correlated with human selection frequency, based on the available ground-truth labels in ImageNetV2 (Recht et al., 2019a), which can be regarded as a proxy of human uncertainty perception.
3) We empirically show that the top-1 class predictions of DRL are more calibrated than the empirical risk minimization (ERM) and the temperature scaling on Office31 (Saenko et al., 2010), Office-Home (Venkateswara et al., 2017), and VisDA (Peng et al., 2017). We measure the level of calibration using expected calibration error (ECE), Brier Score and reliability plots.
4) We integrate our method as a plug-in module in downstream applications such as unsupervised domain adaption and semi-supervised learning, leading to significant improvements. For example, incorporating self-training (Zou et al., 2019) with DRL leads to state-of-the-art performance on VisDA-2017 (Peng et al., 2017) and a 6% improvement on hard examples. Incorporating Fix-
match (Sohn et al., 2020) with DRL improves the original Fixmatch by a relative 17% increase in accuracy under the cross-domain setting.

heading: 2 DISTRIBUTIONALLY ROBUST LEARNING
text: In this section, we first review the preliminaries of DRL (Sec. 2.1), followed by a proposed variant of DRL with class regularization (Sec. 2.2). We then propose an instantiation of DRL with a differentiable density ratio estimation network (Sec. 2.3). Finally, we show the application of DRL in UDA and SSL tasks (Sec. 2.4).

heading: 2.1 PRELIMINARIES
text: Notations and definitions: We denote the input and labels by random variables X and Y . We also use x ∈ Rd and X to represent the realization and sample space ofX . Our goal is to find a predictor
F : Rd 7→ RC , {f(x)|x ∈ X ,f(x) ∈ RC ∩∆} (2)
close to the true underlying Pt(Y |X). Here, d, C and ∆ denote the input dimension, class number, and probabilistic simplex, respectively. We consider the problem with labeled data sampled from a source distribution Ps(X,Y ) and unlabeled data sampled from a target distribution Pt(X), and use ePs(X,Y ) to represent the empirical source distribution. In this work, we consider an important form of domain shift with the covariate shift assumption Ps(X) 6= Pt(X), Ps(Y |X) = Pt(Y |X). Motivation: Traditional empirical risk minimization (ERM) frameworks tend to fail under covariate shift since ERM empirically learns a predictor P̂s(Y |X) from the finite source data that usually fails to generalize to the target distribution. DRL was previously proposed to overcome this issue. DRL can be formulated as a two-player adversarial risk minimization game (Grünwald et al., 2004) with the predictor player minimizing a loss, while the adversary player maximizing the loss. The adversary is allowed to perturb the labels, subject to certain feature-matching constraints to ensure data-compatibility.
Formulation: Under covariate shift, DRL (Liu & Ziebart, 2014) deals with the mismatch between the expected loss and the training data, and is defined on the target data distribution:
P̂t(Y |X) = argmin F max G∈Σ Ex∼Pt(X)L (f(x), g(x)) , (3)
where f(x), g(x) ∈ RC are the conditional label distributions given an input x and F , G are the entire distribution over all the input. Ex∼PtL(·) is an expected log loss on the target input defined:
Ex∼Pt(X)L(f(x), g(x)) , Ex∼Pt(X)[−g(x) · log f(x)]. (4)
F is the predictor player minimizing the loss function while G is the adversary maximizing the loss function. After solving this game, F is our estimate of P̂t(Y |X), which we will use for the classification on the target domain.
Eq. 3 is defined on the target domain only. How could a predictor be properly trained while there are no target labels available? The answer is that the adversary G is implicitly constrained by the source features. We use the following constraints to make sure thatG is close to Ps(Y |X):
Σ = {G|∑igyφ(xi) = ∑ iI[yi = y]φ(xi),∀y}, (5)
where xi ∼ Ps(X) and gy is the y-th dimension of g. Eq. 5 is a necessary but not sufficient condition for G = Ps(Y |X), thus serving as an implicit constraint for G to be close to the true Pt(Y |X) under the covariate shift assumption. Given a predefined feature function φ, when the adversary perturbs the conditional label distribution, certain aggregate function of φ on g should equal to the counterpart on the empirical source data.
From DRL to density ratio: When using the expected target logloss in Eq. 3, Eq. 1 is derived by solving the predictor F . Here we refer the derivation details to (Liu & Ziebart, 2014) but emphasize the important properties of the prediction: representation-level conservativeness.
Representation-level conservativeness: The predictions have higher certainty for inputs closer to the source domain when Ps(x)/Pt(x) is large. On the contrary, when Ps(x)/Pt(x) is small, the prediction is more uncertain. This property reflects the model’s ability to convey information about what it does not know through the model uncertainty.

heading: 2.2 CLASS-REGULARIZED DISTRIBUTIONALLY ROBUST LEARNING
text: Inspired by label smoothing (Szegedy et al., 2016) and regularization of neural network outputs (Pereyra et al., 2017), we can further add class-regularization to the conservative predictions in Eq. 1. Instead of doing it in a post-hoc way, we study how to incorporate class-regularization into the DRL framework. We propose to use a weighted logloss to penalize high confidence in the adversary’s label prediction:
P̂t(Y |X) = argmin F max G∈Σ Ex∼Pt(X)[−g(x) · log f(x)]− rEx∼Pt(X) [y g(x) · log f(x)] (6)
where y is the one-hot class vector, is the element-wise product, and r ∈ [0, 1] is a hyperparameter that controls the level of regularization. Σ here is the same as in 3. We observe the whole loss function is a convex-concave function in terms of f and g. According to the strong duality, we switch the order of the min and max. With a fixed g, f = g is the optimal solution of the inner min problem. So we have the following lemma (we refer all the proofs to appendix Sec. A):
Lemma 1. Eq. 6 can be reduced to a regularized maximum entropy problem with the estimator constrained:
max F∈Σ
Ex∼Pt(X)[−f(x) · log f(x)]− rEx∼Pt(X) [y f(x) · log f(x)] (7)
where Σ is the same as Eq. 5, meaning F should be close to the empirical source Ps(Y |X). Theorem 1. The solution of Eq. 7 for takes the form: fθ,r(y|x) ∝ exp ( Ps(x) Pt(x) θy·φ(x)+rI(y) rI(y)+1 ) ,
where θ represents the model parameters and I(y) is the yth dimension of the one-hot encoding y.
The proof of Theorem 1 follows the same principles when Eq. 1 is derived from Eq. 3. The training process of this model also follows the same procedure in (Liu & Ziebart, 2014), except that we use Theorem 1 to compute the prediction results. In training, we can compute the gradients using source labels where y is the one-hot encoding of each class. In testing, we set y to be an all-one vector.
Class-level regularization: Hyper-parameter r adjusts the smoothness of g’s label prediction in Eq. 6. It translates to the ry terms in the prediction form. Intuitively, this regularization increases the correct label’s prediction result when it is small and decreases it when it is large, using a threshold of 1 to discriminate between ‘large’ and ‘small’. This provides additional regularization and smoothness to the conservative prediction.

heading: 2.3 DIFFERENTIABLE DENSITY RATIO ESTIMATION
text: Estimating Ps(x)/Pt(x) (Sugiyama et al., 2012) can be challenging, especially in high-dimensional spaces. Usually a plug-in estimator is used. But the plug-in estimates are usually sub-optimal for downstream tasks due to the different objectives. We propose an end-to-end training procedure for DRL such that the density ratio estimator is trained together with the target classifier. The key insight here is that we use two neural networks for classification and differentiating the two domains, respectively. See Fig. 1(a). We now first introduce differentiable density ratio estimation before introducing joint training loss and the parameter learning of the two networks.
Differentiable density ratio estimation: Based on the Bayes’ rule, Ps(x)/Pt(x) can be computed from a conditional domain classifier (Bickel et al., 2007): Pt(x)Ps(x) = P (x|t) P (x|s) = P (t|x)P (s) P (s|x)P (t) . Concretely speaking, they can be estimated through binary classification using unlabeled source and target data with P (s)P (t) as a constant relating to number of source and target samples. On the other hand, we observe that Ps(x)/Pt(x) can be a trainable weight for each sample, and can receive gradients from the training objective of DRL. Therefore, we propose to train a discriminative neural network to differentiate the two domains, which receives training signals from both the target classification loss in DRL and the binary classification loss. The weights trained this way lost their original properties as density ratios, but still reflects the relation between the two domains.
Joint training loss: Assume φ(x,wr) is the representation learning neural network with parameter wr. We further define Pd(X) as the joint distribution of both source data and target data with their domain labels D = {d(x)}. We denote τ (x,wd) = (τs, τt) (where τs + τt = 1) as the probability output of the source and target domains from a domain classifier with parameter wd. Our joint
training loss is defined as:
min wr,θ,wd
Ex∼Pt(X) [−gt(x) · log f(x;wr,θ,wd)] + Ex∼Pd(X) [−d(x) · log τ (x,wd)] (8)
where gt(x) = Pt(Y |X = x) and f(x;wr,θ, rd) takes the form in Theorem 1:
f(x;wr,θ, rd) ∝ exp (( τs(x,wd)
τt(x,wd) θ · φ(x,wr) + rI(y)
) /(rI(y) + 1) ) (9)
Parameter learning: Note that gt in the first loss term of Eq. (8) concerns the conditional label distribution on target which is assumed to be not available. However, with the DRL formulation, we can compute and evaluate the gradients of θ and φ(x,wr) directly with the change of measure in the derivation of the gradients (Liu & Ziebart, 2014), since the gradients are only associated with the source data and labels. We show this in Fig. 1(a) that the representation learning network φ only has source data as the input. We then update θ and wr using the computed gradients and also directly back-propagate from the second loss term to update wd. Finally, we treat the densities as trainable variables and derive gradients for them from the first loss term (details shown in appendix Sec. B). By the Bayes rule, Ps(x)Pt(x) = τsP (t) τtP (s)
, where P (t) and P (s) are the amount of unlabeled data from each of the domain during the training process. Since we usually use the same amount of source and target data in each batch, they cancel out by following P (s)P (t) = 1. Then f in Theorem 1 is reduced to Eq. 9. Therefore, besides the binary classification loss, the parameter wd of the discriminative network is also trained with gradients from the first loss term. Algorithm 1 summarizes the procedure.
Xs <latexit sha1_base64="T720vG0aPWB+ZpEp6vdMuimbvgs=">AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUDbbSbt0swm7G6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTAXXxvO+ndLa+sbmVnm7srO7t3/gHh61dJIphk2WiER1QqpRcIlNw43ATqqQxqHAdji+nfntJ1SaJ/LRTFIMYjqUPOKMGis9dPq671a9mjcHWSV+QapQoNF3v3qDhGUxSsME1brre6kJcqoMZwKnlV6mMaVsTIfYtVTSGHWQz0+dkjOrDEiUKFvSkLn6eyKnsdaTOLSdMTUjvezNxP+8bmai6yDnMs0MSrZYFGWCmITM/iYDrpAZMbGEMsXtrYSNqKLM2HQqNgR/+eVV0rqo+V7Nv7+s1m+KOMpwAqdwDj5cQR3uoAFNYDCEZ3iFN0c4L86787FoLTnFzDH8gfP5Az74jcI=</latexit><latexit sha1_base64="T720vG0aPWB+ZpEp6vdMuimbvgs=">AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUDbbSbt0swm7G6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTAXXxvO+ndLa+sbmVnm7srO7t3/gHh61dJIphk2WiER1QqpRcIlNw43ATqqQxqHAdji+nfntJ1SaJ/LRTFIMYjqUPOKMGis9dPq671a9mjcHWSV+QapQoNF3v3qDhGUxSsME1brre6kJcqoMZwKnlV6mMaVsTIfYtVTSGHWQz0+dkjOrDEiUKFvSkLn6eyKnsdaTOLSdMTUjvezNxP+8bmai6yDnMs0MSrZYFGWCmITM/iYDrpAZMbGEMsXtrYSNqKLM2HQqNgR/+eVV0rqo+V7Nv7+s1m+KOMpwAqdwDj5cQR3uoAFNYDCEZ3iFN0c4L86787FoLTnFzDH8gfP5Az74jcI=</latexit><latexit sha1_base64="T720vG0aPWB+ZpEp6vdMuimbvgs=">AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUDbbSbt0swm7G6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTAXXxvO+ndLa+sbmVnm7srO7t3/gHh61dJIphk2WiER1QqpRcIlNw43ATqqQxqHAdji+nfntJ1SaJ/LRTFIMYjqUPOKMGis9dPq671a9mjcHWSV+QapQoNF3v3qDhGUxSsME1brre6kJcqoMZwKnlV6mMaVsTIfYtVTSGHWQz0+dkjOrDEiUKFvSkLn6eyKnsdaTOLSdMTUjvezNxP+8bmai6yDnMs0MSrZYFGWCmITM/iYDrpAZMbGEMsXtrYSNqKLM2HQqNgR/+eVV0rqo+V7Nv7+s1m+KOMpwAqdwDj5cQR3uoAFNYDCEZ3iFN0c4L86787FoLTnFzDH8gfP5Az74jcI=</latexit><latexit sha1_base64="T720vG0aPWB+ZpEp6vdMuimbvgs=">AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUDbbSbt0swm7G6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTAXXxvO+ndLa+sbmVnm7srO7t3/gHh61dJIphk2WiER1QqpRcIlNw43ATqqQxqHAdji+nfntJ1SaJ/LRTFIMYjqUPOKMGis9dPq671a9mjcHWSV+QapQoNF3v3qDhGUxSsME1brre6kJcqoMZwKnlV6mMaVsTIfYtVTSGHWQz0+dkjOrDEiUKFvSkLn6eyKnsdaTOLSdMTUjvezNxP+8bmai6yDnMs0MSrZYFGWCmITM/iYDrpAZMbGEMsXtrYSNqKLM2HQqNgR/+eVV0rqo+V7Nv7+s1m+KOMpwAqdwDj5cQR3uoAFNYDCEZ3iFN0c4L86787FoLTnFzDH8gfP5Az74jcI=</latexit> Ys <latexit sha1_base64="3uEUweR2+ghDMRTs3mEW/tkeVbM=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48V7Ye0oWy2m3bpZhN2J0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GNzO//cS1EbF6wEnC/YgOlQgFo2il+8e+6ZcrbtWdg6wSLycVyNHol796g5ilEVfIJDWm67kJ+hnVKJjk01IvNTyhbEyHvGupohE3fjY/dUrOrDIgYaxtKSRz9fdERiNjJlFgOyOKI7PszcT/vG6K4ZWfCZWkyBVbLApTSTAms7/JQGjOUE4soUwLeythI6opQ5tOyYbgLb+8SloXVc+teneXlfp1HkcRTuAUzsGDGtThFhrQBAZDeIZXeHOk8+K8Ox+L1oKTzxzDHzifP0B+jcM=</latexit><latexit sha1_base64="3uEUweR2+ghDMRTs3mEW/tkeVbM=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48V7Ye0oWy2m3bpZhN2J0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GNzO//cS1EbF6wEnC/YgOlQgFo2il+8e+6ZcrbtWdg6wSLycVyNHol796g5ilEVfIJDWm67kJ+hnVKJjk01IvNTyhbEyHvGupohE3fjY/dUrOrDIgYaxtKSRz9fdERiNjJlFgOyOKI7PszcT/vG6K4ZWfCZWkyBVbLApTSTAms7/JQGjOUE4soUwLeythI6opQ5tOyYbgLb+8SloXVc+teneXlfp1HkcRTuAUzsGDGtThFhrQBAZDeIZXeHOk8+K8Ox+L1oKTzxzDHzifP0B+jcM=</latexit><latexit sha1_base64="3uEUweR2+ghDMRTs3mEW/tkeVbM=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48V7Ye0oWy2m3bpZhN2J0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GNzO//cS1EbF6wEnC/YgOlQgFo2il+8e+6ZcrbtWdg6wSLycVyNHol796g5ilEVfIJDWm67kJ+hnVKJjk01IvNTyhbEyHvGupohE3fjY/dUrOrDIgYaxtKSRz9fdERiNjJlFgOyOKI7PszcT/vG6K4ZWfCZWkyBVbLApTSTAms7/JQGjOUE4soUwLeythI6opQ5tOyYbgLb+8SloXVc+teneXlfp1HkcRTuAUzsGDGtThFhrQBAZDeIZXeHOk8+K8Ox+L1oKTzxzDHzifP0B+jcM=</latexit><latexit sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit><latexit sha1_base64="FF2fC21nUkIXnKU2Zf2WCEresp8=">AAAB33icbZDNSgMxFIXv1L9aq1a3boJFcFVm3OhScOOyom2VdiiZ9E4bmskMyR2hDH0ENy4U8a3c+TamPwttPRD4OCch954oU9KS7397pY3Nre2d8m5lr7p/cFg7qrZtmhuBLZGq1DxG3KKSGlskSeFjZpAnkcJONL6Z5Z1nNFam+oEmGYYJH2oZS8HJWfdPfduv1f2GPxdbh2AJdViq2a999QapyBPUJBS3thv4GYUFNySFwmmll1vMuBjzIXYdap6gDYv5qFN25pwBi1PjjiY2d3+/KHhi7SSJ3M2E08iuZjPzv6ybU3wVFlJnOaEWi4/iXDFK2WxvNpAGBamJAy6MdLMyMeKGC3LtVFwJwerK69C+aAR+I7jzoQwncArnEMAlXMMtNKEFAobwAm/w7inv1ftY1FXylr0dwx95nz8jp4xu</latexit><latexit sha1_base64="FF2fC21nUkIXnKU2Zf2WCEresp8=">AAAB33icbZDNSgMxFIXv1L9aq1a3boJFcFVm3OhScOOyom2VdiiZ9E4bmskMyR2hDH0ENy4U8a3c+TamPwttPRD4OCch954oU9KS7397pY3Nre2d8m5lr7p/cFg7qrZtmhuBLZGq1DxG3KKSGlskSeFjZpAnkcJONL6Z5Z1nNFam+oEmGYYJH2oZS8HJWfdPfduv1f2GPxdbh2AJdViq2a999QapyBPUJBS3thv4GYUFNySFwmmll1vMuBjzIXYdap6gDYv5qFN25pwBi1PjjiY2d3+/KHhi7SSJ3M2E08iuZjPzv6ybU3wVFlJnOaEWi4/iXDFK2WxvNpAGBamJAy6MdLMyMeKGC3LtVFwJwerK69C+aAR+I7jzoQwncArnEMAlXMMtNKEFAobwAm/w7inv1ftY1FXylr0dwx95nz8jp4xu</latexit><latexit sha1_base64="yNbPz3toPjc7ETYH7tUeomunJqM=">AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0m86LHoxWNF+yFtKJvtpl262YTdiVBCf4IXD4p49Rd589+4bXPQ1gcDj/dmmJkXplIY9Lxvp7S2vrG5Vd6u7Ozu7R+4h0ctk2Sa8SZLZKI7ITVcCsWbKFDyTqo5jUPJ2+H4Zua3n7g2IlEPOEl5ENOhEpFgFK10/9g3fbfq1bw5yCrxC1KFAo2++9UbJCyLuUImqTFd30sxyKlGwSSfVnqZ4SllYzrkXUsVjbkJ8vmpU3JmlQGJEm1LIZmrvydyGhsziUPbGVMcmWVvJv7ndTOMroJcqDRDrthiUZRJggmZ/U0GQnOGcmIJZVrYWwkbUU0Z2nQqNgR/+eVV0rqo+V7Nv/Oq9esijjKcwCmcgw+XUIdbaEATGAzhGV7hzZHOi/PufCxaS04xcwx/4Hz+AD8+jb8=</latexit><latexit sha1_base64="3uEUweR2+ghDMRTs3mEW/tkeVbM=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48V7Ye0oWy2m3bpZhN2J0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GNzO//cS1EbF6wEnC/YgOlQgFo2il+8e+6ZcrbtWdg6wSLycVyNHol796g5ilEVfIJDWm67kJ+hnVKJjk01IvNTyhbEyHvGupohE3fjY/dUrOrDIgYaxtKSRz9fdERiNjJlFgOyOKI7PszcT/vG6K4ZWfCZWkyBVbLApTSTAms7/JQGjOUE4soUwLeythI6opQ5tOyYbgLb+8SloXVc+teneXlfp1HkcRTuAUzsGDGtThFhrQBAZDeIZXeHOk8+K8Ox+L1oKTzxzDHzifP0B+jcM=</latexit><latexit sha1_base64="3uEUweR2+ghDMRTs3mEW/tkeVbM=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48V7Ye0oWy2m3bpZhN2J0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GNzO//cS1EbF6wEnC/YgOlQgFo2il+8e+6ZcrbtWdg6wSLycVyNHol796g5ilEVfIJDWm67kJ+hnVKJjk01IvNTyhbEyHvGupohE3fjY/dUrOrDIgYaxtKSRz9fdERiNjJlFgOyOKI7PszcT/vG6K4ZWfCZWkyBVbLApTSTAms7/JQGjOUE4soUwLeythI6opQ5tOyYbgLb+8SloXVc+teneXlfp1HkcRTuAUzsGDGtThFhrQBAZDeIZXeHOk8+K8Ox+L1oKTzxzDHzifP0B+jcM=</latexit><latexit sha1_base64="3uEUweR2+ghDMRTs3mEW/tkeVbM=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48V7Ye0oWy2m3bpZhN2J0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GNzO//cS1EbF6wEnC/YgOlQgFo2il+8e+6ZcrbtWdg6wSLycVyNHol796g5ilEVfIJDWm67kJ+hnVKJjk01IvNTyhbEyHvGupohE3fjY/dUrOrDIgYaxtKSRz9fdERiNjJlFgOyOKI7PszcT/vG6K4ZWfCZWkyBVbLApTSTAms7/JQGjOUE4soUwLeythI6opQ5tOyYbgLb+8SloXVc+teneXlfp1HkcRTuAUzsGDGtThFhrQBAZDeIZXeHOk8+K8Ox+L1oKTzxzDHzifP0B+jcM=</latexit><latexit sha1_base64="3uEUweR2+ghDMRTs3mEW/tkeVbM=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48V7Ye0oWy2m3bpZhN2J0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GNzO//cS1EbF6wEnC/YgOlQgFo2il+8e+6ZcrbtWdg6wSLycVyNHol796g5ilEVfIJDWm67kJ+hnVKJjk01IvNTyhbEyHvGupohE3fjY/dUrOrDIgYaxtKSRz9fdERiNjJlFgOyOKI7PszcT/vG6K4ZWfCZWkyBVbLApTSTAms7/JQGjOUE4soUwLeythI6opQ5tOyYbgLb+8SloXVc+teneXlfp1HkcRTuAUzsGDGtThFhrQBAZDeIZXeHOk8+K8Ox+L1oKTzxzDHzifP0B+jcM=</latexit><latexit sha1_base64="3uEUweR2+ghDMRTs3mEW/tkeVbM=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48V7Ye0oWy2m3bpZhN2J0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GNzO//cS1EbF6wEnC/YgOlQgFo2il+8e+6ZcrbtWdg6wSLycVyNHol796g5ilEVfIJDWm67kJ+hnVKJjk01IvNTyhbEyHvGupohE3fjY/dUrOrDIgYaxtKSRz9fdERiNjJlFgOyOKI7PszcT/vG6K4ZWfCZWkyBVbLApTSTAms7/JQGjOUE4soUwLeythI6opQ5tOyYbgLb+8SloXVc+teneXlfp1HkcRTuAUzsGDGtThFhrQBAZDeIZXeHOk8+K8Ox+L1oKTzxzDHzifP0B+jcM=</latexit><latexit sha1_base64="3uEUweR2+ghDMRTs3mEW/tkeVbM=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48V7Ye0oWy2m3bpZhN2J0IJ/QlePCji1V/kzX/jts1BWx8MPN6bYWZekEhh0HW/ncLa+sbmVnG7tLO7t39QPjxqmTjVjDdZLGPdCajhUijeRIGSdxLNaRRI3g7GNzO//cS1EbF6wEnC/YgOlQgFo2il+8e+6ZcrbtWdg6wSLycVyNHol796g5ilEVfIJDWm67kJ+hnVKJjk01IvNTyhbEyHvGupohE3fjY/dUrOrDIgYaxtKSRz9fdERiNjJlFgOyOKI7PszcT/vG6K4ZWfCZWkyBVbLApTSTAms7/JQGjOUE4soUwLeythI6opQ5tOyYbgLb+8SloXVc+teneXlfp1HkcRTuAUzsGDGtThFhrQBAZDeIZXeHOk8+K8Ox+L1oKTzxzDHzifP0B+jcM=</latexit>
DRL model Training
Target
Source
DRL Losses
DRL model PredictionPseudoLabels Unsupervised LossesXt<latexit sha1_base64="ZlNa6HE035kY5qF8jlZM06/qAlE=">AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUDbbTbt0swm7E6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTKUw6HnfTmltfWNzq7xd2dnd2z9wD49aJsk0402WyER3Qmq4FIo3UaDknVRzGoeSt8Px7cxvP3FtRKIecZLyIKZDJSLBKFrpodPHvlv1at4cZJX4BalCgUbf/eoNEpbFXCGT1Jiu76UY5FSjYJJPK73M8JSyMR3yrqWKxtwE+fzUKTmzyoBEibalkMzV3xM5jY2ZxKHtjCmOzLI3E//zuhlG10EuVJohV2yxKMokwYTM/iYDoTlDObGEMi3srYSNqKYMbToVG4K//PIqaV3UfK/m319W6zdFHGU4gVM4Bx+uoA530IAmMBjCM7zCmyOdF+fd+Vi0lpxi5hj+wPn8AUB8jcM=</latexit><latexit sha1_base64="ZlNa6HE035kY5qF8jlZM06/qAlE=">AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUDbbTbt0swm7E6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTKUw6HnfTmltfWNzq7xd2dnd2z9wD49aJsk0402WyER3Qmq4FIo3UaDknVRzGoeSt8Px7cxvP3FtRKIecZLyIKZDJSLBKFrpodPHvlv1at4cZJX4BalCgUbf/eoNEpbFXCGT1Jiu76UY5FSjYJJPK73M8JSyMR3yrqWKxtwE+fzUKTmzyoBEibalkMzV3xM5jY2ZxKHtjCmOzLI3E//zuhlG10EuVJohV2yxKMokwYTM/iYDoTlDObGEMi3srYSNqKYMbToVG4K//PIqaV3UfK/m319W6zdFHGU4gVM4Bx+uoA530IAmMBjCM7zCmyOdF+fd+Vi0lpxi5hj+wPn8AUB8jcM=</latexit><latexit sha1_base64="ZlNa6HE035kY5qF8jlZM06/qAlE=">AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUDbbTbt0swm7E6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTKUw6HnfTmltfWNzq7xd2dnd2z9wD49aJsk0402WyER3Qmq4FIo3UaDknVRzGoeSt8Px7cxvP3FtRKIecZLyIKZDJSLBKFrpodPHvlv1at4cZJX4BalCgUbf/eoNEpbFXCGT1Jiu76UY5FSjYJJPK73M8JSyMR3yrqWKxtwE+fzUKTmzyoBEibalkMzV3xM5jY2ZxKHtjCmOzLI3E//zuhlG10EuVJohV2yxKMokwYTM/iYDoTlDObGEMi3srYSNqKYMbToVG4K//PIqaV3UfK/m319W6zdFHGU4gVM4Bx+uoA530IAmMBjCM7zCmyOdF+fd+Vi0lpxi5hj+wPn8AUB8jcM=</latexit><latexit sha1_base64="ZlNa6HE035kY5qF8jlZM06/qAlE=">AAAB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUDbbTbt0swm7E6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTKUw6HnfTmltfWNzq7xd2dnd2z9wD49aJsk0402WyER3Qmq4FIo3UaDknVRzGoeSt8Px7cxvP3FtRKIecZLyIKZDJSLBKFrpodPHvlv1at4cZJX4BalCgUbf/eoNEpbFXCGT1Jiu76UY5FSjYJJPK73M8JSyMR3yrqWKxtwE+fzUKTmzyoBEibalkMzV3xM5jY2ZxKHtjCmOzLI3E//zuhlG10EuVJohV2yxKMokwYTM/iYDoTlDObGEMi3srYSNqKYMbToVG4K//PIqaV3UfK/m319W6zdFHGU4gVM4Bx+uoA530IAmMBjCM7zCmyOdF+fd+Vi0lpxi5hj+wPn8AUB8jcM=</latexit>

heading: 2.4 APPLICATIONS TO UDA AND SSL
text: We show how to incorporate end-to-end DRL within a framework that takes unlabeled data for training. We introduce the general setting with two examples: self-training UDA and cross-domain SSL, followed by a new self-training algorithm DRST and a new SSL algorithm DRSSL.
General settings: In many cases, the source domain where models are trained may have abundant labels but the target domain lacks enough labels. Typical problems under this setting include unsupervised domain adaptation and (Zou et al., 2019) and semi-supervised learning (Sohn et al., 2020). In both cases, a common strategy is to treat the prediction results on the target data as pseudo labels to train the model on target data. In these methods, model confidence (softmax output) is often leveraged as the proxy to rank the reliability of pseudo labels, with the underlying assumption that there exists a positive correlation between model confidence and pseudo label quality. However, such an assumption requires accurate uncertainty estimation to avoid false usage of wrong pseudo labels which may poison the training. To this end, we incorporate DRL into these frameworks with to provide calibrated uncertainties. An illustration of this setting is shown in Figure 2.
Distributionally robust self-training: In UDA, we are given labeled source data and unlabeled target data and aim to achieve adaptation from the source to the target domain. Self-training is an effective method for UDA (Zou et al., 2019), where the training procedure in Fig. 2 is conducted multiple times. Here we propose DRST to plug the class-regularized DRL model into self-training.
The idea is to regard each training epoch as a new domain shift problem in DRL. After each training epoch, we make predictions on the target domain and select more confident data and generate pseudo-labels for them to merge into the source training data. Then the labeled source data and the newly pseudo-labeled target data become the new source set for the next training epoch.
Distributionally robust semi-supervised learning: In cross-domain SSL, we have few labeled source data and many unlabeled target data. We aim to utilize unlabeled data in the target domain to help representation learning and save the effort needed for labeling. One simple and effective strategy is to use pseudo-labels generated from weakly-augmented data to supervise stronglyaugmented data (Sohn et al., 2020). Here ‘weakly’ means simple flip-and-shift data augmentation while ‘strongly’ follows the same strategy with (Sohn et al., 2020). Using DRL’s prediction, we propose DRSSL, which assigns pseudo-labels more conservatively. We plug DRL into Fixmatch (Sohn et al., 2020). Here the unsupervised loss in Fig. 2 is the ‘consistency loss’ in (Sohn et al., 2020): Lu = 1M ∑M m=1 I(max(P̂ (ywt |xwt )) > τ)H(ŷwt , P̂ (yst |xst )), where xwt and ywt represent the weakly-augmented target data, xst and y s t represent the strongly-augmented version of the same image data, and τ is a threshold for generating pseudo-labels ŷwt .

heading: 3 EXPERIMENTS
text: We evaluate our method on benchmark datasets. DRL is evaluated as a method providing more calibrated uncertainties (Sec. 3.1), DRST as an effective UDA method (Sec. 3.2) and DRSSL as a cross-domain SSL method. We show additional results and details in appendix Sec. C, D.
Datasets and methods: We use Office31 (Saenko et al., 2010), Office-Home (Venkateswara et al., 2017) and VisDA (Peng et al., 2017) for evaluating DRL’s uncertainties. We compare DRL with temperature scaling (TS), VADA (Shu et al., 2018) and source-only. We also train models using ImageNet (Deng et al., 2009) as the source domain and ImageNetV2 (Recht et al., 2019a) as the target domain to check the relationship between our estimated weights and the human selection frequencies (HSF). VisDA2017 is used to evaluate DRST, for which we compare with (1) traditional UDA baselines: MMD (Long et al., 2015), MCD (Saito et al., 2018b) and ADR (Saito et al., 2018a); (2) recent self-training UDA baselines: CBST (Zou et al., 2018) and CRST (Zou et al., 2019); (3) other uncertainty quantification or UDA methods combined with self-training baselines: AVH (Chen et al., 2020a) + CBST and DeepCORAL (Sun & Saenko, 2016)+CBST. In addition, we use CIFAR10, STL10 (Coates et al., 2011), MNIST (Lecun & Bottou, 1998) and SVHN (svh, 2011) to construct settings with few source labeled data and much unlabeled target data and show DRSSL’s advantages in cross-domain SSL over Fixmatch (Sohn et al., 2020).
Evaluation metrics: Apart from accuracy, we also use Brier score (Brier, 1950), expected calibration error (ECE) (Guo et al., 2017a), and reliability plot (Dawid, 1982; Guo et al., 2017a; Naeini et al., 2015) to evaluate the performance of our proposed method and the baselines. Brier score measures the mean squared difference between the predicted probability assigned to the possible outcome and the actual outcome. Despite potential prob-
lems with ECE (Nixon et al., 2019), it is still the most prevalent metric for the top-1 prediction. ECE is defined as the sum of average difference between prediction accuracy and confidence of different confidence bins (we use 15 bins in practice). For both the Brier score and ECE, the lower the score, the better the model is calibrated.
Experiment setup: For Office31 and Office-Home tasks, we use ResNet50 (He et al., 2016) as the backbone for all models to make the comparison fair. We train by SGD for 100 epochs and set the learning rate to 0.001. For VisDA, we use the ResNet101 (He et al., 2016) backbone and also the SGD optimizer. During the 20 epochs of training, the initial learning rate is set as 10−5 and the weight decay parameter is set as 5 × 10−4. For ImageNet, we follow the standard training process of AlexNet (Krizhevsky et al., 2012) and VGG-19 (Simonyan & Zisserman, 2014), where the initial learning
rate is 0.01 and we decay the learning rate by a factor of 10 every 30 epochs.

heading: 3.1 CALIBRATED UNCERTAINTIES FROM DRL
text: DRL’s calibrated confidence: Fig. 4 demonstrates our more calibrated uncertainty. DRL tends to be underconfident and conservative but still stays closer to the calibration line (dashed line). Table 1 also shows our better ECE on ImageNetV2 in different human selection frequency (HSF) bins. Our additional results in appendix Sec. C show that the accuracy of DRL is also competitive.
Density ratio v.s. human selection frequency: Fig. 3 shows that the estimated weight is positively correlated with the human selection frequency (HSF) in ImageNetV2 (Recht et al., 2019b) under different neural network architectures. Here we regard [0, 0.2] as low HSF and (0.2, 1] as high HSF. Images with low HSF have smaller weights, indicating that they are visually harder for humans.

heading: 3.2 UNSUPERVISED DOMAIN ADAPTATION
text: Initialization with ASG: The quality of the source model has a significant impact on the final performance. The recently proposed automated synthetic-to-real generalization (ASG) model using ImageNet pretrained knowledge can be used to improve synthetic training (Chen et al., 2020b). Initializing with ASG trained model leads to significant improvements in self-training. Hence, we use the ASG pretrained model as an initialization, with results shown in Table 2’s last four lines.
Accuracy and calibration: Fig. 5(a) and Table 2 show that DRST performs best in accuracy. Our vanilla version of DRST outperforms CRST by over 5% with ASG initialization. We improve the SOTA self-training accuracy on VisDA by over 1%. In the rest of the results, we use DRST to represent DRST-ASG. Fig. 5(b) shows that DRST also achieves higher calibrated confidence.
Ablation study: Fig. 5(a)(b) include two ablation methods. In the first ablation, we set r to 0 so that there is no class regularization in DRL (“r = 0”). The prediction then follows the form in Eq. (1). In the second ablation, we set the weights to 1 to mute the differentiable density ratio estimation in our method so that there is no representation level conservativeness (“R = 1”). DRST achieves the best performance when both components are present.
Covariate shift: Fig. 5(c) shows Ps(φ(x))/Pt(φ(x)) − Ps(φ(x),y)/Pt(φ(x),y) using discriminative density ratio estimators (per class). This serves as a proxy of covariate shift as it becomes 0 when covariate shift holds. We can see that the gap decreases with self-training, which shows that even though covariate shift may not hold in the beginning, self-training helps to promote this assumption with better aligned domains and more discriminative feature distributions.
Improvement on hard examples: Fig. 5 (d) demonstrates that DRST achieves larger performance gain compared to the baselines on target samples with smaller weights. Recall that data is not wellrepresented in the source domain when Ps(x)/Pt(x) is small (Fig. 1(b)). Therefore, DRST provides robust performance on harder examples in the target domain.
Density ratios: In Fig. 1(b), a harder example obtains a lower density ratio due to its vague shape. Our density ratios reflect the closeness of a sample to the two domains. More examples are in appendix Sec. C. Moreover, the magnitude of our estimated density ratios are modest in general within the range [0.1, 10]. This is due to the regularization by the other network’s learning signals.
Improved attention: Fig. 6 visualizes the network attention of DRST and the baselines using GradCAM (Selvaraju et al., 2017), where DRST renders improved attention with better object coverage.

heading: 3.3 DISTRIBUTIONALLY ROBUST SEMI-SUPERVISED LEARNING
text: We use two pairs of source and target domains in this experiment: Source: CIFAR10 / Target: STL10 and Source: MNIST / Target: SVHN. Table 3 show that DRL-powered SSL method improves the original Fixmatch significantly in
cross-domain semi-supervised learning tasks. Note our setting is different from UDA where source labeled data is abundant. SSL focuses on using unlabeled augmented data to help learning from few labeled data. In the CIFAR10 to STL10 case, we only have 4k labeled source data. For MNIST to SVHN, we have 40k labeled source data. The results show that DRL is beneficial for generating high-quality pseudo supervision for unlabeled data under the cross-domain SSL setting.

heading: 4 RELATED WORK
text: Distributionally robust learning: Several different learning algorithms can be derived from this framework (Liu & Ziebart, 2014; 2017; Chen et al., 2016; Fathony et al., 2016; 2018). One limitation of this line of work is that it is only suitable for low-dimensional data. The other inconvenience is that it requires the density ratio to be estimated beforehand. Our work instead deals with highdimensional data and estimates the density ratio along with the learning process. Another line of work perturbs the covariate variable (Ben-Tal et al., 2013; Shafieezadeh Abadeh et al., 2015; Sinha et al., 2018; Hu et al., 2018; Najafi et al., 2019) and focuses on model robustness against adversarial perturbations. In our work, we make DRL practical in high-dimensional vision tasks by introducing an end-to-end trained domain classifier.
UDA and SSL: There is a rich literature in UDA focusing on representation alignment between the source and target domains (Fernando et al., 2014; Pan et al., 2010; Sun et al., 2016; Wang et al., 2018; Baktashmotlagh et al., 2013; Na et al., 2021). It is also prevalent to use a discriminator network to differentiate two learned representations (adversarially) to locate such a feature space (Ganin et al., 2016; Tzeng et al., 2017; Ajakan et al., 2014; Long et al., 2018; Sankaranarayanan et al., 2018). Some other works focus on more specific shift assumptions, such as covariate shift (Shimodaira, 2000) and label shift (Lipton et al., 2018; Azizzadenesheli et al., 2019). For the covariate shift case, even though various density ratio estimation methods were investigated before (Sugiyama et al., 2012), only few works explore the possibility to apply them to high-dimensional data (Khan et al., 2019; Moustakides & Basioti, 2019; Park et al., 2020). Recently, multiple self-training methods have been proposed for UDA (Zou et al., 2018; 2019) and many theoretical understandings of selftraining are also developed (Kumar et al., 2020; Chen et al., 2020c). Similarly, rich literature also exist in pseudo-labeling for SSL (Berthelot et al., 2019; Arazo et al., 2020; Miyato et al., 2018; Sohn et al., 2020). In this paper, we focus on utilizing DRL’s model uncertainty to help with choosing better pseudo-labeled data from the target domain in UDA and cross-domain SSL.
Uncertainty quantification with deep models: Bayesian methods largely contribute to complementing deep learning with uncertainty quantification (Gal, 2016; Welling & Teh, 2011; Kingma & Welling, 2013; Louizos & Welling, 2017; Blundell et al., 2015; Riquelme et al., 2018; Gal & Ghahramani, 2015; Gal et al., 2017; Kendall & Gal, 2017; Mandt et al., 2017). However, with a large number of parameters, Bayesian deep learning may suffer from computational inefficiency. Other calibration methods include temperature scaling (Platt et al., 1999; Guo et al., 2017b), deep ensembles (Lakshminarayanan et al., 2017), calibration regression (Kuleshov et al., 2018), quantile regression (Romano et al., 2019) and trainable calibration measure (Kumar et al., 2018). These methods do not apply to our setting because we do not have access to the target labels. Recent work on calibrating or uncertainties under domain shift either only focus on “using” but not “generating” the uncertainties (Han et al., 2019; Lee & Lee, 2020; Kurmi et al., 2019) or focus on the importance weighting setup (Park et al., 2020; Wang et al., 2020). In this paper, we generate more calibrated uncertainties using the DRL framework, where the density ratios are inherent.

heading: 5 CONCLUSION AND DISCUSSION
text: In this paper, we have studied uncertainty estimation under distribution shift using the distributionally robust learning framework. We demonstrate that density estimation can be integrated into the learning process by using a domain classifier. We propose differentiable density ratio estimation and develop end-to-end training techniques for our proposed method. Using DRL’s more calibrated model confidence helps to generate better pseudo-labels for self-training in UDA and cross-domain SSL. We also empirically demonstrate that the density ratios learned from our domain classifier reflect the hardness of an image, showing a positive correlation with the human selection frequencies. In the future, we are interested in relaxing the assumptions made and study different shifts in the DRL framework.
Density ratio estimation: Our density ratios are estimated from a differentiable domain classifier and is not guaranteed to match the true density ratios. Thus our density ratios should not be used in circumstances where true density ratios are absolutely required. However, we show that our density ratios do reflect the relation between the two distributions and benefit the downstream tasks.

Review summaries:
This paper focuses on the problem of uncertainty calibration under distribution shift. By using a domain classifier in the distributional robust learning (DRL) framework, the authors estimate the density ratio between the source and target domain to achieve well-calibrated predictions under domain shift. A regularized DRL framework is further proposed to promote smoothed model prediction and improve the calibration. Experiments on Office31, Office-Home, and VisDA-2017 demonstrate the superiority of DRL over empirical risk minimization (ERM) and the temperature scaling method measured by expected calibration error (ECE), Brier Score, and reliability plots.

This paper proposes scaled uncertainty prediction in the context of unsupervised domain adaptation. The problem in unsupervised domain adaptation is to obtain predictions in a target domain where no (or few) labels are available. A secondary problem then is the calibration of predictive uncertainties in the target domain. This paper proposes a model of uncertainty proportional to the (log) of density ratios between domains. Results include ECE and Brier scores on two common Domain Adaptation datasets.