D ISTRIBUTIONALLY R OBUST L EARNING FOR U NCERTAINTY C ALIBRATION UNDER D OMAIN S HIFT
Anonymous authors
Paper under double-blind review
A BSTRACT
We propose a framework for learning calibrated uncertainties under domain shifts.
We consider the case where the source (training) distribution differs from the target (test) distribution.
We detect such domain shifts through the use of binary domain classiﬁer and integrate it with the task network and train them jointly endto-end.
The binary domain classiﬁer yields a density ratio that reﬂects the closeness of a target (test) sample to the source (training) distribution.
We employ it to adjust the uncertainty of prediction in the task network.
This idea of using the density ratio is based on the distributionally robust learning (DRL) framework, which accounts for the domain shift through adversarial risk minimization.
We demonstrate that our method generates calibrated uncertainties that beneﬁt many downstream tasks, such as unsupervised domain adaptation (UDA) and semi-supervised learning (SSL).
In these tasks, methods like self-training and FixMatch use uncertainties to select conﬁdent pseudo-labels for re-training.
Our experiments show that the introduction of DRL leads to signiﬁcant improvements in cross-domain performance.
We also demonstrate that the estimated density ratios show agreement with the human selection frequencies, suggesting a match with a proxy of human perceived uncertainties.
I NTRODUCTION
Uncertainty estimation is an important machine learning problem that is central to trustworthy
AI (Tomsett et al., 2020; Antifakos et al., 2005).
In addition, many important downstream applications rely on the correct estimation of uncertainties.
This includes unsupervised domain adaptation (UDA) (Zou et al., 2019) and semi-supervised learning (SSL) (Sohn et al., 2020) where they are used to solicit conﬁdent pseudo-labels for re-training.
In these applications, reliable pseudo-labels help avoid error propagation and catastrophic failures in early iterations (Kumar et al., 2020).
Obtaining reliable uncertainty estimation is challenging.
In contrast to human annotation of labels, obtaining the ground-truth uncertainties from real-world data can be costly or even infeasible.
It is also known that commonly used uncertainty proxies in deep neural networks, such as the softmax output, tend to give overconﬁdent estimates (Guo et al., 2017a).
This overconﬁdence is further ampliﬁed under domain shifts, where the target (test) domain and the source training domain differ signiﬁcantly.
Such distributional shifts tend to aggravate the existing issues in uncertainty estimation, leading to wrong but overconﬁdent predictions on unfamiliar samples (Li & Hoiem, 2020).
Many methods have been proposed to calibrate the conﬁdence of deep learning models so that the uncertainty level of a model prediction reﬂects the likelihood of the true event (Guo et al., 2017a).
Label smoothing is a popular approach to reduce overconﬁdence and to promote more uniform outputs (Szegedy et al., 2016).
Temperature scaling is another method where the logit scores are rescaled by a calibrated temperature (Platt et al., 1999).
Approaches such as Monte-Carlo sampling (Gal & Ghahramani, 2016) and Bayesian inference (Blundell et al., 2015; Riquelme et al., 2018) model uncertainties from a Bayesian perspective but are computationally more expensive.
Even though these methods lead to more calibrated uncertainties, recent studies show that their results cannot be fully trusted under domain shift (Snoek et al., 2019).
Our approach.
To handle domain shifts, we characterize the “overlap” between the source training data and the test data.
Intuitively, if the test sample is highly unlikely in the training distribution,
[Table]
+srpXXKxubW9s75u5eW8WpJLRFYh7LuwArypmgLWDA6V0iKY4CTjvB8GpS79xTqVgsbmGcUC/CfcFCRjBo5JvHbigxyZp+5gIdQabyvDY6yX8AzIBvVu26PZW1aJzCVFGhpm9+ur2YpBEVQDhWquvYCXgZlsAIp3nFTRVNMBniPu1qK3BElZdND8qtI016VhhL/QRYU/p7IsORUuMo0J0RhoGar03gf7VuCuGFlzGRpEAFmS0KU25BbE3SsXpMUgJ8rA0mkum/WmSAdUKgM6zoEJz5kxdN+7Tu2HXn5qzauCziKKMDdIhqyEHnqIGuURO1EEEP6Am9oFfj0Xg23oz3WWvJKGb20R8ZH99sCp4S</latexit>
RepresentationLearning
DiscriminativeLearning
[Figure]
then the resulting conﬁdence levels should be lowered.
We incorporate this idea through the density ratio between the two domains and employ it for conﬁdence calibration.
To make our idea more concrete, recall that the probability output of a standard neural network for classiﬁcation can be expressed as P ( y | x ) ∝ exp ( θ y · φ ( x )) , where x is the predictor input, φ ( x ) is the data feature, and θ y is the model parameter of the y -th class.
Instead, we propose the following predictive form for our neural network:
[Equation]
where P s ( x ) and P t ( x ) are the densities of a data sample under the source and target distributions, respectively.
When a target sample is close to the source domain (large P s ( x ) /P
t ( x ) ), the prediction is conﬁdent.
However, when a target sample x is far away from the source distribution (small
P s ( x ) /P t ( x ) ), the conﬁdence is lowered and the prediction is closer to a uniform distribution.
This intuition is analogous to incorporating a sample-wise temperature to adjust the conﬁdence according to the closeness of a test sample to the training distribution.
Eq.
(1) is based on the distributionally robust learning (DRL) framework.
DRL is an adversarial risk minimization framework that involves a two-player minimax game between a predictor and an adversary (Gr¨unwald et al., 2004).
While many previous DRL methods (Liu et al., 2020; Nakka et al., 2020) operate in low dimensional spaces using kernel density estimators for the density ratio estimation, we develop a DRL method to scale to real-world computer vision tasks, which is able to produce calibrated uncertainties under domain shift.
Our contributions are :
1) We propose a DRL method for uncertainty estimation under domain shift.
We introduce a binary domain classiﬁer network which learns to predict the density ratios between source and target domains (See Fig. 1(a)).
The domain classiﬁer and the target classiﬁer are trained simultaneously in an end-to-end fashion.
We also introduce a regularized DRL framework to further promote smoothed model prediction and improve the calibration.
We show that the estimated density ratio reﬂects the distance of a test sample from both training and test distributions (Fig. 1(b)).
Our
experiments further empirically show that these estimates are also correlated with human selection frequency, based on the available ground-truth labels in ImageNetV2 (Recht et al., 2019a), which can be regarded as a proxy of human uncertainty perception.
3) We empirically show that the top-1 class predictions of DRL are more calibrated than the empirical risk minimization (ERM) and the temperature scaling on Ofﬁce31 (Saenko et al., 2010),
Ofﬁce-Home (Venkateswara et al., 2017), and VisDA (Peng et al., 2017).
We measure the level of calibration using expected calibration error (ECE), Brier Score and reliability plots.
We integrate our method as a plug-in module in downstream applications such as unsupervised domain adaption and semi-supervised learning, leading to signiﬁcant improvements.
For example, incorporating self-training (Zou et al., 2019) with DRL leads to state-of-the-art performance on VisDA-2017 (Peng et al., 2017) and a 6% improvement on hard examples.
Incorporating Fix- match (Sohn et al., 2020) with DRL improves the original Fixmatch by a relative 17% increase in accuracy under the cross-domain setting.
D ISTRIBUTIONALLY ROBUST LEARNING
In this section, we ﬁrst review the preliminaries of DRL (Sec. 2.1), followed by a proposed variant of DRL with class regularization (Sec. 2.2).
We then propose an instantiation of DRL with a differentiable density ratio estimation network (Sec. 2.3).
Finally, we show the application of DRL in
UDA and SSL tasks (Sec. 2.4).
2. P RELIMINARIES
Notations and deﬁnitions: We denote the input and labels by random variables X and Y .
We also use x ∈ R d and X to represent the realization and sample space of X .
Our goal is to ﬁnd a predictor
[Equation]
close to the true underlying P t ( Y | X ) .
Here, d , C and ∆ denote the input dimension, class number, and probabilistic simplex, respectively.
We consider the problem with labeled data sampled from a source distribution P s ( X, Y ) and unlabeled data sampled from a target distribution P t ( X ) , and use eP s ( X, Y ) to represent the empirical source distribution.
In this work, we consider an important form of domain shift with the covariate shift assumption
P s ( X ) (cid:54) =
P t ( X ) , P s ( Y | X ) =
P t ( Y | X ) .
Motivation: Traditional empirical risk minimization (ERM) frameworks tend to fail under covariate shift since ERM empirically learns a predictor ˆ P s ( Y | X ) from the ﬁnite source data that usually fails to generalize to the target distribution.
DRL was previously proposed to overcome this issue.
DRL can be formulated as a two-player adversarial risk minimization game (Gr ¨ unwald et al., 2004) with the predictor player minimizing a loss, while the adversary player maximizing the loss.
The adversary is allowed to perturb the labels, subject to certain feature-matching constraints to ensure data-compatibility.
Formulation: Under covariate shift, DRL (Liu & Ziebart, 2014) deals with the mismatch between the expected loss and the training data, and is deﬁned on the target data distribution:
[Equation]
where f ( x ) , g ( x ) ∈ R C are the conditional label distributions given an input x and F , G are the entire distribution over all the input.
E x ∼ P t L ( · ) is an expected log loss on the target input deﬁned:
[Equation]
F is the predictor player minimizing the loss function while G is the adversary maximizing the loss function.
After solving this game, F is our estimate of ˆ P t ( Y | X ) , which we will use for the classiﬁcation on the target domain.
Eq. 3 is deﬁned on the target domain only.
How could a predictor be properly trained while there are no target labels available?
The answer is that the adversary G is implicitly constrained by the source features.
We use the following constraints to make sure that G is close to P s ( Y | X ) :
[Equation]
where x i ∼ P s ( X ) and g y is the y -th dimension of g .
Eq.
5 is a necessary but not sufﬁcient condition for G
=
P s ( Y | X ) , thus serving as an implicit constraint for G to be close to the true
P t ( Y | X ) under the covariate shift assumption.
Given a predeﬁned feature function φ , when the adversary perturbs the conditional label distribution, certain aggregate function of φ on g should equal to the counterpart on the empirical source data.
From DRL to density ratio: When using the expected target logloss in Eq. 3, Eq. 1 is derived by solving the predictor F .
Here we refer the derivation details to (Liu & Ziebart, 2014) but emphasize the important properties of the prediction: representation-level conservativeness.
Representation-level conservativeness:
The predictions have higher certainty for inputs closer to the source domain when P s ( x ) /P
t ( x ) is large.
On the contrary, when P s ( x ) /P t ( x ) is small, the prediction is more uncertain.
This property reﬂects the model’s ability to convey information about what it does not know through the model uncertainty.
2. C LASS - REGULARIZED DISTRIBUTIONALLY ROBUST LEARNING
Inspired by label smoothing (Szegedy et al., 2016) and regularization of neural network outputs (Pereyra et al., 2017), we can further add class-regularization to the conservative predictions in
Eq. 1.
Instead of doing it in a post-hoc way, we study how to incorporate class-regularization into the DRL framework.
We propose to use a weighted logloss to penalize high conﬁdence in the adversary’s label prediction:
[Equation]
where y is the one-hot class vector, (cid:12) is the element-wise product, and r ∈ [0 , 1] is a hyperparameter that controls the level of regularization.
Σ here is the same as in 3.
We observe the whole loss function is a convex-concave function in terms of f and g .
According to the strong duality, we switch the order of the min and max .
With a ﬁxed g , f = g is the optimal solution of the inner min problem.
So we have the following lemma (we refer all the proofs to appendix Sec.
A):
Lemma 1.
Eq.
6 can be reduced to a regularized maximum entropy problem with the estimator constrained:
[Equation]
where Σ is the same as Eq. 5, meaning F should be close to the empirical source P s ( Y | X ) .
Theorem 1.
The solution of Eq.
7 for takes the form: f θ,r ( y | x ) ∝ exp
[Equation]
I ( y )+ (cid:19) , where θ represents the model parameters and I ( y ) is the y th dimension of the one-hot encoding y .
The proof of Theorem 1 follows the same principles when Eq. 1 is derived from Eq.
The training process of this model also follows the same procedure in (Liu & Ziebart, 2014), except that we use
Theorem 1 to compute the prediction results.
In training, we can compute the gradients using source labels where y is the one-hot encoding of each class.
In testing, we set y to be an all-one vector.
Class-level regularization: Hyper-parameter r adjusts the smoothness of g ’s label prediction in
Eq.
It translates to the r y terms in the prediction form.
Intuitively, this regularization increases the correct label’s prediction result when it is small and decreases it when it is large, using a threshold of 1 to discriminate between ‘large’ and ‘small’.
This provides additional regularization and smoothness to the conservative prediction.
2. D IFFERENTIABLE DENSITY RATIO ESTIMATION
Estimating P s ( x ) /P
t ( x ) (Sugiyama et al., 2012) can be challenging, especially in high-dimensional spaces.
Usually a plug-in estimator is used.
But the plug-in estimates are usually sub-optimal for downstream tasks due to the different objectives.
We propose an end-to-end training procedure for
DRL such that the density ratio estimator is trained together with the target classiﬁer .
The key insight here is that we use two neural networks for classiﬁcation and differentiating the two domains, respectively.
See Fig. 1(a).
We now ﬁrst introduce differentiable density ratio estimation before introducing joint training loss and the parameter learning of the two networks.
Differentiable density ratio estimation:
Based on the Bayes’ rule, P s ( x ) /P t ( x ) can be computed from a conditional domain classiﬁer (Bickel et al., 2007):
P t ( x ) P s ( x ) = P ( x | t ) P ( x | s ) =
P ( t | x ) P ( s ) P ( s | x ) P ( t ) .
Concretely speaking, they can be estimated through binary classiﬁcation using unlabeled source and target data with P ( s ) P ( t ) as a constant relating to number of source and target samples.
On the other hand, we observe that P s ( x ) /P
t ( x ) can be a trainable weight for each sample, and can receive gradients from the training objective of DRL.
Therefore, we propose to train a discriminative neural network to differentiate the two domains, which receives training signals from both the target classiﬁcation loss in DRL and the binary classiﬁcation loss.
The weights trained this way lost their original properties as density ratios, but still reﬂects the relation between the two domains.
Joint training loss: Assume φ ( x , w r ) is the representation learning neural network with parameter w r .
We further deﬁne P d ( X ) as the joint distribution of both source data and target data with their domain labels D
= { d ( x ) } .
We denote τ ( x , w d ) =
( τ s , τ t ) (where τ s + τ t = 1 ) as the probability output of the source and target domains from a domain classiﬁer with parameter w d .
Our joint training loss is deﬁned as:
[Equation]
where g t ( x ) =
P t ( Y | X = x ) and f ( x ; w r , θ , r d ) takes the form in Theorem 1:
[Equation]
Parameter learning : Note that g t in the ﬁrst loss term of Eq.
(8) concerns the conditional label distribution on target which is assumed to be not available.
However, with the DRL formulation, we can compute and evaluate the gradients of θ and φ ( x , w r ) directly with the change of measure in the derivation of the gradients (Liu & Ziebart, 2014), since the gradients are only associated with the source data and labels.
We show this in Fig. 1(a) that the representation learning network φ only has source data as the input.
We then update θ and w r using the computed gradients and also directly back-propagate from the second loss term to update w d .
Finally, we treat the densities as trainable variables and derive gradients for them from the ﬁrst loss term (details shown in appendix Sec. B).
By the Bayes rule, P s ( x ) P t ( x ) =
τ s P ( t )
τ t P ( s ) , where P ( t ) and P ( s ) are the amount of unlabeled data from each of the domain during the training process.
Since we usually use the same amount of source and target data in each batch, they cancel out by following P ( s ) P ( t )
= 1 .
Then f in Theorem 1 is reduced to Eq.
Therefore, besides the binary classiﬁcation loss, the parameter w d of the discriminative network is also trained with gradients from the ﬁrst loss term.
Algorithm 1 summarizes the procedure.
[Table]
based UDA or SSL methods with DRL.
The unsupervised losses represent the loss function user can impose on the unlabeled target data.
DRST conducts this procedure for multiple iterations, while DRSSL minimizes the unsupervised losses on the augmented target data.
2. A PPLICATIONS TO UDA AND SSL
We show how to incorporate end-to-end DRL within a framework that takes unlabeled data for training.
We introduce the general setting with two examples: self-training UDA and cross-domain
SSL, followed by a new self-training algorithm DRST and a new SSL algorithm DRSSL.
General settings: In many cases, the source domain where models are trained may have abundant labels but the target domain lacks enough labels.
Typical problems under this setting include unsupervised domain adaptation and (Zou et al., 2019) and semi-supervised learning (Sohn et al., 2020).
In both cases, a common strategy is to treat the prediction results on the target data as pseudo labels to train the model on target data.
In these methods, model conﬁdence (softmax output) is often leveraged as the proxy to rank the reliability of pseudo labels, with the underlying assumption that there exists a positive correlation between model conﬁdence and pseudo label quality.
However, such an assumption requires accurate uncertainty estimation to avoid false usage of wrong pseudo labels which may poison the training.
To this end, we incorporate DRL into these frameworks with to provide calibrated uncertainties.
An illustration of this setting is shown in Figure 2.
Distributionally robust self-training: In UDA, we are given labeled source data and unlabeled target data and aim to achieve adaptation from the source to the target domain.
Self-training is an effective method for UDA (Zou et al., 2019), where the training procedure in Fig.
2 is conducted multiple times.
Here we propose DRST to plug the class-regularized DRL model into self-training.
The idea is to regard each training epoch as a new domain shift problem in DRL.
After each training epoch, we make predictions on the target domain and select more conﬁdent data and generate pseudo-labels for them to merge into the source training data.
Then the labeled source data and the newly pseudo-labeled target data become the new source set for the next training epoch.
Distributionally robust semi-supervised learning: In cross-domain SSL, we have few labeled source data and many unlabeled target data.
We aim to utilize unlabeled data in the target domain to help representation learning and save the effort needed for labeling.
One simple and effective strategy is to use pseudo-labels generated from weakly-augmented data to supervise stronglyaugmented data (Sohn et al., 2020).
Here ‘weakly’ means simple ﬂip-and-shift data augmentation while ‘strongly’ follows the same strategy with (Sohn et al., 2020).
Using DRL’s prediction, we propose DRSSL, which assigns pseudo-labels more conservatively.
We plug DRL into Fixmatch (Sohn et al., 2020).
Here the unsupervised loss in Fig.
2 is the ‘consistency loss’ in (Sohn et al., 2020): L u
[Equation]
)) , where x wt and y wt represent the weakly-augmented target data, x st and y st represent the strongly-augmented version of the same image data, and τ is a threshold for generating pseudo-labels ˆ y wt .
E XPERIMENTS
We evaluate our method on benchmark datasets.
DRL is evaluated as a method providing more calibrated uncertainties (Sec. 3.1), DRST as an effective UDA method (Sec. 3.2) and DRSSL as a cross-domain SSL method.
We show additional results and details in appendix Sec.
C, D.
Datasets and methods: We use Ofﬁce31 (Saenko et al., 2010), Ofﬁce-Home (Venkateswara et al., 2017) and VisDA (Peng et al., 2017) for evaluating DRL’s uncertainties.
We compare DRL with temperature scaling (TS), VADA (Shu et al., 2018) and source-only.
We also train models using
ImageNet (Deng et al., 2009) as the source domain and ImageNetV2 (Recht et al., 2019a) as the target domain to check the relationship between our estimated weights and the human selection frequencies (HSF).
VisDA2017 is used to evaluate DRST, for which we compare with (1) traditional UDA baselines: MMD (Long et al., 2015), MCD (Saito et al., 2018b) and ADR (Saito et al., 2018a); (2) recent self-training UDA baselines: CBST (Zou et al., 2018) and
CRST (Zou et al., 2019); (3) other uncertainty quantiﬁcation or UDA methods combined with self-training baselines:
AVH (Chen et al., 2020a)
+ CBST and DeepCORAL (Sun & Saenko, 2016)+CBST.
In addition, we use CIFAR10, STL10 (Coates et al., 2011), MNIST (Lecun & Bottou, 1998) and SVHN (svh, 2011) to construct settings with few source labeled data and much unlabeled target data and show
DRSSL’s advantages in cross-domain SSL over Fixmatch (Sohn et al., 2020).
[Table]
Evaluation metrics: Apart from accuracy, we also use Brier score (Brier, 1950), expected calibration error (ECE) (Guo et al., 2017a), and reliability plot (Dawid, 1982;
Guo et al., 2017a; Naeini et al., 2015) to evaluate the performance of our proposed method and the baselines.
Brier score measures the mean squared difference between the predicted probability assigned to the possible outcome and the actual outcome.
Despite potential problems with ECE (Nixon et al., 2019), it is still the most prevalent metric for the top-1 prediction.
ECE is deﬁned as the sum of average difference between prediction accuracy and conﬁdence of different conﬁdence bins (we use 10 bins in practice).
For both the Brier score and ECE, the lower the score, the better the model is calibrated.
Experiment setup: For Ofﬁce31 and Ofﬁce-Home tasks, we use
ResNet50 (He et al., 2016) as the backbone for all models to make the comparison fair.
We train by SGD for 100 epochs and set the learning rate to 0 .
For VisDA, we use the ResNet101 (He et al., 2016) backbone and also the SGD optimizer.
During the 20 epochs of training, the initial learning rate is set as 10 − 5 and the weight decay parameter is set as 5 × 10 − 4 .
For ImageNet, we follow the standard training process of AlexNet (Krizhevsky et al., 2012) and
VGG-19 (Simonyan & Zisserman, 2014), where the initial learning rate is 0 .
and we decay the learning rate by a factor of 10 every 30 epochs.
[Figure]
3. C ALIBRATED UNCERTAINTIES FROM DRL
DRL’s calibrated conﬁdence: Fig.
4 demonstrates our more calibrated uncertainty.
DRL tends to be underconﬁdent and conservative but still stays closer to the calibration line (dashed line).
Tablealso shows our better ECE on ImageNetV2 in different human selection frequency (HSF) bins.
Our additional results in appendix Sec.
C show that the accuracy of DRL is also competitive.
Density ratio v.s. human selection frequency: Fig.
3 shows that the estimated weight is positively correlated with the human selection frequency (HSF) in ImageNetV2 (Recht et al., 2019b) under different neural network architectures.
Here we regard [ 0 , 0 . 2 ] as low HSF and (0 . 2 , 1] as high HSF.
Images with low HSF have smaller weights, indicating that they are visually harder for humans to select.
3. U NSUPERVISED DOMAIN ADAPTATION
Initialization with ASG:
The quality of the source model has a signiﬁcant impact on the ﬁnal performance.
The recently proposed automated synthetic-to-real generalization (ASG) model using
ImageNet pretrained knowledge can be used to improve synthetic training (Chen et al., 2020b).
Initializing with ASG trained model leads to signiﬁcant improvements in self-training.
Hence, we use the ASG pretrained model as an initialization, with results shown in Table 2’s last four lines.
Accuracy and calibration: Fig. 5(a) and Table 2 show that DRST performs best in accuracy.
Our vanilla version of DRST outperforms CRST by over 5% with ASG initialization.
We improve the
SOTA self-training accuracy on VisDA by over 1%.
In the rest of the results, we use DRST to represent DRST-ASG.
Fig.
5(b) shows that DRST also achieves higher calibrated conﬁdence.
[Table]
[Figure]
Ablation study: Fig.
5(a)(b) include two ablation methods.
In the ﬁrst ablation, we set r to 0 so that there is no class regularization in DRL (“r = 0”).
The prediction then follows the form in Eq.
In the second ablation, we set the weights to 1 to mute the differentiable density ratio estimation in our method so that there is no representation level conservativeness (“R = 1”).
DRST achieves the best performance when both components are present.
Covariate shift: Fig. 5(c) shows P s ( φ ( x )) /P
t ( φ ( x ))
− P s ( φ ( x ) , y ) /P
t ( φ ( x ) , y ) using discriminative density ratio estimators (per class).
This serves as a proxy of covariate shift as it becomes 0 when covariate shift holds.
We can see that the gap decreases with self-training, which shows that even though covariate shift may not hold in the beginning, self-training helps to promote this assumption with better aligned domains and more discriminative feature distributions.
Improvement on hard examples: Fig.
5 (d) demonstrates that DRST achieves larger performance gain compared to the baselines on target samples with smaller weights.
Recall that data is not wellrepresented in the source domain when P s ( x ) /P
t ( x ) is small (Fig. 1(b)).
Therefore, DRST provides robust performance on harder examples in the target domain.
Density ratios: In Fig. 1(b), a harder example obtains a lower density ratio due to its vague shape.
Our density ratios reﬂect the closeness of a sample to the two domains.
More examples are in appendix Sec.
C. Moreover, the magnitude of our estimated density ratios are modest in general within the range [0 .
This is due to the regularization by the other network’s learning signals.
Improved attention: Fig.
6 visualizes the network attention of DRST and the baselines using GradCAM (Selvaraju et al., 2017), where DRST renders improved attention with better object coverage.
3. D ISTRIBUTIONALLY ROBUST SEMI - SUPERVISED L EARNING
[Table]
We use two pairs of source and target domains in this experiment: Source: CIFAR10 / Target:
STL10 and Source: MNIST / Target: SVHN.
Table 3 show that DRL-powered SSL method improves the original Fixmatch signiﬁcantly in cross-domain semi-supervised learning tasks.
Note our setting is different from UDA where source labeled data is abundant.
SSL focuses on using unlabeled augmented data to help learning from few labeled data.
In the CIFAR10 to STL10 case, we only have 4k labeled source data.
For MNIST to
SVHN, we have 40k labeled source data.
The results show that DRL is beneﬁcial for generating high-quality pseudo supervision for unlabeled data under the cross-domain SSL setting.
R ELATED WORK
Distributionally robust learning: Several different learning algorithms can be derived from this framework with different loss functions and feature constraints (Liu & Ziebart, 2014; 2017; Chen et al., 2016; Fathony et al., 2016; 2018).
One limitation of this line of work is that it is usually only suitable for low-dimensional data.
The other inconvenience is that it requires the density ratio to be estimated beforehand.
Our work instead deals with high-dimensional data and estimates the density ratio along with the learning process.
Another line of work perturbs the covariate variable (Ben-Tal et al., 2013; Shaﬁeezadeh Abadeh et al., 2015; Sinha et al., 2018;
Hu et al., 2018; Najaﬁ et al., 2019) and focuses on model robustness against adversarial perturbations.
In our work, we focus on the beneﬁts of DRL’s uncertainty estimation for downstream tasks like UDA and SSL.
We apply neural networks and propose an end-to-end learning method to handle image data.
UDA and SSL: There is a rich literature in UDA focusing on representation alignment between the source and target domains (Fernando et al., 2014; Pan et al., 2010; Sun et al., 2016;
Wang et al., 2018; Baktashmotlagh et al., 2013; Gretton et al., 2012; Na et al., 2021).
It is also prevalent to use a discriminator network to differentiate two learned representations (adversarially) to locate such a feature space (Ganin et al., 2016; Tzeng et al., 2017; Ajakan et al., 2014; Long et al., 2018;
Sankaranarayanan et al., 2018).
Some other works focus on more speciﬁc shift assumptions, such as covariate shift (Shimodaira, 2000) and label shift (Lipton et al., 2018;
Azizzadenesheli et al., 2019).
For the covariate shift case, even though various density ratio estimation methods were investigated before (Sugiyama et al., 2012), only few works explore the possibility to apply them to high-dimensional data (Khan et al., 2019; Moustakides & Basioti, 2019; Park et al., 2020).
Recently, multiple self-training methods have been proposed for UDA (Zou et al., 2018; 2019) and many theoretical understandings of self-training are also developed (Kumar et al., 2020; Chen et al., 2020c).
Similarly, rich literature also exist in pseudo-labeling for SSL (Berthelot et al., 2019; Arazo et al., 2020; Miyato et al., 2018; Sohn et al., 2020).
In this paper, we focus on utilizing DRL’s model uncertainty to help with choosing better pseudo-labeled data from the target domain in UDA and cross-domain SSL.
Uncertainty quantiﬁcation with deep models: Bayesian methods largely contribute to complementing deep learning with uncertainty quantiﬁcation (Gal, 2016;
Welling & Teh, 2011; Kingma & Welling, 2013; Louizos & Welling, 2017; Blundell et al., 2015; Riquelme et al., 2018; Gal &
Ghahramani, 2015; Gal et al., 2017; Kendall & Gal, 2017; Mandt et al., 2017).
However, with a large number of parameters, Bayesian deep learning may suffer from computational inefﬁciency.
Other calibration methods include temperature scaling (Platt et al., 1999; Guo et al., 2017b), deep ensembles (Lakshminarayanan et al., 2017), calibration regression (Kuleshov et al., 2018), quantile regression (Romano et al., 2019) and trainable calibration measure (Kumar et al., 2018).
These methods do not apply to our setting because we do not have access to the target labels.
Recent work that focus on calibrating or uncertainties under domain shift either only focus on “using” the uncertainties not “generating” the uncertainties (Han et al., 2019; Lee & Lee, 2020; Kurmi et al., 2019) or focus on the importance weighting setup (Park et al., 2020; Wang et al., 2020), where the density ratio reweights each example with target over source densities.
In this paper, we utilize the model conﬁdence from the DRL framework and introduce the domain classiﬁer for differentiating source and target domains, which can be fulﬁlled by end-to-end learning.
C ONCLUSION AND DISCUSSION
In this paper, we have studied uncertainty estimation under distribution shift using the distributionally robust learning framework.
We demonstrate that density estimation can be integrated into the learning process by using a domain classiﬁer.
We propose differentiable density ratio estimation and develop end-to-end training techniques for our proposed method.
Using DRL’s more calibrated model conﬁdence helps to generate better pseudo-labels for self-training in UDA and cross-domain
SSL.
We also empirically demonstrate that the density ratios learned from our domain classiﬁer reﬂect the hardness of an image accurately, matching the human selection frequencies.
Density ratio estimation: Our density ratios are estimated from a differentiable domain classiﬁer and is not guaranteed to match the true density ratios.
Thus our density ratios should not be used in circumstances where true density ratios are absolutely required.
However, we show that our density ratios do reﬂect the relation between the two distributions and beneﬁt the downstream tasks.
Covariate shift assumption: Covariate shift is a relatively strong assumption in real-world applications.
However, this can be partially remediated with stronger and more compositional features φ ( x ) .
We analyzed how well this assumption holds in Fig. 5(c).
In practice, users are encouraged to judge whether the assumption holds to see whether the proposed method is applicable.