Title: UNSUPERVISED TEMPERATURE SCALING: ROBUST POST-PROCESSING CALIBRATION FOR DO-MAIN SHIFT

Abstract: The uncertainty estimation is critical in real-world decision making applications, especially when distributional shift between the training and test data are prevalent. Many calibration methods in the literature have been proposed to improve the predictive uncertainty of DNNs which are generally not well-calibrated. However, none of them is specifically designed to work properly under domain shift condition. In this paper, we propose Unsupervised Temperature Scaling (UTS) as a robust calibration method to domain shift. It exploits unlabeled test samples instead of the training one to adjust the uncertainty prediction of deep models towards the test distribution. UTS utilizes a novel loss function, weighted NLL, which allows unsupervised calibration. We evaluate UTS on a wide range of model-datasets to show the possibility of calibration without labels and demonstrate the robustness of UTS compared to other methods (e.g., TS, MC-dropout, SVI, ensembles) in shifted domains.
Under review as a conference paper at ICLR 2020 2018) are three examples of applications that have expensive labeling procedure with high risk of labeling noise that need senior experts to label them.
In this work, we propose a new approach called Unsupervised Temperature Scaling (UTS) with similar framework of TS and using unlabeled test samples for calibrating the pre-trained model. This novel idea brings the chance of robust calibration to the domain shift. Possibility of using the test samples to calibrate, makes UTS a proper solution not only for domain shift but also for many practical calibration problems like calibrating off-the-shelf-models. More specifically, UTS is proposed with following contributions and foreseen impacts:
• Unsupervised post-processing calibration: UTS brings a new look to NLL loss function which is used as the calibration loss in several post-processing methods. UTS approximates a weight function to estimate the per class distribution of data to compute NLL. This new way of computing NLL makes it independent of the labels. In addition, computing weighted NLL has the same order of time and memory complexity as the classic NLL, without any additional hyper-parameter to fine-tune. • Robustness to the domain shift: UTS is a robust calibration solution to shifted domains.
It adjusts the the model uncertainty based on the test and not the training domain. Therefore by change of distribution in the test domain, UTS can follow the distribution shift easily. • Calibration of off-the-shelf models: The pre-trained models for classification tasks are usually only trained to achieve higher accuracy rate without paying attention to the predictive uncertainty. In fact, many of them are released without the training data that removes the possibility of retraining them to calibrate, like available Pytorch pre-trained models. UTS brings a chance to use these models for decision-making applications by calibrating them on a test data without need of labeling them.
Calibration of predictive uncertainty for DNNs are widely investigated in recent literature. Calibration methods can be categorized in two groups: probabilistic or post-processing approaches.

1. INTRODUCTION:

The predictive distributions provided by Deep Neural Networks (DNNs) have been increasingly used for decision-support systems, for applications ranging from medical diagnoses assistance (Esteva et al., 2017) to self-driving cars (Bojarski et al., 2016) . In DNNs, the predictive distributions usually corresponds to the output of a softmax layer, which is typically interpreted as the confidence over the different classes. The i.i.d hypothesis made in learning usually assumes that the data distributions over the classes are the same at learning and inference time. However, in real-world applications, the distribution of data at inference time (i.e., the test data) may shift and actually be different from the original training distribution -corresponding to distribution shift in representation of data which we refer that as domain shift. For instance, in image classification problem, domain shift happens when the test images are different in illumination, view point, resolution, background or intensity noise from the training set. However, they are the same classification problem with the same objects occurance rate. Arguably, building DNNs that are robust to the domain shift problem is necessary for its safe deployment in decision-making systems. Dealing with this, predictive uncertainty is the key to obtain a meaningful estimation for practitioners to know when prediction accuracy is degrading and allows a system to abstain from making decisions due to low confidence.

The predicted uncertainty in DNNs usually are not calibrated with a tendency to be overconfident. Many probabilistic and post-processing calibration methods have been proposed under i.i.d assumptions to adjust the certainty of DNNs. In recent studies (Ovadia et al., 2019; Maddox et al., 2019) , uncertainty under domain shift condition gets more attention and the common calibration methods have been assessed regarding to the domain shift, although they are not designed to be robust under such condition. In this paper, for the first time, we specifically focused on calibration for the domain shift in image classification. We show post-processing calibration approaches that use Negative Log Likelihood (NLL) as the calibration loss like Temperature Scaling (TS) (Guo et al., 2017 ) may become robust to the domain shift problem if they calibrate the model using the test samples. However, they need labels of the samples to apply calibration. Labeling the test samples even for the small set is not always an easy task and need human experts effort which can be accompanied with the labeling noise and huge time burden. Neuron cells classification taken by electron microscope (Ostroff & Zeng, 2015) , pathology images (Khosravi et al., 2018) and skin disease classification (Kolkur et al., Post-processing approaches are much less complex, albeit less accurate compared to probabilistic calibration. In post-processing approaches, the main idea is to decrease the miscalibration of the network by minimizing a calibration loss (Gneiting & Raftery, 2007 ) such as NLL. In order to train the neural network, NLL is used to simultaneously increase accuracy and decrease miscalibration. However, it easily gets overfitted to confidence and makes the network overconfident (Guo et al., 2017) . Post-processing approaches like TS, Platt-Scaling (Platt et al., 1999) , Histogram Binning (Zadrozny & Elkan, 2001) , Isotonic Regression (Zadrozny & Elkan, 2002) , and Bayesian Binning into Quantiles (Naeini et al., 2015) fine-tunes the softmax layer by keeping the DNNs' weights unchanged. They do not need to retrain the deep network from scratch and they only need to find the best parameter of softmax softening function by minimizing a calibration loss (like NLL) on a small validation set. Temperature Scaling is the state-of-the-art among the post-processing approaches which uses NLL as the loss function. It only uses one parameter T to rescale the logit layer and soften the softmax output. Therefore with keeping the accuracy unchanged it can calibrate the model with the minimum time and memory complexity. These features leads us to focus on TS and try to propose a robust post-processing solution for domain shift based on TS framework.

Robustness to the domain shift: Previously, the results of calibrated model were also reported for different domains such as Out-Of-Distribution (OOD) and Adversaries (Lakshminarayanan et al., 2017; Ritter et al., 2018b) to show the model is uncertain about what it does not learn before. Recently, people get into importance of domain shift problem in calibration and assess how the calibrated methods would behave under domain shift condition (Ovadia et al., 2019; Maddox et al., 2019) . Domain shift concept is different from adversaries and OOD. In the case of OOD, training and test domains are completely different in task distributions and in the case of adversaries the distribution shifts between the training and test is made with the goal of fooling the classifier. In domain shift, the training and test domains are distributionally different but related. The relation between two domains can be used as the prior knowledge to help improving the accuracy or having better calibration. In the literature of calibration, to the best of our knowledge, there is no work that specifically designed to calibrate the model considering domain shift assumptions. In this paper we will focus on Covariate shift as the most famous domain shift setting in image classification and propose UTS as a robust calibration method accordingly.

3. PRELIMINARIES:

In this section, we define the domain shift and calibration setup to clarify UTS objectives. Then, we explain why NLL can be used as a calibration loss and when optimizing NLL will lead to a calibrated model toward the domain shift settings. Finally, we bring a deep analysis of the postprocessing method TS which uses NLL as a calibration loss. We show TS can be a robust calibration solution to the domain shift if it uses labeled samples from the test domain to apply calibration. We discuss TS sensitivity to the labels of samples as the preliminaries to propose UTS method in the next section.

3.1. PROBLEM SETUP:

Considering the domain shift assumptions, the goal of calibration in this work is to improve uncertainty estimation of a pre-trained model for different shifted domains. In this setting, q s (x, y) is considered as the ground-truth distribution of the source domain and q t (x, y) is considered as the ground-truth distribution of the target domain where x ∼ X ∈ R d and y ∈ {1, 2, . . . , K}.

In the setting of domain shift, the source and target domains have different but related distributions. The relation between the domains is defined by Covariate Shift assumption (Adel & Wong (2015) ) which is: the data distributions q s (x, y) = q t (x, y) where the conditional distribution q s (y|x) = q t (y|x), and the task and marginal distributions q s (y) = q t (y) and q s (x) = q t (x), respectively. Let d(x) = {S y (x), ŷ} denotes the pre-trained model in which ŷ is the class prediction and S y (x) is its associated confidence. In domain shift setting, for deep neural networks, model d(•) is a DNN trained on the source domain and would be tested on the target domain. In this setting, S y (x) is the output of the softmax layer which is calibrated when S y (x) = q t (y|x).

Miscalibration of DNNs can be explained in different ways. Temperature Scaling models the miscalibration as the rescaled logit layers by scaling factor T * . TS objective is to find T * value to rescale the logit layer back and makes the model calibrated. More specifically, the calibrated output of TS is defined as S y (x; T * ) = exp(

fy(x) T * )/ K j=1 exp( fj (x) T * ) where f (x) = [f 1 (x), f 2 (x), . . . , f k (x)]

is the logit layer of model d(x). In this paper, considering the same definition of miscalibration as TS, we propose UTS under domain shift condition. UTS objective is to find the scaling factor T * that S y (x; T * ) = q t (y|x), given that we have access to the source pre-trained model d(•), unlabeled calibration set C = {x i } L i=1 ∼ q t (x) and the known task distribution q s (y).

3.2. ROBUSTNESS TO DOMAIN SHIFT WITH NLL LOSS FUNCTION:

To calibrate a model, first we need to evaluate the quality of predicted uncertainty of the model. Evaluating the quality of predictive uncertainty is challenging, as the ground-truth of the uncertainty estimate is usually not available. Accordingly, scoring rules are defined to measure the quality of predictive uncertainty (Gneiting & Raftery (2007) ). Scoring rules are numerical scores that rank the distribution prediction p θ (y|x) by giving lower score to better prediction of true distribution q(y|x).

Let a scoring rule be a function R(p θ , (x, y)) that evaluates the quality of the predictive distribution p θ (y|x) based on the samples (x, y) ∼ q(x, y) where q(x, y) is the true distribution of the data.

The expected scoring rule is defined by R(p θ , q) = q(x, y)R(p θ , (x, y))dydx. A proper scoring rule function is one where R(p θ , q) ≤ R(q, q) with equality if and only if p θ (y|x) = q(y|x) for all samples.

Negative Log Likelihood (NLL) is a proper scoring rule based on Gibbs inequality i.e., R(p θ , q) = E q(x) q(y|x) log p θ (y|x) ≤ E q(x) q(y|x) log q(y|x). Therefore minimizing NLL w.r.t θ on the samples generated from q(x, y) distribution, leads to p θ (y|x) → q(y|x). Under the domain shift assumption, as the training and test domains have different distributions, the final goal of calibration is p θ (y|x) = q t (y|x). In the case of using NLL as the loss function, if we minimize NLL on the samples that are generated from the test domain, we will have p θ (y|x) → q t (y|x) and makes the method robust to the domain shift. One of the post-processing methods that uses NLL as the loss function is TS. Therefore, TS has the ability to get robust to the domain shift.

3.3. TEMPERATURE SCALING ANALYSIS:

TS (Guo et al. (2017) ) is the state-of-the-art post-processing approach which rescales the logit layer of a deep model by parameter T that is called temperature. TS is used to soften the output of the softmax layer and makes it more calibrated. The best value of T will be obtained by minimizing NLL loss function (explained in Sec.(3.2), why minimizing NLL leads to more calibrated model) respecting to T conditioned by T > 0 on the calibration set as defined in Eq. (1):

EQUATION

where S yi (x i ; T ) = exp(

fy i (xi) T )/ K j=1 exp( fj (xi)

T ), is the softed version of softmax by applying parameter T on the logit layer f j (x). TS has the minimum time and memory complexity with order of O(1) among calibration approaches as it only optimizes one parameter T on small labeled calibration set. Having only one parameter helps TS not only to be efficient and practical but also not to get overfitted to NLL loss function when it is optimized on small calibration set. TS previously is applied for calibration (Guo et al. (2017) ), distilling the knowledge (Hinton et al. (2015) ) and enhancing the output of DNNs for better discrimination between the in and out distribution samples (Liang et al. (2017) ). TS models the uncalibration as the rescaling factor in the logit layer. By computing the derivative of NLL respecting to T in Eq. ( 1), and find the minimum, we will have:

EQUATION

It shows regarding to the true label of the samples, TS selects the T value which maximizes the S k (x i , T ) for k = y i and minimize S k (x i , T ) for all the other k = y i . Therefore for correctly classified samples that y i = arg max y S y (x i ), T approaches 0 to increase the confidence of that class toward 1 and for the misclassified samples, T goes toward ∞ to decrease the confidence of predicted label. The balance between the correctly classified and misclassified samples brings back the optimal value of T .

TS uses NLL as the loss function then by selecting the calibration set from the test domain instead of the training one, we can make TS robust to the domain shift problem. We refer to this approach as TS-Target. TS is highly dependent on the labels of the samples. however, Labeling the test samples is a challenging task. When the calibration set contains data with labeling noise or unlabeled samples, TS loses the balance to find the optimal T * T S and cannot calibrate the network successfully. Later, in Sec. (5.3), we will show even a small portion of the noise has big distortion in TS results. This brings the idea of Unsupervised Temperature Scaling which makes TS independent of labeled data and robust to the domain shift.

4. UNSUPERVISED TEMPERATURE SCALING:

Considering the assumptions in Sec. (3.1), its main objective is to find T * U T S for the unlabeled calibration set. UTS the same as TS uses NLL as the proper scoring rule to minimize the calibration gap and finds the optimal T value. The first step of using NLL in UTS is to make NLL independent of labeled data. Considering NLL loss function, we can rewrite it with focus on per class distribution, formalized as:

EQUATION

In Eq. ( 3), NLL is the summation of K different sample sets, generated from class distribution q(x, y = k) where k ∈ {1, 2, . . . , K}. When the labels of the samples are available, they can be used as the guide to select the samples set for each class distribution. But in the absence of the labels, the main question is how to select the samples generated from q t (x, y = k) and calculate NLL. To come along with this challenge, instead of selecting the samples by labels, UTS applies weights on the samples. The weight function Ŵk (x; w * ) represents the probability that sample x is drawn from q(x, y = k). Later, in Sec. (4.1) we will give specific details of the weight function Ŵk (x; w * ) and how to approximate it. By applying weights on the samples, the UTS loss function is defined as the Weighted NLL (WNLL) which is:

T * U T S = arg min T WNLL - K k=1 L i=1 Ŵk (x i ; w * ) log (S k (x i ; T )) s.t : T > 0, {x i } L i=1 ∈ C ∼ q(x) (4) 4.1 WEIGHT FUNCTION Ŵk (•; •)

We start the discussion about the weight function by introducing a fact from the Bayes rule (Jin et al. ( 2017)):

q(x, y = k) = q(y = k|x) q(y = k|x) q(x, y = k) (5)

When a sample has a true label of y = k it is drawn from distribution q(x, y = k). However, Eq.( 5) shows that even samples with true label of y = k can be used as the samples drawn from the distribution q(x, y = k) by applying weight of q(y = k|x)/q(y = k|x). Therefore, we can simply use the weight of 1 for the samples with true label of y = k and the weight of q(y = k|x)/q(y = k|x) for the samples y = k to change all the samples in the calibration set to the samples drawn from q(x, y = k). Therefore, we define the weight function as Eq.( 6):

W k (x i ) = 1, if x i ∼ q(x, y = k) q(y=k|xi)

1-q(y=k|xi) , otherwise.

(6)

Algorithm 1: Unsupervised Temperature Scaling Require: q s (y): task distribution Require: d(•): the pre-trained model Require: C ∼ q t (x): unlabeled calibration set derived from test domain 1: Find the optimal w * = arg min w (

K k=1 L i=1 Ŵk (x i ; w) -q(y = k)) 2 , 2: Find the optimal T * U T S = arg min T - K k=1 L i=1 Ŵk (x i ; w * ) log (S k (x i ; T )) 3: Calibrate softmax output of model d(•) by: S y (x; T * U T S ) = exp( fy(x) T * U T S )/ K j=1 exp( fj (x) T * U T S

) .

To compute W k (•) for the samples, we need the ground-truth distribution q(x, y = k) that in UTS setting is not available. However we can approximate it empirically referring to UTS assumptions (Sec. (3.1)).

Proposition 1: Let d(•) be a model which gets miscalibrated with rescaled logit layer by factor w * . Then, with known q t (y), the empirical approximation of W k (•) is equal Ŵk (•; •) which is defined as:

EQUATION

)

where w * is:

EQUATION

Proposition 1 valid for the domain shift setting with Covariate shift assumption and also for the case that there is no distribution shift between the training and test datasets. The validity of Proposition 1 for both settings are provided in Appendix A. Fig. 1 illustrates a schematic view of the weight function Ŵk (•; •) in the feature space. The color hue is correlated with the weight values. For the samples that are classified as ŷ = k and for the samples with ŷ = k located near to the decision boundary, the weight is equal to 1. The weight would decrease as the samples fall further from the decision boundary which shows they are less probable to be drawn from distribution q(x, y = k).

Time Complexity of UTS: Computing Ŵk (•; •) is a one parameter optimization which has the time complexity of O(1). After approximating the weight function Ŵk (•; •), UTS minimizes WNLL (Eq.( 4)) with the same time complexity to find the optimal T * U T S which leads to have UTS with the total time complexity of O(1). Algorithm 1 summarizes UTS.

Validity of UTS in Practice: UTS is valid when there is no domain shift or when the there is Covariate Shift between domains. When the test and training datasets are different in representation but keeps the same proportions of each class occurrence, it is categorized as Covariate shift assumption. In many applications like medical image classifications, the probability of happening a class of object is staying the same during the training and test phases which means q s (y) = q t (y) but the illumination, capturing noise, resolution, and image size and viewpoint can vary between two domains which means q s (x) = q t (x). Therefore, in classification problems with Covariate Shift assumption or without any shift, UTS only needs to calculate empirically the number of occurrence of each class to the total number of samples in the training set and use it as q t (y) to calibrate the model.

5. EXPERIMENTS:

We conduct the experiments to analyze the behavior of UTS in comparison to the other methods for two different calibration scenarios. First, we compare UTS with several post processing methods that use NLL as the loss function in the experiment with the same training and test domain distributions. This experiment is designed to be a proof of concept to show that weighted NLL of UTS can indeed calibrate the model without accessing the labels. Second, in order to show the success of UTS in calibrating the model under domain shift condition, we compare UTS, TS and We also study the results of TS-Target that is a TS which selects the calibration set from the target (test) domain. TS-Target has the most accurate uncertainty prediction among all other baselines for the shifted domain distributions. However, we will show in the third section of Experiments part, it suffers from the labeling noise which justifies our try to make TS unsupervised.

5.1. CALIBRATION WITH THE SAME TRAINING AND TEST DOMAINS:

Here we consider the training and the test domains have the same distribution. Our goal is to show UTS can calibrate the models without labels in the case of no domain shift. 2011)). We use all the data pre-processing, training procedures and hyper-parameters tuning for each dataset as described in each mentioned reference. To setup the calibration set, we randomly select 20% of the test dataset. Then, we consider the rest to be evaluated as a test set. We repeat each experiment 20 times independently and report the mean of NLL as a calibration metric. More explanation of experiment setup and the baselines, datasets and calibration metrics are provided in Appendix B.

Experiment Setup:



Results:

In Table 1 , the calibration results based on NLL are compared between TS and UTS, which have only one parameter for fine-tuning the softmax output layer, and Matrix and Vector scaling, which apply a linear function on logit layers. In all cases, TS calibrates the network better than all the other methods. Although the results of UTS are not better than TS, UTS shows improvement in calibration for all dataset-models. It means the weighted NLL as the approximation of NLL with unlabeled samples can work properly to calibrate the model even though it is not as accurate as the NLL with access to labels. Although Matrix and Vector Scaling can define more complex functions to soften the softmax layer, they suffer from over-fitting w.r.t the validation set in confidence. We also provide the complete results with mean and standard deviation for accuracy as well as other standard calibration metrics (NLL, ECE and Brier Score) in Appendix C.1. The explanation about the ECE and Brier are given in Appendix B.3.

5.2. CALIBRATION UNDER DOMAIN SHIFT SETTING:

In this section, we divide the experiments into two parts. First, we compare UTS to the uncalibrated model (UNC), TS and TS-Target on different domain shift scenarios. The goal of this experiment is to compare the gap of calibration between UTS and TS-Target that can be considered as the ground- truth for UTS when the labels are available. Later, we also evaluate the robustness of UTS, which was specifically designed to domain shift, and several probabilistic approaches, which only consider the case of calibration for the same distribution setting. The goal of the experiment is to show that UTS can be indeed robust for different shifting domain scenarios.

Experiment Setup We follow the same experimental setup as Sec (5.1) but with different domain shift assumptions. We use the benchmark proposed specifically for domain shift problem in (Ovadia et al. (2019) ). They model the distribution shift by applying different operations like rotation, translation (rolling), and with different severity levels of intensity corruptions proposed in (Hendrycks & Dietterich (2019) ). In the first part of this section, we compare the result of the UTS to uncalibrated, TS and TS-Target on MNIST and CIFAR10 datasets applying rotation, pixel translation and Gaussian noise to the test domain. In the second part, we add more probabilistic baselines such as LL-Dropout (Gal & Ghahramani (2016) ), SVI (Blundell et al. (2015) ) and Ensemble (Lakshminarayanan et al. ( 2017)) with more variation of the domain shifts in the sense of corruption of intensities. Specific details of baselines and experiment can be found in Appendix B.1

Results As we can see in Fig. 2 , the accuracy of the model degrades by the effect of domain shift. TS family approaches does not change the accuracy during the calibration therefore, all the methods have the same accuracy as the uncalibrated one. TS-Target has the same setting as TS with the difference that it uses the labeled calibration set from the target domain. Then, in ideal situation, UTS uncertainty prediction would reach the TS-Target performance. We can see that UTS is working better than uncalibrated model (UNC) and TS which uses the source data for calibration under the domain shift condition. The gap between UTS and TS-Target is interestingly small in the sense of Brier Score. More results for other domain shifts are provided in Appendix C.2. We also provide the analysis of UTS sensitivity to number of calibration samples in compared to TS and TS-Target in Appendix C.4 which shows UTS obtains stable results in decreasing the number of samples from 20% to 2.5% of test detest size. In Fig. 3 we compare probabilistic approaches to UTS, uncalibrated model (UNC), TS and TS-Target. As all the calibration metrics are dependent on the accuracy of the models, controlling the accuracy to have fair comparison between the methods is important. Otherwise, the better calibration can be as the result of having better accuracy and not the calibration itself. Accordingly, we apply the shifts to the datasets and check the accuracy of the UTS with other approaches, and select the domain shift settings that UTS accuracy is near to the others. As we can see for different combination of model and datasets, UTS can achieve better results than any other probabilistic approaches and has a small gap with TS-Target which achieves the best results. It shows that using the test samples to fine-tune the calibration toward test distribution can help the model to be robust to the domain shift problem. As mentioned before, labeling the test samples is not a trivial task, therefore, using directly TS-Target might not be possible in many cases which justifies the importance of an unsupervised approaches like UTS. In the next section, we show that for a weakly supervision of the test samples, TS-Target cannot be successful in calibration and it needs exact labeling of the test domain samples that might be impractical in many cases.

5.3. TS SENSITIVITY TO LABELING NOISE:

When the labeled samples are available for calibration, TS shows the best results with and without domain shifts. In this section, we will investigate the sensitivity of TS with labeling noise. We apply different rates of random altering the labels only for the calibration set and evaluate the calibration success of TS, UTS and uncalibrated methods accordingly. As we can see in Fig. 4 , TS is extremely sensitive to the noise of labeling. Therefore, in order to have a successful calibration for TS in shifted domains, the exact labeling of test samples is essential which might be not feasible for many applications. UTS is robust to the labeling noise as it is an unsupervised calibration method and it can remove the challenge of labeling the test samples, completely. More results for more datasetsmodels are provided in Appendix C.3.

6. DISCUSSION AND FUTURE WORK:

In this paper, we propose UTS as a robust unsupervised post-processing method to the domain shift calibration challenge. UTS is a member of TS family approaches which have low time and memory complexity, and can calibrate with few number of samples while preserving the accuracy intact. UTS utilizes a new calibration loss function, weighted NLL which is independent of the labels. The computational complexity of weighted NLL is in the same order of NLL which makes UTS a fast and practical calibration solution. Since UTS uses the test samples to adjust the uncertainty, we show it is robust to domain shift and can make off-the-shelf models calibrated when their training samples are not available anymore.

Recent studies (Maddox et al., 2019; Kumar et al., 2018) mentioned that using TS with probabilistic approaches can even improve the uncertainty prediction of already calibrated models. Therefore, we believe this work can be extended in the direction of combining UTS with such approaches in order to achieve more robust domain shift solutions. We also consider another direction of this work towards exploring UTS for more variant domain shift assumptions. In this paper, we study UTS only for Covariate Shift assumption, however, it can be extended to other shifting scenarios such as OOD and adversaries, in future.

A PROOF OF PROPOSITION 1:

First we investigate the validity of Proposition 1 for the settings with no distribution shift. Proposition 1: let W k (•) be a weight function defined as:

W k (x i ) = 1, if x i ∼ q(x, y = k) q(y=k|xi)

1-q(y=k|xi) , otherwise.

and let d(•) be a model which gets miscalibrated with rescaled logit layer by factor w * . Then, with known task distribution q(y), the empirical approximation of W k (•) is equal Ŵk (•; •) which is defined as:

Ŵk (x i , w * ) = 1, if ŷi = k 1/exp( 1 w * log(S y=k (x i ) -1 -1)), otherwise.

where w * is:

w * = arg min w K k=1 L i=1 Ŵk (x i ; w) -q(y = k) 2 x i ∈ C ∼ q(x)

Proof: For simplicity we split the proof into two parts. First, we show that for a known value of w * , Ŵk (•; •) is the approximation W k (•). In other words, 1/exp( 1 w * log(S y=k (x) -1 -1)) = q(y = k|x)/q(y = k|x):

The softmax output of an uncalibrated model d(•) with rescaled logit layer by a factor w * , can be formulated as:

S y=k (x) = exp(w * f k (x i )) K j=1 exp(w * f j (x i )) S y =k (x) = 1 -S y=k (x)

Therefore calibrated output will be defined as:

S y=k (x; w * ) = exp(f k (x i )) K j=1 exp(f j (x i ))

= q(y = k|x) S y =k (x, w * ) = 1 -S y=k (x, w * ) = q(y = k|x) Considering these definitions:

1 exp 1 w * log(S y=k (x) -1 -1) = 1 exp 1 w * log K j=1 exp(w * fj (xi))-exp(w * f k (xi)) exp(w * f k (xi)) = 1 exp log K j=1 exp(fj (xi))-exp(f k (xi)) exp(f k (xi)) = 1 K j=1 exp(fj (xi))-exp(f k (xi)) exp(f k (xi)) = exp(f k (x i )) K j=1 exp(f j (x i )) -exp(f k (x i )) = exp(f k (xi)) K j=1 exp(f k (xi)) 1 -exp(f k (xi)) K j=1 exp(fj (xi)) = S y=k (x; w * ) 1 -S y=k (x; w * ) = q(y = k|x) 1 -q(y = k|x) = q(y = k|x) q(y = k|x)

We consider ŷi = k as the rough estimation of condition x i ∼ q(x, y = k) . The accuracy of the uncalibrated model is the same as calibrated one when it gets uncalibrated by rescaled logit layer. Therefore we can use the prediction output of uncalibrated model to have a rough estimation of the samples that x ∼ q(x, y = k).

Now in the second part, we show that w * = arg min w K k=1 L i=1 Ŵk (x i ; w) -q(y = k) 2 where x i ∈ C ∼ q(x)

Referring to W k (•) definition, we can simply show:

x W k (x)q(x, y)dx = q(y = k)

Which means E q(x,y) [W k (x)] = q(y = k), and as Ŵk (•; •) is equal to W k (•), empirically we can show x c i ∈C Ŵk (x i ; w * ) = q(y = k). In this problem setting, we assume q(y) is known. Therefore, w * can be found easily by minimizing

K k=1 L i=1 Ŵk (x i ; w) -q(y = k) 2 .

Now we show the validity of Proposition 1 under Covariate Shift assumptions.

Corollary 1: Considering the same weight function W k (•) defined in Proposition 1, let d(•) be a model which gets miscalibrated with rescaled logit layer by factor w * toward the target domain q t (x, y). Assume covariate shift where q s (y|x) = q t (y|x), q s (y) = q t (y) and q s (x) = q t (x). Then, with known q s (y), the empirical approximation of W k (•) is equal Ŵk (•; •) which is defined as:

Ŵk (x i , w * ) = 1, if ŷi = k 1/exp( 1 w * log(S y=k (x i ) -1 -1)), otherwise.

where w * is:

w * = arg min w K k=1 L i=1

Ŵk (x i ; w) -q s (y = k)

2 x i ∈ C ∼ q t (x)

Proof: The first part of the proof is exactly the same as Proposition 1 with considering that w * with new assumptions is the scaling factor that makes the model uncalibrated towards the target domain. Therefore, we will conclude:

= S y=k (x; w * ) 1 -S y=k (x; w * ) = q t (y = k|x) 1 -q t (y = k|x) = q t (y = k|x) q t (y = k|x)

For the second part of the proof, we have:

x W k (x)q s (x, y)dx = q s (y = k)

Referring to covariate shift assumption where q s (y|x) = q t (y|x) and q s (y) = q t (y), we can deduce:

x W k (x)q t (x, y)dx = q t (y = k)

Which means E qt(x,y) [W k (x)] = q t (y = k), and as Ŵk (•; •) is equal to W k (•), empirically we can show

x c i ∈C Ŵk (x i ; w * ) = q t (y = k). In this problem setting, we assume q s (y) is known that is equal to q t (y). Therefore, w * can be found easily by minimizing • Matrix and Vector Scaling (Platt et al. (1999) ): Matrix Scaling applies a linear transformation on the logits to soften them:

S y= ŷi (x i ; θ, b) = max k σ(θ.f (x i ) + b) (k) ŷi = arg max k σ(θ.f (x i ) + b) (k) (9)

Where σ is the softmax function which takes logit layer f (x) as an input. The parameters θ K×K and b K are optimized with respect to NLL on the validation set. Vector Scaling is the relaxed version of Matrix Scaling in which θ K×K is a diagonal matrix. • ll-Dropout Monte-Carlo Dropout (Gal & Ghahramani (2016) ), A pre-trained model which is trained with dropout rate p = 0.5 only on the activation function before the last layer, and keeping the dropout active during the test with the same rate. • Ensembles Ensembles of 10 networks trained independently on the entire dataset using the random initialization(Lakshminarayanan et al. ( 2017)). • SVI Stochastic Variational Bayesian Inference for deep learning (Blundell et al. (2015) ; Wen et al. (2018) with the specific settings of training mentioned in (Ovadia et al. (2019) )

B.2 DATASETS:

We apply the calibration method on different image classification datasets. For each experiment, the size of validation set is 20% of the test set which is selected randomly. For all the model-dataset we have trained them on the training set. 

EQUATION

)

where N is the total number of samples. In this paper, we consider B = 15 to report the ECE error. ECE is not derivable function. Therefore mostly it is ignored as the loss function of the 0.0040 +/-0.0000 0.0039 +/-0.0000 0.0039 +/-0.0000 0.0048 +/-0.0001 0.0052 +/-0.0002 CIFAR100 DenseNet100 0.0037 +/-0.0000 0.0033 +/-0.0000 0.0034 +/-0.0000 0.0040 +/-0.0000 0.0040 +/-0.0001 CIFAR100 ResNet110 0.0043 +/-0.0000 0.0040 +/-0.0000 0.0040 +/-0.0000 0.0048 +/-0.0001 0.0051 +/-0.0001 CIFAR100 WideResNet40-4 0.0030 +/-0.0000 0.0029 +/-0.0000 0.0030 +/-0.0000 0.0031 +/-0.0000 0.0030 +/-0.0000

C.2 COMPARING DIFFERENT TS FAMILY APPROACHES TO UTS FOR DOMAIN SHIFT SCENARIOS:

In this section, we provide more results to compare the behavior of UTS with TS and TS-Target in different domain shift applied to CIFAR10 dataset. Details of shifting operation can be found in (Hendrycks & Dietterich, 2019) . We can see in Fig. 5 that TS-Target has the most calibration robustness to the domain shift and followed by UTS with small gap.

C.3 MORE EXPERIMENTAL RESULTS FOR SENSITIVITY OF TS TO THE LABELING NOISE:

In this section, we provide more experiments for more models and datasets. As we can see in Fig. 3 , TS shows a huge sensitivity to the noise of labeling in calibration set. UTS is completely robust to this noise as it is an unsupervised method. It shows if TS wants to calibrate the model without available labeled data, the labeling phase should be handled precisely. Otherwise TS results are not reliable.

C.4 ANALYSIS OF UTS SENSITIVITY TO NUMBER OF SAMPLES:

In this experiment, we show the impact of number of available calibration samples on TS, UTS and TS-Target. As we can see in Fig. 7 , TS and UTS vary significantly when the number of samples are really few (between 30 ∼ 50) while TS-Target is not affected severely (the variance of TS-Target comparing to other methods is small then it is not properly shown in the image). However, by increasing the number of samples to 500, UTS reaches to the optimal T * U T S and after that increasing the number of samples does not consequences better results. 

:



Figures and Tables:

Figure 1: A color view of the weight function in a three class classification problem. For the samples which are classified as class ŷ = k and for the samples that ŷ = k but located near to the decision boundary, the Ŵk (x, w * ) = 1 which is shown with darker hue. For the samples that ŷ = k and are far from the decision boundary Ŵk (x, w * ) → 0, which is with lighter hue. The decision boundary is illustrated by the black line.

We compare UTS with several post-processing baselines which are Temperature Scaling (TS)(Guo et al. (2017)) and Matrix and Vector Scaling(Platt et al. (1999)) on a wide range of different state-of-the-art deep convolutional networks with variations in depth which are ResNet(He et al. (2016)), WideResNet(Zagoruyko & Komodakis (2016)), DenseNet(Iandola et al. (2014)),LeNet (LeCun et al. (1998a)), and VGG(Simonyan & Zisserman (2014)). We test the methods on different datasets such as CIFAR-10 and CIFAR-100(Krizhevsky & Hinton (2009)), SVHN(Netzer et al. (2011)),MNIST (LeCun et al. (1998b)), and Calthec-UCSD Birds(Wah et al. (

Figure 2: Results show accuracy and Brier Score↓ in different degree of shifting for MNIST and CIFAR10 datasets. Comparing TS and TS-Target results shows TS-Target is more robust to domain shift than TS. UTS with small gap is following TS-Target that is the labeled version of UTS and can be considered as the groundtruth for it. The pre-trained models are trained on MNIST and CIFAR10 datasets and tested on different degree of shifted MNIST and CIFAR10 datasets to model domain shift.

Figure4: Sensitivity of TS approach to the labeling noise. Even with small percentage of noise in the labels of samples TS cannot calibrate the model. UTS is completely robust to the labeling noise as it is an unsupervised approach. Shaded regions represent std over 20 runs.

x i ; w) -q s (y = k) Temperature Scaling(Guo et al. (2017)): It is explained in Sec.(3.3)

CIFAR-10 (Krizhevsky et al. (2009)): It contains 60000, 32×32 color images of 10 different objects, with 6000 images per class. The size of training and test sets are 50000 and 10000 respectively. 2. CIFAR-100 (Krizhevsky et al. (2009)): With the same setting as CIFAR-10, except it has 100 classes of different objects containing 600 images in each class. 3. SVHN (Netzer et al. (2011)): It contains 32×32 color images of numbers between 0 to 9 that has 73257 digits for training, 26032 digits for testing. 4. MNIST (LeCun et al. (1998b)): It contains 28×28 gray-scale images of numbers between 0 to 9. It has 60,000 images for training, and 10,000 images for test. 5. Calthec-UCSD Birds (Wah et al. (2011)): It contains 11,788 color images of 200 different birds species. We divided randomly into 7073 training, and 4715 testing samples. B.3 CALIBRATION METRICS Expected Calibration Error (ECE) ECE (Naeini et al. (2015)) measures the average gap between the accuracy and predicted probabilities. Based on this definition of calibration, ECE is proposed as empirical expectation error between the accuracy and confidence. It is calculated by partitioning the range of confidence between [0 , 1] into B equally-spaced confidence bins and then assign the samples to each bin B b where b = {1, . . . , B} by their confidence range. Later it calculates the weighted absolute difference between the accuracy and confidence for each subset B l . More specifically: ECE

Figure6: Sensitivity of TS method to the labeling noise in calibration set in compare to UTS and uncalibrated methods. UTS is not sensitive to the labeling noise in calibration sets at all as it is an unsupervised approach. Shaded regions represent std over 20 runs.

Figure 7: Comparing the stability of different TS family approaches to the number of calibration samples. TS-Target has the best stability. However UTS converges quickly to the optimal solution. Consider that collecting samples for UTS comparing to TS-Target is much less expensive task as it does not need labeled samples. The number of samples in calibration set C from top row to the bottom is 30, 50, 500, 1000, and 2000.

The results of NLL↓ for UTS and other post-processing approaches with the same training and test domains. In all cases UTS can calibrate the model without labeled samples, however it does not achieve the best results. This experiment shows UTS can calibrate off-the-shelf-models with only test samples. The mean are reported in 20 independent runs.

Table content: <table><tr><td>Dataset</td><td>Model</td><td>Uncalibrated</td><td>TS</td><td>UTS</td><td>Vector Scaling</td><td>Matrix Scaling</td></tr><tr><td>Birds</td><td>ResNet50</td><td>0.9383</td><td>0.9287</td><td>0.9313</td><td>0.9355</td><td>7.1423</td></tr><tr><td>MNIST</td><td>Lenet-5</td><td>0.1044</td><td>0.0243</td><td>0.0429</td><td>0.0988</td><td>0.1187</td></tr><tr><td>SVHN</td><td>ResNet110</td><td>0.2107</td><td>0.1534</td><td>0.1566</td><td>0.1936</td><td>0.2070</td></tr><tr><td>SVHN</td><td>DenseNet100</td><td>0.1803</td><td>0.1608</td><td>0.1735</td><td>0.1687</td><td>0.1754</td></tr><tr><td>CIFAR10</td><td>DenseNet40</td><td>0.2895</td><td>0.2221</td><td>0.2848</td><td>0.2912</td><td>0.2883</td></tr><tr><td>CIFAR10</td><td>DenseNet100</td><td>0.1973</td><td>0.1559</td><td>0.1635</td><td>0.2006</td><td>0.1965</td></tr><tr><td>CIFAR10</td><td>ResNet110</td><td>0.3134</td><td>0.2069</td><td>0.2341</td><td>0.2907</td><td>0.3103</td></tr><tr><td>CIFAR10</td><td>VGG16</td><td>0.2608</td><td>0.2036</td><td>0.2099</td><td>0.2503</td><td>0.2573</td></tr><tr><td>CIFAR100</td><td>DenseNet40</td><td>1.0964</td><td>1.0064</td><td>1.0217</td><td>1.2876</td><td>1.4914</td></tr><tr><td>CIFAR100</td><td>DenseNet100</td><td>1.1285</td><td>0.8751</td><td>0.8889</td><td>1.2470</td><td>1.2761</td></tr><tr><td>CIFAR100</td><td>ResNet110</td><td>1.2442</td><td>1.0466</td><td>1.0636</td><td>1.3869</td><td>1.5358</td></tr><tr><td>CIFAR100</td><td>WideResNet40-4</td><td>0.8751</td><td>0.8192</td><td>0.8439</td><td>0.8757</td><td>0.8609</td></tr></table>

ECE↓ for UTS and other post-processing approaches. The mean and std are reported in 20 independent runs.

Table content: <table><tr><td>Dataset</td><td>Model</td><td>Uncalibrated</td><td>TS</td><td>UTS</td><td>VS</td><td>MS</td></tr><tr><td>Birds</td><td>ResNet50</td><td colspan="5">0.0601 +/-0.0015 0.0397 +/-0.0025 0.0350 +/-0.0024 0.0576 +/-0.0019 0.2940 +/-0.0089</td></tr><tr><td>MNIST</td><td>Lenet-5</td><td colspan="5">0.0074 +/-0.0004 0.0027 +/-0.0009 0.0199 +/-0.0014 0.0074 +/-0.0004 0.0077 +/-0.0005</td></tr><tr><td>SVHN</td><td>ResNet110</td><td colspan="5">0.0266 +/-0.0004 0.0044 +/-0.0009 0.0127 +/-0.0036 0.0247 +/-0.0003 0.0264 +/-0.0005</td></tr><tr><td>SVHN</td><td>DenseNet100</td><td colspan="5">0.0160 +/-0.0004 0.0059 +/-0.0009 0.0106 +/-0.0013 0.0139 +/-0.0003 0.0156 +/-0.0005</td></tr><tr><td>CIFAR10</td><td>DenseNet40</td><td colspan="5">0.0409 +/-0.0007 0.0070 +/-0.0009 0.0351 +/-0.0039 0.0410 +/-0.0021 0.0419 +/-0.0008</td></tr><tr><td>CIFAR10</td><td>DenseNet100</td><td colspan="5">0.0262 +/-0.0010 0.0073 +/-0.0017 0.0157 +/-0.0040 0.0281 +/-0.0033 0.0262 +/-0.0017</td></tr><tr><td>CIFAR10</td><td>ResNet110</td><td colspan="5">0.0431 +/-0.0009 0.0088 +/-0.0009 0.0226 +/-0.0021 0.0435 +/-0.0023 0.0439 +/-0.0012</td></tr><tr><td>CIFAR10</td><td>VGG16</td><td colspan="5">0.0419 +/-0.0010 0.0169 +/-0.0009 0.0297 +/-0.0043 0.0402 +/-0.0011 0.0419 +/-0.0011</td></tr><tr><td colspan="2">CIFAR100 DenseNet40</td><td colspan="5">0.0861 +/-0.0011 0.0109 +/-0.0018 0.0390 +/-0.0062 0.1312 +/-0.0054 0.1553 +/-0.0102</td></tr><tr><td colspan="2">CIFAR100 DenseNet100</td><td colspan="5">0.1223 +/-0.0016 0.0152 +/-0.0011 0.0372 +/-0.0063 0.1416 +/-0.0020 0.1452 +/-0.0039</td></tr><tr><td colspan="2">CIFAR100 ResNet110</td><td colspan="5">0.1265 +/-0.0006 0.0102 +/-0.0027 0.0447 +/-0.0048 0.1567 +/-0.0045 0.1740 +/-0.0074</td></tr><tr><td colspan="7">CIFAR100 WideResNet40-4 0.0894 +/-0.0012 0.0402 +/-0.0027 0.0689 +/-0.0033 0.0839 +/-0.0037 0.0850 +/-0.0023</td></tr></table>

Brier Score↓ for UTS and other post-processing approaches. The mean and std are reported in 20 independent runs.

Table content: <table><tr><td>Dataset</td><td>Model</td><td>Uncalibrated</td><td>TS</td><td>UTS</td><td>VS</td><td>MS</td></tr><tr><td>Birds</td><td>ResNet50</td><td colspan="5">0.0017 +/-0.0000 0.0017 +/-0.0000 0.0017 +/-0.0000 0.0017 +/-0.0000 0.0043 +/-0.0001</td></tr><tr><td>MNIST</td><td>Lenet-5</td><td colspan="5">0.0015 +/-0.0001 0.0012 +/-0.0001 0.0014 +/-0.0000 0.0015 +/-0.0001 0.0016 +/-0.0001</td></tr><tr><td>SVHN</td><td>ResNet110</td><td colspan="5">0.0066 +/-0.0001 0.0061 +/-0.0001 0.0062 +/-0.0001 0.0065 +/-0.0001 0.0066 +/-0.0001</td></tr><tr><td>SVHN</td><td>DenseNet100</td><td colspan="5">0.0066 +/-0.0001 0.0065 +/-0.0001 0.0065 +/-0.0001 0.0063 +/-0.0001 0.0064 +/-0.0001</td></tr><tr><td>CIFAR10</td><td>DenseNet40</td><td colspan="5">0.0118 +/-0.0001 0.0109 +/-0.0001 0.0115 +/-0.0002 0.0121 +/-0.0002 0.0118 +/-0.0001</td></tr><tr><td>CIFAR10</td><td>DenseNet100</td><td colspan="5">0.0080 +/-0.0002 0.0075 +/-0.0002 0.0076 +/-0.0002 0.0084 +/-0.0005 0.0080 +/-0.0003</td></tr><tr><td>CIFAR10</td><td>ResNet110</td><td colspan="5">0.0108 +/-0.0002 0.0098 +/-0.0001 0.0100 +/-0.0001 0.0110 +/-0.0003 0.0108 +/-0.0002</td></tr><tr><td>CIFAR10</td><td>VGG16</td><td colspan="5">0.0108 +/-0.0002 0.0098 +/-0.0003 0.0102 +/-0.0003 0.0106 +/-0.0002 0.0107 +/-0.0002</td></tr><tr><td colspan="2">CIFAR100 DenseNet40</td><td/><td/><td/><td/><td/></tr></table>

Peer review (iclr2020):
The authors present an algorithm for postprocessing neural networks to ensure calibration under domain shift. Calibration under domain shift is an interesting challenge that has been receiving increasing attention and tackling this in an unsupervised manner is an interesting approach. However, I have 2 major concerns regarding the approach presented by the authors. What makes calibration under domain shift useful and appealing is that the model is then robust against any changes in the test distribution that can occur during the life cycle of a model. These often include erroneous/samples (corresponding to truly OOD samples), but also gradual domain shift, where the test distribution continuously moves away from the training distribution (e.g. due to a continuous drift in user behaviour/change in customer base) or unforeseen changes. My first major concern is regarding the requirements for UTS, which render this approach not very useful in many of these practical applications: UTS first requires knowledge of and access to the test distribution; in addition it assumes that the distribution of the labels remains unaffected under domain shift. These assumptions are violated in the practical applications described above, in particular those where a gradual, continuous domain shift occurs - in this case, access to the test distribution is difficult since it changes continuously. On this note I also would have liked to see some analysis on how performance depends on the number of samples that are available from the test set, since in practice this might be substantially smaller than the full test set used. Furthermore, I find the assumption that the distribution of labels remains unchanged problematic (q_s(y) = q_t(y) and even q_s(y|x)=q_t(y|x)): once sufficiently out-of-domain, labels become meaningless and predictions for truly OOD samples should have maximum entropy. Even for small domain shifts in practical applications it is not clear why q_s(y|x)=q_t(y|x) should hold and it would have been useful to see a discussion and some robustness analysis on this. Finally, the algorithm requires re-calibration whenever the test distribution changes, which in practice is often not clear (and part of the reason why dealing with predictions under domain shift is so challenging). In addition to doubts on practical applicability, my second major concern is regarding the depth of the evaluation. First, while the authors present some comparisons to probabilistic methods, I am missing a crucial comparison to Evidential Deep Learning (Sensoy et al, NeurIPS 2018), which results in far superior performance than deep ensembles, SVI or dropout. Importantly, the comparisons to probabilistic approaches presented by the authors are very limited. The big advantage of those approaches is that, once trained, no further recalibration is necessary and well calibrated predictions can be made for any level of domain shift, whereas UTS requires a recalibration step for very level of domain shift. That is why I think it is crucial to not only show one arbitrarily picked level of domain shift for each dataset/perturbation, but calibration across all levels of domain shift, as for TS and TS-Target; since no recalibration is required for those probabilistic approaches this is very straight-forward and would be very informative - especially since e.g Figure 5 shows that UTS has only very minor advantages over TS in many settings. I appreciate that the authors report some performance in terms of ECE in the supplement, but I think it would be very informative to report performance in terms of ECE for all domain-shift experiments: The Brier score conflates accuracy with calibration (see eg the 2 component decomposition), whereas ECE directly quantifies calibration and is hence easier to interpret and arguably the more meaningful measure when quantifying calibration. Minor: I find the manuscript lacks clarity. Aspects such as the definition of calibration as well as implications and interpretation of Proposition 1 should be described in more detail in the manuscript.