Assessing Model Out-of-distribution Generalization with Softmax Prediction Probability Baselines and A Correlation Method
Authors: Weijie Tu
Weijian Deng
Tom Gedeon
Liang Zheng
Venue: Submitted to ICLR 2023 (2023)

Abstract: This paper studies the use of Softmax prediction to assess model generalization under distribution shift. Specifically, given an out-of distribution (OOD) test set and a pool of classifiers, we aim to develop a Softmax prediction-based measure which has a monotonic relationship with OOD generalization performance. We first show existing uncertainty measures (e.g., entropy and maximum Softmax prediction) are fairly useful of predicting generalization in some OOD scenarios. We then move ahead with proposing a new measure, Softmax Correlation (SoftmaxCorr). To obtain the SoftmaxCorr score for a classifier, we compute the class-class correlation matrix from all the Softmax vectors in a test set, and then its cosine similarity with an identity matrix. We show that the class-class correlation matrix reveals significant knowledge about the confusion matrix: its high similarity with the identity matrix means predictions have low confusion (uncertainty) and evenly cover all classes, and vice versa. Across three setups including ImageNet, CIFAR-10, and WILDS, we show that SoftmaxCorr is well predictive of model accuracy on both in-distribution and OOD datasets.

Under review as a conference paper at ICLR 2023
ASSESSING MODEL OUT-OF-DISTRIBUTION GENER- ALIZATION WITH SOFTMAX PREDICTION PROBABIL- ITY BASELINES AND A CORRELATION METHOD
Anonymous authors Paper under double-bling blind review
ABSTRACT
This paper studies the use of Softmax prediction probabilities to assess model generalization under distribution shift. Specifically, given an out-o--distribution (OOD) test set and a pool of classifiers, we aim to develop prediction probability- based measure which has a monotonic relationship with OOD generalization per- formance. We first show existing uncertainty measures (e.g., entropy and max- imum Softmax prediction probability) are fairly useful of predicting generaliza- tion in some OOD scenarios. We then move ahead with proposing new measure, Softmax Correlation (SoftmaxCorr). To obtain the SoftmaxCorr score for a classi- fier, we compute the class-class correlation matrix from all the Softmax vectors in a test set, and then its cosine similarity with an identity matrix. We show that the class-class correlation matrix reveals significant knowledge about the confusion matrix: its high similarity with the identity matrix means predictions have low confusion (uncertainty) and evenly cover all classes, and vice versa. Across three setups including ImageNet, CIFAR- 10, and WILDS, we show that SoftmaxCori is well predictive of model accuracy on both in-distribution and OOD datasets.
INTRODUCTION
Understanding the generalization of deep neural networks is an essential problem in deep learning.
There is substantial interest in predicting ID generalization gap via complexity measures (Jiang
al. 2020a Neyshabur et al. 2015) [artlett et al.] 2017] Keskar et al., 2017 Nagarajan Kolter,
Chuang et al. [2021] Jiang al. 2020b Smith & Le [018 Arora et al. 2017). Although significant, developing measures to characterize OOD generalization remains under-explored. In fact, the test environment in the real world often undergoes distribution shift due to factors like sample bias and non-stationarity. Ignoring the potential distribution shift can lead serious safety concerns in self-driving cars (Kuutti al.
2019
Neyshabur et al. 22017;
ctt al.
2018: Dziugaite & Roy
2020) and histopathology (Bandi et al. 2018). etc.
Softmax prediction probability is found to be useful to analyze the test environment (Hendrycks) & Gimpel 2016 Guillory et al. [021 Deng et al.) 2022a Liang et al.) 20188) Garg et al. 2022). For example. Hendrycks & Gimpel (016)) Liang et al. (2018) utilize maximum Softmax prediction probability to identily samples from open-set classes. This gives us a hint: models" prediction probabilities may be informative to reflect their OOD performance. Therefore, we are interested in measures based on prediction probability and conduct large-scale correlation study to explore whether they are useful to characterize generalization of various models under distribution shift. Concretely, given various deep models, we aim to study and develop prediction probability-based measures which monotonically relate to model generalization.
To this end, we construct catalog of empirical prediction probability based measures and create a wide range of experimental setups. We collect 502 different classification models ranging from standard convolutional neural network to Vision Transformers. We cover 19 ID and OOD datasets with various types of distribution shift, such as ImageNet-V2 (Recht et al. 2019) with dataset repro- duction shift and stylized- ImageNet (Geirhos et al. 2019) with style shift.
Based on experimental results, we observe that empirical uncertainty measures based on prediction probabilities (e.g., entropy) useful in characterizing OOD generalization to some extent. How-

---

Under review as a conference paper at ICLR 2023
ever, they are limited in leveraging class-wise relationship encoded in prediction probabilities. We thus further propose Softmax correlation (SoftmaxCorr) an effective prediction probability-basee measure describing for each classifier to what extent classes correlate with each other. Specifically, for each classifier we compute a class correlation matrix from all prediction vectors in a test set. Then, we calculate its cosine similarity with identity matrix to evaluate whether this classifier makes diverse and certain predictions. We show that class-class correlation effectively uncovers knowledge of confusion matrix, thus better reflecting overall accuracy on OOD test set. The broad correlation study shows the efficacy of SoftmaxCorr. Our contributions are summarized below.
We observe that Softmax prediction probability- measures generally give good base- line indicators of the OOD accuracy of various classification models.
Furthering this finding, we propose SoftmaxCorr which assesses model generalization by explicitly calculating class-class correlation, a new angle to leverage the prediction proba- bility. A wide range of experiment suggests the effectiveness this method.
RELATED WORK
Predicting generalization in deep learning studies the ID generalization gap (i.e., the difference between training and test accuracy) of deep neural networks. Representative methods develop com- plexity measures based on model parameters and training set (Jiang et al. 2020a bj Neyshabur et al.)
2015 Keskar et al., 2017 Bartlett et al. 2017; Neyshabur et al. 2018 Liang et al.) 2019; Chuang et al. 2021; Smith & Le 2018 Arora et al. [018]] Dziugaite & Roy |20 Dinh et al. (017). such as distance of training weights from initialization Nagarajan & Kolter 2019), the product of norms of weights across layers (Neyshabur et al. 2017) and the change of model accuracy with respect to different perturbation levels in training data (Schifi et al., a (22TT) These methods assume that training and test data come from the same distribution and do not incorporate characteristics of test data, so we can unlikely make reasonable predictions on a different distribution. To mitigate this limitation, we investigate the model generalization under distribution shift by developing measures that reflect models' generalization property on OOD datasets.
OOD generalization. Machine learning models should generalize from training distribution to OOD datasets (Djolonga al.) [22T]] Koh et al., (202T). To study the problem of OOD generaliza- tion, several benchmarks are proposed (Hendrycks & Dietterich 2019 Koh et al. 2021 Gulrajani & Lapez-Paz, 202T). such as corruption benchmark (Hendrycks & Dietterich] 2019) and domain
generalization testbed (Gulrajani & Lopez-Paz 2021). Moreover, several methods are proposed to
improve model OOD generalization (Volpi et al. 2018 Zhao et al. 2020 Sagawa et al. 2020 Liu
ct al. 202 la Mansilla et al. 2021 Shi al. [022]] such as adversarial domain augmentation (Volpi
ct al.
2018 Qiao & Peng 2021) and domain gradient matching (Shi et al. 2021).
There are few works study the characterization of model OOD generalization. Ben- David et al. (010) provide upper bounds of OOD generalization error for domain adaptation. Some works further bound the OOD generalization error based on the divergence between the two distributions (Acuna et al. 2021 Zhang et al.) 2019 Tachet des Combes et al., (020). However, as suggested by Miller al. (001)) when the shift becomes larger, the above bounds on OOD performance become looser. Moreover, Vedantam et al. (2021) report that the adapting theory from domain adaptation limited in predicting OOD generalization. In this work, we aim to assess model generalization under distribution shift by OOD measures.
detection aim detect test data that do not belong to any of classes modeled in
training distribution (Hendrycks & Gimpel, 20166) Yang et al.) al.] 202 Many methods are developed based on model outputs (Hendryeks & Gimpel| 2016]] Liang al. 2018 Hendrycks et al. 2022). For example, Hendrycks & Gimpel| (2016) use maximum prediction probability to identify OOD samples. Hendryeks et al. (2022) further show that, without being normalized by Softmax function, maximal model logit is also a strong baseline for OOD detection. Liu et al. (2020b) calculate an
energy score based on Softmax probability, which is regarded as a simple yet effective replacement for maximum prediction probability. Other methods investigate the predictive uncertainty L et al.
2020a Van Amersfoort et al. 2020). Different from the above works, this research does not aim to detect OOD data, but to explore Softmax-based measures assess model generalization.
2

---

Under review as a conference paper at ICLR 2023
3
TASK FORMULATION
Task definition. We consider a K-way classification task, and let 3 (1, K} denote the label space and X E Rd denote the input space. We are given a labeled training set DS = N that contains N₂ data i.i.d drawn from a source distribution Ps. and an OOD test set {1e9, uz) =
D" {(12, ):)) that contains N data i.i.d drawn from another distribution Pr (Ps # Pr). We train M neural network classifiers {dm J.M on DS Given a sample (æ, y) from D the classifier om : X' + A generates Softmax prediction probabilities for æ on K classes, where A A denote K dimensional unit simplex. By testing on D", Φm yields prediction matrix E RNNK whose each row represents prediction probabilities of a test data. Specifically, the prediction matrix N. satisfies Σ Pag vi 1...N N and P.j 2 0 Vi E 1 1....N.) E the probability that i-th sample is predicted to the j-th class.
1...K, where P.y indicates
The dataset has an evaluation metric (e.g., accuracy) to obtain ground-truth generalization GT, of classifier Om. The goal is to design measure to calculate score Sm for cach classifier om with- out access to data annotations. The calculated scores {5mm m== ideally should well correlate with {G, 1M=1, that can assess the OOD generalization of models based the scores.
Evaluation metrics. We use Spearman's 's Rank Correlation coefficient p (Kendall 1948) to measure monotonicity between calculated scores and model generalization. We additionally the weighted variant Kendall's rank correlation Tup which shown to be useful when selecting the best ranked item is of interest (You et al., 202Tb. Both coefficients range from [-1,1]. A value closer to -1 or 1 indicates a strong negative or positive correlation, respectively, and 0 means no correlation.
Similar to Miller al. (202I); Back al. (2022), we apply the same probit scale to both accuracy and SoftmaxCorr in our experiment for better linear fit.
SOFTMAX PREDICTION PROBABILITY-BASEI OOD MEASURES
4.1
WHY USE SOFTMAX PREDICTION PROBABILITY?
Deep neural networks tend to make over-confident predictions (Ovadia al. 2019 Minderer al., Guo et al. [017]] Hein et al., 2019), which at the first look makes it less reliable to use Softmax Prediction as an uncertainty measure on the test data. However, existing works show that it is indeed informative in analyzing test environments. For example, Hendrycks & Gimpel (2016) show that the maximum Softmax prediction probability (MaxPred) of correct samples tends to higher than that of incorrect or OOD samples. Guillory et al. (2021): Garg et al. (2022) suggest it is useful to estimate accuracy trained classifier by calculating its MaxPred on test samples.
2021
Proof of concept. The above works imply the prediction probabil- ity likely be effective in measuring OOD performance of a pool of models. Given an OOD test set (ImageNet-R) and various Ima- geNet models, we conduct correlation study between MaxPred and classification accuracy. In Fig. [ we show that there is moderate correlation between MaxPred and model accuracy (p = 0.613 and To 0. 0.776) ImageNet-R. This indicates that MaxPred is feasi- ble in assessing OOD performance. Based on this observation, we empirical prediction probability- sures and develop a more effective measure which exploits more semantics reflected in the prediction probability.
ImaeeNes-R
0.613
Accuracy (%)
Maximum prediction probabilly
further explore
Figure 1: Correlation study between MaxPred and accu- racy (%) on ImageNet-R.
4.2 EXPLORING MORE EMPIRICAL MEASURES
In addition to MaxPred, we investigate other empirical measures below.
Softmax Gap (SoftGap). MaxPred only utilizes the maximal value of Softmax vectors while disre- garding values on other entries. We introduce SoftGap based on MaxPred which further considers the second largest entry in prediction vector. Specifically, it calculates the average of difference between the largest and second largest Softmax prediction probabilities over all samples. A larger Softgap indicates prediction is certain, suggesting better generalization.
3

---

Under review as a conference paper at ICLR 2023
comslation matri
confusion matri
corrllation matrix
contusion
correlation matrix
confusion matrix
LoNet
5
aicuracy:
SimpleDLA 6.
acouragy:
MobilaNet
Rocuracy: 71.35%
917
Figure 2: Comparison between the correlation matrix computed from Softmax vectors and confusion matrix. We show comparisons obtained by four classifiers: LeNet, SimpleDLA and MobileNet. The test set is CINIC. For each pair of matrices, we use representational similarity analysis (r₂) measure their similarity. The high r, > 0.85 indicate that the class correlation matrix reveals knowledge of confusion matrix and therefore model accuracy.
Negative Predictive Entropy (Entropy) (Pereyra et al. 2017 Vedantam et al. 2021). This computes negative entropy of P by - En H(P..). where H-)) is Shannon entropy Shannon 1948). A larger Entropy means a model makes more confident predictions.
Information Maximization (InfoMax).. It a loss function used in unsupervised domain adaptation algorithms (Shi & Sha ch12 Liang al. 2020). Its formula is H Σ P..) ( x/ Σ (P..)) The idea is that a well-generalized model has diverse predictions over all samples (first term) while being certain individuals (second term). Specifically, accurate model tends to have large entropy of the average prediction probability of all samples and small entropy on each data point.
4.3 OUR METHO::: SOFTMAX CORRELATION (SOFTMAXCORR)
Given the prediction matrix P E RNNK predicted by @m, class correlation matrix C E R Rx can be computed by C P. An entry Ci,, indicates the correlation between prediction probabilities of class and class j over all samples, and is computed by Ci.; En PriPnag We define the sum of diagonal entries the correlation matrix as intra-class correlation (IntraCorr) and off- diagonal elements as inter-class correlation (InterCorr). Note that, the sum of C is N.
Based on class correlation matrix C. we develop SoftmaxCorr to take into account the two char- acteristics. First, whether the model produces confident predictions and thus its computed class correlation matrix C' has a high intra-class correlation. Second, whether the model gives diverse predictions where all classes are predicted. This detects the trivial model solution where all data confidently predicted as the same class. To achieve this, we define SoftmaxCorr as the cosine simi- larity between the class-class correlation matrix C' and an identity matrix Ix: cos = |CI-I higher similarity score means that model gives 1) high prediction certainty (intra-class correla- tion) and 2) high prediction diversity (the diagonal elements of class correlation matrix is uniformly distributed). We note that existing studies (Yang et al. 2021b Wang & Isola 2020 Asano et al. 2019 also report that the two characteristics are important for learning discrimintive Teatures.
P.
In Fig. 2 we compare the computed class correlation matrices and ground-truth confusion matrices of LeNet (LeCun et al. 1998). SimpleDLA (Yu et al. 2018) and MobileNet (Howard al. 2017) on
CINIC. We use representational similarity analysis (ra) (Kriegeskorte et al. 2008 Dwivedi & Roig 2019] to calculate the similarity between each pair of class correlation and confusion matrices. We observe that r. scores are high (r, > 0.85). This suggests that correlation matrix encodes critical and informative knowledge of the level of confusion between classes, thus directly relating to accuracy.
EXPERIMENT
In this section, we first describe the ImageNet, CIFAR-10 and WILDS setups. Then, we analyze the experiment results of prediction probability- measures on three setups. After that, we compare SoftmaxCor with other characterizations of correlation matrix. Moreover, we test whether Soft- maxCorr can model checkpoints along the training trajectory and assess models trained by different domain adaption algorithms. Lastly, study the correlation between SoftmaxCors and accuracy of
4

---

Under review as a conference paper at ICLR 2023
Table 1: Method comparison on ImageNet, CIFAR-IO, WILDS and DomainNet. We compare our SoftmaxCorr with four empirical Softmax prediction probability-basee measures: MaxPred, Soft- Gap, Entropy and InfoMax. We use Spearman's rank correlation (p) and weighted Kendall's corre- lation (Tw.) for comparison. The highest correlation in each row is highlighted in bold. We show that our method is stable and yields the highest average correlation on ImageNet, CIFAR-10 and WILDS setup and competitive correlation on DomainNet setup.
MaxPred
SoftGap
Entropy
InfoMax
SoftmaxCort
Setup
Dataset
p
Tu
p
Tw
p
p
Tw
ImageNet- Val
0.542
0.515
0.665
0.568
0.142
0.286
0.145
0.288
0.919
0.752
ImageNettV2-A
0.645
0.574
0.747
0.622
0.217
0.327
0.222
0.330
0.922
0.792
ImageNet-V2-BB
0,506
0.483
0.636
0.554
0.109
0.256
0.113
0.266
0,912
0.785
ImageNet-V2-C
0,577
0.528
0.699
0.594
0.169
0.303
0.173
0,306
0.917
0.795
ImageNet
ImageNet-R
0,613
0.776
0.797
0.839
0.295
0.623
0.423 0,688
0.925
0.880
ImageNet-S
0.784
0.798
0.839
0.825
0.471
0.652
0.614 0.730
0.875
0.883
Stylized-Imagee ImageNet
0,580
0.609
0.661
0.656
0.233
0.431
0.512 0.613
0.828
0.834
ObjectNet
0.912
0.814
0.934
0.848
0,820
0.735
0.836
0,761
0,927
0.897
ImageNet-Blur
0,877
0.828
0.898
0.843
0.780
0.702
0.850
0.807
0.967
0.929
Average
0.671
0.665
0.764
0.705
0.361
0.480
0.433
0.520
0.910
0.839
CIFAR-I0- Val
0.860
0.512
0.871
0.551
0.844
0.494
0.844
0.494
0.865
0.536
CIFAR-10.2 10.2
0.857
0.696
0.868
0.705
0,826
0.663
0.832
0.669
0.884
0.694
CIFAR-10
CINIC
0,753
0.375
0.777
0.402
0.722
0.331
0.735
0.348
0.844
0.578
CIFAR-IO-Noise
0,032
-0.082
0.041
-0.077
0.035
-0.090
0.896
0.750
0.813
0.771
Average
0.626
0.375
0.639
0.395
0.607
0.350
0.827
0.565
0.851
0.645
iWildCam-ID
0.972
0.889
0.969
0.887
0.967
0.885
0.960
0,810
0.945
0.792
iWildCam- -OOD
0.917
0.840
0.936
0.846
0.853
0.798
0.883
0,792 0.916
0.839
WILDS
Camelyon 7-ID
0.847
0.718
0.847
0.718
0.728
0.611
0.833
0.653
0.893
0.746
Homelyon17-OO0
0.192
0.320
0.192
0.320
0.167
0.299
0.695
0.635
0.689
0.492
Average
0.732
0.691
0.736
0.692
0.679
0.647
0.842
0.725
0.861
0.717
DomainNet-ID
0.725
0.570
0,744
0.597
0.677
0.523
0.698
0.539
0.747
0.607
DomainNet
DomainNet-OOD
0.403
0.274
0.407
0.258
0.386
0.253
0.763
0.721
0.693
0.566
Average
0.564
0.422
0.576
0.428
0.532
0.388
0.731 0.630 0.720 0.587
single model on various OOD test sets. We note that for the completeness of experiment, we also include the performance of Softmax measures on ID test sets.
5.1 EXPERIMENTAL SETUP
ImageNet setup. We collect 173 models publicty accessible from TIMM (Wightman) 2019). They
are trained or fine-tuned on ImageNet (Deng et al.] (209) and vary in architectures and training strategies. We use both ID and OOD datasets for correlation study. Specifically, ImageNet- Val is ID test set. OOD datasets are: (1) ImageNet-V2 V2 (Recht et al. 2019): (2) ObjectNet (Barbu et al. 2019); (Wang et al. 2 (4) ImageNet et Blur severity 5 (Hendrycks & Dietterich 2019; (5) ImageNet-R(enditonnn
(3) ImageNet-
(Hendrycks et al., 202T (6) Stylized-ImageNer (Geirhos et al., 2019.. Note that, ImageNet-Ve V2 has three versions (ImageNet- VV-AA/CVV resulting from different
sampling strategies. ImageNet-R and ObjectNet contain 200 and 113 ImageNet classes respectively.
CIFAR-10 setup. We collect 65 networks trained with the scheme provided by Wightman (2017) on CIFAR-10 training set (Krizhevsky et al. 2209). 1TFAR-1O-Vallidaiie OOD datasets, we use (I) CIFAR-10.2 (Recht et al.) 2018) (2) CINIC (Darlow et al. 2018 (3)
is the ID test set. For
CIFAR-10-Noise with severity 5 (Hendrycks & Dietterich 2019
WILDS setup. We follow the same training scheme provided by Koh et al. (2021) to train or fine- tune models for 1 (Bandi et al. 2018) and iWildCam (Beery et al. (021). Camelyon17 is a binary classification dataset where the objective is to classify whether a slide contains tumor issue. We use 45 models varying in architectures and random seeds. iWildCam is 182-way
5

---

Under review as a conference paper at ICLR 2023
ImtgeNat-Val Fu=
ImageNel-V2- = +.922 0.792
ImageNel-R
Accuracy (%)
73
= 0.925 0.880
ImageNet
20
69
53
43
30
13
0.94
0.85
0.7
5.85
0.37
0.62
SoftmaxCort
CIFAR-10-Val 008865 -0.536
50
CIFAR-10.Z
CIFAR-10-NOEE
+0.884
0.813
0.894
0.771
CIFAR-10
Accuracy
60
0.7
SoftmaxCor
0.48
MilldCam-ID 0.945
iWidCam-OOD
0.916
Camelyon17-00
0.792
0.839
0.482
WILDS
Macro-FF
5.25
02
0.15
0.1
0.42
0.57
0.76
0.41
0.96
SoftmaxCort
Figure 3: SoftmaxCorr vs. model generalization under ImageNet, CIFAR-10 and WILDS se- tups. For ImageNet setup, ID test ImageNet-R. For CIFAR-10 10 setup, ID test set is CIFAR- 1--AR----ail Val: OOD test sets are CIFAR-10.2 and CIFAR-IO-Noise For WILDS, ID test set iWildCam-ID; OOD test sets are iWildCam- Camelyon17-OOD. The y-axis is top-1 accuracy, top-1 accuracy and macro-F1 for the three se- tups, respectively. In each subfigure, each point denotes a model trained for the corresponding task. Straight lines are fit with robust linear regression (Huber) 2011). Axes are probit scaled as described Section 3 We observe that SoftmaxCorr is a reliable and effective metric. Particularly on Ima- geNet, SoftmaxCom is predictive of model generalization with strong performance (p 0.92).
ImageNet-Vaad Val: OOD test are ImageNet- V2-A and
and
animal classification dataset. We collect 66 models the variations of which results from different network architectures and learning rates. Model performance is measured by macro-F1 F1 score.
5.2 MAIN OBSERVATIONS
SoftmaxCort exhibits strong correlation with model generalization. In Fig. 3 and Table we observe that SoftmaxCort is indicative of model performance under the three setups. Particularly on the ImageNet setup, SoftmaxCort has consistently stronger correlations than the other empirical measures. For example, the average Spearman's 's Rank Correlation coefficient, p is 0.910, 0.671 and 0.764, and 0.361, and 0..43 for our method, MaxPred, SoftGap, Entropy and InfoMax, respectively. test sets it does not present the strongest correlation,
On CIFAR-10 10 and WILDS, while still think that Softmax a preferred measure because it has higher or very competitive average correlation scores 0.851, 0.861 and т., = 0.645, 0.717).
SoftmaxCorr gives more stable correlation, while the other four measures have mixed per- formance different test sets. On CIFAR-1O-Noise,
find that SoftmaxCorr correlates well with model performance (p 0.813 and Tu = 0.771). In contrast, Entropy, MaxPred and Soft- Gap show correlation (p < 0.05 and The < 0). Although InfoMax has higher rank correlation

---

Under review as a conference paper at ICLR 2023
Dataset
Diag-sum
Diag-std SoftmaxCorr
Dataset
1%
5%
10%
30 %
100% %
ImageNet-V224 V2-A
0.503
0.099
0.922
ImageNet- -2-A
0.820 0.853 0.865 0,901 0.922
ImageNet-R
0.669
0.314
0.925
ImageNct-R
0.873 0.928 0.925 0.926 0.925
CIFAR-10.2
0.854
0.688
0.884
CIFAR-Noie
0.734 0.785 0.814 0.813 0.813
CINIC
0.752
0.781
0.844
CINIC
0.655 0.777 0.827 0.857 0.844
iWildCam- OOD
0.916
-0,696
0,916
iWildCam- OOD
0.826 0.874 0.896 0,903 0.916
Camelyon 17-00D
0.174
0.581
0.689
Camelyon17-OOD 0.708 0.701 0.690 0.694 0.689
average
0.645
0.294
0.863
Table 3: Sensitivity analysis of SoftmaxCorr
Table 2: Comparison between SoftmaxCorr and two variants (Diag-sum and Diag-std). Rank cor- relation (p) is used as the metric.
on test set sizes. We test four sampling ratios and report p on six datasets. We show that SoftmaxCori is relatively stable given rea- sonable number of samples.
than SoftmaxCort (p = 0.896 vs. 0.813), it does not correlate with accuracy on ImageNet- Val, MmageNet-V2-A/BCC (p 0.25 and Tw < 0.35). On ID and OOD sets of iWildCam, SoftGap and MaxPred have higher correlations than SoftmaxCorr. However, they fail to measure the macro-F1 on Mamelyon17-OO1 (p 0.192). While in some cases SoftmaxCorr does not present the highest correlation, we emphasize that overall gives more stable and stronger correlations. Thus, we think that SoftmaxCorr is generally more indicative of model generalization than other measures.
Compared to SoftGap and InfoMax, SoftmaxCort better utilizes Softmax prediction prob- abilities. We SoftGap top of MaxPred
simple approach to explieitly consider more entries (the second largest probability) in Softmax predictions. Based on the entropy, InfoMax fur- ther considers the prediction diversity (its first term) and thereby it also leverages more information of Softmax predictions. As shown in Table [1 both methods achieve higher correlation results than MaxPred. This indicates that using more information of Softmax predictions is useful. Compared with SoftGap and InfoMax, the llasswwise correlation considered in our Softmax better reveals the knowledge encoded by Softmax predictions. This is supported by the higher average correlation and more stable performance of SoftmaxCort under three setups.
5.3 DISCUSSION ON THE CHARACTERIZATION OF CORRELATION MATRIX
To investigate the importance of prediction diversity and certainty for predicting OOD generaliza- tion, we compare SoftmaxCorr with two variants: (1) the sum of diagonal entries in the class cor- relation matrix (Diag-sum); (2) the negative of standard deviation of diagonal elements in the class correlation matrix (Diag-ste std). In Table 2 on six datasets from three setups. We see that SoftmaxCom is more predictive of OOD generaliza- tion than Diag-sum and Diag-std (p 0.863 vs. 0.645 vs. 0.294). This means that it is important to measure both prediction diversity and certainty for OOD generalization assessment.
we present Spearman's rank correlation of three methods
5.4 SENSITIVITY ANALYSTS ON TEST SET SIZF
We study the sensitivity of SoftmaxCor to test set size. Specifically, we reduce the dataset size by randomly sampling 1%, 5%, 10% and 30% of the original data. We report the averaged Spearman's 's correlation of three random runs on six datasets (e.g., ImageNet- V2-A, ImageNet-R, CINIC and (WildCam-OOD). As shown in Table g observe that when the number of test data is very small (1%), the correlation of SoftmaxCorr drops. When the dataset size increases (2 5%), Softmax- Corr exhibits stable and high correlation with model performance. This suggests that SoftmaxCorr requires reasonable number of samples to capture the model OOD generalization.
5.5 EVALUATING GENERALIZATION ALONG A TRAINING TRAJECTORY
In previous sections, SoftmaxCorr utilized to measure the performance of models varying in different architectures and training strategies. In practice, we are sometimes interested in evaluating the models at different training check points. Hence, we also analyze whether SoftmaxCort is helpful in this case. We collect prediction probabilities of CINIC every 10 epochs along the training process of 20, DenseNet- 121 (Huang et al,, 2017), VGG11 (Simonyan & Zisserman 2014) and MobileNet (Howard al. 2017) trained on the CIFAR-10. In Fig. 4 observe that SoftmaxCorr

---

Under review as a conference paper at ICLR 2023
Rashet-z Bit
VoG-11
Mazaleke R BET
Acouraayy (%g
0.916
20
03B 2.0
1.5
330
082
SoftmaxCorr
SoflmsxCorf
SoftnesCort
SoftmaxCorr
Figure 4: Correlation analysis: SoftmaxCor and accuracy CINIC. Each point represents check- point. We consider CIFAR-10 models: ResNet-20, DenseNet-1211 121, VGG- and MobileNet. Axes probit scaled in Section 3 In each subfigure, every point means checkpoint of the model along the training process. For four models, we see strong correlations (p > 0.93). This suggests that SofumaxCorr is helpful in assessing checkpoints along the training process.
Ser--ral 2-0 204
VT-Eooo
0.557
#0881
pm ELitS
-. 0014
Acouracy
22
SoftmaxCorr
SottmaxCorr
SoftmaxCoom
SoftmaxCor
Figure 5: SoftmaxCorr E.S. accuracy on ImageNet-C benchmark. In every subfigure, each dot indicates dataset of ImageNet-C. We observe strong linear and rank correlation for ImageNet models: ConyNext-Base, Swin-Small, EfficientNet-B2 and ViT-Base.
has high rank correlation (p 0.93) with model performance for four networks. This that we can potentially apply SoftmaxCorr to assay model generalization along the training process.
5.6 ASSESSING PREDICTIVE ABILITY OF SOFTMAXCORR UNDER DOMAIN ADAPTATION.
On ImageNet, CIFAR and WILDS setups, all models are trained by standard empirical risk min- imization and do not use the unlabeled OOD samples for training. In some scenarios, domain adaptation (DA) algorithms
employed for learning target-adaptive models with additional unla- beled OOD samples [Kow & Loog 2019 Zhou et al. 2022 To explore whether SoftmaxCorr still effective to
conduct correlation study under DomainNet setup (Peng et al. 2019 Sagawa et al 202T. The models are trained by different DA algorithms (e.g.. DecpCORAL [un & Saenko 2016). DANN Gaain al. 2016 In Table I we observe that SoftmaxCor: performs reasonably on DomainNet-ID and DomainNet-OOD. We also notice that InfoMax better than SoftmaxCorr on DomainNet-OOD. InfoMax commonly used as regularization in DA (Shi & Sha 2012 Liang et al.] 2020 We speculate it might better reflect models' target-adaptation ability and exhibit higher correlation on DomainNet-OOD.
the generalizan ization of these models,
5.7 SOFTMAXCORR V.S. GENERALIZATION ON VARIOUS OOD TEST SETS
We investigate how given trained model generalize to various OOD datasets. In detail, we eval- uate single model on all test sets of ImageNet-C benchmark (Hendrycks Dietterich, 2019) and conduct correlation study between accuracy and SoftmaxCorr. We additionally use Pearson's 's corre- lation (r) to measure overall linear trend. This coefficient varies in [-1, 1]. value closer to -1/1 indicates better negative/positive linearity and 0 means correlation. We use ImageNet networks: ConvNeXt--Bl Base (Liu et al. 2022). Swin-Small (Liu et al. 20216). EfficientNet-Bi (Tan & Le 2019) and TiT-Base (Dosovitskiy al. 2020). Figure shows strong linear relationship and high rank correlation (r 0.85 and > 0.91). indicates that with a linear regressor SoftmaxCort can also help estimate the accuracy of given model various test sets.
8

---

Under review as a conference paper at ICLR 2023
6 DISCUSSION AND POTENTIAL DIRECTIONS
What makes OOD measures interesting? Miller et al. 2021 report an accurac--ontthe-line phe- nomenon where there exists very strong linear correlation between probit-scaled ID and OOD generalization. It means ID generalization is a good predictor of OOD generalization. There are three reasons that make it meaningful to investigate OOD measures. First, Softmax prediction prob- ability can be obtained with just unlabeled data. This makes prediction probability measures of important practical value as indicators of OOD generalization. Second, Miller all. (202T) show that accuracy-on- oo-ttoe--tteeeee is not universal. That is, on some datasets, ID and OOD generalization do not show clear positive correlation. This finding is further discussed in recent work (Wenzel et al,)
2022). Specifically, Wenzel et al. (2022) suggest two patterns preventing this phenomenon: (1) un- derspecification (e.g., Camelyon 1 77) where same ID performance leads to different OOD behavior; (2) models do not transfer information from ID to OOD domains (e.g., DomainNet). That is, despite of various ID performance, all models perform poorly on OOD datasets. Paradoxically, identifying the patterns itself requires labeled OOD data. Therefore, while ID performance is indicative in some scenarios, we think that SoftmaxCort is overall a good alternative when labeled ID data is inaccessi- ble seccurac--on-th--laee
does not hold. Third, it is demanding and sophisticated to design ID test sets (Engstrom et al. 2020), which are expected to be unbiased and representative of distribution to effectively measure model ID accuracy. Further, it is a trade-off' split a full dataset into training, validation and test sets in terms of training and evaluation quality. Last, since SoftmaxCorr requires no held-out validation set, we can use all available data for training models.
Discussion on imbalanced test sets. SoftmaxCort is defined as the cosine similarity between cor- relation matrix C' and an identity matrix Kx. This implicitly assumes the test set is balanced: the number of samples in each class is roughly equal. This definition may not optimal when the test set is extremely imbalanced (e.g., some classes may only have one image being predicted into). For this point, note that some test sets in our study are moderately imbalanced, such as ImageNet-R, Ob- jectNet, iWildCam and DomainNet. We empirically observe that SoftmaxCor still maintains a high correlation these datasets. Under extremely imbalanced case, the definition needs to consider the importance weighting of each class. It would be helpful to use techniques of label shift estimation Lipton et al. 2018 Garg et al. 2020 Sun et al. 2022), ) and we leave it as our future work.
Potential OOD measures. This work proposes SoftmaxCorr to use class-wise relationship encoded by Softmax prediction probabilities. Here, discuss other potential ways. First, SoftGap com- putes the difference of largest and second largest prediction probabilities. We show that SoftGap exhibits stronger correlation with performance compared to MaxPred. would be interesting to improve SoftGap by utilizing more probabilities (e.g., top five probabilities). Second, for per- feetly calibrated model, its MaxPred over test set corresponds to its accuracy. Yet, calibration methods seldom exhibit desired calibration performance under the distribution shift (Ovadia et al.] 2019. That said, it would be promising to study post-hoc calibration methods for OOD datasets, which benefits MaxPred for assessing model generalization. Third, in some application scenarios, we might able to obtain a small number of labeled test data. It would be helpful to study the way of using them to improve the OOD generalization assessment. Last, this work focuses on Softmax prediction probability. We tested our method based on logits but no obvious correlation is exhibited. This may be because the logits of different models vary in significantly different ranges. Moreover, studying other model statistics (e.g., weights and feature representations) would be interesting.
7 CONCLUSION
This paper studies an under-explored problem of assaying model generalization under distribution shift. To this end, we explore the use of Softmax prediction probability for developing OOD mea- sures. We start by finding that maximum Softmax prediction probability is to some extent useful to measure the OOD performance. We then propose Softmax Correlation (SoftmaxCorr) which lever- ages class confusion encoded by class-class correlation matrix and thus better reflects the overall quality of the classifier predictions. To validate the usefulness of SoftmaxCorr, we compare with four other empirical measures across 19 datasets under ImageNet, CIFAR-10 and WILDS setups. We observe SoftmaxCorr generally presents more stable and higher correlation with model gener- alization on both ID and OOD datasets. This paper establishes some baseline usage of Softmax prediction probability and a specific improvement, and more investigation will be made in future.
9

---

Under review as a conference paper at ICLR 2023
REFERENCES
David Acuna, Guojun Zhang, Marc T Law, and Sanja Fidler. f-domain adversarial learning: Theory and algorithms. In International Conference on Machine Learning, pp. 66-75, 2021.
Sanjeev Arora, Rong Ge, Behnam Neyshabur, and Yi Zhang. Stronger generalization bounds for deep nets via a compression approach. In International Conference on Machine Learning, pp- 254-263, 2018.
Yuki Markus Asano, Christian Rupprecht, and Andrea Vedaldi. Self-labelling via simultancous clustering and representation learning. In International Conference on Learning Representations, 2019.
Christina Baek, Yiding Jiang, Aditi Raghunathan, and Zico Kolter. Agreement-on-the-lmne Predict- ing the performance of neural networks under distribution shift. Advances in Neural Informa- fion Processing Systems, 2022.
Robert Baldock, Hartmut Maennel, and Behnam Neyshabur. Deep learning through the lens of ex- ample difficulty. In Advances in Neural Information Processing Systems, volume 34, pp. 10876- 10889, 2021.
Peter Bandi, Oscar Geessink, Quirine Manson, Marcory Van Dijk, Maschenka Balkenhol, Meyke Hermsen, Babak Ehteshami Bejnordi, Byungjae Lee, Kyunghyun Paeng, Aoxiao Zhong, et al. From detection of individual metastases to classification of lymph node status at the patient level: the camelyon17 challenge. IEEE Transactions on Medical Imaging, 2018.
Andrei Barbu, David Mayo, Julian Alverio, William Luo, Christopher Wang, Dan Gutfreund, Josh Tenenbaum, and Boris Katz. Objectnet: A large-scale bias-controlled dataset for pushing the limits of object recognition models. In Advances in Neural Information Processing Systems, 2019.
Peter L Bartlett, Dylan Foster, and Matus Telgarsky. Spectraaly- margin bounds for neural networks. In Advances in Neural Information Processing Systems, 2017.
Sara Beery, Arushi Agarwal, Elijah Cole, and Vighnesh Birodkar. The iwildcam 2021 competition dataset. arXiv preprint arXiv: 105x:2005.03444, 2021.
Shai Ben-David, John Blitzer, Koby Crammer, and Fernando Pereira. Analysis of representations for domain adaptation. In Advances in Neural Information Processing Systems, 2006.
Shai Bev-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wort- man Vaughan. A theory of learning from different domains. Machine Learning, 79(1)1511-755, 2010.
Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Un- supervised learning of visual features by contrasting cluster assignments. In Advances in Neural Information Processing Systems, volume 33, pp. 9912-9924, 2020.
Ching Yao Chuang, Youssef Mroueh, Kristjan Greenewald, Antonio Torralba, and Stefanie Jegelka. Measuring generalization with optimal transport. Advances in Neural Information Processing Systems, pp. 8294-8306, 2021.
Luke N Darlow, Elliot Crowley, Antreas Antoniou, and Amos Storkey, Cinic- is not imagenet or cifar-10. arXiv preprini urXiv:1810.088 2018.
Ailin Deng, Shen Li, Miao Xiong, Zhirui Chen, and Bryan Hooi. Trust, verify: Using self- supervised probing to improve trustworthiness. In European Conference Computer Vision, 2022a.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In Proceedings of the IEEE Conference on Computer Vision and Patlern Recognition, pp. 255, 2009.
10

---

Under review as a conference paper at ICLR 2023
Weijian Deng, Stephen Gould, and Liang Zheng. On the strong correlation between model invari- ance and generalization. In Advances in Neural Information Processing Systems, 2022b.
Laurent Dinh, Razvan Pascanu, Samy Bengio, and Yoshua Bengio. Sharp minima generalize for deep nets. In International Conference on Machine Learning pp. 1019-1028, 2017.
Josip Djolonga, Jessica Yung, Michael Tschannen, Rob Romijnders, Lucas Beyer, Alexander Kolesnikov, Joan Puigcerver, Matthias Minderer, Alexander D' A'mour, Dan Moldovan, et al. On robustness and transferability of convolutional neural networks. In Proceedings of the IEEE Con- ference on Computer Vision and Pattern Recognition, pp. 16458-16468, 2021.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An im- age is worth 16x16 words: Transformers for image recognition at scale. In International Confer- ence on Learning Representations, 2020.
Kshitij Dwivedi and Gemma Roig. Representation similarity analysis for efficient task taxonomy & transfer learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12387-12396, 2019.
Gintare Karolina Dziugaite and Daniel M Roy. Computing nonvacuous generalization bounds for deep (stochastic) neural networks with many more parameters than training data. In Proceedings of the Thirty-Third Conference on Uncertainty in Artificial Intelligence, 2017.
Logan Engstrom, Andrew lyas, Shibani Santurkar, Dimitris Tsipras, Jacob Steinhardt, and Alek- sander Madry. Identifying statistical bias in dataset replication. In International Conference on Machine Learning. pp. 2922 2922-2932. PMLR, 2020.
Ustinova Ganin, Germain Ajakan, Laviolette Larochelle, Marchand, and Lempitsky. Domain- adversarial training of neural networks. In Journal of Machine Learning Research, 2016.
Saurabh Garg. Yifan Wu, Sivaraman Balakrishnan, and Zachary Lipton. A unified view of label shift estimation. In Advances Neural Information Processing Systems, pp. 3290-3300, 2020.
Saurabh Garg, Sivaraman Balakrishnan, Zachary C Lipton, Behnam Neyshabur, and Hanie Sedghi. Leveraging unlabeled data to predict out-of-diee of--istribuuiinn performance. In International Conference on Learning Representations, 2022.
Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix Wichmann, and Wieland Brendel. Imagenet-trained cans are biased towards texture; increasing shape bias im- proves accuracy and robustness. In International Conference on Learning Representationss 2019.
Devin Guillory, Vaishaal Shankar, Sayna Ebrahimi, Trevor Darrell, and Ludwig Schmidt. Predicting with confidence on unseen distributions. Proceedings of the IEEF/CVF International Confer- Computer Vision, pp. 1134-1144, 2021.
Ishaan Guirajani and David Lopez-Paz. In search of lost domain generalization. In International Conference on Learning Representations, 2021.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International Conference Machine Learning, pp. 1321-1330, 2017.
Matthias Hein, Maksym Andriushchenko, and Julian Bitterwolf. Why relu networks yield high- confidence predictions far away from the training data and how to mitigate the problem. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 41- 50, 2019.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common cor- ruptions and perturbations. International Conference Learning Representations, 2019.
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of distribution examples in neural networks. In International Conference Learning Representations, 2016.

---

Under review as a conference paper at ICLR 2023
Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. The many faces of robustness: A critical analysis of out-of-distributionn generalization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8340-8349, 2021.
Dan Hendrycks, Steven Basart, Mantas Mazeika, Mohammadreza Mostajabi, Jacob Steinhardt, and Dawn Song. Scaling out-of-distriuution detection for real-world settings. In International Con- ference Machine Learning, 2022.
Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.048661 2017.
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 4700-4708, 2017.
Peter J Huber. Robust statistics. In International Encyclopedia of Statistical Science, pp. Springer, 2011.
Yiding Jiang, Pierre Foret, Scott Yak, Daniel Roy, Hossein Mobahi, Gintare Karolina Dziugaite, Samy Bengio, Suriya Gunasekar, Isabelle Guyon, and Behnam Neyshabur. Neurips 2020 compe- tition: Predicting generalization in deep learning. arXiv preprint arXiv:2012.07976 2020a.
Yiding Jiang, Behnam Neyshabur, Hossein Mobahi, Dilip Krishnan, and Samy Bengio. Fantas- tic generalization measures and where to find them. In International Conference on Learning Representations, 2020b.
Maurice George Kendall. Rank correlation methods. Griffin, 1948.
Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Pe- ter Tang. On large-batch training for deep learning: Generalization gap and sharp minima. In International Conference on Learning Representations, 2017.
Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Bal- subramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, et al. Wilds: A benchmark of in-the-wild distribution shifts. In International Conference on Machine Learning, pp. 5637-5664, 2021.
Wouter M Kouw and Marco Loog. A review of domain adaptation without target labels. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(3):766-785, 2019.
Nikolaus Kriegeskorte, Maricke Mur, and Peter A Bandettini. Representational similarity analysis- connecting the branches of systems neuroscience. Frontiers in Systems Neuroscience, pp. 4, 2008.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. Citeseer, 2009.
Sampo Kuutti, Richard Bowden, Yaochu Jin, Phil Barber, and Saber Fallah. survey of deep learn- ing applications to autonomous vehicle control. IEEE Transactions Intelligent Transportation Systems, 22(2):712-733, 2020.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):228823244 1998.
Dong-Hyun Lee et al. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on Challenges in Representation Learning, number 2, pp- 896. 2013.
Jian Liang, Dapeng Hu, and Jiashi Feng. Do really need access the source data? source hypothesis transfer for unsupervised domain adaptation. In International Conference on Machine Learning, pp. 6028-660 60399, 2020.
12

---

Under review as a conference paper at ICLR 2023
Shiyu Liang, Yixuan Li, and Rayadurgam Srikant. Enhancing the reliability of image detection in neural networks. In International Conference on Learning Representations, 2018.
Tengyuan Liang, Tomaso Poggio, Alexander Rakhlin, and James Stokes. Fisher-rao metric, ge- ometry, and complexity of neural networks. In The 22nd International Conference Artificial Intelligence and Statistics, pp. 888-896, 2019.
Zachary Lipton, Yu-Xiang Wang, and Alexander Smola. Detecting and correcting for label shift with black box predictors. International Conference on Machine Learning, pp. 3122-3130, 2018.
Evan Z Liu, Behzad Haghgoo, Annic S Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. Just train twice: Improving group robustness without training group information. International Conference on Machine Learning, pp- 6781-6792, 2021a.
Jeremiah Liu, Zi Lin, Shreyas Padhy, Dustin Tran, Tania Bedrax Weiss, and Balaji Lakshmi- narayanan. Simple and principled uncertainty estimation with deterministic deep learning via distance awareness. In Advances in Neural Information Processing Systems, pp. 7498-7512, 2020a.
Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. Energy-based outto--distributton detec- tion. In Advances in Neural Information Processing Systems, pp. 11464-214755 2020b.
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 10012-10022, 2021b.
Zhuang Liu, Hanzi Mao, Chao- Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 11976-10 11986, 2022.
Lucas Mansilla, Rodrigo Echeveste, Diego H Milone, and Enzo Ferrante. Domain generalization via gradient surgery. In Proceedings of the IEEE International Conference on Computer Vision, 2021.
John Miller, Rohan Taori, Aditi Raghunathan, Shiori Sagawa, Pang Wei Koh, Vaishaal Shankar, Percy Liang, Yair Carmon, and Ludwig Schmidt. Accuracy the line: the strong correlation between out-of-distribution and in-distribution generalization. In International Conference on Machine Learning, pp. 7721-7735, 2021.
Matthias Minderer, Josip Djolonga, Rob Romijnders, Frances Hubis, Xiaohua Zhai, Neil Houlsby, Dustin Tran, and Mario Lucie. Revisiting the calibration of modern neural networks. In Advances in Neural Information Processing Systems, pp. 15682-15694, 2021.
Vaishnavh Nagarajan and Zico Kolter. Generalization in deep networks: The role of distance from initialization. arXiv preprini arXiv: 19010016290.. 2019.
Behnam Neyshabur, Ryota Tomioka, and Nathan Srebro. Norm-based capacity control in neural networks. In Conference on Learning Theory, pp. 1376-1401, 2015.
Behnam Neyshabur, Srinadh Bhojanapalli, David McAllester, and Nati Srebro. Exploring general- ization deep learning. In Advances in Neural Information Processing systems, 2017.
Behnam Neyshabur, Srinadh Bhojanapalli, and Nathan Srebro. A pac-bayesian approach to spectrally margin bounds for neural networks. International Conference on Learn ing Representations, 2018.
Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan, and Jasper Snock. Can you trust your model's ' uncertainty? evaluating predictive uncertainty under dataset shift. In Advances Neural Information Process- ing Systems, 2019.
13

---

Under review as a conference paper at ICLR 2023
Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment matching for multi-source domain adaptation. In Proceedings of the IEEE International Conference on Computer Vision, pp. 1400--145,, 2019.
Gabriel Pereyra, George Tucker, Jan Chorowski, Lukasz Kaiser, and Geoffrey Hinton. Regularizing neural networks by penalizing confident output distributions. In International Conference on Learning Representations, 2017.
Fengchun Qiao and Xi Peng. Uncertainty-guided model generalization unseen domains. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6790- 6800, 2021.
Benjamin Recht, Rehecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do cifar-10 classifiers generalize to cifar-10? arXiv preprint arXiv: 1006:01806000451,, 2018.
Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do imagenet classifiers generalize to imagenet? In International Conference on Machine Learning, pp. 5389-5400, 2019.
Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generaliza- tion. In International Conference on Learning Representations, 2020.
Shiori Sagawa, Pang Wei Koh, Tony Lee, Irena Gao, Sang Michael Xie, Kendrick Shen, Ananya Kumar, Weihua Hu, Michihiro Yasunaga, Henrik Marklund, et al. Extending the wilds benchmark for unsupervised adaptation. arXiv preprint arXiv: 221::21220000 1200000 2021.
Yair Schiff, Brian Quanz, Payel Das, and Pin-Yu Chen. Predicting deep neural network generaliza- tion with perturbation response curves. In Advances in Newral Information Processing Systems, pp. 21176-21188, 2021.
Claude Elwood Shannon. A mathematical theory of communication. The Bell System Technical Journal, 27(3))379-423 1948.
Yuan Shi and Fei Sha. inoommationthhoretiicn learning of discriminative clusters for unsupervised domain adaptation. In International Conference on Machine Learning, 2012.
Yuge Shi, Jeffrey Seely, Philip HS Torr, N Siddharth, Awni Hannun, Nicolas Usunier, and Gabriel Synnaeve. Gradient matching for domain generalization. arXiv preprint arXiv:2104.099:) 2021.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv: 1556, 2014.
Samuel L Smith and Quoe V Le. A bayesian perspective on generalization and stochastic gradient descent. In International Conference on Learning Representations, 2018.
Kihyuk Sohn, David Berthelot, Chun-Liang Li, Zizhao Zhang, Nicholas Carlini, Ekin D Cubuk, Alex Kurakin, Han Zhang, and Colin Raffel. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. arXiv preprint arXiv:2001.0768: 2020.
Baochen Sun and Kate Saenko. Deep coral: Correlation alignment for deep domain adaptation. In European Conference on Computer Vision, pp. 443-450, 2016.
Tao Sun, Cheng Lu, and Haibin Ling. Prior knowledge guided unsupervised domain adaptation. In European Conference on Computer Vision, 2022.
Remi Tachet des Combes, Han Zhao, Yu-Xian Xiang Wang, and Geoffrey J Gordon. Domain adapta- tion with conditional distribution matching and generalized label shift. In Advances in Neural Information Processing Systems, pp. 19276-19289, 2020.
Mingxing Tan and Quoc Le. Efficientnet: Rethinking model scaling for convolutional neural net-
works. In International Conference Machine Learning, pp. 6105-6114 6114, 2019.
Joost Van Amersfoort, Lewis Smith, Yee Whye Teh, and Yarin Gal. Uncertainty estimation using single deep deterministic neural network. International Conference on Machine Learning, pp- 9690-9700, 2020.
14

---

Under review as a conference paper at ICLR 2023
Vladimir Vapnik. Principles of risk minimization for learning theory. In Advances in Neural Infor- mation Processing Systems, 1991.
Ramakrishna Vedantam, David Lopez-Paz, and David J Schwab. An empirical investigation of domain generalization with empirical risk minimizers. In Advances in Neural Information Pro- cessing Systems, pp. 28131-28143, 2021.
Riccardo Volpi, Hongseok Namkoong. Ozan Sener, John Duchi, Vittorio Murino, and Silvio Savarese. Generalizing to unseen domains via adversarial data augmentation. In Advances in Neural Information Processing Systems, 2018.
Haohan Wang, Songwei Ge, Zachary Lipton, and Eric P Xing. Learning robust global representa- tions by penalizing local predictive power. In Advances in Neural Information Processing Sys- tems, 2019.
Tongzhou Wang and Phillip Isola. Understanding contrastive representation learning through align- ment and uniformity on the hypersphere. In International Conference on Machine Learning, pp. 9929-999999 9939, 2020.
Florian Wenzel, Andrea Dittadi, Peter Vincent Gehler, Carl-Johann Simon-Gabriel, Max Horn, Do- minik Zietlow, David Kernert, Chris Russell, Thomas Brox, Bernt Schiele, et al. Assaying out- of-distribution generalization in transfer learning. In Advances in Neural Information Processing Systems, 2022.
Ross Wightman. Train cifar10 with pytorch.
httpp://ggtiibb..om//uuaggg
pytorch-cifad 2017.
Ross Wightman. Pytorch image models. 2019.
http:///gtthub..cmmm/i.ttt
Qizhe Xie, Minh-Thang Luong, Eduard Hovy, and Quoc V Le. Self-training with noisy student improves imagenet classification. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 11687-10698, 2020.
Ruijia Xu, Guanbin Li, Jihan Yang, and Liang Lin. Larger norm more transferable: An adaptive fea- ture norm approach for unsupervised domain adaptation. In The IEEE International Conference on Computer Vision, 2019.
Jingkang Yang. Kaiyang Zhou, Yixuan Li, and Ziwei Liu. Generalized out-of-diisributioo detection: A survey, arXiv preprint 1rXiv:2110.11333 2021a.
Shiqi Yang. Joost van de Weijer, Luis Herranz, Shangling Jui, et al. Exploiting the intrinsic neighbor- hood structure for nuuree-free domain adaptation. In Advances in Neural Information Processing Systems, pp. 29393-29405, 2021b.
Kaichao You, Yong Liu. Jianmin Wang, and Mingsheng Long. Logme: Practical assessment of pre-trained models for transfer learning. International Conference on Machine Learning, pp. 22133-12143 2021.
Fisher Yu, Dequan Wang, Evan Shelhamer, and Trevor Darrell. Deep layer aggregation. In Pro- ceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2403-2412, 2018.
Yuchen Zhang, Tianle Liu, Mingsheng Long, and Michael Jordan. Bridging theory and algorithm for domain adaptation. In International Conference on Machine Learning, pp. 7404-7413, 2019.
Long Zhao, Ting Liu, Xi Peng, and Dimitris Metaxas. ceximmum-entrope adversarial data augmen- tation for improved generalization and robustness. In Advances Neural Information Processing Systems, 2020.
Kaiyang Zhou, Ziwei Liu, Yu Qiao, Tao Xiang, and Chen Change Loy. Domain generalization: A survey. JEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
15

---

Under review as a conference paper at ICLR 2023
A APPENDIX
A.1 EXPERIMENTAL SETUP
A.1.1 IMAGENET SETUP
(1) ImageNet Models
There 173 models used in experiment. All of models trained fine-tuned on ImageNet- 1k (Deng et al. 2009). The weights of the models are publicly avaiable via TIMM-1.5 (Wight- man 2019). For the choice of the models, we refer the supplementary materials of Deng et al. b Miller et al. (202T); Baek et al.] (2022). Moreover, we additionally include fol- lowing models: 'resnerlOr' 'resmetl4r'. 'darknet53', "csidarknet.m', cs3darkmer,focusaa "cs3darknet I', "cs3darknet focus I', 'regnety 040', 'regnety 064', 'regnety 080', 'regnetv 040', 'regnetv 064', 'regnetz 040', 'regnetz 040h')
(2) ImageNet Datasets
The datasets we use are publicly available standard benchmarks. We have double checked their license. We list their open-source as follows:
ImageNet-Validatin (Deng et al. 2009)
(https//www.image-nm)
ImageNet-V2-A/BIC (Recht et al. 2019) http:///whhtbb.ooooo////////////
ImageNet-Corruption (Hendrycks Dietterich 2019)
ImageNet-Sketch (Wang al.) 2019))
http://gihhu..oo/HHHohoo Wang/lmageeNet-Skechh
ImageNet-Rendition (Hendrycks et al. (021) (https://github.comm)
ObjectNet (Barbu et 20.19)019) (https//obbeettnetdee))
(Geirhos et al. 2019) (ttp::///ttillbooo//g/g/o///////
A.1.2 CIFAR-10 SETUP
(1) CIFAR-10 Models
Follow the practice in (Miller al. 2021), train 65 CIFAR models using the imple- mentations from https: /ggithu.ccom/ wooom/uuangiini wungkuuu///yyoooccccffaaa The models span a range of manually designed architectures and the results of automated architecture searches. Specif- ically, we use:
'Densenet-cifar' 'DLA', 'DPN26/92', 'Ef- ficientNetBO', 'GoogLeNet'. 'LeNer', 'MobileNet', 'MobileNerV2'. 'PNASNetA/B', 'PreActRes- Ner18/34//0/110/1552 RegNeeXX2000F/4000 http:///---/kkA//222A4/////--/// PleDLA', VGG-11/13/16/19),
'RegNetY-400MF'; 'ResNer- Re530er1813455000111//2 'SENet/8", 'Shuflleeri22', "ShufleNerG2/G33. 'Sim-
Furthermore, we use the trained models publicly provided by https//gghhubbcmmm/caaaaaaaapppp cifar- They are {'cifarl0-ni "ccfar100mmoblleeeen22213 mobilenenv2-xl-00,, "cffur10-mobbieeenn2=-144 'cifarl--repvggaa)" 'cifarl0-repvgg-al" 'cifarl0-
10 'cifarl0-
repvgg-a2', 'iffar10-resnee20", "cifar10-resnett3) cifaa10--resne444. 'cifar10-resnet560: 'cifarl0-
Shufflenen2-x00-5,,, Shtta-00-sufffeeeee2-ex x1-O', Sharl00-suuffenen2-x1--5: Siftr10-shuffennee
x2-0'. 'cifar10-vggll-be 'cifar10-vgg13: 3-nbb''' 'cifar10-vggl6 'iffar0--vggI--nn'
(2) CIFAR-10 Datasets
The datasets are publicly available benchmarks. Their source are as follows:
CIFAR-10 (Krizheskyy et al. 2009) (https://www.. http://wwwwwwwww.o...oooooo kriz/cifar.html):
CIFAR-10-C (Hendrycks & Dietterich 2019) https:/ggitubb.om//enttt/s/////p
CIFAR-10.1 (Recht et al. (2018) http:///hhbbbooooooooyyyaaact/c/ htt.a///.gggg//gggo/oooooooooooa
CINIC (Darlow al. 2018) https///thhhhccom//yyyyaaaahhhhh
A. 1.3 WILDS SETUP
(1) iWildCam Models
We use the publicly available code (https://gtthub.com hat.d//llmd///widds to finetune 66 models that pretrained on ImageNet. Following the practice in (Miller al. 202T). model variation results
16

---

Under review as a conference paper at ICLR 2023
from model architecture and learning rate. We train for 12 epochs with batch size 16 using Adam and sweep over learning rate in the grid {le 3. 1ele 1e. The other Adam parameters were set to the Pytorch defaults. The models are: { 'alexnet', 'mobilenet v2', 'resnet18', 'resnet34', 'resnet50", 'vggl1', "densenet121", efficientne bO, efficientnet bl, 'regnet 400mf', 'regnet 800mf', 'shuf- 12 x2huftenet
(2) Camelyon17 Models
We use the publicly available code (https://github.com/// httphhhbbcooomp-lmmdddiiids lambda/wills) finetune 45 models that are pretrained on ImageNet. Models vary in model architecture and random seed. The model archi- tectures are: d'alexnet, 'vggll', 'densenet221' 'resnet18", 'resnet34', 'resnet50", 'resnet101", 'resnex:50 _0 _3xx4d' 'wide_resnet502" 2' 'mobilenet.v3 _argee, 'regnet.y-166ff" "shufflenet.22xx0.5,, 'shufflenet v2 x1 5', 'shufflenet v2 x2 0', 'regnet y-400ff'' yregnet 800mf', }.
(3) WILDS Datasets
iWildCam (Beery al., 2021 http://www..gggeccmm/////lmaaaaa Camelyon17 (Bandi et al. 2018) httpss//cmmelyyo//a/gaaadlalllll
A.1.4 DOMAINNET
Domain adaptation algorithms. We use publicly accessible weights provided in: https://workshects.. https//wworshhccs..ddub.oooo are trained by following optimization algorithms: ERM (Vapnik 1991). DeepCORAL (Sun &
laa50a. These models
Saenko, 2016), DANN (annin et al. (016)) AFN (Xu et al. 2019), PseudoLabel (Lee et al. 2013).
NoisyStudent (Xie et al. 2020). FixMatch (Sohn et al. 2020). SWAV (Caron et al. 2020).
DomainNet Dataset (Peng et al.
2019) http://ti.bu.ccd/M333X4.
A.2 COMPUTATION RESOURCES
All experiment is run on one 2080Ti and the CPU AMD Ryzen Threadripper 2950X 16-Core Pro- cessor. We use Py PyTorch (1.10.2+cu102) for all the experiments.
A.3 DISCUSSION ON LOW PERFORMANCE OF SOFTMAXCORR ON CAMBLYON17-0OD
Camelyon17 17 is a binary image classification task where each example is a tissue patch. Its images metastasized breast cancer tissue samples collected from different hospitals. As noted by Koh
ct al. 2021 the held-out OOD test set contains tissue samples from a hospital not seen in the
training The distribution shift largely arises from differences in staining and imaging protocols across hospitals.
In our study, we observe similar underspecification phenomenon Miller et al. 202 Wenzel let al. (2022;: y very similar ID performances lead to very different OOD performances. This indicates that the models cannot learn robust patterns that can be transferred OOD data. Moreover, the similarity between images from the same slide or hospital is very high. This probably exacerbates the underspecification. Based on the above, we speculate that Softmaxcorr is less effective detecting the inderspecification issue on Camelyon 17-00D and thus has low correlation. That said, it would be interesting to further study this phenomenon future research.
A.4 COMPARED WITH MORE BASELINES
We additionally add three baselines: MaxLogit HaxLoggieendrrykss et al. 2022 Energy Liu et al.
2020b
Baldock and Adversarial Input Maldoncalldock et al. 2021p.
MaxLogits calculates the average of maximal unnormalized logits of all test samples.
Energy calculates the average of energy scores of all samples. Its formula for a single input æ E = log Σ3 (x)/T, where i' the temperature scaling hyper-parametee and ø (a)) indicates the logit returned by (xx) corresponding to jth class.
17

---

Under review as a conference paper at ICLR 2023
Table 4: Method comparison on three OOD test sets. We compare our SoftmaxCorr with three measures: MaxLogit, Energy and Complexity. We use Spearman's 's rank correlation (p) and weighted Kendall's correlation (Tw) for comparison. We show that our method stable and yields the highest average correlation across four datasets.
MaxL.ogi
Energy
Complexity
SoftmaxCorr
Dataset
Tw
Tw
Tw
Tw
ImageNet- V2-A
-0.267
-0.237
0.350
0.158
0.341
0.282
0.922
0.792
ImageNet-S
0.024
0,422
0.087
-0.180
0.329
0,417
0.875
0.883
CIFAR-10.2
0.210
-0.032
-0.180
-0.180
0.680
0.524
0.884
0.694
Adversarial Input Margin (Complexity) is proposed to measure the model complexity. It computes the smallest norm required for an adversarial perturbation in the input to change the model's class predictions. It estimates adversarial input margin y by linear approximation. Its formula is given by: y = minjæti |= [=,()) where x is the input, i means predicted class and Rg denotes the logit returned by the network for class j. We then take the average of ry over all data.
observe that Logtt and Energy exhibit weak correlation. While model complexity achieves some correlation on three datasets, it is still inferior to SoftmaxCorr.
As shown in Table 4
A.5 COMPARE SOFTMAXCORR WITH ACCURACY-ON-TII-LIN
We report the Spearman's 's rank correlations of accuracy-on-the-BBe and SoftmaxCort in Table We observe that accuracy-on-the con-th--line shows strong correlations on many datasets. However, we have similar observations the authors of accuracy-on-ther -inee--iee report: it exhibits weak correlation or even no correlation on CIFAR- 10-Nse,,, Camelyon 17-OOD and DomainNet-OOD (p = 0.003, -0.021 and 0.350). In comparison, SoftmaxCorr remains relatively informative and exhibits mod- crately high correlation (p = 0.813, 0.689 and 0.693) on the three test sets.
shows very strong correlations, Softmax is also competitive. Therefore, we think that SoftmaxCorr is overall a good alternative when labeled ID data is inaccessible or accuracy-on oo-tte---- does not hold.
We further note that when accuracy-on-the
A.6 AN ORACLE BASELINE: CONFUSION MATRIX
We may label a small number of OOD test data and compute the confusion matrix. Then, we calculate the cosine similarity between the computed confusion matrix and an identity matrix to assess model OOD generalization on the whole dataset. Specifically, we randomly label 1%. 5% and 10% of data and compute the cosine similarity between the confusion matrix and identity matrix. We report the average of Spearman's 's rank correlation over three random runs in Table 0
This oracle exhibits strong correlation given 5% of labeled test data. This suggests that exploring class confusion information for OOD generalization assessment is feasible. Specifically, if the cal- culated class-class correlation matrix ideally matches the true confusion matrix, then OOD general- ization can be perfectly predicted. Yet, we would like to discuss that this oracle may be impractical in some scenarios. First, data annotation could be very expensive and challenging. For example, precisely labeling wildlife categorization of iWildCam images is laborious and requires expert-level knowledge; annotating cancer tissue images of Camelyon 7 also requires expert-level knowledge. Second, when the test distribution is changed, we need to label data again which is also laborious.
A.7 SENSITIVITY ANALYSIS ON TEST SET SIZE OF ALL METHODS
We conducted a stratified analysis of all five methods on four OOD datasets: ImageNet-R, ImageNet-Blur, CINIC and iWildCam-OOD. Specifically, by randomly sampling 1%, 5%, 10%, 30% of test data. For each method under each test set size, we report the Spearman's 's rank correlation (p) of the average results of three random runs.
test different size of OOD datasets
18

---

Under review as a conference paper at ICLR 2023
Dataset
ID-Acc
SoftmaxCorr
Dataset
1 %
5%
10%
ImageNet-R
0.932
0,925
ImageNet-R
0.984
0.993
0.993
ImageNet-v2-A
0.995
0,922
ImageNet-v2-A
0.947
0.981
0.990
ImageNet-v2-B
0.994
0,912
ImageNet-v2-B
0.857
0.956
0.974
ImageNet-v2-C
0,995
0,917
ImageNet-vv-CC
0.922
0.979
0.986
ImageNet-S
0.934
0,875
ImageNet-S
0.990
0.997
0.998
Stylized-ImageNet
0.800
0.828
Boylized-ImageNNe
0.986
0.996
0.998
ObjectNet
0.976
0.927
ObjectNet
0.972
0,981
0.984
ImageNet-Blur
0.902
0.967
ImageNet -lur
0.989
0.993
0.995
CIFAR-10.2
0.958
0.884
CIFAR-10.2
0.350
0.617
0.706
CINIC
0.917
0.844
CINIC
0,842
0.928
0.943
CIFAR-IO-Noise
0,003
0.813
CIFAR-IO-Noise
0.842
0.950
0.956
iWildCam-OOe
0.944
0,916
iWildCam- OOD
0.848
0.870
0.868
Camelyon 17-0OD
-0.021
0.689
Camelyon 17-OOD
0.936
0.945
0.949
DomainNet-OOD
0.350
0.693
DomainNct-OOD 0.955
0.977
0.968
Table Comparison between Accuracy-on- the-line (ID-Acc) and SoftmaxCorr. Rank cor- relation (p) is the metric.
Table 6: Results of Confusion matrix. Rank cor- relation (p) is the metric.
IetgeNet-R
1%
5%
10 %
30%
100 %
ImageNet-Blur
1%
5%
10%
30 %
100%
MaxPred
0.618
0.623
0.612
0.615
0.613
MaxPred
0.863
0.877
0.878
0.878
0.877
SoftGap
0.798
0.803 0.794 0.799
0.797
SoftGap
0.883
0.895 0.897 0.898
0.898
Entropy
0.302
0.305 0.295
0.296
0.295
Entropy
0.767
0.778
0.779 0.780
0.780
InfoMax
0.407
0.431
0.426
0.426
0.423
InfoMax
0.830
0.848
0.850
0.851
0.850
SoftmaxCorr 0.873 0.928 0.925
0.926 0.925
SoftmaxCorr
0,948 0.966 0.967 0.967
0.967
Table 7: Sensitivity analysis of all methods on test set sizes. We report and find that SoftmaxCorr is relatively stable.
Table 8: Sensitivity analysis of all methods on test set sizes. We report and find that SoftmaxCorr is relatively stable.
CINIC
1%
5%
10% %
30%
100 %
iWildCam-OOD
1%
5%
10%
30%
100%
MaxPred
0.764
0.750
0.755
0.747
0.753
MaxPred
0.820 0.873
0.893
0.906
0.917
SoftGap
0.780
0.772 0.781
0.772
0.777
SoftGap
0.850
0.902 0.918 0.926
0.936
Entropy
0.749
0.733
0.733
0.717
0.722
Entropy
0.734 0.793 0.821 0.839
0.853
InfoMax
0.753
0.740
0.745
0.732
0.735
InfoMax
0.784 0.832 0,859
0.869
0.883
SoftmaxCor
0.655
0.777
0.827
0.857
0.844
SoftmaXCorr
0.826 0.874 0.896 0.903
0.916
Table 9; Sensitivity analysis of all methods on test set sizes. We report r and find that SoftmaxConr is relatively stable.
Table 10:
Sensitivity analysis of all methods test set sizes. We report r and find that Softmax- Corr is relatively stable.
In Tables and 10 we observe that 1) when test size is very small (2 5%). all methods have slightly lower correlation strength. For example, on ImageNet-R, InfoMax and SoftmaxCorr drop 0.024 and 0.055, respectively; 2) all methods have stable correlation results when test data is moderately sufficient. Specifically, when the correlation sampled ratio is larger than 10%.
methods becomes robust when the
19

---

Under review as a conference paper at ICLR 2023
A.8 RESULTS OF SOFTMAX CORRELATION
2.11
- 002
5312
D 017
340 040
031
045 3:1 2.00
037
0.5
troggNett- 122
Imsgelat-5
OApectMet
0.822
527
0.500
-.454
035
6:30
3.5
640
3.24
03806 37 055
6.0
(3035 CS7
053
031 120 304
036
059
0.813
+0.45
0.44
055
112
0.53
330
0.00
0.3
0.33
3:11
0.2
811
0.1
5.7
033
E 291
= Eno
0.000
6.7
0.00
034
0.31
6.07
030
31
Figure 6: CortmaxCorr vs. model generalization under ImageNet, CIFAR-10 and WILDS setups. The y-axis is accuracy, accuracy and macro-F1 for three setups, respectively. The straight lines fit with robust linear regression (Huber 2011). p. Axes are probit scaled as described in Section 3
20

---

Under review as a conference paper at ICLR 2023
A.9 RESULTS OF MAXIMUM PREDICTION PROBABILITY
5.542
8.1
031
041
5.1
0.11
0.3
031
3710
370
Trage 813
IrageNet-5
7S4
0
0112
0.977
0.0900
613
621
045
833
334 BM
6.1
548 635
347
3.99
049 0.5
0.9
6.11
0,222
0.872
0.17
122
035
313
048
662
6.00
0.15
0.90
6.2
3.5
875
0.0
030
0.55
043
0.7
6.0
6.90
3.00 3.81
5.2)
053
8.55
+. 400
647
124
3.52
0.55
0.54
Figure 7: MaxPred vs. model generalization under ImageNet, CIFAR-10 10 and WILDS setups. The y-axis is accuracy, accuracy and macro-Fl F1 for three setups. respectively. The straight lines are fit with robust linear regression (Huber 201 Axes are probit scaled as described in Section
21

---

Under review as a conference paper at ICLR 2023
A.10
RESULTS OF SOFTMAX GAP
5.000
& AGG
in EAM
0441
310
60
033
10
0.1
013
1.0
031
1.0
6.
Trage 797
tragehet- 232
Stpizad iragehot 2.601 2ym
Osjecitet
0.271
= 0.500
= 0.777
033
6.2 238
66. 039 0.7
0.3
355
6.54
8.23
8.0
0.1
8.50
8.51
5.55
1.0
0.041
0.369
= 2-00
=0 CAT
226
030
000
0.5
3.70
0.0
3.47
3.5
8.1
035 0.0 2.5
Candpoo17.000
0.112
2.44
* 4.77
Te.
S
237
2.47
353
03 24
Figure 8: SoftGap vs. model generalization under ImageNet, CIFAR- 10 and WILDS setups. The y-axis is accuracy, accuracy and macro-F1 for three setups. respectively. The straight lines are fit with robust linear regression (Huber 2011). Axes are probit scaled as described in Section
22

---

Under review as a conference paper at ICLR 2023
A.11
RESULTS OF NEGATIVE PREDICTION ENTROPY
D D217
B
235
012
1.9
315 E1)
22
072
043
100 114
036
817
8.0
Bikaadimageree 02233
291
D44
+0
0.790
0.4
-.722
4.00
4.59
6.00
4.1
4.3-4 032 4:35
4.89
-248 4.30 4034
4.3
4.40
4.30
422
0008
#B 75a
Eye 0.255
122
000
318
244 231
2.0
233 A31 1220
4:38 633 431
438
4.08
243
A37
A35
A31
DompirictD 0520
Domgi
187
0,526
240
-125
4.3
-85
Figure 9: Entropy vs. model generalization under ImageNet, CIFAR-10 and WILDS setups. The y-axis is accuracy, accuracy and macro-Fl for three setups. respectively. The straight lines are fit with robust linear regression (Hube 2011). Axes are probit scaled as described in Section
23

---

Under review as a conference paper at ICLR 2023
A.12
RESULTS OF INFORMATION MAXIMIZATION
D. Hb
2.223
0.3
3.3
641
3.10
1.1 012
6A1
011
030
tragghhtt c2!
D.14
U.512
0.0
000
0.222
0
6.5
880 036 023
543
343 0-1
042
6.00
035
1.00
0.833
0.0
04
1.3
054
8.55
8.25
9.5
044
034
633
140
649
035
0.2
3.20
0.3
640
3.0
655
0.0
6.0
371
1.
Damarhisl1 Nat-1
Dom30Ne10000 b721
U 505
0.63
037
642
3.31
3.00
6.7
Figure 10: InfoMax vs. model generalization under ImageNet, CIFAR-10 and WILDS setups. The y-axis is accuracy, accuracy and macro-F1 for three setups, respectively. The straight lines are fit with robust linear regression (Huber 2011). Axes are probit scaled as described in Section
24


Reviews:
[{"review_id": "drvwfzzjpW", "reviewer": "Reviewer_hesX", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work studies qualitative measures using prediction probabilities that can predict the out-of-domain generalization. They argue that softmax probabilities are informative of accuracy in new domains and moreover measures that maximally exploit structure in prediction probability are at an advantage. ", "summary_of_the_review": "The paper is mostly well-written and does a good job with its thorough systematic experiments. But some of the results and their method needs more explanation as further explained by my questions.\n\n--------------\n**Response to author comments**   \nI thank the authors for providing detailed clarification. Some of my concerns related to correctness of their results (such as questions 3, 4) are now addressed. I have also read other reviews and author responses. Reviewer 8Aud and I share similar concerns regarding comparison with other baselines, which the authors added in response. Accuracy-on-the-line explains accuracy poorly on 3/14 datasets while SoftmaxCorr is more consistent. However, it's unclear why the SoftmaxCorr is more consistent, even intuitively. Also, the definition of SoftmaxCorr is still not clear, what is $C.I$, elementwise product? \n\nIn summary, SoftmaxCorr is thoroughly evaluated and performs consistently well. Yet, I do not find the paper exciting. It is known that prediction probabilities are a good proxy for predicting accuracies and this work suggests that one could perform even better by also considering diversity of predictions. SoftmaxCorr is motivated from symptoms of a poor classifier: uncertain and skewed classification. So, a random classifier could as well be expected to have good performance on any dataset, which is why it is important to characterize and explicitly state the assumptions on the classifier and OOD datasets (and its relation to training data). The author's response to when SoftmaxCorr works is answered with more numbers on why it is important to look at diversity of predictions (along with confidence), and hence does not address my question. Also, the response on why SoftmaxCorr suffers on Camelyon-OOD is unconvincing. They explain that it could be due to underspecification, but why is underspecification only seen on this dataset and why does SoftmaxCorr fail for *underspecifed* setting despite not using ID datasets for accuracy predictions? \n\nThis work does not contain any surprising findings that I would like to eagerly share with a broader audience. For this reason, I recommend rejection and retain my original score.  \n\n", "strength_and_weaknesses": "Strengths.\n1. Developing predictive measures for accuracy is important and has several practical applications from ML safety to adaptation. \n2. Their method is simple and computationally inexpensive to derive. \n\n\nWeakness/questions.\n1. Cosine-similarity of matrices is not formally defined and there is no standard definition of the same either. \n2. Infomax vs SoftmaxCorr. Both the methods of estimation use almost similar information. They both look at diversity and confidence of predictions, but why is SoftmaxCorr so much better than InfoMax?\n3. Variant B and SoftmaxCorr of Table 2. They are very similar, yet on wilds dataset, variant-b is far worse (even negative). Variant-B, however, considers only the diagonal elements and I expect it should be even closer to the Identity matrix when compared with SoftmaxCorr. \n4. In all the figures, the authors should clarify what different data points (dots) in each plot are. Are these evaluations from different models? If so, how are they obtained and how do they differ?\n5. Lack of understanding. In the discussion section, authors remark about how in-domain validation is not always informative of out-domain and that SoftmaxCorr could be more universal. But I do not see what merits SoftmaxCorr to be universal, on one of the WILDS dataset, we see softmaxCorr does not predict accuracy well. I would like to see the authors comment on when and why SoftmaxCorr is expected to work. \n6. More evaluation. There have been several empirically derived measures of accuracy prediction, but the authors only compared with simple approaches that exploit strictly lower information from softmax probabilities (such as max probability or difference between highest two probabilities). Authors should compare with the most recent related work on this problem. The case of low performance on Camelyon-OOD should be further investigated and explained. \n7. The motivation of Section 5.5 is unclear, especially in the last sentence.  \n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "JcssoVlSvfF", "reviewer": "Reviewer_8Aud", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors propose a method, SoftmaxCorr, which is an easy to compute metric to evaluate the generalization performance of a neural network on an OOD dataset. The metric is computed as the cosine similarity between the identity matrix and a correlation matrix of the matrix of predictions on the OOD dataset. The metric empirically is shown to approximate well the confusion matrix. SoftmaxCorr compares favourably to other metrics which make use only of the network predictive probabilities to compute an OOD generalization metric, when evaluated on OOD datasets sharing the same label space as the ID dataset.", "summary_of_the_review": "The paper is well written and presents a clear, simple and effective metric for predicting OOD generalization performance. SoftmaxCorr compares well to other methods which only make use of the OOD softmax probabilities. However I have two major concerns: (i) I would have liked to have seen evaluation against other practical baselines which do not only use the softmax probabilities and (ii) I would have liked to see evaluation against far-OOD datasets and OOD datasets with different label spaces to the ID dataset, as there is existing evidence that ID accuracy is a good predictor of OOD performance for the near OOD datasets used in the paper.", "strength_and_weaknesses": "**Strengths**\n\n- The paper is well written and clear.\n- SoftmaxCorr is easy to compute and the underlying idea is simple and intuitive.\n- The comparison to the confusion matrix is compelling, Fig. 2.\n- SoftmaxCorr outperforms other metrics computed solely from the softmax predictive probabilities.\n\n**Weaknesses**\n\n*Comparison to non-softmax probability baselines*\n\nThe motivation for restricting the baselines to metrics computed solely on the softmax predictive probabilities is not clear to me. It would be understandable if the authors compared to other methods which may have additional requirements such as architectural changes, availability of ID labels, etc. and then stated that such methods have disadvantages due to these extra requirements. But given that these requirements may not be unreasonably prohibitive it seems to me that it is necessary to add such baselines.\n\nIn particular:\n\n- It would have been useful to compare against methods which make use of ID performance. W.r.t. \"we emphasize that SoftmaxCorr is overall a good alternative when labeled ID data is inaccessible or accuracy-on-the-line does not hold\", I regard it as atypical that labeled ID data would not be available so it would be reasonable to evaluate against such methods e.g. Miller et al. (2021) and/or Wenzel et al. (2022).\n- SNGP, Liu et al. (2020) and DUQ, Van Amersfoort et al. (2020), are competing methods. While they require architectural changes, it would be important to contextualize the SoftmaxCorr results by comparing against these methods.\n- A further interesting baseline, would be to use the confusion matrix directly (i.e. cosine similarity of the confusion matrix with the identity matrix). Clearly on new unlabelled data points we will not have labels with which to compute the confusion matrix. But given that the SoftmaxCorr metric must be computed on a full dataset rather than individual samples and that it effectively assumes a stationary OOD distribution, then it would not seem unreasonable to label a small number of these samples and then compute the confusion matrix directly. I am open to arguments as to why this may be impractical or undesirable in certain scenarios and naturally not requiring labels is an advantage of SoftmaxCorr in any case, nonetheless it would be useful to have this oracle and potentially practical method benchmarked in the experiments section.\n\n*Evaluation on far-OOD and OOD datasets with labels spaces different to the ID dataset*\n\nIdentifying OOD samples which are very far from the ID dataset, so far as not belonging to the same label space, is a key application of OOD detection metrics. Methods such as SNGP and DUQ have the advantage of being applicable to such scenarios. One major concern I have with the paper is that the OOD datasets are not very far from the ID datasets, for example ImageNet variants for ImageNet ID dataset and CIFAR variants for the CIFAR-10 training dataset. In such cases ID test accuracy has been shown to be a good predictor of OOD generalization Minderer, et al (2021).\n\nThe SoftmaxCorr metric could be computed on OOD datasets that do not share a label space with ID dataset. It is possible that SoftmaxCorr by approximating the confusion matrix is simply a good proxy for ID test set accuracy or similar metrics and that these metrics and SoftmaxCorr will not generalize to far OOD datasets. I would have liked the authors to have stress tested their method by evaluating it on datasets such as OOD=SVHN when ID=CIFAR-10 as done in Liu et al. (2020) and Van Amersfoort et al. (2020), and similarly more challenging label shifted datasets for the ID=ImageNet experiments.\n\n*Dataset level metric*\n\nThe SoftmaxCorr metric seems to be computed at the dataset level, this is quite a restrictive setting, meaning that individual samples cannot be scored and the metric cannot be directly applied to online prediction problems where inputs may come from a distribution shifting over time. I would be interested in the authors commenting on this disadvantage of the method. An interesting sensitivity analysis would be to progressively reduce the size of OOD dataset size N and measure the impact on the Table 1 results.\n\n\n\n\nLiu, Jeremiah, et al. \"Simple and principled uncertainty estimation with deterministic deep learning via distance awareness.\" Advances in Neural Information Processing Systems 33 (2020): 7498-7512.\n\nVan Amersfoort, Joost, et al. \"Uncertainty estimation using a single deep deterministic neural network.\" International conference on machine learning. PMLR, 2020.\n\n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "CD-YfPP8af", "reviewer": "Reviewer_Ku6q", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes an approach for estimating the out-of-distribution performance of a model using unlabeled test examples. The approach is relatively simple and involves computing the class correlation matrix based on the model's predictions on the text examples. Across a wide range of problems, models, and even training checkpoints, the proposed method shows strong empirical performance compared to well established baselines.", "summary_of_the_review": "In summary, the paper is reasonably well written and the experiments are sound, and I have recommended a few additions that will further strengthen the paper. I am recommending weak accept and am happy to discuss further.\n\nEdit after author response\n---\n\nApologies for the late reply. I appreciate the authors' comments, clarifications, and revisions, and I am maintaining my recommendation of acceptance.", "strength_and_weaknesses": "Strengths\n---\n\n- The paper makes experimental comparisons across a wide range of problems and models.\n- The paper is reasonably well written and easy to follow.\n- To the best of my knowledge, the proposed method is novel and addresses an important problem.\n\nWeaknesses\n---\n\n- The proposed method is not always the best performing, and test sample size is not considered (see below).\n- The paper would benefit from additional motivation and discussion around the method itself, which currently is explained only very briefly.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject