What Does Softmax Probability Tell Us about Classifiers Ranking Across Diverse Test Conditions?
Authors: Weijie Tu
Weijian Deng
Liang Zheng
Tom Gedeon
Venue: Accepted by TMLR ()

Abstract: This work aims to develop a measure that can accurately rank the performance of various classifiers when they are tested on unlabeled data from out-of-distribution (OOD) distributions. We commence by demonstrating that conventional uncertainty metrics, notably the maximum Softmax prediction probability, possess inherent utility in forecasting model generalization across certain OOD contexts. Building on this insight, we introduce a new measure called Softmax Correlation (SoftmaxCorr). It calculates the cosine similarity between a class-class correlation matrix, constructed from Softmax output vectors across an unlabeled test dataset, and a predefined reference matrix that embodies ideal class correlations. A high resemblance of predictions to the reference matrix signals that the model delivers confident and uniform predictions across all categories, reflecting minimal uncertainty and confusion. Through rigorous evaluation across a suite of datasets, including ImageNet, CIFAR-10, and WILDS, we affirm the predictive validity of SoftmaxCorr in accurately forecasting model performance within both in-distribution (ID) and OOD settings. Furthermore, we discuss the limitations of our proposed measure and suggest avenues for future research.

Published in Transactions on Machine Learning Research (05/2024)
What Does Softmax Probability Tell Us about Classifiers Ranking Across Diverse Test Conditions?
Weijie Tu
weijie. teijie.tu@anu.co edu..ou
Australian National University
Weijian Deng
weijjan.dengSSunu.ccck..
Australian National University
Liang Zheng
Beeng.zheng@annuecd.cc
Australian National University
Tom Gedeon
lom.gedeon@ccurtine edu..o.eeeuu
Australian National University
Curtin University
University of Obuda
Reviewed on OpenReview: https: // openreview. net/ peet/for forum? id= idggtsD
Abstract
This work aims to develop a measure that can accurately rank the performance of various classifiers when they are tested on unlabeled data from out-of-distribution (OOD) distribu- tions. We commence by demonstrating that conventional uncertainly metrics, notably the maximum Softmax prediction probability, possess inherent utility in forecasting model gener- alization across certain OOD contexts. Building this insight, we introduce a new measure called Softmax Correlation (SoftmaxCorr). It calculates the cosine similarity between class-class ion matrix, constructed from Softmax output vectors across an unlabeled test dataset, and a predefined reference matrix that embodies ideal class correlations. A high resemblance of predictions to the reference matrix signals that the model delivers confident and uniform predictions across all categories, reflecting minimal uncertainty and confusion. Through rigorous evaluation across suite of datasets, including ImageNet, CIFAR-10, and WILDS, we affirm the predictive validity of SoftmaxCorr in accurately forecasting model performance within both in-distribution (ID) and OOD settings. Furthermore, we discuss the limitations of our proposed measure and suggest avenues for future research.
1 Introduction
Machine learning (ML) models typically excel on test sets coming from the same distribution as the training set. However, this assumption seldom holds in real-world deployments, where the test environments often experience distribution shifts caused by factors such as sample bias and non-stationarity. Recognizing this challenge, there pressing ncod assoss MI model performance in unlabeled testing environments where traditional evaluation metrics which require data. annolations, such as accuracy and F1 score, may become infeasible. Hence, our focus shifts towards the vital yet under-explored task of ranking models under these conditions. Specifically, given a pool of trained models and an unlabeled test set, the objective is to efficiently select the most suitable model for utilization.
Various complexity measures have been proposed to predict the in-distribution (ID) accuracy models
(Jiang al., 2020a; Neyshabur al., 2015; Bartlett et al., 2017; Keskar al., 2017; Nagarajan & Kolter, 2019; Neyshabur al., 2017: Chuang et al., 2021; Jiang et al., 2020b; Smith & Le, 2018; Arora al., 2018; Dziugaite & Roy, 2017; Dinh et al., 2017). For the OOD test sets, classical domain adaptation theory provides

---

Published in Transactions on Machine Learning Research (05/2024)
partial answer by quantifying the distance between the ID and OOD distributions (Ben-David et al., 2006). Moreover, the "accuracy-on-the-line phenomenon Miller et al. (2021) shows the linear correlation in the probit scale between ID and OOD performance. However, such phenomenon does not always hold on some distributions (Teney et al., 2022). Building on previous research, our objective is to develop a robust measure capable of ranking models on both ID and OOD datasets without lesting labels.
Softmax prediction probability has been shown to be useful in analyzing test data in several tasks, such as open-set data detection (Hendrycks & Gimpel, 2016), accuracy estimation (Guillory et al., 2021; Garg et al., 2022), and misclassified imput detection (Deng et al., 2022). For example, Hendrycks & Gimpel (2016) and Liang et al. (2018) utilize maximum Softmax prediction probability to identify samples from open-set classes. Driven by these insights, we develop OOD measures based on Softmax probability. Concretely, given various deep models, we aim to develop probability- based measures that monotonically relate to OOD generalization. To validate the feasibility, we conduct extensive and large-scale correlation studies using various models and different types of dataset shifts. We construct a catalog of empirical prediction probability-base based measures and create wide range of experimental setups. We collect 573 different classification models ranging from standard convolutional neural networks to Vision Transformers. We cover 11 ID and OOD datasets with various types of distribution shift, such as ImageNet-V2 (Rocht et al., 2019) with dataset reproduction shift. and ImageNet-R (Hendrycks et al., 2021a) with style shift.
Based on experimental results, we first show the demand for measures to rank model performance beyond in-distribution accuracy (Miller et al., 2021). Then, we observe that empirical uncertainty measures based on prediction probabilities (e.g., maximum softmax probability) are useful in characterizing OOD generalization to some extent. However, we note that these measures solely account for prediction certainty. In response, we introduce SoftmaxCorr, a method designed to harness both prediction certainty and diversity. Confidence pertains to the certainty of individual predictions, whereas dispersity indicates the spread of predictions across all categories. Specifically, for each classifier, we compute a class-class correlation matrix from all prediction vectors in a test set. Then, we calculate its cosine similarity with a prodefined reference matrix, designed to represent desirable prediction patterns, to evaluate whether this classifier makes diverse and certain predictions. The broad correlation study shows the efficacy of SoftmaxCorr in ranking models.
2 Related Work
Predicting generalization in deep learning studies the ID generalization gap (i.e., the difference between training and test accuracy) of deep neural nelworks. Representative methods develop complexity measures based on model parameters and training set (Jiang et al., 2020a;b; Neyshabur et al., 2015; Keskar al., 2017; Bartlett et al., 2017; Neyshabur et al., 2018; Liang et al., 2019; Chuang et al., 2021; Smith & Lo, 2018; Arora el al., 2018; Dziugaite & Roy, 2017; Dinh et al., 2017; Dziugaite et al., 2020), such as distance of training weights from initialization (Nagarajan & Kolter, 2019), the product of norms of weights across layers (Neyshabur et al., 2017) and the change of model accuracy with respect to different perturbation levels in training data (Schiff et al., 2021). The above methods assume that training and test data come from the same distribution and do not incorporate characteristics of test data, so we can unlikely make reasonable predictions on a different distribution. To mitigate this limitation, we investigate the model generalization under distribution shift by developing measures that reflect the models' generalization ability on OOD datasets.
OOD generalization. Machine learning models should generalize from training distribution to OOD datasets (Djolonga et al., 2021; Koh et al., 2021; Kirsch & Gal, 2022). To study this problem, several benchmarks are proposed (Hendrycks & Dietterich, 2019; Koh et al., 2021; Gulrajani & Lopez-Paz, 2021), such as corruption benchmark (Hendrycks & Dictterich, 2019) and domain generalization testbed (Gulrajani & Lopez-Paz, 2021). Moreover, several methods are proposed to improve model OOD generalization (Volpi et al., 2018; Zhao et al., 2020; Sagawa et al., 2020; Liu et al., 2021; Mansilla et al., 2021; Shi et al., 2021; Krishnamachari et al., 2023; Zhang et al., 2021; Huang et al., 2023; Forct et al., 2020; Huang et al., 2022), such as adversarial domain augmentation (Volpi et al., 2018; Qiao & Peng, 2021; Alhamoud al., 2022) and inter-domain gradient matching (Shi et al., 2021).
2

---

Published in Transactions on Machine Learning Research (05/2024)
Prior research has established theoretical frameworks for assessing classifier performance amid distribution shifts. Ben-David et al. (2006) introduced the first. VC-dimension-basel the difference in classifier error between source and target distributions using a divergence measure. Mansour et al. (2009) extended this analysis to more general loss functions, providing refined generalization bounds via. Rademacher complexity. Other studies (Blitzer et al., 2007; Hoffman et al., 2018; Mansour el al., 2008) expanded upon these findings to include multiple source domains. Some works further bound the OOD generalization error based on the divergence between the two distributions (Acuna al., 2021; Zhang et al., 2019; Tachet des Combes et al., 2020) However, as suggested by Miller et al. (2021), when the distribution shift. becomes larger, the above bounds on OOD performance become looser. In addition, Vedantam et al. (2021) report that the adapting theory from domain adaptation is limited in predicting OOD generalization.
bound, quantifying
Unsupervised accuracy estimation (UAE) aims to predict the performance of a given model on various unlabelled out-of-distribution datasets. One line of research utilizes the model outputs on the test sets (Guillory et al., 2021; Garg et al., 2022; Deng et al., 2023; Tu et al., 2023b). For instance, Guillory et al. (2021) uses the maximum value of prediction probability to estimate model accuracy. A parallel line of works predicts model performance by gauging distribution discrepancy between training and test sets using image features (Deng & Zheng, 2021; Tu et al., 2023a; Xie et al., 2024; Lu et al., 2024). For example, Deng & Zheng (2021) uses the first- and second-order statistics of image features and Fréchet Distance to jointly measure the model performance. In contrast, other studies investigate this task through model weights (Yu et al., 2022).
While UAE predicts OOD accuracy of a single model across various test sets, our work focuses on ranking multiple models' OOD performance on an unlabeled test dataset. UAE methods may be deployed in this task by predicting the performance of each single model. However, to derive the estimated performance, most UAE methods (e.g., BoP (Tu et al., 2023a) or Dispersion score (Xie al., 2024)) require training an accuracy predictor a number of datasets. These operations make the deployment of methods computational expensive in our lasks. We have included efficient Softmax-based UAE methods in our study
3 Task Formulation
Task definition. We consider a K-way classification task, and let y = {1,.... K} denote the label space and X E dEd denote the input space. We are given a labeled training set DS that contains data i.idd drawn from a source distribution Ps, and an OOD lest set D" == (.) that contains data i.i.d drawn from another distribution Pr (Ps # Pr). We train M neural network classifiers [ on DS. Given a sample (a., y) from DJ, the classifier @m : X - A gives Softmax probabilities for on K classes, where A denote K- 1-1 dimensional unit simplex. By testing on D"" 2m yields prediction matrix P E R whose each row represents prediction probabilities of a test data. Specifically, the prediction matrix satisfies E Pto- = 1 Vi E I...N N and P.j 2 0 Vi E N.j E E1... 1...K, where P.g indicates the probability that i-th sample is predicted to the j-th class.
M
The dataset has an evaluation metric (e.g., accuracy) to obtain ground-truth generalization Gm of classifier Om. The goal to design a measure to calculate a score Sm for each classifier @m without access to data annotations. The calculated scores (S,m. JM idcally should correlate well with [Gm)M that we rank the OOD generalization of models based on the scores.
Evaluation metrics. We use Spearman's Rank Correlation coefficient p (Kendall, 1948) to measure monotonicity between calculated scores and model generalization. In addition, we also compute the weighted variant of Kendall's rank correlation Tw, which is shown to be a useful measure when selecting the best-ranked item of interest (You et al., 2021). Both coefficients range from [-1, A value closer to -1 or 1 indicates a strong negative or positive correlation, respectively, and 0 mcans no correlation. Similar to Miller al. (2021) and Back et al. (2022), we apply the same probit scale to both accuracy and SoftmaxCorr in our experiment for better linear fit.
3

---

Published in Transactions on Machine Learning Research (05/2024)
mmggeNos-S 1.505 0.467
imge 4525
Imagehet-S P 0.864
Imagelvvt-
110
0,529
Accuracy
0.431
0.684
Non-VLM
400
VLM
27%
633 644
346
0.86
633 241 648
430 a oão 2h1
C28 850
094
Accuracy on ImageNet-Val
Maximum prediction probebiiity
Figure 1: Corrclation study between MaxPred and accuracy (%) on ImageNet-S and ImageNet-R. Every point denotes classifier. We use 173 ImageNet models and 89 vision language models introduced in Section 5. The straight line is fit with robust linear regression (Huber, 2011) and the shadow means the 95% Clopper-Pearson confidence intervals. We show that MaxPred exhibits a moderate correlation with accuracy, while accuracy on ImageNee--validatton shows a relatively low correlation with performance on ImageNet-R. Moreover, Vision-Tanguage Models (VLMs) exhibit varying linear trends in terms of their ID and OOD accuracy compared to standard supervised models.
4 Softmax Probability-based OOD Measures
4.1 What Makes OOD Measures Interesting?
(AoL). Miller et al. (2021) report an AoL phenomenon where there exists a strong linear correlation between probit-scaled ID and OOD generalization. This implies that ID accuracy is good predictor of OOD generalization. However, delving deeper into OOD measures is warranted for three compelling reasons. First, Miller et al. (2021) discuss that AoL is not universal. That is, on some datasets, ID, and OOD accuracy do not show a clear positive correlation. This point further discussed by Wenzel al. (2022). Specifically, Wenzel et al. (2022) suggest two patterns preventing this phenomenon: (1) mndorpeecffiation (c.g., Camelyon17) where the same ID performance leads to different OOD behavior; (2) models do not transfer information from ID to OOD domains (e.g., DomainNet). That is, despite various ID performance, all models perform poorly on OOD datasets. Paradoxically, identifying the failure patterns itself requires labeled OOD data (Teney et al., 2024). Second, it is demanding and sophisticated to design ID test sets (Engstrom et al., 2020), which are expected to be unbiased and representative of distribution to effectively measure model TD accuracy. Further, it is a trade-off to split a full dataset into training, validation and test sets terms of training and evaluation quality. Third, recent advancements vision-language models, such as CLIP (Radford et al., 2021) and BLIP (Li et al., 2022), have achieved remarkable zero-shot classification performance. This means that collecting a dataset for specific task to train such models is not required. Due to their diverse and large-scale training data, it might not be suitable to use a proxy dataset (e.g., ImageNet validation set) to reflect their performance. As shown in Figure 1, Vision-Language models (VLM) exhibit varying linear trends in lerms of their ID and OOD accuracy compared to standard supervised models. Furthermore, even among VLMs themselves, consistency in these linear trends is not always guarantcod. These findings suggest that AoL alone does not suffice to accurately rank CLIP models.
Beyond Accuracy-on-the-Liin
Why Use Softmax Prediction Probability? Deep neural networks often exhibit tendency to provide overly confident predictions, as evidenced by various studies (Ovadia et al., 2019; Minderer et al., 2021; Guo et al., 2017: Hein et al., 2019). Initially, this characteristic might raise doubts about the reliability of using Softmax Prediction as a measure of uncertainty on test data. However, existing research has shed light on its informative nature when analyzing test environments. For instance, Hendrycks & Gimpel (2016) demonstrated that the maximum Softmax prediction probability (MaxPred) for correctly classified samples tends to be higher than that of incorrectly classified or out-of-distribne distribution (OOD) samples. This observation has paved the way for utilizing MaxPred in error arror/suceess prediction and ID/OOD detection. Building upon this insight, Guillory et al. (2021) and Garg et al. (2022) have proposed leveraging MaxPred to estimate the accuracy of trained classifiers on test samples.
Morcover, Softmax probabilities can be efficiently computed without requiring any changes to the network architecture or additional data preprocessing. Additionally, they can be derived solely from unlabeled data. These characteristies endow prediction probability-basel measures with substantial practical value reliable
4

---

Published in Transactions on Machine Learning Research (05/2024)
indicators of OOD generalization. Inspired by the above discussion, this work aims to validate the feasibility of using model output-based methods (such as softmax probability) for assessing and ranking models.
Proof of concept. The above works imply the prediction probability is likely to be effective in measuring the OOD performance of a pool of models. Given an OOD test set (ImageNet-R) and various ImageNet models, we conduct a correlation study between MaxPred and classification accuracy. In Figure 1, we show that there is a relatively strong correlation between MaxPred and model accuracy (p 0.829 and To 0.884) on ImageNet-R. This indicates that MaxPred is feasible in ranking OOD performance. Based this observation, we further explore more empirical prediction probability- based measures and develop a more effective mcasure that exploits more semantics reflected in the prediction probability.
4.2 Exploring More Empirical Measures
In addition to accuracy-on-the-lnne (AoL) and maximum prediction probability (MaxPred), we investigate other empirical measures as follows:
Average Thresholded Confidence with Maximum Confidence (ATC-MC) (Garg et al., 2022). This method is used to predict the performance of a specific trained model under distributional shift. Here, we deploy it to rank various models' performance on one particular OOD test set. It firstly identifies a threshold on DS such that the number of samples with confidence score lower than is equal to the model's error. Then, the ATC on D is given by the number of points whose confidence is less than t. The formula is: ATC =
BznD"
[arrg max < d]], where [[F] is the binary indicator of event E.
Softmax Gap (SoftGap). MaxPred only uses the maximal value of Softmax vectors while disregarding values on other entries. Inspired by Baldock et al. (2021), we introduce Soft based on MaxPred which further considers the second-largest entry in a prediction vector. Specifically, calculates the average difference between the largest and second-largest Softmax prediction probabilities over all samples. A high Gop indicates confident, prediction, while a low SoftGap suggests confusion between the two classes corresponding to the highost and sccond-highest probabilities.
4.3 Ours: Softmax Correlation (SoftmaxCorr)
Class-class correlation matrix Given the prediction matrix P RNNK predicted by om, a class correlation matrix E KK K is computed by C P.P. An entry Ca,, indicates a correlation between prediction probabilities of class and class j over all samples. We define the sum of diagonals of the correlation matrix as intra-class correlation (Ia) and the sum of off-diagonals as inter-class correlation (Ie). The sum of C is 1, which means La- = 11-ee Ie- Moreover, trace( PP = EN E P? = ||||| where ||-I1 is the N. According to Cui et al. (2020), P|| has strict
Frobenius norm of a matrix. Thus, we can derive L, = LP opposite monotonicity, and the minimum of the entropy H (P) and the maximum of P|| could be achievod at the same value. Therefore, Ia reflects the certainty of predictions.
Motivation: In domain genoralization, it has been observed that class-class correlation encodes class confusion, thereby offering the potential for regularization to enhance model generalization ability (Chen et al., 2022; Jin et al., 2020; Li et al., 2021; Cui et al., 2020). Additionally, existing literature underscores the significance of both high prediction certainty and prediction diversity in identifying discriminative fcatures (Yang et al., 2021; Wang& & Isola, 2020; Asano et al., 2019).
Inspired by these insights, we can propose an OOD metric, which leverages a class-class correlation matrix to take into account the two characteristics:
Prediction certainty: the model's ability to produce confident predictions. Given the strict opposite monotonicity between Ia and entropy HPP), prediction certainty can be reflected by high Ia and consequently a low 1. = 11 11-1, La within the class correlation matrix C.
Prediction diversity: relying solely on prediction confidence may be inadequate. For instance, a model may exhibit. high certainty yet be biased towards single class, making the diagonal entries of C skewed.
5

---

Published in Transactions on Machine Learning Research (05/2024)
In such cases, while prediction confidence is high, it does not necessarily indicate high model performance. Driven by this, we further evaluate prediction diversity: each class should be predicted and involved in predictions. To measure this, we expect the diagonals of C to be well-distributed across all classes instead of biased towards few classes.
Combining two characteristics, a desirable class correlation matrix has zeros on the off-diagonal entries, and values on diagonal elements match the distribution of classes. The similarity between the class correlation matrix of an evaluated model and the desirable class correlation matrix reflects the model performance.
Implementation: To achieve this, we propose SoftmaxCorr and define as the cosine similarity between
the class-class correlation matrix C and reference matrix R: cos(C, R) Σ,, (C,(C)))
[|||||| where @ is the element-wise product between matrices. The reference matrix captures the essence of the desirable class correlations: it is a diagonal matrix whose off-diagonal elements are 0 while its diagonal entries mirror the class marginal distribution. To approximate this class distribution, we use the average prediction probability across test. data generated by zero-shot vision-language model SoftmaxCorr is [0, 1]. For a fixed reference matrix, the maximum is obtained 1) when a model confidently assigns probability 1 to the predicted class and 2) the proportion of predictions for each class matches the class distribution. The minimal value is achieved when a model is extremely biased lowards one specific class with the prediction probability of one, while the estimated distribution for such a class is zero
The range of the
5 Experiments
In this section, we first describe three setups including ImageNet, CIFAR-10, and WILDS. Then, we analyze the experiment results of prediction probability-base -based measures on three setups. After that, we study the study whether
impacts of class distribution estimator on predictive ability of SoftmaxCorr. Furthermore, SoftmaxCorr can rank the performance of model checkpoints along the training trajectory. Also, validate the effectiveness of moftmaxCorr under the domain adaption setting. Lastly, we study the correlation between SoftmaxCorr and the accuracy of a single model on various OOD test sets.
5.1 Experimental Setup
ImageNet setup. We collect 173 models publicly accessible from TIMM (Wightman, 2019). They are trained or fine-tuned on ImageNet (Deng et al., 2009) and have various architoctures, training strategics and training paradigns. We use five OOD datasets for the correlation study. Specifically, OOD datasets are: (1) ImageNet- V2 (Recht al., 2019); (2) ObjectNet (Barbu al., 2019); (3) ImageNet-S(kketee (Wang et al., 2019); (4) ImageNet-Bllu severity (Hendrycks & Dietterich, 2019); (5) LmageNet-RRendiine (Hendrycks et al., 2021a); ImageNet-R and ObjectNet contain 200 and 113 ImageNet classes respectively.
CIFAR-10 setup. We collect 65 networks trained with the scheme provided by Wightman (2017) on CI training set (Krizhevsky al., 2009). These models have different model architectures. CIF Val(idation) is the ID) test set. For OOD datasets, we use (1) CIFAR-10.2 (Recht et al., 2018b) (2) CINIC (Darlow et al., 2018) (3) CIFAR-10-Noise with severity 5 (Hendrycks & Dietterich, 2019). We use accuracy the metric of model generalization.
WILDS setup. We consider a classification tasks of this setup: Camelyon17 (Bandi et al., 2018). It is a binary classification datasct where the objective to classify whether slide contains a tumor issuc. We use 45 models varying in architectures and random seeds. ID and OOD datasets are the default ID validation set and OOD test set respectively. For each task, we follow the same training scheme provided by Koh et al. (2021) to train or fine-tune models.
Zero-shot vision language models. In addition to models which are trained on the ID training datasct, we also consider 89 zero-shot vision-language models, including CLIP (Radford et al., 2021), SigLIT (Zhai et al., 2023), BLIP (Li et al., 2022), BLIIP-2 (Li et al., 2023) and Flava (Singh et al., 2022). We use default.
6

---

Published in Transactions on Machine Learning Research (05/2024)
Validation Required
Validation Free
Sctup
Dataset
AoL
ATC-MC
MaxPred
Gopt
SoftmaxCore
P
p
p
p
Tw
ImageNet-VV V2
0.954
0.911
0.991
0.961
0.711
0.597
0.796
0.641
0.921
0.758
ImageNet-A
0.593
0.636
0.830
0.895
0.756
0.813
0.805
0.846
0.961
0.915
ImageNet-R
0.452
0.431
0.950
0.954
0.829
0.884
0.902
0.911
0.951
0.928
ImageNet
ImageNet-S
0.505
0.467
0.981
0.959
0.862
0.864
0.887
0.871
0.935
0.909
ObjectNett
0.619
0.545
0.961
0.821
0.883
0.849
0.908
0.865
0.963
0.895
ImageNet-Blur
0.637
0.576
0.937
0.905
0.816
0.821
0.844
0.845
0.961
0.907
Average
0.627
0.594
0.942
0.916
0.810
0.805
0.857
0.830
0.949
0.885
CIFAR-10.2
0.983
0.949
0.992
0.966
0.837
0.809
0.854
0.818
0.894
0.836
CINIC
0.866
0.890
0.952
0.902
0.665
0.663
0.690
0.681
0.821
0.763
CIFAR-10
CIFAR-1O-Noise
0.641
0.765
0.219
0.461
0.051
0.131
0.139
0.213
0.931
0.917
Average
0.830
0.868
0.721
0.776
0.518
0.534
0.561
0.571
0.892
0.846
Mamellynl7-OOO1
-0.021
-0.022
-0.111
00.75
0.192
0.320
0.192
0.320
0.420
0.560
WILDS
DotO
0.350
0.219
0.513
0.254
0.403
0.274
0.407
0.258
0.740
0.680
Average
0.165
0.074
0.201
0.217
0.298
0.597
0.300
0.289
0.580
0.620
Average three setups
0.598
0.575
0.747
0.751
0.637
0.693
0.675
0.609
0.864
0.824
Table 1: Method comparison on ImageNet, CIFAR-10, WILDS and DomainNet. We compare SoftmaxCorr with four measures: accuracy-on-thh--lne (AoL) (Miller et al., 2021), average thresholded confidence with maximum confidence (ATC-MC) (Garg et al., 2022), MaxPred (Hendrycks & Gimpel, 2016) and SoptGap (Baldock et al., 2021). Spearman's 's rank correlation (p) and weighted Kendall's correlation (Tw) are metrics. The highest correlation in each row highlighted bold and the sccond highest in blue. We show that our method is stable and yields the highest average correlations over three setups.
prompt set for corresponding models. the default prompt sets are not provided, "A picture of {class}." is deployed. Unless specified, we use ViT-bigG/144CIIPPA to estimate class distribution for all setups.
5.2 Main Observations
Albeit requiring no access to validation/proxy set, SoftmaxCorr exhibits a strong correlation with model generalization. In Figure 2 and Table 1, we observe that SoftmaxCort is indicative of model performance under the three setups. Particularly on the ImageNet setup, SoftmaxCorr has consistently strong correlations with models' OOD performance. For example, the average Spearman's 's Rank Correlation coefficient p is 0.949, 0.627, 0.942, 0.810, and 0.857 for our method, AoI., ATC-MC, MaxPred and Soft respectively. We also notice that AoL exhibits very strong correlation with model accuracy on ImageNet- V2 (p 0.954), but such correlation drops noticcably insufficient to rank both supervised models and vision-language models based solely this phenomenon. On C-FAR-10 and WILDS, while on some test sets it does not present the strongest correlation, we still think that SoftmaxCorr is a preferred measure because it has very competitive average correlation scores. For Camelyon17, we see that all methods are less useful. We speculate this is caused by the siccffspeciiinttion phenomenon (Wenzel et al., 2022) where the model relies on spurious features.
other OOD datasets. This implics that it is
SoftmaxCorr gives stable correlation, while the other four measures have mixed performance on different test sets. On CIFAR-10-Noise, we find that SoftmaxCorr correlates well with model performance (p = 0.931 and Tar 0.917). In contrast, AoL, ATC-MC, MaxPred, SoftGap show a weaker correlation. Although ATC-MC has a higher average weighted Kendall's correlation than SoftmaxCorr (p = 0,916 us. 0.885) on ImageNet sctup, it exhibits a weaker correlation on CIFAR-10 and WILDS setups. While in some cases SoftmaxCorr does not present the highest correlation, we emphasize that it overall gives more stable and stronger correlations. Thus, we think SoftmaxCorr is generally indicative of model generalization.
7

---

Published in Transactions on Machine Learning Research (05/2024)
ObjectNet = 0. Tu. 0.895
ImageNet-S
ImageNet-Blur =0.961 Tup
Image setup
=0.935 Ty 0.909
61%
Accuracy
55%
45%
43%
37%
31%
19%
14%-
SoftmaxCorr
CIFAR-10.2 0.894 Tip
CINIC
CIFAR- 10-Noise
CIFAR-10 10 setup
00.821 7,
7pe 0.917
=0931
Accuracy
66%-
7856
50%
30%
43%
0.610.7 5.87
0.6
SoftmaxCorr
Camelyon 17 /
48%-
DomainNet-OOD 0.740 0.680
Wilds setup
80%-
Ty= 0.560
Non-VLM
Macro-F1
Accuracy
60%
128-
X
Linear models
0.70
0.79
0.00
0.7
0.2
0.42
0.61
SoftmaxCorr
Figure 2: SoftmaxCorr us. model generalization under ImageNet, CIFAR-10 and WILDS setups. In each subfigure, each point denotes a model trained for the corresponding task. For ImageNet setup, OOD test sets are ObjectNet and ImageNet-S and ImageNet-Blur. For CIFAR-10 setup, OOD test sets are CIFAR- 10.2, CINIC and CIFAR-1O-Noise. For WILDS, OOD test sets Camelyon17-OOD and ComainNet-OODD The y-axis is top-1 accuracy, top-1 accuracy and macro-F1 for the three setups, respectively. Straight lines are fit with robust linear regression (Huber, 2011). Axes are probit scaled as described in Section 3. We observe that SoftmaxCorr is a reliable and effective metric. Particularly on ImageNet, SoftmaxCorr is predictive of model generalization with strong performance (p > 0.92).
Compared to MaxPred and SoftGap, SoftmaxCorr better utilizes Softmax prediction probabil- ities. We use Gopt on top of MaxPred as a simple approach to explicitly consider more entries (the second largest probability) Softmax predictions. As shown in Table 1, SoftmaxCort and SoftGap both achieve higher correlation results than MaxPred (p = 0.864, 0.675 and 0.637, respectively). This indicates that it is helpful to analyze the distribution of softmax outputs. Compared with MaxPred and Soft the class-wit wise correlation considered in our SoftmaxCorr better reveals the knowledge encoded by Softmax predictions. This is supported by the higher average correlation and more stable performance of SoftmaxCorr.
5.3 Sensitivity Analysis on Test Set Size
We study the sensitivity of SoftmaxCorr to test set size. Specifically, reduce the dataset size by randomly sampling 1%, 5%, 10% and 30% of the original data. We report the averaged Spcarman's 's correlation of three random runs on six datasets (e.g., ImageNet-V2, ImageNet-R, ObjectNet, CINIC, CIFAR-1O-Noise and DomainNet-OOD), As shown in Table 3, we observe that when the number of test data is very small (1%), the correlation SoftmaxCorr drops. When the datasct size increases (2 10%), SoftmaxCorr exhibits a stable and high correlation with model performance. This suggests that SoftmaxCorr requires reasonable number of samples to capture model generalization.
8

---

Published in Transactions on Machine Learning Research (05/2024)
Dataset
Certainty Diversity SoftmaxCorr
Dataset
1%
5%
10%
30%
100%
ImageNet-V22 V2-A
0.619
0.551
0.921
Image.Net-V2
0.851
0.791
0.817
0.862
0.921
ImageNet-R
0.851
0.743
0.951
ImageNet-RR
0.913
0.949
0.945
0.951
0.951
ObjectNet
0.881
0.812
0.963
ObjectNet
0,906
0.950
0.955
0.963
0.963
C-FAR-10.2
0.881
0.766
0.894
CIFAR-1O-Noise
0.910
0.933
0.930
0.930
0.931
CINIC
0.643
0.543
0.821
CINIC
0.596
0.838
0.837
0.832
0.821
DomainNet-OOU
0.412
0.139
0.740
DomainNet-OOD 0.758 0.784 0.789 0.789 0.740
Table 2: Comparison between SoftmaxCorr and two variants (Certainty and Diversity). Rank correlation (p) is used the metric.
Table 3: Scnsitivity analysis of SoftmaxCorr on test set sizes. We Lest four sampling ratios and report p on six datasets. SoftmaxCorr stable given a reasonable number of samples.
Method
ImageNet- V2-A ImageNet-R ObjectNet
CIFAR-10.2
CINIC DomainNet-OOD
Disagreement 0.379
0.746
0.021
0.264
0.314
00.083
SoftmaxCorr
0.921
0.951
0.963
0.894
0.821
0.740
Table 4: Comparison between SoftmaxCorr and Disagreement. Rank correlation (p) is used as the metric.
5.4 Impacts of Class Distribution Estimator
In previous experiments, we solely distribution. Another baseline usage of such reference model is to measure the disagreements between the predictions of evaluated models and the reference model (Disagreement). In Table 4, we compare Spearman's rank correlation of Disagreement and SoftmaxCorr. We observe that SoftmaxCorr consistently gives a stronger correlation than Disagreement. Disagreement explicitly measures the similarity between the evaluated models and the reference model, so the performance of the reference model significantly influences the predictive ability of Disagreement. In contrast, SoftmaxCorr only uses it for estimating the empirical class distribution, which constitutes the diagonal elements of the reference matrix R, with all off-diagonal elements set to zero.
vision-language model the reference model to estimate class
To study the influence caused by the estimator, further use less accurate zero-shot vision-languaag model (ViT-H-14) and an ideal estimator for calculating the marginal class distribution. We evaluate them on ObjectNet, because it has imbalanced class distribution. In Figure 3, we observe that with a moderately accurate ViT-H-14, SoftmaxCorr remains predictive, and the reference estimator further enhances SortmaxCorr. Note that, we only use the average of softmax prediction probability to estimate class weights. Some better-designed algorithms (Lipton et al., 2018; Garg al., 2020; Sun al., 2022) may improve SoftmaxCorr's stability, and we leave it as future work.
5.5 Different Characterizations of Correlation Matrix
To investigate the importance of prediction diversity and certainty for predicting OOD generalization, we compare SoftmaxCorr with two variants: (1) the sum of diagonal entries in the class correlation matrix (Certainty). It measures whether the values in diagonal entries are large, indicating prediction certainty; (2) the Euclidean distance between diagonal elements in the class correlation matrix and estimated class distribution (Diversity). It measures whether models make diverse predictions whose distribution matches the estimated distribution of each class. In Table 2, we present Spearman's rank correlation three methods on six datasets from three setups. Both variants give weaker correlation strength than SoftmaxCorr. Specifically, SoftmaxCorr is more predictive of OOD generalization than Certainty and Diversity (p = 0.821 us. 0.643 us. 0.543) on CINIC. This indicates that it important measure both prediction diversity and certainty for OOD generalization assessment.
5.6 Effectiveness SoftmaxCorr for Domain Adaptation
On ImageNet, CIFAR and WILDS setups, all models are trained by standard empirical risk minimization and do not use the unlabeled OOD samples from training. In some scenarios, domain adaptation (DA) algorithms
9

---

Published in Transactions on Machine Learning Research (05/2024)
VIT-H-144 = =0.448 0.835
79%
ViT-bigG 1M-1G-14-CLIPA
0.959
75%
Ground truth
009661
60%
0.900
60%
Ty
Accuracy
42%-
22%-
22%
0.35
6.57 0.76
0.35
0:31
0.55
0.77
0.36
0.14
991
SoftmaxCorr
SoftmaxCorr
SoftmaxCorr
(a) Impacts of class distribution estimator
71%
EfficienNer-B22 p 0.981
61%
ViT-Base 0.9799
CLIP.RNI50
Accuracy
69%
41%
P 0.9799
0:30 90 5.00
0.41 0.72
0.64 9.
SoftmaxCorr
SoftmaxCom
SoftmaxCorr
(b) Unsupervised model evaluation
Figure 3: (a) Impacts of class distribution estimator, use three estimators: ViT-H-14, ViT-bigG-11- CLIPA and the ground truth. We find SoftmaxCort is fairly stable. (b) SoftmaxCorr u.s. accuracy on ImageNet-C benchmark. In every subfigure, each dot indicates a dataset of ImageNet-C.. We see strong correlations between SoftmaxCorr and OOD accuracy on various test set.
are employed for learning target-adaptive models with additional unlabeled OOD samples (Kouw & Loog, 2019; Zhou al., 2022). To explore whether SoftmaxCorr is still effective in assessing the generalization of these models, we conduct correlation study under DomainNet setup (Peng et al., 2019; Sagawa al., 2021). The models are trained by different DA algorithms (e.g., DeepCORAL (Sun & Saenko, 2016), DANN (Ganin et al., 2016)). In Table 1, we observe that morr performs reasonably on DomainNel-OOD We also notice that ID accuracy-basee methods (AoL and ATC-MC) become less useful. DA algorithms likely focus on improving OOD performance, while ID accuracy may not be enhanced accordingly.
5.7 CoftmaxCorr Reflects Model's Generalization on Various 00D Test Sets
We investigate how a given trained model generalizes to various OOD datasets. In detail, we evaluate single model on all test sets of ImageNet-C benchmark (Hendrycks & Dietterich, 2019) and conduct. a correlation study between accuracy and SoftmaxCorr. We additionally use Pearson's correlation (r) to measure the overall linear trend. This cocfficient varies in [-1, 1]. A value closer 1111 indicates better negative ppoivii linearity and 0 means correlation. We use ImageNet net EfficientNet-B2 (Tan & Le, 2019), ViT-Base (Dosovitskiy et al., 2020) and CLIP-RN RN/50 (Radford et al., 2021). Figure 3 shows strong lincar relationship and high-rank correlation (r > 0.95 and > 0.97). It indicates that with linear regression, SoftmaxCort can also help estimate the accuracy of given model on various test sets.
5.8 Evaluation Along Training Trajectory
In previous sections, SoftmaxCorr is used to measure the performance of models varying in different architectures and training strategics. In practice, we are sometimes interested in evaluating the models at different training checkpoints. Hence, we analyze whether SoftmaxCorr is helpful in this case. We collect prediction probabilities on CINIC every 10 epochs along the training process ResNet-20, DenseNet- 121 (Huang et al., 2017), VGG11 (Simonyan & Zisserman, 2014) and MobilcNet (Howard al., 2017) trained on CIFAR-10. Fig. 4, we observe SoftmaxCorr has a high-rank correlation (p 0.93) with model performance. This means we can potentially apply SoftmaxCor to assay model generalization along the training process.
10

---

Published in Transactions on Machine Learning Research (05/2024)
Rashvl-33 0.540
DansaNat-121
Accuracy (%)
0.831
=0.958
0.9115
72
0.537
76
73
62
33
0.75 2. 0.44 0.23 1.00
0.22
1.50
0.23
0.27
0.03 0.33
0.7
0.01
6.02
SoftmaxCorr
SoftmaxCorr
SoftmaxCort
SoftmaxCort
Figure 4: Correlation analysis: SoftmaxCorr and accuracy on CINIC. Each point represents a chockpoint. We consider CIFAR-10 models: ResNet-20, DenscNet-121, VGG-11 and MobileNet. Axes are probit scaled as in Section 3. In each subligure, every point. means checkpoint of the model along the training process. For four models, we see strong correlations (p > 0.93). This suggests that SoftmaxCorr is helpful in assessing checkpoints along the training process.
6 Discussion and Potential Directions
Discussion on imbalanced test sets. SoftmaxCorr is defined cosine similarity between correlation matrix C' and reference matrix Rx- The reference matrix is diagonal matrix and each entry represents the corresponding class distribution. To derive class distribution, a zero-shot vision-language model is used for efficient deployment without training and shows strong robustness towards distribution shifts. Consoquently, the predictive ability of SoftmaxCort correlates to the deployed vision-language model and the method used to estimate the distribution. Hence, it would be beneficial to use advanced label shift estimation techniques (Lipton et al., 2018; Garg al., 2020; Sun et al., 2022) and employ more performant vision-language models, and we leave it future work.
Potential OOD measures. This work proposes SoftmaxCorr to use class-wise relationships encoded by Softmax prediction probabilities. Here, we discuss other potential ways. First, Soft computes the difference between the largest and second-largest prediction probabilities. We show that SoftGap exhibits a stronger correlation with performance compared to MaxPred. Tt. would be interesting to improve SoftGap by utilizing more probabilities (e.g., top five probabilities). Second, for a perfectly calibrated model, its MaxPred over test set corresponds to its accuracy. Yet, calibration methods seldom exhibit desired performance under distribution shift (Ovadia et al., 2019). That said, it would be promising to study post-hoc calibration methods for OOD datasets, which benefits MaxPred for assessing model generalization. Last, this work focuses on max prediction probability. We tested our method based on logits but no obvious correlation is exhibited. This may be because the logits of different models vary in significantly different ranges. We also think that studying other model statistics (e.g., weights and fcature reprosentations) would be interesting.
7 Conclusion
This paper studies an important problem of assaying and ranking model generalization under distribution shifts. To this end, we explore the use of Softmax prediction probability for developing OOD measures. We start by identifying the demand for OOD measures beyond eccuracy-on-the-ling and finding that maximum Softmax prediction probability is to some extent useful to mcasure the OOD performance. We then propose Softmax Correlation (SoftmaxCorr) which leverages class confusion encoded by the class-class correlation matrix and thus better reflects the overall quality of the classifier predictions. To validate the usefulness of SoftmaxCorr, we compare with four other empirical measures across 11 datasets under ImageNet, CIFAR-I1 10 and WILDS sctups. We observe SoftmaxCorr generally presents a stable and high correlation with model performance on different OOD datasets. This paper establishes some baseline usage of Softmax prediction probability and specific improvement, and more investigation will be made in the future.
Acknowledgements
We sincerely thank all the anonymous reviewers and area chairs for their constructive comments and valuable suggestions, which have greatly helped in enhancing this paper.
11

---

Published in Transactions on Machine Learning Research (05/2024)
References
David Acuna, Guojun Zhang, Mare T Law, and Sanja Fidler. f-domain adversarial learning: Theory and algorithms. In International Conference on Machine Learning, pp. 66-75, 2021.
Kumail Alhamoud, Hasan Abed Al Kader Hammoud, Motasem Alfarra, and Bernard Ghanem. Generalizability of adversarial robustness under distribution shifts. Transaction on Machine Learning Research, 2022.
Sanjeev Arora, Rong Ge, Behnam Neyshabur, and Yi Zhang. Stronger generalization bounds for deep nets via a compression approach. In International Conference on Machine Learning, pp. 254-263, 2018
Yuki Markus Asano, Christian Rupprecht, and Audrea Vedaldi. Self-labelling via simultaneous clustering and representation learning. In International Conference on Learning Representations, 2019.
Christina Back, Yiding Jiang, Aditi Raghunathan, and Zico Kolter. Agreement-on-the-le Predicting the performance of neural networks under distribution shift. In Advonces in Neurol Information Processing Systems, 2022.
Robert Baldock, Hartmut Maennel, and Behnam Neyshabur. Deep learning through the lens of example difficulty. In Advances in Neural Information Processing Systems, volume 34, pp. 10876 10889, 2021.
Peter Bandi, Oscar Geessink, Quirine Manson, Marcory Van Dijk, Maschenka Balkenhol, Meyke Hermsen, Babak Ehteshami Bejnordi, Byungjae Lee, Kyunghyun Paeng, Acxiao Zhong, et al. From detection of individual metastases to classification of lymph node status at the patient level: the camelyon17 challenge. IEEE Transactions on Medical Imaging, 2018.
Andrei Barbu, David Mayo, Julian Alverio, William Luo, Christopher Wang, Dan Gutfreund, Josh Tenenbaum, and Boris Katz. Objectnet: A large-scale bias-controlled dataset for pushing the limits of object recognition models. In Advances in Neural Information Processing Systems, 2019.
Peter T. Bartlett, Dylan J Foster, and Matus J Telgarsky. poectrallynorrmaliine margin bounds for neural networks. In Advances in Neural Information Processing Systems, 2017.
Shai Ben-David, John Blitzer, Koby Crammer, and Fernando Pereira. Analysis of representations for domain adaptation. In Advances in Neural Information Processing Systems, 2006.
John Blitzer, Koby Crammer, Alex Kulesza, Fernando Percira, and Jennifer Wortman. Learning bounds for domain adaptation. In Advances in newral information processing systems, 2007.
Lin Chen, Huaian Chen, Zhixiang Wei, Xin Jin, Xiao Tan, Yi Jin, and Enhong Chen. Reusing the task- specific classifier discriminator: Discriminator-fres adversarial domain adaptation. In Proceedings of the IEEE//VF Conference on Computer Vision and Pattern Recognition, 2022.
Ching-Yac Yao Chuang, Youssef Mroueh, Kristjan Greenewald, Autonio Torralba, and Stefanie Jegelka. Measuring generalization with optimal transport. In Advances in Neurol Information Processing Systems, pp. 8294 8306, 2021.
Shuhao Cui, Shuhui Wang, Junbao Zhuo, Liang Li, Qingming Huang, and Qi Tian. Towards discriminability and diversily: Batch nuclear-norm maximization under label insulficient situations. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recogrétion, pp. 3941 3950, 2020.
Luke N Darlow, Elliot J Crowley, Antreas Antoniou, and Amos J Storkey. Cinic- 10 is not imagenet or cifar- 10.
Xiviu preprint arXiv: 03505, 2018.
Ailin Deng, Shen Li, Miao Xiong, Zhirui Chen, and Bryan Hooi. Trust, but verify: Using self-supervised probing to improve trustworthiness. In European Conference Computer Vision, 2022.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hicrarchical image database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 248 255, 2009.
12

---

Published in Transactions on Machine Learning Research (05/2024)
Weijian Deng and Liang Zheng. Are labels always necessary for classifier accuracy evaluation? In Proceedings of the IERE/CVF Conference on Computer Vision and Pallern. Recognition, pp. 15069-15078, 2021.
Weijian Deng, Yumin Suh, Stephen Gould, and Liang Zheng. Confidence and dispersity speak: Characterising prediction matrix for unsupervised accuracy estimation. In International Conference on Machine Learning, 2023.
Laurent. Dinh, Razvan Pascanu, Samy Bengio, and Yoshua Bengio. Sharp minima can generalize for deep nets. In International Conference Machine Learning, pp. 1019-1024 2017.
Josip Djolonga, Jessica Yung, Michael Tschannen, Rob Romijnders, Lucas Beyer, Alexander Kolesnikov, Joan Puigcerver, Matthias Minderer, Alexander D' D'Amour, Dan Moldovan, et al. On robustness and transferability of convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pottern Recognition, pp. 16458 16458-16468, 2021.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference Learning Represenlations, 2020.
Gintare Karolina Dziugaite and Daniel M Roy. Computing nonvacuous generalization bounds for deep (stochastic) neural networks with many more parameters than training data. In Proceedings of the Thirty-Ther Third Conference on Uncertainty in Artificial Intelligence, 2017.
Gintare Karolina Dziugaite, Alexandre Drouin, Brady Neal, Nitarshan Rajkumar, Ethan Caballero, Linbo Wang, loannis Mitliagkas, and Daniel M Roy. In search of robust measures of generalization. In Advances in Neural Information Processing Systems, pp. 11723-11733, 2020.
Logan Engstrom, Andrew Tyas, Shibani Santurkar, Dimitris Tsipras, Jacob Steinhardt, and Alcksander Madry. Identifying statistical bias in dataset replication. International Conference on Machine Learning, pp. 2922 29322. PMLRR, 2020.
Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-aware minimization for efficiently improving generalization. In International Conference Learning Representations, 2020.
Ustinova Ganin, Germain Ajakan, Laviolette Larochelle, Marchand, and Lempitsky. Domain-advorsaria
training of neural networks. In Journal of Machine Learning Research, 2016.
Saurabh Garg, Yifan Wu, Sivaraman Balakrishnan, and Zachary Lipton. A unified view of label shift. estimation. In Advances in Neural Information Processing Systems, pp. 3290 3300, 2020.
Saurabh Garg, Sivaraman Balakrishnan, Zachary C Lipton, Behnam Neyshabur, and Hanie Sedghi. Leveraging unlabeled data to predict out-of-distribution performance. In International Conference Learning Representations, 2022.
Robert Geirhos, Kantharaju Narayanappa, Benjamin Mitzkus, Tizian Thieringer, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. Partial success in closing the gap between human and machine vision. In Advances in Neural Information Processing Systems, 2021.
Devin Guillory, Vaishaal Shankar, Sayna Ebrahimi, Trevor Darrell, and Ludwig Schmidt. Predicting with confidence unseen distributions. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 1134-1144, 2021.
Ishaan Gulrajani and David Lopez-Paz. In search of lost domain generalization. In International Conference on Learning Representations, 2021.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. International Conference on Machine Learning, pp. 1321-1330) 2017.
13

---

Published in Transactions on Machine Learning Research (05/2024)
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEER/CVF Conference on Compuler Vision and Pallern Recognition, 2022.
Matthias Hein, Maksym Andriushchenko, and Julian Bitterwolf. Why relu networks yield high-confiddence predictions far away from the training data and how to mitigate the problem. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognation, pp. 41-50, 2019.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. International Conference on Learning Representations, 2019.
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. In International Conference on Learning Representations, 2016.
Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. The many faces of robustness: A critical analysis of out-of-distribution generalization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.
2021a.
Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15262 15271, 2021b.
Judy Hoffiman, Mehryar Mohri, and Ningshan Zhang, Algorithms and theory for multiple-source adaptation. In Advances in neural information processing systems, 2018
Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arxiv preprint arXiv:1 770.04861,, 2017.
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Denscly connected convolutional networks. In Proceedings of the IEEE Conference on Compuler Vision and Poltern Recognition, pp. 4700 4708, 2017.
Zhuo Huang, Xiaobo Xia, Li Shen, Bo Han, Mingming Gong, Chen Gong, and Tongliang Liu. Harnessing dittr--iiittiiutionn examples via augmenting content and style. In International Conference on Learning Representations, 2022.
Zhuo Huang, Muyang Li, Li Shen, Jun Yu, Chen Gong, Bo Han, and Tongliang Liu. Winning prize comes from losing tickets: Improve invariant learning by exploring variant parameters for out-of-distributioo generalization. arXio preprint arXiw:2310.16391, 2023.
Peter J Huber. Robust statistics. In International Encyclopedia of Statistical Science, pp. 1248-122 1251. Springer, 2011.
Gabriel Ilharco, Mitchell Wortsman, Ross Wightman, Cade Gordon, Nicholas Carlini, Rohan Taori, Achal Dave, Vaishaal Shankar, Hongsook Namkoong, John Miller, Hannanch Hajishirzi, Ali Farhadi, and Ludwig Schmidt. Openelip, July 2021. URL https://doff or./o0. httgg//../////eee. please cite it below.
w0w.//eeooooo.o.o 5143773. If you use this software,
Yiding Jiang, Pierre Foret, Scott Yak, Daniel M Roy, Hossein Mobahi, Gintare Karolina Dziugaite, Samy Bengio, Suriya Gunasekar, Isabelle Guyon, and Behnam Neyshabur. Neurips 2020 competition: Predicting generalization in deep learning. arXiv preprint w:Xiw:2012.07911 2020a.
Yiding Jiang, Behnam Neyshabur, Hossein Mobahi, Dilip Krishnan, and Samy Bengio. Fantastic gencralization measures and where to find them. In International Conference on Learning Representations, 2020b.
Ying Jin, Ximei Wang, Mingsheng Long, and Jianmin Wang. Minimum class confusion for versatile domain adaptation. In European Conference Computer Vision, pp. 464-480, 2020.
14

---

Published in Transactions on Machine Learning Research (05/2024)
Maurice George Kendall. Rank correlation methods. Griffin, 1948.
Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Peter Tang. On large-batch training for deep learning: Generalization gap and sharp minima. In International Conference on Learning Representations, 2017.
Andreas Kirsch and Yarin Gal. A note on' assessing generalization of sgd via. disagrecment'. Transaction Machine Learning Research, 2022.
Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xic, Marvin Zhang, Akshay Balsubramani,
Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, et al. Wilds: benchmark of in-the-wild distribution shifts. In International Conference on Machine Learning, pp. 5637-5664, 2021.
Wouter M Kouw and Marco Loog. A review of domain adaptation without target labels. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(3):766-775, 2019.
Kiran Krishnamachari, Sce-Kiong Ng, and Chuan-Shong Foo. Mitigating real-world distribution shifts in the fourier domain. Transactions on Machine Leurning Research, 2023.
Alex Krizhevsky, Geoffrey Hinton, al. Learning multiple layers of features from tiny images. Citeseer, 2009.
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-iman pre-training for unified vision-language understanding and generation. In International Conference on Machine Learning, 2022.
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. BLIP-2: bootstrapping languago-image pro-training with frozen image encoders and large language models. In International Conference on Machine Learning, 2023.
Shuang Li, Fangrui Lv, Binhui Xie, Chi Harold Liu, Jian Liang, and Chen Qin. Bi-classifier determinacy maximization for unsupervised domain adaptation. In Proceedings of the AAAI conference on artificial intelligence, pp. 8455-8464, 2021.
Shiyu Liang, Yixuan Li, and Rayadurgam Srikant. Enhancing the reliability of out-of-distribution image detection in neural networks. In International Conference on Learning Representations, 2018.
Tengyuan Liang, Tomaso Poggio, Alexander Rakhlin, and James Stokes. Fisher-rao metric, geometry, and complexity of neural networks. In The 22nd International Conference on Artificial Intelligence and Statistics, pp. 888 896, 2019.
Zachary Lipton, Yu-Xiang Wang, and Alexander Smola. Detecting and correcting for label shift with black box predictors. In International Conference on Machine Learning, pp. 3122-33 22300, 2018.
Evan Liu, Behzad Haghgoo, Annie Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. Just train twice: Improving group robustness without training group information. In International Conference on Machine Learning, pp. 6781-6792, 2021.
Yuzhe Lu, Yilong Qin, Runtian Zhai, Andrew Shen, Ketong Chen, Zhenlin Wang, Sohcil Kolouri, Simon Stepputtis, Joseph Campbell, and Kalia Sycara. Characterizing out-of-disttibatiio error via optimal transport. In Advances in Neural Information Processing Systems, 2024.
Lucas Mansilla, Rodrigo Echeveste, Diego Milone, and Enzo Ferrante. Domain generalization via gradient surgery. In Proceedings of the IEEE International Conference Computer Vision, 2021.
Yishay Mansour, Mehryar Mohri, and Afshin Rostamizadeh. Domain adaptation with multiple sources. In Advances in neural information processing systems, 2008.
Yishay Mansour, Mchryar Mohri, and Afshin Rostamizadeh. Domain adaptation: Learning bounds and algorithms. Xiv preprint ar +xXix00922.4430 2009.
15

---

Published in Transactions on Machine Learning Research (05/2024)
John P Miller, Rohan Taori, Aditi Raghunathan, Shiori Sagawa, Pang Wei Koh, Vaishaal Shankar, Percy Liang, Yair Carmon, and Ludwig Schmidt. Accuracy on the line: on the strong correlation between out-of of------isttbttionnn and in-distribution generalization. In International Conference on Machine Learning, pp. 7721-7735, 2021.
Matthias Minderer, Josip Djolonga, Rob Romijnders, Frances Hubis, Xiaohua Zhai, Neil Houlsby, Dustin Tran, and Mario Lucic. Revisiting the calibration of modern neural networks. In Advances in Newral Information Processing Systems, pp. 18682-15094, 2021.
Vaishnavh Nagarajan and J Zico Kolter. Generalization in deep networks: The role of distance from initialization. arXiv preprint Xrxim:1011 1901.0177,, 2019.
Behnam Neyshabur, Ryota Tomioka, and Nathan Srebro. Norm-based capacity control in neural networks. In Conference on Learning Theory, pp. 1376-1401, 2015.
Behnam Neyshabur, Srinadh Bhojanapalli, David McAllester, and Nati Srebro. Exploring generalization in doep learning. In Advances in Neural Information Processing systems, 2017.
Behnam Neyshabur, Srinadh Bhojanapalli, and Nathan Srebro. A pac-bayesian approach to spectrally- normalized margin bounds for neural networks. In International Conference on Learning Representations, 2018.
Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Takshminnarayaaan, and Jasper Snoek. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. In Advances in Newral Information Processing Systems, 2019.
Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment matching for multi-source domain adaptation. In Proceedings of the IEEE International Conference on Computer Vision, pp. 1406 1415, 2019.
Fengchun Qiao and Xi Peng. Uncertainty-guided model generalization to unseen domains. In Proceedings of the IEEE Conference Compuler Vision and Pallern Recogmilion, pp. 6790-688688 2021.
Alec Radford, Jong Wook Kim, Chris Hallacy, Adilya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International Conference Machine Learning, 2021.
Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do cifar-10 classifiers generalize cifar-10? arXiv preprint Xriiv:1806. 00451, 2018a.
Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do cifar-10 classifiers generalize to cifar-10? Xrxiv preprint Xrxiv:1866.00// 2018b.
Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do imagenet classifiers generalize to imagenet? In International Conference on Machine Learning, pp. 5389 5400, 2019.
Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang, Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. In International Conference on Learning Representations, 2020.
Shiori Sagawa, Pang Wei Koh, Tony Lee, Irena Gao, Sang Michael Xie, Kendrick Shen, Ananya Kumar, Weihua Hu, Michihiro Yasunag,, Henrik Marklund, et al. Extending the wilds benchmark for unsupervised adaptation. arXiv preprint arxiv:2112 12.0000 2021.
Yair Schiff, Brian Quanz, Payel Das, and Pin- Yu Chen. Predicting deep neural network generalization with perturbation response curves. In Advances in Neural Information Processing Systems, pp. 21176-21188, 2021.
Yuge Shi, Jeffrey Scely, Philip HS Torr, N Siddharth, Awni Hannun, Nicolas Usunier, and Gabriel Synnaeve. Gradient matching for domain generalization. arXiv preprint ar Xrx::22104 0et:2144409997 2021.
16

---

Published in Transactions on Machine Learning Research (05/2024)
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arviv preprint 1rx: 1556, 2014.
Amanpreet Singh, Ronghang Hu, Vedanuj Goswami, Guillaume Couairon, Wojciech Galuba, Marcus Rohrbach, and Douwe Kiela. Flava: A foundational language and vision alignment model. In Proceedings of the /EEE/CVF Conference on Computer Vision and Pattern Recognation, 2022.
Samuel L Smith and Quoe V Le. A bayesian perspective on generalization and stochastic gradient descent. In International Conference on Learning Representations, 2018.
Baochen Sun and Kate Saenko. Deep coral: Correlation alignment for deep domain adaptation. In European Conference on Computer Vision, pp. 443 450, 2016.
Tao Sun, Cheng Lu, and Haibin Ling. Prior knowledge guided unsupervised domain adaptation. In European Conference on Compuler Vision 2022.
Remi Tachet des Combes, Han Zhao, Yu-Xiang Wang, and Geoffrey J Gordon. Domain adaptation with conditional distribution matching and generalized label shift. In Advances in Neural Information Processing Systems, pp. 19276 19289,, 2020.
Mingxing Tan and Quoc Le. Efficientnel: Rethinking model scaling for convolutional neural networks. In International Conference on Machine Learning, pp. 1105-6114 2019.
Damien Teney, Seong Joon Oh, and Ehsan Abbasnejad. Id and ood performance are sometimes inversely correlated on real-world datasets. arXiv preprint arXiv:22090000 2022.
Damien Teney, Yong Lin, Seong Joon Oh, and Ehsan Abbasnejad. Id and ood performance are sometimes inversely correlated on real-world datasets. In Advances in Neural Information Processing Systems, 2024.
Weijie Tu, Weijian Deng, Tom Gedeon, and Liang Zheng. A prrotttyotss representation for dataset-level applications. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognation, 2023a.
Weijie Tu, Weijian Deng, Tom Gedeon, and Liang Zheng. A bag-of-prototypes representation for dataset-level applications. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 2881-2892, 2023b.
Ramakrishna Vedantam, David Lopez-Paz, and David J Schwab. An empirical investigation of domain generalization with empirical risk minimizers. In Advances in Neural Information Processing Systems, pp- 28131 28143, 2021.
Riccardo Volpi, Hongseok Namkoong, Ozan Sener, John Duchi, Vittorio Murino, and Silvio Savarese. Generalizing unseen domains via adversarial data augmentation. In Advances in Neural Information Processing Systems, 2018.
Haohan Wang, Songwei Gc, Zachary Lipton, and Eric P Xing. Learning robust global representations by penalizing local predictive power. In Advances in Newral Informalion Processing Systems, 2019.
Tongzhou Wang and Phillip Isola. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In International Conference on Machine Learning, pp. 9929 9939, 2020.
Florian Wenzel, Andrea Dittadi, Peter Vincent Gehler, Carl-Johann Simon-Gabriel, Max Horn, Dominik Zietlow, David Kernert, Chris Russell, Thomas Brox, Bernt Schiele, et al. Assaying generalization in transfer learning. In Advances in Neural Information Processing Systems, 2022.
Ross Wightman. Train 10 with pytorch.
2017.
Ross Wightman. Pytorch image models. http:://gttto..ooorrgggaaaaaaaat
2019.
Renchunzi Xie, Hongxin Wei, Lei Feng, Yuzhou Cao, and Bo An. On the importance of feature separability in predicting out-of-distributiio error. Tn Advances in Neural Information Processing Systems, 2024.
17

---

Published in Transactions on Machine Learning Research (05/2024)
Shiqi Yang, Joost van de Weijer, Luis Herranz, Shangling Jui, al. Exploiting the intrinsic neighborhood structure for source-free domain adaptation. In Advances in Neumal Information Processing Systems, pp- 29393 29405, 2021.
Kaichao You, Yong Liu, Jianmin Wang, and Mingsheng Long. Logme: Practical assessment of pre-trained models for transfer learning. In International Conference Machine Learning, pp. 12133-12143, 2021.
Yaodong Yu, Zitong Yang, Alexander Wei, Yi Ma, and Jacob Steinhardt. Predicting out-of-distribution error with the projection norm. In International Conference on Machine Learning, 2022.
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In International Conference on Compuler Vision, 2023.
Dinghuai Zhang, Kartik Ahuja, Yilun Xu, Yisen Wang, and Aaron Courville. Can subnetwork structure be the key to out-of-distribution generalization? In International Conference Machine Learning, pp- 18856-12361 2021.
Yuchen Zhang, Tianle Liu, Mingsheng Long, and Michael Jordan. Bridging theory and algorithm for domain adaptation. In International Conference Machine Learning, pp. 7404-7413, 2019.
Long Zhao, Ting Liu, Xi Peng, and Dimitris Metaxas. Maximum-entropy adversarial data augmentation for improved generalization and robustness. Advances in Neural Information Processing Systems, 2020.
Kaiyang Zhou, Ziwei Liu, Yu Qiao, Tao Xiang, and Chen Change Loy. Domain gencralization: A survey. IEEE Transactions on Pallern Analysis and Machine Intelligence, 2022.
18

---

Published in Transactions on Machine Learning Research (05/2024)
A
Appendix
In the appendix, we first introduce the experimental details including access to models, datasets, and computation resources.
A.1
Experimental Setup
A.1.1 Datasets
We carcfully check the licenses of all datasets used in the experiment and list the open sources to them.
ImageNet (Deng et al., 2009):
hutts://www.immgeenn
ImageNet-C (Hendrycks & Dietterich, 2019):
TiaggNNet-V2(RRei) (Recht et al., 2019):
hutp:://jittub.ccommoolettggpggt Net
Imagghlett-Adveersarial (Hendrycks et al., 2021b):
ltttss//gittuub.oomm/bedddyyy hatp://aaaaaa---eaaaaaaa
Imagellet-Renddttion (Hendrycks al., 2021a):
ImagellettSSketch (Wang et al., 2019):
ObjectNet (Barbu et al., 2019):
hhtpps//obieectneee.ee// http://dddeee/oooooooa...m..ma
CIFAR-10 (Krihhevsky et al., 2009):
https://www.cstoorntto..om cduttoo..du/// krrz//cifar.hmi)
CIFAR-10-C (Hendrycks & Dietterich, 2019):
http://gthhb..co///nnnnss///////
CIFAR-10 2 (Recht et al., 2018a):
Iutops://gtthub.com/ http://dddttyassssCIiie (IFAacchht//(IAR-10000))
CINIC (Darlow et al., 2018):
Camelyon17 (Bandi et al., 2018):
DomainNet (Peng et al., 2019):
A.1.2 Model Pool
ImageNet. models are publicly available via TIMM (Wightman, 2019). Models pre-trained by contrastive learning can be used from model-vv-hhuman project (Geirhos et al., 2021). Some MAE models (He et al., 2022) are accessible from
PIRL
InsDis
MoCo
MoCoV2
InfoMin
simclr_resnet500
simclr_resnetóie
simcir_resnet500
resnet50_122_eps1
resnet50_11_oopo__5
resnet50 12_
resnet:50_12_eps0) 12__eps0_22
19

---

Published in Transactions on Machine Learning Research (05/2024)
resnet50 _e_petts______p__00_____
resnet50_12 121550_112___0 _____________111111_____________
ens ans_adv_inceptii sew_iinception_reeett v2
adv adv_inception_v?
__efffccienteet_b.. _p
resnet550_traine__oo ______n__
resnet50_trraieedd on___________________
Latpeedddo_SS_____ __N and_ IN IN_ faaeen_finttued_oo
@esntt50_traimed_om
Saaett50_augmii
eeseet50ccutmix
geenet.50_eeepaugee
resnett5__featur__untii
resnet500_pixmiz
convnext wwtpeexxxxxagee_eeeee _large_384___ 1erre__844_n222fte_ in22ftik
convnext Seevnex_xxlaree_____
xlarge_in22fftll in22ft1k
resnetv2 resnetv2_152x: 1e_____22___b_______
resnetv2_500 50x1_bitm
bitm
convnext_ban base_ _atee_______0 _ii__222t
______e___nnftik convnext_bane in222ftkk
resmlp_ _ec_l__bi__2__ ___________m__ _e4__n22ftik
resmlp_big.20 1al___24_disiillee 224
tf ifffiiccieettnettes___.hlflf
tf [___=f_ficcatttttrr2_____lhiti
tf efffficcloottttr222....hhfff
tf dfgefficccatttetv2aaa__...f. vit vit_larree_patcch16.3884
beit beit_largge_aacchh66_522
beit boit_large__atcch16_888
beit_ beit_laarge_aacchh6.24
beit_ beit_basee_aacchl._.
vit_basse_patch16_ 384
vit_small_r226__ Be1_6__s2_____00 _2___________4______2222
vit_tiny_patch16 384
vit_large _50
______50_______________ a32_384
mixer_b16 22__222_r___ _miil
resmlp_big 212222_2222222222 _24
ig_resnextt01__2xx16d
ig.resnexti11_3223322
ig_resmextt11__3xx8
ig_rrannext111_324488
resnext101_39xx16d, wsl
tf df.efficcientnne_12___...
tf tf_effiicionnnet_____1 tf efffciiienttttttbbbe b6 ns
tf sf.efficientnett_e b5 ns
ssl_resnext101__22
9sl_resnext111_32xbb
swsl_resste 1t__rremext1001_10001 32x8d
swal swsl_reamexteeeee _0__reeeeett01__22110
ssl 1s__rennext111_2xx4d
881 1eeeeeett033xx44 32x4d
20

---

Published in Transactions on Machine Learning Research (05/2024)
as1 asl_resnet50
swsl
swal_reeneee
[es__rrsnext5__32x44
swel_resnet50
swtn.small_patctt__vvrooorr/
deit_base_patcchi6__2 224
deit ceit_small_distilled httpeeee_ppahhh1____11__1 224
resmlp_36_222
cait_s36_384
cait_s24_ 224
convit_base
convit_tiny
base
eca_nfnet_Ii
acit_tiny_24_pte dist
officientnet_bi
efficientnet_Be b3
efficientnet_bbe b4
tf sf_efficientnet,
d__efficientnet_liit convnext_base
convnext_small
resnetrs350
pit_xs att_x__disee
_ix_x__dsssllle__222
crossvit_small_240
botnet26t_2266
tinynet_e
tinynet_d
repvgg_b2g4
mmasnet_smal]
dla46x_c
lcnet_0s 050
tv res
tv vv_resnet50
wv_resnet101
vv_reneet12
densenet121
inception_ve v4
resnet26d
mobilenetv2_140
hrnet_w40
xception
xception41
reanet18
resnet34
seresnet50
mobilenetv__050
seresnet33ts
wide ride_resnet50_2 2
wide res_eeteee1122
resnet18d
hrnet_w18_sman small
gluon_resnet152 vid
21

---

Published in Transactions on Machine Learning Research (05/2024)
hrnet_w43
hrnet_ w44
repvgg_ _2
densenet201
hrnet_wis
resnet101d
gluon_resnett01__1ee
_id
gluoo_resnet100_.1ss
gluon_xcepttiom65
gluon_eereennextt__32x
gluon_seneti14
gluon_inception_.v
gluuon_resnet111__1e
tf if_inception__33
tv_densenet121
bv_resnext50_33244
repvgg_big4
resmext26ts
ghostnet_ 100
crossvit_e 9_240
deit deit_base Brtt_aas__pacchh__0_0 patch16__84
rexnet_ _50
rexnet_130 130
resnetrs50
resnet50d
resnet50
resnetv2_50
resnetrs152
resnetrs101
resnet50_aa
@esnet50_fasstar
resnet50_randas
vgg19_bn
vgg16_bb
vgg13_
vggl1_bs
vgg11
vgg11_bn bn
vgg16
vgg19
resnet10t
resnet14t
darknet53
cs3darknet_m
@e3darknet_focus__
cs3darknet_1
cs3darkmet_
regnety_040
regnety_0o 064
regnety_oe _000
regnetv 040
regnetv 064
regnetz 040
22

---

Published in Transactions on Machine Learning Research (05/2024)
regnetz_040h
The training script for training trajectory experiment borrows from (yytorcccc/tooor.ccfar)..
CIFAR-10 models can be downloaded through
VGG16
VGG13
ResNet34
ResNet50
ResNet101
ResNet152
ShuffleNetG2
ShuffleNetG3
PreActResNet16
PreActResaNet34
PreActResNet550
PreActResNNet01
densenet_ ciffrr
DenseNett11
DenseNet 169
DenseNet201
DenseNett11
ResNeXt22_ 8x64d
ResNeXt29_
MobileNetV2
BagNetX_200Mg
DLA
DPN
PNASNetB
RegNetX_40000 4
BegNetY_400MS 400MF
MobileNet
ResNet18
VGG11
SimpleDLA
ResNeXt29 4x
DesNeXt29_2xxar
BfficientNetB0
SENet18
ShuffleNetV2
GoogLeNeet
DPN92
ResNet:34-160
ResNet.34-170
ShuffleNettG2-170
Shuff1eNNetG2-180
ShuffleNeetG2-190
SimpleDLA-90
SimpleDLA-105
SimpleDLA-120
SimpleDLA-135
SimpleDLA-150
SENet18-60
SENet18-75
23

---

Published in Transactions on Machine Learning Research (05/2024)
SENet18-90
SENet18-120
SENet18-135
SENet18-150
SENet18-165
ShuffleNetG2-110
ShuffleeNetG2-180
ShuffleNNet02-190
densenet_ccifar1113
densenet_ciifa-115
densenet_cciaa-110
MobileNeetV2-135
15b11
MobileNetV2-165
LeNet
PreActResNet112
WILDS models (Camelyon17 and DomainNet) are trained by github https///gtthhb.oom/m/maaaaaa///a
Zero-shot vision-language models are provided in OpenCLIP (Ilharco al., 2021). They are listed follows in the pattern (architecture, source): (RN50, openai) (RN50, yfcc15m)
(RN50, cc12m)
[NN0--qicckellu, yfcc15m)
[RN5O-quickgelu, cc12m)
(RN101, openai)
(RN101, yfcc15m)
IN101-quickgelu, yfcc15m)
(RN50x4, openai) )
|RN50x16, openai)
(RN50x64, openai)
(ViT-B-32, openai)
(ViT-B-32, laion400m_e32)
(ViT-B-16, openai)
(ViT-B-16, laion400m_ees e32)
(ViT-L-14, openai)
(ViT-L-14, 1aiinib_ss2___bbnlbbba b82k)
ViT-L-144336, openai)
(ViT-H-14,
1anto12b,,i11__________ s34b b88k)
(ViT-bigG-14, laion2b s39b 1aion2b_s99b_b11000
convnext__ase 2aion400mms13b_bt
convnext_base_ laion_ saion_asttherric__33b..222
convnext_base__2 320, laion httpaaahchccccccssee2222,.
augreg)
Convnext__large_d, laion2b, s26b httpaa_aaaaa________aa_aaaaaa_a
convnext_large_d__222 laion2b desiin2b_s29bbb1333
(ViT-B-32,
1a- pt)
(ViT-B-16, Model-B-16 0tt.::aaaoaaa----10aaaaaaa--a--
1r-1e-3_bss-88 pt)
(ViT-L-14, htt.::aaaaaaaa.atat.aa.oaaaaaaaa
(ViT-B-32,
datacomp____s22m__bba.
(ViT-B-32,
24

---

Published in Transactions on Machine Learning Research (05/2024)
(ViT-B-32, commonpool_ http:/oooo_o_aooon__e___eeeeeeee _4kk
(ViT-B-32, commonpool m_ _______________________ imaaimmaaa__aee s12emm____mm__m_1mmmm__mm___m b4k)
(ViT-B-32, gaamooppol___text___tt
b4k)
(ViT-B-32,
b4k)
(ViT-B-32,
commonpool_m_s1220 _ b4k)
(ViT-B-32,
datacomp_s_s13mm
(ViT-B-32,
sammoooool___cippp s13mmp__3m___m b4k)
(ViT-B-32,
commonpool___laion smmmmmm__mmmmmmmmmmm s13m_b4k)
(ViT-B-32,
secpoppool_s_imaggeee _1m_ma__eamm____________ _133__4kk)
(ViT-B-32,
Sttpoppool___tet_______ b4k)
(ViT-B-32,
basic_si3m b4k)
(ViT-B-32,
commonpool___s13n, b4k)
(ViT-B-16,
V 0atpa
E
Sww001-g=l4-pluus_eereedd..... b114k)
EVA02-L-11, mergddd2 34b_b13kk
EVA02-L-114-336,mergeee
b61k)
1ainon__ b115k)
1aa02--E-1--luus_aaainn _ab_bi___nn____bb42 b144k)
VaT----6G-uucckeelnnmteeeee
V
)
V datacomp1b)
Vi1----4--CLIPA-3366. datacomp1b)
Vi1--biGG-14-CLIPA4 datacompp))
V datacomp1b)
V+T-BB-6G-Sig.IPPeebb11
W.1T-B-6G-SigLII--2556, webli)
VxTTL-1O-SSgLLPP-844eell
V webli)
V webli)
http::/ppggucceglll.mmmtcc.pl..p fullcc)
VviT-B-6--quickeelu, metaclip_fullca
VViT-L-14-quickell, metaclip_fullor
BLIP (Li et al., 2022) and BLIP-2 (Li et al., 2023) models are from LAVIS library
htttps://gihubbcoom / LAVIS (AAAV///ere//amn))
A.1.3 Computation Resources
PyTorch version is c.10.2+ All experiments run on one 2080Ti and the CPU AMD Ryzen Threadripper 2950X 16-Core Processor.
25

---

Published in Transactions on Machine Learning Research (05/2024)
Models ImageNet-
V2-A V2-A ImageNet-R. ObjectNet
Models ImageNet- V2-A ImageNet-R ObjectNet
20%
0.705
0.940
0.856
20%
0.540
0.841
0.786
50%
0.746
0.934
0.894
50%
0.542
0.877
0.846
70%
0.778
0.928
0.906
70%
0.579
0.867
0.8/1
100%
0.758
0.928
0.895
100%
0.597
0.884
0.849
Table 5: Correlation strength of SoftmaxCorr with different number models. Weighted Kendall correlation (Tw) is used as the metric.
Table 6: Correlation strength of MaxPred with different number models. Weighted Kendall correlation (Tw) is used as the metric.
A.2 Sensitivity analysis on Tw
We use Scipy implementation of weighted Kendall's rank correlation (Tw)', which does not provide a p-value indicate the statistical significance. It is because the mull distribution the statistic unknown even in the additive hyperbolic case. Thus, to examine whether the evaluation metric is stable, conduct sensitivity analysis on Tu by randomly dropping some models from the model pool. The results summarized in Table 5 and Table 6. We find that has consistent assessment on two methods with different numbers of models, suggesting the stability of Tw.
26


Reviews:
[{"review_id": "ObO3FC7djc", "reviewer": "Reviewer_wfZb", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \n1. This paper proposes a method to rank the performance of classifiers on OOD data without using annotation. \n2. A lot of empirical results demonstrate the effectiveness of the proposed method.\n\nWeaknesses:\n\n1. Though the paper proposes SoftmaxCorr, its advantages compared with previous methods are unclear. Why is SoftmaxCorr better than previous methods? How does the proposed method utilize the two characteristics described in the paper  (Prediction certainty and Prediction diversity)?\n2. The writing of the paper is poor. See the Requested Changes.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a method (SoftmaxCorr) to rank the performance of classifiers on unlabeled out-of-distribution data. Experiments on various datasets demonstrate the effectiveness of SoftmaxCorr.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "No", "audience_explanation": "", "requested_changes": "1. Figure 1 should be more clear. For example, the way to obtain the different classifiers should be clarified (using different network structures or different training methods?)  How many types of VLMs have been tested? What is the meaning of the line with shadow?\n2. The equation on Section 4.3 is wrong $C_{i,j}=\\sum_{n=1}^NP_{i,n}P_{n,j}$, instead of $C_{i,j}=\\sum_{n=1}^NP_{n,i}P_{n,j}$.\n3. What is the relationship between $\\mathbf{R}_K$ and $\\mathbf{R}$?\n4. The advantages of introducing SoftGap compared with MaxPred should be discussed in the paper.\n5. Missing the sensitive test with the evaluation metric Kendall’s rank correlation $\\tau_w$.", "broader_impact_concerns": "This work does not present direct ethical concerns or adverse broader societal impacts.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "yLfsDhCAPg", "reviewer": "Reviewer_RkLL", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths.** This submission presents an extensive set of experiments over many datasets, models, settings, and OOD performance ranking baselines. Furthermore, rank correlations are computed in a few settings: (a) both model and ID/OOD datasets change, (b) datasets change but model is fixed, and (c) datasets are fixed but model changes. The central claim that SoftmaxCorr is better correlated with OOD performance on average is well-supported. To my best understanding, there are enough details to reproduce experiments.\n\n\n**Weakness #1: Insufficient justification for SoftmaxCorr definition.** I find that SoftmaxCorr definition is not sufficiently motivated and expanded upon. Below are a few questions and concerns about it.\n* In Section 4.3, the “class-class correlation matrix” $C$ is defined as $P^T P$. There should be a $1/n$ factor to counter the growth with $n$.\n* Note that $P^T P$ is not a correlation matrix. It is not even a covariance matrix, because columns are not centered. Why is $C$ defined like this, but not like a correlation matrix?\n* Why is the reference matrix $R$ picked to be diagonal? Why is it not defined like $C$ but for the reference model?\n* I am not familiar with the cosine similarity between two matrices in the way the authors write:\n$$\n\\mathrm{cos}(C,F) = \\frac{\\sum\\_{i,j} (CR)\\_{i,j}}{||C||\\_F ||R||\\_F}.\n$$\nI have seen definitions that flatten the matrices into vectors and then compute cosine similarity. That looks like this:\n$$\n\\mathrm{cos}(C,F) = \\frac{\\sum\\_{i,j} C\\_{i,j} R\\_{i,j}}{||C||\\_F ||R||\\_F},\n$$\nand shows up in other areas of ML (e.g., measuring alignment between two kernel matrices). Please elaborate on your definition of cosine similarity. What range of values does it take? For a fixed $R$, which $C$ maximizes it? Furthermore, since $R$ is picked to be diagonal, I suggest simplifying SoftmaxCorr definition instead of stating it for a generic $R$.\n\n**Weakness $2: Reliance on a strong reference model.** SoftmaxCorr relies on a strong reference model to estimate class prevalences reliably. It is likely that this gives a significant advantage to SoftmaxCorr over other baselines that do not use such a reference model. If such strong reference models are allowed, a natural way to estimate OOD accuracy is to compute disagreement between main model predictions and reference model predictions (i.e., treating reference model predictions as ground truth).\n\n\n**Weakness #3: There are many vague statements [minor].**\n* In the introduction, there is a sentence “We thus propose SoftmaxCorr is a prediction probability-based metric that quantifies the extent to which class predictions made by classifiers get confused with each other“. I do not see how this is exact in a technical sense. I suggest adding a small paragraph in later sections showing this connection.\n* Throughout the work, the authors refer to $C$ as a correlation matrix, which it is not. Furthermore, there are places where it is treated like a confusion matrix, which it is also not.\n* In Section 4.3, “A model scoring high on the SoftmaxCorr measure excels in two critical parts: 1) It demonstrates a high level of prediction certainty, ensuring that its intra-class correlations are strong and reliable. 2) It achieves a broad prediction diversity, indicated by the alignment of the diagonal elements in its class correlation matrix with the class marginal distribution.” These two points are unclear to me. I think it would be nice to expand SoftmaxCorr definition and show explicitly how these two effects arise.\n* In Section 5.5, “(2) the euclidean distance between diagonal elements in the class correlation matrix and estimated class distribution (Diversity). It measures whether models make diverse predictions whose distribution matches the estimated distribution of each class.” How is this interpreted as diversity?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work focuses on a setting where one is given various classifiers trained on some source domain and the goal is to rank them according to their performance on a target domain using only unlabeled data from the target domain.\nIt has been observed previously that a few quantities correlate reasonably with out-of-distribution (OOD) performance.\nExamples of such quantities are in-distribution (ID) accuracy, average prediction confidence, and average confidence gap, where the latter two depend on predicted class probabilities (i.e., the softmax outputs) on test examples. \n\nThe authors of this submission aim to find a better way of using predicted class probabilities that goes beyond maximum or second maximum probabilities and incorporates “correlations” between predicted probabilities of different classes.\nLet $P \\in \\mathbb{R}^{n\\times k}$ be the matrix of predicted probabilities on $n$ test examples for a $k$-way classification tasks.\nLet $C$ denote the uncentered class-class covariance matrix, $C = P^T P \\in \\mathbb{R}^{k \\times k}$.\nLet $R$ be a diagonal matrix where $R_{i,i}$ is the prevalence of the $i$-th class estimated using a strong vision-language model (e.g., a CLIP-like model).\nThe authors propose a new score called SoftmaxCorr that computes a similarity between $C$ and $R$ as follows:\n$$\n\\mathrm{SoftmaxCorr} = \\frac{\\sum_{i,j} (CR)_{i,j}}{||C||_F ||R||_F}.\n$$\n\nWith extensive experiments over many dataset pairs, models, and settings, the authors demonstrate that on-average SoftmaxCorr has the best rank correlation with OOD performance, outperforming baselines like average prediction confidence, average confidence gap, ID accuracy, and average thresholded confidence.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. This submission will benefit significantly from added experiments that compare SoftmaxCorr and a simple baseline that estimates OOD accuracy by treating the reference model predictions as ground truth.\n2. Justifying SoftmaxCorr definition better (please see my comments above).\n\n**Minor improvements**\n* In the introduction, “...there is a pressing need to assess ML model performance in unlabeled testing environments where traditional evaluation metrics may prove inadequate.” I suggest clarifying which traditional evaluation metrics are referred to here. The traditional evaluation metrics I think of require labels.\n* I suggest adding (Dzuigate et al., 2020) to the list of references where the goal is to predict in-distribution generalization.\n* In “0.949, 0.627, 0.934, 0.810, and 0.857” the number 0.934 does match with Table 1. Please correct either the sentence or Table 1.\n* “well correlate” → “correlate well”\n* In the abstract: “matrix–constructed”  → “matrix constructed”\n* In the related work: “this work investigate” → “this work investigates”\n* In the related work, “There are few works consider the characterization of model OOD generalization”. A “that\" is missing. Furthermore, it is not true that few works consider bounds for OOD generalization.\n* “Machine learning models should generalize from training distribution to OOD dataset.” The papers cited in the context of this sentence seem to be not directly related. I don’t think this sentence needs citations. If one decides to cite, there are 100s of papers that should be cited here.\n* In the related work, “This work investigate a different task from unsupervised accuracy estimation.”. One can argue that unsupervised accuracy estimation is very close to the ranking problem, because any method of estimation is also a method of ranking.\n* The abbreviations “IntraCorr” and “InterCorr” are never used.\n* The notation for source examples is never used and can be removed.\n* Section 4.1, “Paradoxically, identifying the failure patterns itself requires labeled OOD data.”. Could you please add citation(s) for this claim?\n* Section 5.2, “On OOD sets of Camelyon17, all methods correlate weakly with OOD performance, which may be caused by under-specification (Wenzel et al., 2022).“ This message is already conveyed in the paragraph above.\n\n\n**Related work**\n\n[1] Dziugaite GK, Drouin A, Neal B, Rajkumar N, Caballero E, Wang L, Mitliagkas I, Roy DM. In search of robust measures of generalization. Advances in Neural Information Processing Systems. 2020;33:11723-33.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "3cHQswt77H", "reviewer": "Reviewer_6Wtr", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:  \n- This paper is well-written and logically organized.  \n- The experimental results are sufficiently good.  \n\nWeaknesses:  \n- The motivation is not clear. Why existing metrics are not good enough for OOD scenarios is not discussed in either the Abstract or Introduction. The authors just present such a novel metric and their motivation is unknown.   \n- Moreover, why the proposed SoftmaxCorr is designed this way is not explained. Why using the class-class correlation matrix C can help to capture both certainty and diversity is not justified. The proposed claims should have either theoretical or empirical evidence.   \n- Missing OOD generalization references:  \n  - Zhang et al., Can Subnetwork Structure be the Key to Out-of-Distribution Generalization? In ICML 2021.\n  - Huang et al., Winning Prize Comes from Losing Tickets: Improve Invariant Learning by Exploring Variant Parameters for Out-of-Distribution Generalization. In arXiv 2023.\n  - Foret et al., Sharpness-Aware Minimization for Efficiently Improving Generalization. In ICLR 2021.\n  - Huang et al., Harnessing Out-Of-Distribution Examples via Augmenting Content and Style. ICLR 2023.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper reveals the drawbacks of common uncertainty metrics related to Softmax prediction. In order to improve the measurement of model performance within both ID and OOD settings, a novel Softmax Correlation criterion is proposed. Specifically, it calculates the cosine similarity between a class-class correlation matrix which is constructed from the Softmax prediction matrix. Such a novel metric can capture both prediction uncertainty and diversity. Through quantitative and qualitative experiments, the effectiveness of SoftmaxCorr is validated on various OOD datasets.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please see weaknesses for details.", "broader_impact_concerns": "No ethical concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: In reviews, the reviewers raise concerns about typos in equations, evaluation metrics, motivation, and assumptions about requiring a strong inference model. After the authors' response, most concerns have been addressed. All reviewers vote for leaning accept. 

By considering that how to properly select the model in the OOD setting and the experiments can be valuable for future research. We decide to accept this paper with minor revision. The authors should follow the advice of  reviewers to further improve the motivation of this paper.
Decision: Accept (poster)