On the Strong Correlation Between Model Invariance and Generalization
Abstract
Generalization and invariance are two essential properties of machine learning models.
Generalization captures a model’s ability to classify unseen data while invariance measures consistency of model predictions on transformations of the data.
Existing research suggests a positive relationship: a model generalizing well should be invariant to certain visual factors.
Building on this qualitative implication we make two contributions.
First, we introduce effective invariance (EI), a simple and reasonable measure of model invariance which does not rely on image labels.
Given predictions on a test image and its transformed version, EI measures how well the predictions agree and with what level of confidence.
Second, using invariance scores computed by EI, we perform large-scale quantitative correlation studies between generalization and invariance, focusing on rotation and grayscale transformations.
From a model-centric view, we observe generalization and invariance of different models exhibit a strong linear relationship , on both in-distribution and out-of-distribution datasets.
From a dataset-centric view, we find a certain model’s accuracy and invariance linearly correlated on different test sets.
Apart from these major findings, other minor but interesting insights are also discussed.
Introduction
Generalization and invariance are two important model properties in machine learning.
The former characterizes how well a model performs when encountering in-distribution or out-of-distribution (OOD) test data [1–5].
The latter assesses how consistent model predictions are on transformed test data [6–12].
Therefore, understanding how the two properties are related would benefit model decision analysis under dynamic environments.
The importance of model invariance on generalization has been qualitatively explored [9, 12–15].
For example, adding rotation invariance to the model improves its in-distribution (ID) classification accuracy [14, 16, 17]; a shift-invariant model is robust to perturbation [9].
In addition, some works provide a neuroscientific perspective to investigate the importance of invariant representations for pattern recognition [13, 15, 18–22].
Furthermore, theoretical investigations suggest that learning invariant features benefits model generalization [12, 22].
However, most existing research is limited to a few ID datasets and model architectures.
As such, the relationship of interest remains unknown on many other scenarios, such as OOD and large-scale test datasets, and other types of models.
Before performing the quantitative study, it is necessary to first quantify generalization and invariance.
For the former, the deep models we consider are well trained, so we simply use the accuracy on the test set, as in many previous works [3, 2, 23].
In comparison, quantifying invariance is not as straightforward.
Some works use model accuracy drop when the test set undergoes transformations to indicate invariance ability [24, 7, 17].
While this strategy is useful for a single model, its effectiveness is limited when comparing invariance of multiple models.
Others resort to consistency, i.e. , models should have the same decision [9, 6], but this method neglects prediction confidence , which we find critical for describing invariance (see discussions in Section 3).
We make two contributions to the community.
First , we propose a new method to measure model invariance, named effective invariance (EI), which considers both the consistency and confidence of predictions.
Given a test image and its transformed counterpart, if the model predicts the same class and with high confidence, the EI value or invariance strength is high.
Otherwise, if the model makes different class predictions or the confidence is low, the EI score will be low.
We show this new measure solves invariance valuation in canonical cases where the commonly used metrics ( e.g. ,
Jensen-Shannon divergence) may fail.
Second , we conduct a broad correlation study to quantitatively understand the relationship between model generalization and invariance.
Specifically, we use 8 test sets with various distribution types, such as the in-distribution ImageNet validation set [25], and outof-distribution ImageNet-Rendition with style shift [4].
We evaluate 150 ImageNet models ranging from traditional convolution neural networks ( VGGs [26]) to the very recent vision transformers ( e.g. ,
BEiT [27]).
Below we list two key observations and example insights.
• For various models , there is a strong correlation between their accuracy and invariance on both in-distribution and out-of-distribution datasets (Sections 5.1 and 5.2).
This finding can be useful for unsupervised model selection, because EI does not require test ground truths.
• On various out-of-distribution datasets , a model’s accuracy and EI scores are also strongly correlated (Section 6).
This observation can be used to predict model accuracy on out-of-distribution datasets without access to ground truths.
• Compared with data augmentation, training with more data seems more effective to improve invariance and generalization (Section 5.6).
Related Work
Predicting generalization gap: a model-centric view.
This task aims to predict the generalization gap of machine learning models on in-distribution data, i.e. , the difference between training and test accuracy.
Most existing works focus on developing complexity measure of trained network parameters and training data [28–36], such as persistent topology [31] and the product of norms of the weights across layers [33].
These methods, assuming the training and test distributions are the same, do not consider the characteristics of the test distribution.
Moreover, they only study limited types of neural networks.
In comparison, we conduct a much more comprehensive study on both in-distribution and out-of-distribution test sets, using various network architectures.
We show that invariance, under our definition, serves as a strong indicator of model generalization ability or accuracy on both in-distribution and out-of-distribution test data.
Closely related to our work, two recent methods [37, 24] predict ID generalization gap based on how a network performs on perturbed data points.
Specifically, Kashyap et al.
[37] use confidence drop to represent invariance, which is less effective for invariance measurement under OOD data.
Schiff et al.
[24] uses accuracy drop to measure invariance, which needs test labels, while EI does not require test labels and is more reasonable than accuracy under OOD data.
In addition, drawing a response curve in [24] is computationally heavy, while our method is relatively efficient.
Further, both studies are limited in their scope: they mainly study ID generalization, has few types of networks and lack large-scale test sets, while our work is much more comprehensive.
Predicting generalization gap: a dataset-centric view.
The overall goal of this task is to predict the performance of a given model on various unlabeled test sets [38–43].
Many methods take into account the statistics of the test set for accuracy prediction [38, 39, 44, 40, 45], such as distribution shift [38], average Softmax score on each test set [39].
We contribute a new solution: using the model’s invariance on the OOD dataset to predict its accuracy. This is supported by our new observation of strong linear correlation between a certain model’s accuracy and invariance on various test sets. Improving robustness with data augmentation. Data augmentation transforms training data to increase its diversity, which helps learn more robust models [46–52].
For example, Mixup [53, 54] and AutoAugment [48] are shown to improve model performance under distribution changes [55, 51]. Instead of using common transformations, adversarial training [56–58] augments training images with an adversarially learned noise distribution. While these works aim to improve corruption robustness with data augmentation, we instead use the latter to analyze model invariance (and generalization). c c c c  0.
c c c c
JS=0.019EI=0.
𝑦" = 𝑦" ! 0.
c c c c  0.
c
c c c f ( 𝒙 ) f ( 𝒙 ! )
JS=0.076EI=0.
𝑦" = 𝑦" !
(a) (b) (d)
JS=0.029EI= 𝑦" ≠ 𝑦" !
JS=0.
EI= 𝑦" ≠ 𝑦" !
(c)
Proposed Effective Invariance (EI)
Notations .
Considering an K -way classification task we define input space X ∈ R d and label space
Y = { 1 , ..., K } .
Given a sample ( x , y ) drawn from an unknown distribution π on X × Y , a neural network classifier f : R d → ∆ K produces a probability distribution for x on K classes, where ∆ K denotes the K − 1 dimensional unit simplex.
Specifically, f i ( x ) denotes the i -th element of the Softmax output vector produced by f .
Then, ˆ y =:
arg max i f i ( x ) is the predicted class, and ˆ p =:
max i f i ( x ) is the associated confidence score.
Image transformation is defined as T : R d → R d .
Then, the transformed image is x ′ =
T ( x ) , and its predicted class is ˆ y t
=: arg max i f i ( x ′ ) with confidence score ˆ p t
=: max i f i ( x ′ ) .
Drawback of existing invariance measures .
A commonly seen strategy is to directly use the distance between the Softmax vectors of two predictions as invariance measure: a lower distance means higher invariance, and vice versa.
Examples of the similarity metrics are Jensen-Shannon divergence (JS) [59, 46] and ℓ 2 distance [60, 61] and Kullback–Leibler divergence [62, 63].
However, they only leverage the global similarity between two Softmax vectors without an explicitly consideration of prediction class consistency and confidence.
We illustrate this drawback taking JS divergence as an example in Fig.
In cases (a) and (b) where the predicted classes are both consistent, JS decides classifier f in (b) has higher, which ignores the low confidence in (b).
In cases (d) where the predicted classes are different, JS still gives high invariance (small JS score), indicating a clear error.
Moreover, in works [46, 60, 61] cases (b) and (d) are discarded when computing the consistency loss.
Definition of EI.
Unlike neuroscience works that study the invariance of an individual neuron of the network [13, 15, 18, 19], we measure the invariance at the network level.
Specifically, given an image and its transformed sample, a model with high invariance should give the same predicted class, and vice versa [6, 9].
In our definition of EI, we further use the prediction confidence.
Our motivation is as follows.
When a model predicts the same class for the two images, if either of the two predictions is of low confidence, we should not consider it as highly invariant but give a penalty.
A model should have a high invariance if and only if it is highly confident in predicting the same class.
Based on these considerations, EI is defined as:
[Equation]
To better understand the soundness of EI, we depict four representative cases in Fig.
In cases (a) and (b), classifier f gives the same predicted class ( ˆ y =
ˆ y t ) on original and transformed images.
Under EI, Classifier f has a higher invariance ability in (a) because it has high confidence scores.
In case (c), the predicted class are different ( ˆ y ̸
= ˆ y t ), and in (d), the predictions are low in confidence (and give different classes), so we define invariance in both cases to be 0.
Computation of EI in practice.
Given a test image, we generate a transformed image using a certain transformation.
Then, we compute the EI score based their Softmax vectors (Eq. 3).
We obtain the model invariance by averaging the EI scores over all the test images.
In this work, we mainly investigate the rotation transformation and grayscale transformation.
For the former, to avoid interpolation that would introduce artifact, we only use three transformation angles ( 90 ◦ , 180 ◦ , 270 ◦ ).
For each rotation angle, we compute an overall invariance score by comparing with the predictions of original data.
By averaging the three EI scores, we obtain rotation invariance on the test set.
For the grayscale transformation, we remove color information and keep only luminous intensity information.
Then, we compare with the predictions of grayscale and original data and compute the overall grayscale invariance on each test set.
Experimental Setup 4.1 Models to Be Evaluated
We considers both very recent and classic image classification models with different architectures, including Convolutional Neural Networks ( e.g. , standard VGGs [26], ResNets [64], and modern
ConvNeXt [65]), Vision Transformers ( e.g. , ViTs [66], Swin [67], and BEiT [27]), and all-MLP architectures [68, 69] ( i.e , MLP-Mixer [69]).
In addition to different architectures, we also cover models with various training and regularization strategies ( e.g. , learning rate schedule [70], label smooth [71] and data augmentation [46, 47, 49, 48]), scaling strategies in model dimension (width, depth and resolution) [72–74], and learning manners (supervised learning, semi-supervised learning [75] and knowledge distillation [76, 77]).
In total, we have 150 models provided by TIMM [78].
They are either trained or fine-tuned on the ImageNet-1k training set [25].
The selected models can be roughly divided into the following three categories:
Standard neural networks.
This category includes 100 models only trained on ImageNet training set.
These networks cover various architectures ranging from VGGs [
] to EfficientNet [73].
Semi-supervised learning.
We include 15 models trained in a semi-supervised learning manner.
They leverage a large collection of unlabelled images of YFCC100M [79] or Instagram 900M [80] to improve the performance.
We use models trained based on a teacher-student paradigm ( e.g. ,
SWSL-ResNet [75] and SSL-ResNet [75]).
Models trained with self-training methods on unlabeled
JFT-300M [81] ( e.g , EfficientNet-L2-NS [82]) are also included.
Pretraining on more data.
We use another 35 models that are pre-trained on significantly larger datasets than the standard ImageNet training set.
Specifically, we consider three pre-training methods: a) weakly supervised pretraining on IG-3.6B ( i.e. , RegNetY [83] and ResNeXt101-WSL [80]); b) supervised pre-training on ImageNet-21K [25] ( e.g. , BiT [84] and Swin [67]); (c) supervised pretraining on JFT-300M [81] ( e.g. , ViT L/16 [66]).
4.2 Test Sets
We use both in-distribution (ID) and out-of-distribution (OOD) datasets for the correlation study.
Specifically, the ImageNet validation set (ImageNet-Val) is used as ID test set.
For OOD test sets, we use of seven datasets, each with a different distribution from standard ImageNet.
Their distribution shift can be divided into the following five types.
Dataset reproduction shift.
ImageNet-V2 [23] is a recollected version of ImageNet-Val.
It contains three versions resulting from different data sampling strategies : Matched-Frequency (A), Threshold0.7 (B), Top-Images (C).
Each version has 10 , 000 images from 1 , 000 classes.
Natural adversarial shift.
ImageNet-Adv(ersarial) [85] is adversarially selected to be misclassified by ResNet-50.
Its natural adversarial examples are unmodified real-world images and have been shown to be hard for other models [85, 3].
It has 7 , 500 samples from 200 ImageNet classes.
Sketch shift.
ImageNet-S(ketch)
[86] consists of sketch-like images and matches ImageNet-Val in categories and scale.
It contains 50 , 000 images and shares the same 1 , 000 classes as ImageNet.
Blur shift.
We use ImageNet-Blur with the highest severity provided by [87].
This dataset is synthesized by blurring ImageNet-Val images with a Gaussian function.
Style shift.
ImageNet-R(endition) [4] contains various abstract visual renditions ( e.g. , art, paintings, and video game) of ImageNet classes.
ImageNet-R has 30 , 000 images of 200 ImageNet classes. [Figure] y [Figure]
T o p - 1 a cc
[Figure] y [Figure] 4.3 Correlation Measures
We use Pearson Correlation coefficient ( r ) [88] and Spearman’s Rank Correlation coefficient ( ρ ) [89] to measure the linearity and monotonicity between invariance and generalization, respectively.
Both coefficients range from [ − 1 , 1] .
A value closer to − 1 or 1 indicates strong negative or positive correlation, respectively, and 0 implies no correlation [88].
To precisely show the correlation, we use logit axis scaling that maps the accuracy range from [0 , 1] to [ −∞ , + ∞ ] , following [3, 2].
Unless noted otherwise, the correlation coefficients are calculated using invariance score and non-linearly scaled accuracy numbers.
Experimental Observations
The experiment is from a model-centric perspective, where we investigate how different models’ accuracy and invariance correlate, and series of observations are made (Section 5.1 - Section 5.6).
5.1 Strong Correlation Between Model’s Rotation Invariance and Accuracy
In Fig.
2, we show the correlation results of rotation invariance and generalization.
We have two observations.
First , we find that for different models, their rotation EI scores have a linear relationship with their classification accuracy.
The correlation holds for both ID test and OOD test sets, various architectures and training strategies.
Specifically, both correlation metrics λ and ρ are higher than 0.840.
This indicates models with higher accuracy numbers are most likely to have stronger rotation invariance (measured by EI), and vice versa.
To our knowledge, it is a very early observation of the quantitative relationship between generalization and invariance (to a certain factor).
Second , training with more data benefits rotation invariance and generalization.
Large datasets contain images with various geometric variations.
When (pre)trained with large datasets, models (blue and orange dots in Fig. 2) adapt to the rotation variations and gains stronger invariance, which, according to our study, likely means a higher generalization accuracy on ID and OOD test sets. [Figure] c l a ss i f i c a t i o n a cc u r a c
[Figure] [Figure] evaluated. Overall we observe relatively solid correlation in all the four cases.
Looking more closely, most standard models (gray dots) are scattered in the low-accuracy region, while models trained with more data (blue and orange ones) move away from this region and exhibit linear trends.
Thus, the overall linear correlation is high ( r > 0 . 880 ), and the overall rank correlation is slightly less consistent ( ρ ranges from 0 .
703 to 0 . 858 )
but still has clear trends.
5.2 Strong Correlation Between Model’s Grayscale Invariance and Accuracy
We now focus on grayscale invariance and report the correlation results in Fig.
We have the following conclusions.
First , among the 150 models, there is strong linear correlation between accuracy and grayscale invariance measured by EI.
Specifically, both correlation coefficients r and ρ are higher than 0 .
820 on all test sets.
Second , we find that pretrained or semi-supervised models tend to have higher grayscale invariance and accuracy than standard models, which again indicates the usefulness of large training sets.
Third and interestingly, the correlation is stronger on ImageNet-R than ImageNet-Val and ImageNet-V2-A ( 0 . 940 vs. 0 .
900 vs. 0 . 868 ).
In fact, ImageNet-R is featured by style shift during its collection [4], so for this test set being invariant to color changes is an important property for a stronger generalization ability.
5.3 Correlation Exists on Very Hard Test Sets
We now study the correlation under very hard test sets and use ImageNet-A and ImageNet-GaussBlur for testing, on which accuracy of 83 models is lower than 20% .
We study rotation invariance and c l a ss i f i c a t i o n a cc u r a c y
[Figure]
i [Figure] n [Figure]
Test
[Table]
man’s Rank Correlation ρ , JS does not show strong correlation on most datasets while EI does on all the four datasets.
grayscale invariance.
In Fig. 4, we find strong linear correlation in the four cases ( r ≤ 0 . 88 ).
The rank correlation is less consistent but still indicates clear trends.
Moreover, most standard models have low accuracy, while models (pre)trained with more data tend to have high accuracy and invariance.
We also notice standard models are scattered differently in the low-accuracy regime of ImageNet-A and ImageNet-Blur, possibly due to their dataset bias introduced during dataset construction [91–93].
5.4 Correlation Holds Under the CIFAR-10 Setup
We conduct correlation study on CIFAR-10 setup.
We collect 90 CIFAR models ranging from LeNet to EfficientNet.
We use the ID CIFAR-10 test set and two OOD test sets.
1) CIFAR-10.1 [94] is a reproduction of the CIFAR-10 test set but with distribution shift arising from changes in data collection.
It contains 2 , 000 test images sampled from TinyImages [95].
2) CINIC-10 test set [96] is an extended alternative for CIFAR-10.
It has has 90 , 000 images sampled from ImageNet.
We show the relationship between model generalization and invariance in Fig.
On all the three test sets, we observe the strong correlation between model accuracy numbers and rotation invariance scores, where both r and ρ are greater than 0 . 90 .
Moreover, when testing grayscale invariance, the strong correlation still exist (both r and ρ are greater than 0 . 83 ). [Figure]
5.5 EI Gives Stronger Correlation Than JS
In the representative cases (Fig. 1), EI shows superiority to JS in measuring model invariance by explicitly considering Softmax consistency and confidence.
Now we compare the invariance scores measured by EI and JS w.r.t their correlation strength with accuracy in Fig.
6 and Table 1.
We find that JS provides good correlation on the ID ImageNet-Val test set, but much weaker correlation on the more difficult OOD tests than EI.
As discussed in Section 3, the advantage of EI is that it not only follows the definition of invariance, but also integrates confidence to strength it, while JS only compares the overall Softmax vector.
In fact, the drawback of JS is primarily reflected in hard test sets, where the Softmax vector usually has a flat shape.
This explains why JS does not give strong correlation on the OOD test sets (see Section 5.3 for EI’s performance on very hard test sets).
We refer readers to the supplementary material for comparisons with other measures.
5.6 Comparing Two Training Manners of Their Generalization and Invariance
Existing works report that using more diverse training data artificially ( i.e. , data augmentation) or naturally ( i.e. , more real-world data) improves model invariance [4, 46–51, 97].
In this study we compare the two strategies of their generalization and invariance abilities.
We use 14 ResNetmodels trained with strong data augmentation such as PixMix [52] and AutoAugment [48].
For comparison, we employ another 3 ResNet-50 models that learn invariance using more training samples: SWSL-ResNet-50, SSL-ResNet-50, and ResNet-50-BiTM.
In Fig.
7, we observe that models with heavy and strong augmentation exhibit linear trends on ImageNet-Val and ImageNet-R, but deviate on ImageNet-A.
In comparison, models (pre)trained with more data follow the linear trend on three test sets.
The latter seem more effective in improving invariance and accuracy compared with data augmentation, especially on ImageNet-A.
To thoroughly understand and extend this initial observation, we will conduct more comprehensive experiment on this specific point in future.
Correlation of A Model’s Invariance and Generalization on OOD Test Sets
From a dataset-centric perspective, we study the relationship between generalization and invariance of a given model on different OOD test sets.
Given a model, we calculate its accuracy on every test test of ImageNet-C [87] and compute its rotation and grayscale invariance scores.
We evaluate ResNet-152, ViT-Base-16, and DenseNet-121 classifiers for the correlation study.
As shown in Fig.
8, there is a very strong correlation between classifier accuracy and invariance on various datasets ( r > 0 .
93 and ρ > 0 . 95 ).
The results indicate that a classifier tends to have a high accuracy on the test set where it has a high EI score.
The above analysis indicates that it is feasible to use EI to access out-of-distribution error of a model. [Figure] o p - [Figure]
a [Figure] u r a c [Figure]
T o pa cc u r a c y ( % )
[Figure]
In all
, we observe a strong negative
Conclusion
This work considers two critical properties of a machine learning model: invariance and generalization.
To study their relationship, we first introduce effective invariance (EI) to more reasonably measure invariance and then provide in-depth and comprehensive correlation experiment.
From a modelcentric perspective, we report accuracy and EI of various models has a strong linear relationship on both ID and OOD datasets, which is validated on many scenarios such as large-scale test sets,
CIFAR-10 and very hard test sets.
From a dataset-centric perspective, we show the accuracy and EI of a model have a strong linear relationship on various OOD datasets.
Limitations and potential directions.
First , some networks with specially designed modules can be highly invariant to some transformations [7–9, 16, 98], such as rotation invariance networks [14, 99, 17].
The rotation invariance scores of these models may present very different correlation from our observations.
That said, their color invariance scores may still exhibit similar correlation with this work.
In future works, it would be interesting to understand how these models deviate from others.
Second , our ImageNet test sets do not include the geographic shift where images are captured from various locations [4].
Recent works show this shift is also a key factor influencing model accuracy [100].
We leave this question to future study.
Moreover , we focus on classification tasks, where models are supervised by image-level annotations.
In other computer vision tasks, models may be trained with different levels of supervision, such as instance-level bounding box annotations [101], pixel-level labels [102] and temporal context supervision [103].
Different types of supervision may lead to invariance ability to various factors, which may exhibit different correlation profiles.
Lastly, in designing EI, we mainly study four representative cases and explain the limitations of some commonly used measures ( e.g. , JS and L2).
Considering more special cases would be beneficial.
Potential negative social impact.
We study fundamental model properties with public classification datasets, which might be misused in certain applications with ethical concerns.
Acknowledgments and Disclosure of Funding
We thank all anonymous reviewers for their constructive comments in improving this paper.
This work was in part supported by the ARC Discovery Early Career Researcher Award (DE200101283) and the ARC Discovery Project (DP210102801).
Stephen Gould is the recipient of an ARC Future
Fellowship (project number FT200100421) funded by the Australian Government.
Weijian Deng is a recipient of Australian Government Research Training Program (RTP) Scholarship.