On the Strong Correlation Between Model Invariance and Generalization
Anonymous Author(s)
AffiliationAddress email
Abstract
Generalization and invariance are two essential properties of machine learningmodels.
Generalization captures a model’s ability to classify unseen data whileinvariance measures consistency of model predictions on transformations of thedata.
Existing research suggests a positive relationship: a model generalizing wellshould be invariant to certain visual factors.
Building on this qualitative implication we make two contributions.
First, we introduce effective invariance (EI), asimple and reasonable measure of model invariance which does not rely on imagelabels.
Given predictions on a test image and its transformed version, EI measureshow well the predictions agree and with what level of confidence.
Second, usinginvariance scores computed by EI, we perform large-scale quantitative correlationstudies between generalization and invariance, focusing on rotation and grayscaletransformations.
From a model-centric view, we observe generalization and invariance of different models exhibit a strong linear relationship , on both in-distributionand out-of-distribution datasets.
From a dataset-centric view, we find a certainmodel’s accuracy and invariance linearly correlated on different test sets.
Apartfrom these major findings, other minor but interesting insights are also discussed.
Introduction
Generalization and invariance are two important model properties in machine learning.
The formercharacterizes how well a model performs when encountering in-distribution or out-of-distribution(OOD) test data [1–5].
The latter assesses how consistent model predictions are on transformedtest data [6–12].
Therefore, understanding how the two properties are related would benefit modeldecision analysis under dynamic environments.
The importance of model invariance on generalization has been qualitatively explored [9, 12–15].
For example, adding rotation invariance to the model improves its in-distribution (ID) classificationaccuracy [14, 16, 17]; a shift-invariant model is robust to perturbation [9]. In addition, some worksprovide a neuroscientific perspective to analyze the importance of invariant representations for patternrecognition [13, 15, 18–22]. Furthermore, theoretical investigations suggest that learning invariantfeatures benefits generalization [12, 22].
Moreover, most existing research is limited to a fewin-distribution datasets and classifier architectures.
As such, the relationship of interest remainsunknown on many other scenarios, such as out-of-distribution and large-scale test data, and othertypes of models.
Before performing the quantitative study, it is necessary to first quantify generalization and invariance.
For the former, the deep models we consider are well trained, so we simply use the accuracy on the test set, as in many previous works [3, 2, 23].
In comparison, quantifying invariance is not asstraightforward.
Some works use model accuracy drop when the test set undergoes transformations toindicate invariance ability [24, 7, 17].
While this strategy is useful for a single model, its effectivenessis limited when comparing invariance of multiple models.
Others resort to consistency, i.e. , modelsshould have the same decision [9, 6], but this method neglects prediction confidence , which we findcritical for describing invariance (see discussions in Section 3).
We make two contributions to the community.
First , we propose a new method to measure modelinvariance, named effective invariance (EI), which considers both the consistency and confidenceof predictions.
Given a test image and its transformed counterpart, if the model predicts the sameclass and with high confidence, the EI value or invariance strength is high.
Otherwise, if the modelmakes different class predictions or the confidence is low, the EI score will be low.
We show thisnew measure solves invariance valuation in canonical cases where the commonly used metrics ( e.g. ,
Jensen-Shannon divergence) may fail.
Second , we conduct a broad correlation study to quantitativelyunderstand the relationship between model generalization and invariance.
Specifically, we use 8 testsets with various distribution types, such as the in-distribution ImageNet validation set [25], and outof-distribution ImageNet-Rendition with style shift [4].
We evaluate 150 ImageNet models rangingfrom traditional convolution neural networks ( VGGs [26]) to the very recent vision transformers ( e.g. ,
BEiT [27]).
Below we list two key observations and example insights.
• For various models , there is a strong correlation between their accuracy and invariance on bothin-distribution and out-of-distribution datasets (Sections 5.1 and 5.2).
This finding can be usefulfor unsupervised model selection, because EI does not require test ground truths.
• On various out-of-distribution datasets , a model’s accuracy and EI scores are also strongly corre- lated (Section 6).
This observation can be used to predict model accuracy on out-of-distribution datasets without access to ground truths
• Compared with data augmentation, training with more data seems more effective to improveinvariance and generalization (Section 5.6)
Related Work
Predicting generalization gap: a model-centric view.
This task aims to predict the generalizationgap of machine learning models on in-distribution data, i.e. , the difference between training andtest accuracy.
Most existing works focus on developing complexity measurement of trained networkparameters and training data [28–36], such as persistent topology [31] and the product of normsof the weights across layers [33].
These methods, assuming the training and test distributions arethe same, do not consider the characteristics of the test distribution.
Moreover, they only studylimited types of neural networks. In comparison, we conduct a much more comprehensive study onboth in-distribution and out-of-distribution test sets, using various network architectures.
We showthat invariance, under our definition, serves as a strong indicator of model generalization ability oraccuracy on both in-distribution and out-of-distribution test data.
Closely related to our work, two recent methods [37, 24] predict ID generalization gap based on howa network performs on perturbed data points.
Specifically, Kashyap et al.
[37] use confidence drop torepresent invariance, which is less effective for invariance measurement under OOD data.
Schiff etal. [24] uses accuracy drop to measure invariance, which needs test labels, while EI does not requiretest labels and is more reasonable than accuracy under OOD data.
Besides, drawing a response curvein [24] is computationally heavy, while our method is efficient.
Further, both studies are limited intheir scope: they mainly study ID generalization, has two types of networks and lack large-scale testsets, while our work is much more comprehensive.
Predicting generalization gap: a dataset-centric view.
The overall goal of this task is to predict theperformance of a given model on various unlabeled test sets [38–43].
Many methods take into accountthe statistics of the test set for accuracy prediction [38, 39, 44, 40, 45], such as distribution shift[38], average Softmax score on each test set [39].
We contribute a new solution: using the model’s
c c c c  0.
c c c c
JS=0.019EI=0.
𝑦" = 𝑦" ! 0.
c c c c  0.
c
c c c f ( 𝒙 ) f ( 𝒙 ! )
JS=0.076EI=0.
𝑦" = 𝑦" !
(a) (b) (d)
JS=0.029EI= 𝑦" ≠ 𝑦" !
JS=0.
EI= 𝑦" ≠ 𝑦" !
(c) invariance on the OOD dataset to predict its accuracy. This is supported by our new observation ofstrong linear correlation between a certain model’s accuracy and invariance on various test sets.
Improving robustness with data augmentation.
Data augmentation transforms training data toincrease its diversity, which helps learn more robust models [46–52].
For example, Mixup [53, 54]and AutoAugment [48] are shown to improve model performance under distribution changes [55, 51].
Instead of using common transformations, adversarial training [56–58] augments training images withan adversarially learned noise distribution.
While these works aim to improve corruption robustnesswith data augmentation, we instead use the latter to analyze model invariance (and generalization).
Proposed Effective Invariance (EI)
Notations .
Considering an K -way classification task, we define input space X ∈ R d and label space
Y = { 1 , ..., K } .
Given a sample ( x , y ) drawn from an unknown distribution π on X × Y , a neuralnetwork classifier f : R d → ∆ K produces a probability distribution for x on K classes, where∆ K denotes the K − 1 dimensional unit simplex.
Specifically, f i ( x ) denotes the i -th element ofthe Softmax output vector produced by f .
Then, ˆ y =:
arg max i f i ( x ) is the predicted class, andˆ p
=: max i f i ( x ) is the associated confidence score.
Image transformation is defined as T : R d → R d .
Then, the transformed image is x ′ =
T ( x ) , and its predicted class is ˆ y t
=: arg max i f i ( x ′ )
withconfidence score ˆ p t =: max
i f i ( x ′ ) .
Drawback of existing invariance measures .
A commonly seen strategy is to directly use the distancebetween the Softmax vectors of two predictions as invariance measure: a lower distance means higherinvariance, and vice versa.
Examples of the similarity metrics are Jensen-Shannon divergence (JS)[59, 46] and ℓ 2 distance [60, 61] and Kullback–Leibler divergence [62, 63].
However, they onlyleverage the global similarity between two Softmax vectors without an explicitly consideration ofprediction class consistency and confidence.
We illustrate this drawback taking JS divergence as anexample in Fig.
In cases (a) and (b) where the predicted classes are both consistent, JS decidesclassifier f in (b) has higher, which ignores the low confidence in (b).
In cases (d) where the predictedclasses are different, JS still gives high invariance (small JS score), indicating a clear error.
Moreover,in works [46, 60, 61] cases (b) and (d) are discarded when computing the consistency loss.
Definition of EI.
Unlike neuroscience works that study the invariance of an individual neuron of thenetwork [13, 15, 18, 19], we measure the invariance at the network level.
Specifically, given an imageand its transformed sample, a model with high invariance should give the same predicted class, andvice versa [6, 9].
In our definition of EI, we further use the prediction confidence.
Our motivation isas follows.
When a model predicts the same class for the two images, if either of the two predictionsis of low confidence, we should not consider it as highly invariant but give a penalty.
A model should have a high invariance if and only if it is highly confident in predicting the same class.
Based on theseconsiderations, EI is defined as:
[Equation]
To better understand the soundness of EI, we depict four representative cases in Fig.
In cases (a)and (b), classifier f gives the same predicted class ( ˆ y
= ˆ y t ) on original and transformed images.
Under EI, Classifier f has a higher invariance ability in (a) because it has high confidence scores.
Incase (c), the predicted class are different ( ˆ y ̸
= ˆ y t ), and in (d), the predictions are low in confidence(and give different classes), so we define invariance in both cases to be 0.
Note that while we use thegeometric mean of ˆ p t and ˆ p , our preliminary experiment shows the arithmetic gives similar effect.
Computation of EI in practice.
Given a test image, we generate a transformed image using acertain transformation.
Then, we compute the EI score based their Softmax vectors (Eq. 3).
Weobtain the model invariance by averaging the EI scores over all the test images.
In this work, wemainly investigate the rotation transformation and grayscale transformation.
For the former, to avoidinterpolation that would introduce artifact, we only use three transformation angles ( 90 ◦ , 180 ◦ , 270 ◦ ).
For each rotation angle, we compute an overall invariance score by comparing with the predictionsof original data.
By averaging the three EI scores, we obtain rotation invariance on the test set.
For the grayscale transformation, we remove color information and keep only luminous intensityinformation.
Then, we compare with the predictions of grayscale and original data and compute theoverall grayscale invariance on each test set.
Experimental Setup
4.1 Models to Be Evaluated
We considers both very recent and classic image classification models with different architectures,including Convolutional Neural Networks ( e.g. , standard VGGs [26], ResNets [64], and modern
ConvNeXt [65]), Vision Transformers ( e.g. , ViTs [66], Swin [67], and BEiT [27]), and all-MLParchitectures [68, 69] ( i.e , MLP-Mixer [69]).
In addition to different architectures, we also cover models with various training and regularizationstrategies ( e.g. , learning rate schedule [70], label smooth [71] and data augmentation [46, 47, 49, 48]),scaling strategies in model dimension (width, depth and resolution) [72–74], and learning manners(supervised learning, semi-supervised learning [75] and knowledge distillation [76, 77]).
In total, wehave 150 models provided by TIMM [78].
They are either trained or fine-tuned on the ImageNet-1ktraining set [25].
The selected models can be roughly divided into the following three categories:
Standard neural networks.
This category includes 100 models only trained on ImageNet training set.
These networks cover various architectures ranging from VGGs [ ] to EfficientNet
Semi-supervised learning.
We include 15 models trained in a semi-supervised learning manner.
They leverage a large collection of unlabelled images of YFCC100M ] or Instagram 900M
to improve the performance.
We use models trained based on a teacher-student paradigm ( e g. ,
SWSL-ResNet [75] and SSL-ResNet [75]).
Models trained with self-training methods on unlabeled
JFT-300M [81] ( e.g , EfficientNet-L2-NS [82]) are also included.
Pretraining on more data.
We use another 35 models that are pre-trained on significantly largerdatasets than the standard ImageNet training set.
Specifically, we consider three pre-training methods:
a) weakly supervised pretraining on IG-3.6B ( i.e. , RegNetY [83] and ResNeXt101-WSL [80]);
b) supervised pre-training on ImageNet- [Figure] y [Figure]
T o p - 1 a cc [Figure] r a c y [Figure]
4.2 Test Sets
We use both in-distribution (ID) and out-of-distribution (OOD) datasets for the correlation study.
Specifically, the ImageNet validation set (ImageNet-Val) is used as ID test set.
For OOD test sets, weuse of seven datasets, each with a different distribution from standard ImageNet.
Their distributionshift can be divided into the following five types.
Dataset reproduction shift.
ImageNet-V2 [23] is a recollected version of ImageNet-Val.
It containsthree versions resulting from different data sampling strategies : Matched-Frequency (A), Threshold0.7 (B), Top-Images (C).
Each version has 10 , 000 images from 1 , 000 classes.
Natural adversarial shift.
ImageNet-Adv(ersarial) [85] is adversarially selected to be misclassifiedby ResNet-50.
Its natural adversarial examples are unmodified real-world images and have beenshown to be hard for other models [85, 3].
It has 7 , 500 samples from 200 ImageNet classes.
Sketch shift.
ImageNet-S(ketch) [86] consists of sketch-like images and matches ImageNet-Val incategories and scale.
It contains 50 , 000 images and shares the same 1 , 000 classes as ImageNet.
Blur shift.
We use ImageNet-Blur with the highest severity provided by [87].
This dataset issynthesized by blurring ImageNet-Val images with a Gaussian function.
Style shift.
ImageNet-R(endition) [4] contains various abstract visual renditions ( e.g. , art, paintings,and video game) of ImageNet classes.
ImageNet-R has 30 , 000 images of 200 ImageNet classes.
4.3 Correlation Measures
We use Pearson Correlation coefficient ( r ) [88] and Spearman’s Rank Correlation coefficient ( ρ )[89] to measure the linearity and monotonicity between invariance and generalization, respectively.
Both coefficients range from [ − 1 , 1] .
A value closer to − 1 or 1 indicates strong negative or positivecorrelation, respectively, and 0 implies no correlation [88].
To precisely show the correlation, we uselogit axis scaling that maps the accuracy range from [0 , 1] to [ −∞ , + ∞ ] , following [3, 2].
Unlessnoted otherwise, the correlation coefficients are calculated using invariance score and non-linearlyscaled accuracy numbers.
[Figure]
pa cc u r a c y ( % )
[Figure]
c l a ss i f
i c a t i o n a cc u r a c y ( % )
[Figure]
Similar to Fig.
2, each dot denotes a model, where different colors denote different training strategies (see Section 4.1).
The subfigures correspond to three ImageNet test sets, respectively.
We observe the correlation is also strong: Pearson’s Correlation r > 0 .
840 , and Spearman’s Correlation ρ > 0 .
Experimental Observations
The experiment is from a model-centric perspective, where we investigate how different models’accuracy and invariance correlate, and series of observations are made (Section 5.1 - Section 5.6).
5.1 Strong Correlation Between Model’s Rotation Invariance and Accuracy
In Fig.
2, we show the correlation results of rotation invariance and generalization.
We have twoobservations.
First , we find that for different models, their rotation EI scores have a linear relationshipwith their classification accuracy.
The correlation holds for both ID test and OOD test sets, variousarchitectures and training strategies.
Specifically, both correlation metrics λ and ρ are higher than0.840.
This indicates models with higher accuracy numbers are most likely to have stronger rotationinvariance (measured by EI), and vice versa.
To our knowledge, it is a very early observation of thequantitative relationship between generalization and invariance (to a certain factor).
Second , training with more data benefits rotation invariance and generalization.
Large datasetscontain images with various geometric variations.
When (pre)trained with large datasets, models(blue and orange dots in Fig. 2) adapt to the rotation variations and gains stronger invariance, which,according to our study, likely means a higher generalization accuracy on ID and OOD test sets.
5.2 Strong Correlation Between Model’s Grayscale Invariance and Accuracy
We now focus on grayscale invariance and report the correlation results in Fig.
We have thefollowing conclusions.
First , among the 150 models, there is strong linear correlation betweenaccuracy and grayscale invariance measured by EI.
Specifically, both correlation coefficients r and ρare higher than 0 .
820 on all test sets.
Second , we find that pretrained or semi-supervised models tendto have higher grayscale invariance and accuracy than standard models, which again indicates theusefulness of large training sets.
Third and interestingly, the correlation is stronger on ImageNet-Rthan ImageNet-Val and ImageNet-V2-A ( 0 . 940 vs. 0 .
900 vs. 0 . 868 ).
In fact, ImageNet-R is featuredby style shift during its collection [4], so for this test set being invariant to color changes is animportant property for a stronger generalization ability. [Figure] [Figure]
5.3 Correlation Exists on Very Hard Test
We now study the correlation under very hard test sets and use ImageNet-A and ImageNet-GaussBlurfor testing, on which accuracy of 83 models is lower than 20% .
We study rotation invariance andgrayscale invariance.
In Fig.
4, we find strong linear correlation in the four cases ( r ≤ 0 . 88 ).
The rankcorrelation is less consistent but still indicates clear trends.
Moreover, most standard models havelow accuracy, while models (pre)trained with more data tend to have high accuracy and invariance.
We also notice standard models are scattered differently in the low-accuracy regime of ImageNet-Aand ImageNet-Blur, possibly due to their dataset bias introduced during dataset construction [91–93].
5.4 Correlation Holds Under the CIFAR-10 Setup
We conduct correlation study on CIFAR-10 setup.
We collect 90 CIFAR models ranging from LeNetto EfficientNet.
We use the ID CIFAR-10 test set and two OOD test sets.
1) CIFAR-10.1 [94] isa reproduction of the CIFAR-10 test set but with distribution shift arising from changes in datacollection.
It contains 2 , 000 test images sampled from TinyImages [95].
2) CINIC-10 test set [96] isan extended alternative for CIFAR-10.
It has has 90 , 000 images sampled from ImageNet.
We show the relationship between model generalization and invariance in Fig.
On all the threetest sets, we observe the strong correlation between model accuracy numbers and rotation invariancescores, where both r and ρ are greater than 0 . 90 .
In the supplementary materials, we show correlationmaintains with EI measured grayscale invariance.
5.5 EI Gives Stronger Correlation Than JS
In the representative cases (Fig. 1), EI shows superiority to JS in measuring model invariance byexplicitly considering Softmax consistency and confidence. Now we compare the invariance scores [Figure] [Figure]
with more data allows models to achieve relatively high accuracy and invariance on three test sets.
measured by EI and JS w.r.t their correlation strength with accuracy in Fig.
6 and Table 1.
We findthat JS provides good correlation on the ID ImageNet-Val test set, but much weaker correlation onthe more difficult OOD tests than EI.
As discussed in Section 3, the advantage of EI is that it notonly follows the definition of invariance, but also integrates confidence to strength it, while JS onlycompares the overall Softmax vector.
In fact, the drawback of JS is primarily reflected in hard testsets, where the Softmax vector usually has a flat shape.
This explains why JS does not give strongcorrelation on the OOD test sets (see Section 5.3 for EI’s performance on very hard test sets).
Werefer readers to the supplementary material for comparisons with other measures.
5.6 Comparing Two Training Manners of Their Generalization and Invariance
Existing works report that using more diverse training data artificially ( i.e. , data augmentation) ornaturally ( i.e. , more real-world data) improves model invariance [4, 46–51, 97].
In this study wecompare the two strategies of their generalization and invariance abilities.
We use 14 ResNet-50models trained with strong data augmentation such as PixMix [52] and AutoAugment [48].
Forcomparison, we employ another 3 ResNet-50 models that learn invariance using more trainingsamples: SWSL-ResNet-50, SSL-ResNet-50, and ResNet-50-BiTM.
In Fig.
7, we observe thatmodels with heavy and strong augmentation exhibit linear trends on ImageNet-Val and ImageNet-R,but deviate on ImageNet-A.
In comparison, models (pre)trained with more data follow the linear trendon three test sets.
The latter seem more effective in improving invariance and accuracy comparedwith data augmentation, especially on ImageNet-A.
To thoroughly understand and extend this initialobservation, we will conduct more comprehensive experiment on this specific point in future.
[Figure]
Correlation of A Model’s Invariance and Generalization on OOD Test Sets
From a dataset-centric perspective, we study the relationship between generalization and invarianceof a given model on different OOD test sets.
Given a model, we calculate its accuracy on every testtest of ImageNet-C [87] and compute its rotation and grayscale invariance scores.
We evaluate ResNet-152, ViT-Base-16, and DenseNet-121 classifiers for the correlation study.
Asshown in Fig. 8, there is a very strong correlation between classifier accuracy and invariance onvarious datasets ( r > 0 .
93 and ρ > 0 . 95 ).
The results indicate that a classifier tends to have a highaccuracy on the test set where it has a high EI score.
The above analysis indicates that it is feasible touse EI to access out-of-distribution error of a model.
Conclusion
This work considers two critical properties of a machine learning model: invariance and generalization.
To study their relationship, we first introduce effective invariance (EI) to more reasonably measureinvariance and then provide in-depth and comprehensive correlation experiment.
From a modelcentric perspective, we report accuracy and EI of various models has a strong linear relationshipon both ID and OOD datasets, which is validated on many scenarios such as large-scale test sets,
CIFAR-10 and very hard test sets.
From a dataset-centric perspective, we show the accuracy and EIof a model have a strong linear relationship on various OOD datasets.
Limitations and potential directions.
First , some networks with specially designed modules canbe highly invariant to some transformations [7–9, 16, 98], such as rotation invariance networks[14, 99, 17].
The rotation invariance scores of these models may present very different correlationfrom our observations.
That said, their color invariance scores may still exhibit similar correlationwith this work.
In future works, it would be interesting to understand how these models deviatefrom others.
Second , our ImageNet test sets do not include the geographic shift where images arecaptured from various locations [4].
Recent works show this shift is also a key factor influencingmodel accuracy [100].
We leave this question to future study.
Lastly , we focus on classification tasks,where models are supervised by image-level annotations.
In other computer vision tasks, models may be trained with different levels of supervision, such as instance-level bounding box annotations [101],pixel-level labels [102] and temporal context supervision [103].
Different types of supervision maylead to invariance ability to various factors, which may exhibit different correlation profiles.
Potential negative social impact.
We study fundamental model properties with public classificationdatasets, which might be misused in certain applications with ethical concerns.