Do deep networks transfer invariances across classes?

1 INTRODUCTION . Good generalization in machine learning models requires ignoring irrelevant details : a classifier should respond to whether the subject of an image is a cat or a dog but not to the background or lighting conditions . Put another way , generalization involves invariance to nuisance transformations that should not affect the predicted output . Deep neural networks can and do learn invariances given sufficiently many diverse examples . For example , we can expect our trained “ cat vs dog ” classifier to be invariant to changes in the background provided the training dataset contains images of cats and dogs with varied scenery . However , if all the training examples for the “ dog ” class are set in grassy fields , our classifier might be confused by an image of a dog in a house ( Beery et al. , 2018 ) . This situation is problematic for imbalanced datasets , in which the amount of training data varies from class to class . Class imbalance is common in practice , as many real-world datasets follow a long-tailed distribution where a few head classes have many examples , and each of the remaining tail classes have few examples . Hence even if the cumulative number of examples in a long-tailed dataset is large , classifiers may struggle to learn invariances for the smaller tail classes . And while augmentation can address this issue by increasing the amount and variety of data in the tail classes , this strategy is not feasible for every nuisance transformation ( e.g. , modifying an image ’ s background scenery ) . On the other hand , many nuisance transformations such as lighting changes are “ class agnostic : ” they apply similarly to examples from any class , and should generalize well across classes ( Hariharan & Girshick , 2016 ) . Ideally a trained model should automatically transfer invariance to class agnostic transformations from larger classes to smaller ones . This observation raises the question : how well do deep neural network classifiers transfer learned invariances across classes ? In this work , we find empirical evidence that neural networks transfer learned invariances poorly across classes , even after applying balancing strategies like oversampling . For example , on a longtailed dataset where every example is rotated uniformly at random , classifiers tend to be rotationally invariant for images from the head classes but not on images from the tail classes . To this end , we present a straightforward method for more effectively transferring invariances across classes . We first train an input conditioned but class-agnostic generative model which captures a dataset ’ s nuisance transformations , where withholding explicit class information encourages transfer between classes . We then we use this generative model to transform training inputs , akin to learned data augmentation training the classifier . We ultimately show that the resulting classifier is more invariant to the nuisance transformations due to better invariance on the tail classes , which in turn leads to better test accuracy on those classes . Contributions . The primary contribution of this paper is an empirical study concerning whether deep neural networks learn invariances in long-tailed and class-imbalanced settings . We analyze the extent to which neural networks ( fail to ) transfer learned invariances across classes , and we argue that this lack of invariance explains poor performance on real-world class-imbalanced datasets . Our analysis suggests that one path to improving imbalanced classification is to develop approaches that better transfer invariances across classes . We experiment with one such approach , which we call Generative Invariance Transfer ( GIT ) , in which we train a generative model of a task ’ s nuisance transformations and then use this model to perform data augmentation for the classifier . We find that combining GIT with existing methods such as resampling improves balanced accuracy on imbalanced image classification benchmarks such as GTSRB and CIFAR-LT . 2 RELATED WORK . Real-world datasets are often class imbalanced , and state-of-the-art classifiers often perform poorly under such conditions . One particular setting of interest is when the class distribution is longtailed , where most of the classes have only a few examples ( Liu et al. , 2019 ; Guo et al. , 2016 ; Thomee et al. , 2016 ; Horn & Perona , 2017 ) . Researchers have proposed many approaches for this setting , including correcting the imbalance when sampling data ( Buda et al. , 2018 ; Huang et al. , 2016 ; Chawla et al. , 2002 ; He & Garcia , 2009 ) , using modified loss functions ( Cao et al. , 2019 ; Lin et al. , 2017 ; Cui et al. , 2019 ) , and modifying the optimizer ( Tang et al. , 2020 ) . These methods generally do not study the question of invariance transfer and are complementary to Generative Invariance Transfer . Yang & Xu ( 2020 ) study the mixed value of imbalanced labels and propose learning class-agnostic information using self-supervised training , while GIT learns class-agnostic transformations to use as data augmentation . Wang et al . ( 2017 ) propose implicit transfer learning from head to tail classes using a meta-network that generates the weights of the classifier ’ s model . Here we quantitatively measure per-class invariances to more precisely explain why transfer learning may help on imbalanced problems , and explicitly transfer across classes with a generative model . Toward understanding the failure modes of classifiers trained on real-world datasets , a rapidly growing body of work has sought to study the robustness of commonly-used machine learning models . Notably , researchers have shown that the performance state-of-the-art models is susceptible to a wide range of adversarial attacks ( Biggio & Roli , 2018 ; Madry et al. , 2017 ) and distributional shifts ( Taori et al. , 2020 ; Hendrycks et al. , 2020 ; Koh et al. , 2021 ) . Ultimately , the fragility of these models is a significant barrier to deployment in safety-critical applications such as medical imaging ( Bashyam et al. , 2020 ) and autonomous driving ( Chernikova et al. , 2019 ) . In response to these fragilities , recent methods have learned the distributional shifts present in the data using generative models and then use these learned transformations for robust training ( Robey et al. , 2020 ; Wong & Kolter , 2020 ) . These works assume pairs or groupings of transformed and untransformed examples , so that a generative model can learn the distribution shift . In our imbalanced setting , generative invariance transfer aims to learn transformations from the head classes that apply to the tail classes of the same dataset , and does not assume the data is paired or grouped . There is a general interest in obtaining invariances for machine learning models ( Benton et al. , 2020 ; Zhou et al. , 2021 ) . Data augmentation ( Beymer & Poggio , 1995 ; Niyogi et al. , 1998 ) can be used to train classifiers to be invariant to certain hand-picked transformations , but requires the practitioner to know and implement those transformations in advance . In search of greater generality , Antoniou et al . ( 2017 ) use a trained GAN as data augmentations for training a downstream classifier . Similarly , Mariani et al . ( 2018 ) use a GAN to generate more examples for the minority classes in imbalanced classification . Learned augmentation can also be done in feature space , to avoid learning a generative model of the high dimensional input space ( Yin et al. , 2018 ; Chu et al. , 2020 ) . In the low-shot setting Hariharan & Girshick ( 2016 ) study the notion of transferring generalizable ( or classagnostic ) variation to new classes , and Wang et al . ( 2018 ) learn to produce examples for new classes using meta-learning . Work in neural style transfer ( Gatys et al. , 2015b ; a ; Johnson et al. , 2016 ) has long studied how to leverage trained neural networks to transfer variation between images or domains . In particular , GIT leverages advances in image-to-image translation ( Isola et al. , 2016 ; Zhu et al. , 2017 ; Huang et al. , 2018 ) as a convenient way to learn nuisance transformations and transfer them between classes . Our work carefully quantifies invariance learning under class imbalance , which can explain why leveraging generative models of transformations can improve performance . There is prior work analyzing how learned representations and invariances are affected by noise and regularization ( Achille & Soatto , 2018 ) , task diversity ( Yang et al. , 2019 ) , and data diversity ( Madan et al. , 2021 ) . The latter is most relevant to our own analysis , which studies how invariance is affected by the amount of data per class . 3 INVARIANCE TRANSFER IN CLASS-IMBALANCED DATASETS . In this section we empirically analyze the invariance of trained classifiers to nuisance transformations , and the extent to which these classifiers transfer invariances across classes . In particular , we first introduce concepts related to invariance in the imbalanced setting , then define a metric for measuring invariance before describing our experimental setup . Finally , we present and analyze the observed relationship between invariance and class size . 3.1 SETUP : CLASSIFICATION , IMBALANCE , AND INVARIANCES . In the classification setting , we have input-label pairs ( x , y ) , with y taking values in { 1 , · · · , C } where C is the number of classes . We will consider a neural network model parameterized by weightsw that is trained to estimate the conditional probabilities P̂w ( y = j|x ) . The classifier selects the class j with the highest estimated probability . Given a training dataset { ( x ( i ) , y ( i ) ) } Ni=1 ∼ Ptrain , empirical risk minimization ( ERM ) minimizes average loss over training examples . But in the classimbalanced setting , the distribution of { y ( i ) } in our training dataset is not uniform , and ERM tends to perform poorly on the minority classes . In real world scenarios we typically want to perform well on all classes , e.g . in order to classify rare diseases properly ( Bajwa et al. , 2020 ) or to ensure fairness in decision making ( Hinnefeld et al. , 2018 ) . Hence we evaluate classifiers using class-balanced metrics , which is equivalent to evaluating on a test distribution Ptest that is uniform over y . To analyze invariance , we assume there is a θ ∼ Q parameterizing a family of nuisance transformations T ( · ; θ ) that do not affect labels , i.e . ( x , y ) and ( T ( x ; θ ) , y ) are the same in distribution . As a simple example , consider an image dataset where images in each class can be rotated in any way with equal likelihood . Then T ( · ; θ ) corresponds to 2D rotation by angle θ ∼ Unif ( 0 , 2π ) . Given that the nuisance transformations are assumed to not impact the labels , we expect a good classifier to be invariant to such transformations , i.e. , P̂w ( ·|x ) = P̂w ( ·|T ( x ; θ ) ) , ∀x , θ . ( 1 ) That is , the estimated conditional class probabilities should be unchanged by these transformations . 3.2 MEASURING LEARNED INVARIANCES . To quantify the extent to which classifiers learn invariances , we measure the expected KL divergence ( eKLD ) between its estimated distributions for untransformed and transformed inputs : eKLD ( P̂w ) = Ex , θ [ DKL ( P̂w ( ·|x ) ||P̂w ( ·|T ( x ; θ ) ) ] ( 2 ) This is a non-negative number , with lower eKLD corresponding to more invariance ; a classifier totally invariant to T would have an eKLD of 0 . We can compute this metric in practice if we have the nuisance transformation T on hand : By sampling transformation parameters { θ ( j ) } and applying them to held out inputs , we can estimate the expectation in Eq . 2 . To study how invariance depends on class size , we can also naturally compute the class-conditional eKLD by restricting the expectation over x to examples from a given class j . Calculating eKLD and studying invariance requires access to a dataset ’ s underlying nuisance transformation family T , but for most real world datasets we do not know T a-priori . Instead , we create synthetic datasets that contain a known nuisance transformation family . Like RotMNIST ( Larochelle et al. , 2007 ) , a common benchmark for rotation invariance , we create these datasets by transforming each example in a natural dataset . This is different from data augmentation , where multiple randomly sampled transformations would be applied to the same image throughout training . We want to test how classifiers learn invariances from a limited amount of data diversity ; providing the transformation as data augmentation artificially boosts the observed data diversity and we use it as an Oracle comparison in later experiments . Since MNIST only contains 10 classes , we use Kuzushiji-49 ( Clanuwat et al. , 2018 ) which has 49 classes allowing for a finer-grained study across different class sizes . We create synthetic datasets using three different transformation families : image rotation ( K49-ROT-LT ) , varying background intensity ( K49-BG-LT ) , and image dilation or erosion ( K49-DIL-LT ) . Appendix Figure 7 shows representative image samples from each of these datasets . To make the training datasets long-tailed ( LT ) , we choose an arbitrary ordering of classes from largest to smallest . Then we selectively remove examples from classes until the frequency of classes in the training dataset follows Zipf ’ s law with parameter 2.0 , while enforcing a minimum class size of 5 . We repeat this process with 30 randomly sampled orderings of the classes from largest to smallest , to construct 30 different long-tailed training datasets for each variant . Each long-tailed training set has 7864 training examples , with the largest class having 4828 and the smallest having 5 . Since we are interested in either per-class or class-balanced test metrics , we do not modify the test set class distribution . Good classifiers for K49-ROT-LT , K49-BG-LT , and K49-DIL-LT should clearly be rotation , background , and dilation/erosion invariant , respectively , for inputs from any class . We can train classifiers and measure their invariance for inputs of each class by estimating per-class eKLD . That is , we first sample more transformations and then measure how predictions change on held-out test inputs . For training we use both standard ERM and CE+DRS ( Cao et al. , 2019 ) , which stands for delayed ( class-balanced ) resampling with standard cross-entropy loss . DRS samples training examples naively just as in ERM for the initial epochs of training , then switches to class-balanced sampling for the later epochs of training . We train a separate classifier using each method on each training dataset , then calculate each classifier ’ s per-class eKLD on held out test examples . Figure 1 shows how the resulting per-class eKLD varies with class size . The eKLD curves for both methods show a clear pattern on all three transformation families : invariance decreases as classes get smaller . This result , while intuitive , shows quantitatively that deep neural networks learn invariances non-uniformly on imbalanced datasets . It suggests that they fail to transfer learned invariances across classes despite the nuisance transformation families being fairly class-agnostic . Although these results use a ResNet architecture , Appendix C.2 shows similar results for non-ResNet CNNs . Note that by construction , smaller classes contain both fewer original K49 examples and less transformation diversity . Yet Appendix 8 shows the same trend for datasets where the number of original K49 examples is the same across all classes , with larger classes only containing more sampled transformations of the same images . This shows that the trend is largely due to observed transformation diversity , rather than the number of original K49 examples . Figure 1 also shows that training classifiers with resampling ( CE+DRS ) results in generally lower eKLD ( more invariance ) for the same class size . This effect may partly explain why resampling can improve class balanced metrics such as balanced test accuracy . However , the eKLD curves show that there is clearly still room for improvement and even DRS does not achieve close to uniform invariance learning across classes . One explanation for this is that DRS can show the classifier the same minority class images more often , but can not increase the transformation diversity of those images ( e.g. , DRS can not by itself create more examples of rotated images from the minority classes ) . This suggests that learning the transformation family T ( · ; θ ) , combined with resampling schemes , can help classifiers achieve uniform invariance across classes .

Summary: ======================= Summary:  This paper studies the problem of how well do neural networks transfer class-agnostic invariances from head classes to tail classes. They found that even a given transformation is class-agnostic, the DNN models still cannot disentangle it from class-specific features, which could partially explain the worse performances on rare classes. To solve this problem, they introduce a GIT method that uses a generative model to augment the tail categories with more diverse samples under certain transformations. It can empirically improve the long-tailed performances and transfer the knowledge of class-agnostic transformations from head to tail.