Do deep networks transfer invariances across classes?

1 INTRODUCTION . Good generalization in machine learning models requires ignoring irrelevant details : a classifier should respond to whether the subject of an image is a cat or a dog but not to the background or lighting conditions . Put another way , generalization involves invariance to nuisance transformations that should not affect the predicted output . Deep neural networks can and do learn invariances given sufficiently many diverse examples . For example , we can expect our trained “ cat vs dog ” classifier to be invariant to changes in the background provided the training dataset contains images of cats and dogs with varied scenery . However , if all the training examples for the “ dog ” class are set in grassy fields , our classifier might be confused by an image of a dog in a house ( Beery et al. , 2018 ) . This situation is problematic for imbalanced datasets , in which the amount of training data varies from class to class . Class imbalance is common in practice , as many real-world datasets follow a long-tailed distribution where a few head classes have many examples , and each of the remaining tail classes have few examples . Hence even if the cumulative number of examples in a long-tailed dataset is large , classifiers may struggle to learn invariances for the smaller tail classes . And while augmentation can address this issue by increasing the amount and variety of data in the tail classes , this strategy is not feasible for every nuisance transformation ( e.g. , modifying an image ’ s background scenery ) . On the other hand , many nuisance transformations such as lighting changes are “ class agnostic : ” they apply similarly to examples from any class , and should generalize well across classes ( Hariharan & Girshick , 2016 ) . Ideally a trained model should automatically transfer invariance to class agnostic transformations from larger classes to smaller ones . This observation raises the question : how well do deep neural network classifiers transfer learned invariances across classes ? In this work , we find empirical evidence that neural networks transfer learned invariances poorly across classes , even after applying balancing strategies like oversampling . For example , on a longtailed dataset where every example is rotated uniformly at random , classifiers tend to be rotationally invariant for images from the head classes but not on images from the tail classes . To this end , we present a straightforward method for more effectively transferring invariances across classes . We first train an input conditioned but class-agnostic generative model which captures a dataset ’ s nuisance transformations , where withholding explicit class information encourages transfer between classes . We then we use this generative model to transform training inputs , akin to learned data augmentation training the classifier . We ultimately show that the resulting classifier is more invariant to the nuisance transformations due to better invariance on the tail classes , which in turn leads to better test accuracy on those classes . Contributions . The primary contribution of this paper is an empirical study concerning whether deep neural networks learn invariances in long-tailed and class-imbalanced settings . We analyze the extent to which neural networks ( fail to ) transfer learned invariances across classes , and we argue that this lack of invariance explains poor performance on real-world class-imbalanced datasets . Our analysis suggests that one path to improving imbalanced classification is to develop approaches that better transfer invariances across classes . We experiment with one such approach , which we call Generative Invariance Transfer ( GIT ) , in which we train a generative model of a task ’ s nuisance transformations and then use this model to perform data augmentation for the classifier . We find that combining GIT with existing methods such as resampling improves balanced accuracy on imbalanced image classification benchmarks such as GTSRB and CIFAR-LT . 2 RELATED WORK . Real-world datasets are often class imbalanced , and state-of-the-art classifiers often perform poorly under such conditions . One particular setting of interest is when the class distribution is longtailed , where most of the classes have only a few examples ( Liu et al. , 2019 ; Guo et al. , 2016 ; Thomee et al. , 2016 ; Horn & Perona , 2017 ) . Researchers have proposed many approaches for this setting , including correcting the imbalance when sampling data ( Buda et al. , 2018 ; Huang et al. , 2016 ; Chawla et al. , 2002 ; He & Garcia , 2009 ) , using modified loss functions ( Cao et al. , 2019 ; Lin et al. , 2017 ; Cui et al. , 2019 ) , and modifying the optimizer ( Tang et al. , 2020 ) . These methods generally do not study the question of invariance transfer and are complementary to Generative Invariance Transfer . Yang & Xu ( 2020 ) study the mixed value of imbalanced labels and propose learning class-agnostic information using self-supervised training , while GIT learns class-agnostic transformations to use as data augmentation . Wang et al . ( 2017 ) propose implicit transfer learning from head to tail classes using a meta-network that generates the weights of the classifier ’ s model . Here we quantitatively measure per-class invariances to more precisely explain why transfer learning may help on imbalanced problems , and explicitly transfer across classes with a generative model . Toward understanding the failure modes of classifiers trained on real-world datasets , a rapidly growing body of work has sought to study the robustness of commonly-used machine learning models . Notably , researchers have shown that the performance state-of-the-art models is susceptible to a wide range of adversarial attacks ( Biggio & Roli , 2018 ; Madry et al. , 2017 ) and distributional shifts ( Taori et al. , 2020 ; Hendrycks et al. , 2020 ; Koh et al. , 2021 ) . Ultimately , the fragility of these models is a significant barrier to deployment in safety-critical applications such as medical imaging ( Bashyam et al. , 2020 ) and autonomous driving ( Chernikova et al. , 2019 ) . In response to these fragilities , recent methods have learned the distributional shifts present in the data using generative models and then use these learned transformations for robust training ( Robey et al. , 2020 ; Wong & Kolter , 2020 ) . These works assume pairs or groupings of transformed and untransformed examples , so that a generative model can learn the distribution shift . In our imbalanced setting , generative invariance transfer aims to learn transformations from the head classes that apply to the tail classes of the same dataset , and does not assume the data is paired or grouped . There is a general interest in obtaining invariances for machine learning models ( Benton et al. , 2020 ; Zhou et al. , 2021 ) . Data augmentation ( Beymer & Poggio , 1995 ; Niyogi et al. , 1998 ) can be used to train classifiers to be invariant to certain hand-picked transformations , but requires the practitioner to know and implement those transformations in advance . In search of greater generality , Antoniou et al . ( 2017 ) use a trained GAN as data augmentations for training a downstream classifier . Similarly , Mariani et al . ( 2018 ) use a GAN to generate more examples for the minority classes in imbalanced classification . Learned augmentation can also be done in feature space , to avoid learning a generative model of the high dimensional input space ( Yin et al. , 2018 ; Chu et al. , 2020 ) . In the low-shot setting Hariharan & Girshick ( 2016 ) study the notion of transferring generalizable ( or classagnostic ) variation to new classes , and Wang et al . ( 2018 ) learn to produce examples for new classes using meta-learning . Work in neural style transfer ( Gatys et al. , 2015b ; a ; Johnson et al. , 2016 ) has long studied how to leverage trained neural networks to transfer variation between images or domains . In particular , GIT leverages advances in image-to-image translation ( Isola et al. , 2016 ; Zhu et al. , 2017 ; Huang et al. , 2018 ) as a convenient way to learn nuisance transformations and transfer them between classes . Our work carefully quantifies invariance learning under class imbalance , which can explain why leveraging generative models of transformations can improve performance . There is prior work analyzing how learned representations and invariances are affected by noise and regularization ( Achille & Soatto , 2018 ) , task diversity ( Yang et al. , 2019 ) , and data diversity ( Madan et al. , 2021 ) . The latter is most relevant to our own analysis , which studies how invariance is affected by the amount of data per class . 3 INVARIANCE TRANSFER IN CLASS-IMBALANCED DATASETS . In this section we empirically analyze the invariance of trained classifiers to nuisance transformations , and the extent to which these classifiers transfer invariances across classes . In particular , we first introduce concepts related to invariance in the imbalanced setting , then define a metric for measuring invariance before describing our experimental setup . Finally , we present and analyze the observed relationship between invariance and class size . 3.1 SETUP : CLASSIFICATION , IMBALANCE , AND INVARIANCES . In the classification setting , we have input-label pairs ( x , y ) , with y taking values in { 1 , · · · , C } where C is the number of classes . We will consider a neural network model parameterized by weightsw that is trained to estimate the conditional probabilities P̂w ( y = j|x ) . The classifier selects the class j with the highest estimated probability . Given a training dataset { ( x ( i ) , y ( i ) ) } Ni=1 ∼ Ptrain , empirical risk minimization ( ERM ) minimizes average loss over training examples . But in the classimbalanced setting , the distribution of { y ( i ) } in our training dataset is not uniform , and ERM tends to perform poorly on the minority classes . In real world scenarios we typically want to perform well on all classes , e.g . in order to classify rare diseases properly ( Bajwa et al. , 2020 ) or to ensure fairness in decision making ( Hinnefeld et al. , 2018 ) . Hence we evaluate classifiers using class-balanced metrics , which is equivalent to evaluating on a test distribution Ptest that is uniform over y . To analyze invariance , we assume there is a θ ∼ Q parameterizing a family of nuisance transformations T ( · ; θ ) that do not affect labels , i.e . ( x , y ) and ( T ( x ; θ ) , y ) are the same in distribution . As a simple example , consider an image dataset where images in each class can be rotated in any way with equal likelihood . Then T ( · ; θ ) corresponds to 2D rotation by angle θ ∼ Unif ( 0 , 2π ) . Given that the nuisance transformations are assumed to not impact the labels , we expect a good classifier to be invariant to such transformations , i.e. , P̂w ( ·|x ) = P̂w ( ·|T ( x ; θ ) ) , ∀x , θ . ( 1 ) That is , the estimated conditional class probabilities should be unchanged by these transformations . 3.2 MEASURING LEARNED INVARIANCES . To quantify the extent to which classifiers learn invariances , we measure the expected KL divergence ( eKLD ) between its estimated distributions for untransformed and transformed inputs : eKLD ( P̂w ) = Ex , θ [ DKL ( P̂w ( ·|x ) ||P̂w ( ·|T ( x ; θ ) ) ] ( 2 ) This is a non-negative number , with lower eKLD corresponding to more invariance ; a classifier totally invariant to T would have an eKLD of 0 . We can compute this metric in practice if we have the nuisance transformation T on hand : By sampling transformation parameters { θ ( j ) } and applying them to held out inputs , we can estimate the expectation in Eq . 2 . To study how invariance depends on class size , we can also naturally compute the class-conditional eKLD by restricting the expectation over x to examples from a given class j . Calculating eKLD and studying invariance requires access to a dataset ’ s underlying nuisance transformation family T , but for most real world datasets we do not know T a-priori . Instead , we create synthetic datasets that contain a known nuisance transformation family . Like RotMNIST ( Larochelle et al. , 2007 ) , a common benchmark for rotation invariance , we create these datasets by transforming each example in a natural dataset . This is different from data augmentation , where multiple randomly sampled transformations would be applied to the same image throughout training . We want to test how classifiers learn invariances from a limited amount of data diversity ; providing the transformation as data augmentation artificially boosts the observed data diversity and we use it as an Oracle comparison in later experiments . Since MNIST only contains 10 classes , we use Kuzushiji-49 ( Clanuwat et al. , 2018 ) which has 49 classes allowing for a finer-grained study across different class sizes . We create synthetic datasets using three different transformation families : image rotation ( K49-ROT-LT ) , varying background intensity ( K49-BG-LT ) , and image dilation or erosion ( K49-DIL-LT ) . Appendix Figure 7 shows representative image samples from each of these datasets . To make the training datasets long-tailed ( LT ) , we choose an arbitrary ordering of classes from largest to smallest . Then we selectively remove examples from classes until the frequency of classes in the training dataset follows Zipf ’ s law with parameter 2.0 , while enforcing a minimum class size of 5 . We repeat this process with 30 randomly sampled orderings of the classes from largest to smallest , to construct 30 different long-tailed training datasets for each variant . Each long-tailed training set has 7864 training examples , with the largest class having 4828 and the smallest having 5 . Since we are interested in either per-class or class-balanced test metrics , we do not modify the test set class distribution . Good classifiers for K49-ROT-LT , K49-BG-LT , and K49-DIL-LT should clearly be rotation , background , and dilation/erosion invariant , respectively , for inputs from any class . We can train classifiers and measure their invariance for inputs of each class by estimating per-class eKLD . That is , we first sample more transformations and then measure how predictions change on held-out test inputs . For training we use both standard ERM and CE+DRS ( Cao et al. , 2019 ) , which stands for delayed ( class-balanced ) resampling with standard cross-entropy loss . DRS samples training examples naively just as in ERM for the initial epochs of training , then switches to class-balanced sampling for the later epochs of training . We train a separate classifier using each method on each training dataset , then calculate each classifier ’ s per-class eKLD on held out test examples . Figure 1 shows how the resulting per-class eKLD varies with class size . The eKLD curves for both methods show a clear pattern on all three transformation families : invariance decreases as classes get smaller . This result , while intuitive , shows quantitatively that deep neural networks learn invariances non-uniformly on imbalanced datasets . It suggests that they fail to transfer learned invariances across classes despite the nuisance transformation families being fairly class-agnostic . Although these results use a ResNet architecture , Appendix C.2 shows similar results for non-ResNet CNNs . Note that by construction , smaller classes contain both fewer original K49 examples and less transformation diversity . Yet Appendix 8 shows the same trend for datasets where the number of original K49 examples is the same across all classes , with larger classes only containing more sampled transformations of the same images . This shows that the trend is largely due to observed transformation diversity , rather than the number of original K49 examples . Figure 1 also shows that training classifiers with resampling ( CE+DRS ) results in generally lower eKLD ( more invariance ) for the same class size . This effect may partly explain why resampling can improve class balanced metrics such as balanced test accuracy . However , the eKLD curves show that there is clearly still room for improvement and even DRS does not achieve close to uniform invariance learning across classes . One explanation for this is that DRS can show the classifier the same minority class images more often , but can not increase the transformation diversity of those images ( e.g. , DRS can not by itself create more examples of rotated images from the minority classes ) . This suggests that learning the transformation family T ( · ; θ ) , combined with resampling schemes , can help classifiers achieve uniform invariance across classes .

Summary: The paper investigates if robustness (or "invariance") to nuisance transformations which do not change the class label such as lighting, rotation etc are learned across all classes or if such robustness is sensitive to the class size. The paper demonstrates that such invariances seem not to transfer across classes: ie the classes with fewer examples suffer more. THe paper proposes a generative model (GIT) to "augment" the less frequent classes which, to some degree, remedies this problem.