Underwhelming Generalization Improvements From Controlling Feature Attribution

1 INTRODUCTION . Overfitting is a common problem in machine learning , particularly when one uses powerful function approximators such as deep neural networks . When training these models with backpropagation , the network will evolve from modelling simple to more complicated functions until it finds salient discriminative features in the data . Once the model has found these , the gradients of the loss do not encourage the model to find other discriminative features in the data , even if they exist ( Reed & Marks , 1999 ) . In the classification case , this can be problematic if there exists some distractor feature xd in the data that is correlated with one of the output classes . This is a common issue in industry data ( e.g. , medical ) where datasets are typically small and there are many confounding variables . Consider the extreme case in a binary classification problem where in the training distribution there exists a confounding distractor element xd of the input data such that for Dtrain , p ( y = 1|xd ) = 1 , while in the validation distribution Dvalid , p ( y = 0|xd ) = 1 ( Figure 1 ) . In this scenario , predicting using xd is easier than predicting using the true features that denote class membership and a classifier trained on Dtrain with traditional classification loss would predict the incorrect class with 100 % probability on Dvalid . This is a textbook example of overfitting ( Goodfellow et al. , 2016 ; Reed & Marks , 1999 ) . The existence of these overfit features is the motivation behind methods seeking to learn domain-invariant representations ( Ganin & Lempitsky , 2014 ; Fernando et al. , 2014 ) , and is a common problem with real-world data ( Badgeley et al. , 2019 ; Zhao et al. , 2019 ; Young et al. , 2019 ) . In this paper , we explore the utility of various methods that allow one to use a mask on the input data to guide the network to avoid predicting from the defined region and penalize the network for attributing a prediction to a distractor . We present a synthetic dataset that encourages all models tested to overfit to an easy to represent distractor instead of a more complicated counting task . We present a novel “ activation difference ” ( actdiff ) regularizer which mitigates this behaviour directly . We also present a method where we train an autoencoder/UNet to reconstruct a masked version of the input , indirectly controlling feature representations used for classification . We compare these methods with the recently-proposed gradmask ( Simpson et al. , 2019b ) , and present an expanded analysis of this algorithm ’ s behaviour . All code for this paper , and this dataset , are available here : https : //github.com/bigtrellis2222/activmask . We compare the real-life performance of these methods on open medical datasets with traditional classifiers , and demonstrate the differences in their feature attributions using saliency maps . Finally , we describe a medical dataset curated from two openly-available X-ray databases , and describe how samples can be drawn from each to generate a dataset biased by a site-diagnosis correlation inspired by previous work ( Zhao et al. , 2019 ) . We demonstrate that , similarly to our synthetic datasets , classifiers are likely to predict using features unrelated to the task , and demonstrate that the proposed methods do mitigate this and often successfully refine the saliency maps to focus on the correct anatomy . However they do not consistently prevent overfitting . 2 RELATED WORK . It is a well-documented phenomenon that convolutional neural networks ( CNNs ) , instead of building object-level representations of the input data , tend to find convenient surface-level statistics in the training data that are predictive of class ( Jo & Bengio , 2017 ) . Previous work has attempted to reduce the model ’ s proclivity to use distractor features by randomly masking out regions of the input ( DeVries & Taylor , 2017 ) . By randomly removing information from the inputs to the network , this method helped the network learn representations that aren ’ t dependent on single feature types in the image . However , this regularization approach gives no control over the kinds of representations learned by the model . Recently , the Gradmask ( Simpson et al. , 2019b ) and CARE methods ( Zhuang et al. , 2019 ) both proposed to control feature representations by penalizing the model for utilizing gradients outside of regions of interest . CARE was additionally designed to deal with class imbalances by increasing the impact of the gradients inside region of interest of the under-represented class . In contrast to these two methods , we propose a new method which does not work with a saliency map , which can be noisy due to the ReLU activations allowing irrelevant features to pass through the activation function ( Kim et al. , 2019 ) . Instead , this method operates directly on the activations themselves , encouraging the model to produce similar activation patterns in the presence of , and absence of , information outside of the region of interest . 3 METHODS . Actdiff Loss : To mitigate the effect of xd , we propose to explicitly regularize the network to ignore xd at test time by minimizing the distance between the network activations of the model when presented with a full input image and one where the information outside of some mask on the image has been corrupted . The model is only directly trained on unmasked examples . This encourages the network to build features which appear inside the masked regions even though it always sees the full image during training . The method requires having access to masks drawn by an expert who can distinguish between interesting and non-interesting discriminatve features , as is often the case in medical imaging . The actdiff regularization term is Lact = 1 n n∑ l=1 ||ol ( xmasked ) − ol ( x ) ||2 , ( 1 ) where ol ( x ) are the pre-activation outputs for layer l of the n-layer encoder f ( x ) when the network is presented with the original data x , and ol ( xmasked ) are the pre-activations outputs for layer l when presented with masked data xmasked . We call this the actdiff penalty . xmasked should be constructed by randomizing the indices of all pixels that fall outside of the mask , destroying any spatial information available in those regions of the image , but retaining the distribution of intensities found in the data . Furthermore to retain important context around a masked region , we always dilate the mask by a set number of pixels . This loss has previously been used in style transfer to transform a random noise image into one containing the same layer-wise activations as some target image Gatys et al . ( 2015 ) . Reconstruction Loss : The pressure from Lact to not represent aspects of the input data x can be balanced with a reconstruction term that encourages the model to represent the data . Concretely , we can employ an auto-encoder architecture x̂ = g ( f ( x ) ) , where f is a convolutional encoder and g is a deconvolutional decoder , and require the model to minimize the reconstruction loss Lrecon = 1m ∑m j=1 ||x ( j ) − x̂ ( j ) ||2 , where j indexes the pixel-wise difference between the input x and reconstruction x̂ . In our experiments a reconstruction term is helpful for guiding the network toward building useful representations of the data while employing Lact . All autoencoder models presented in this paper employed this loss term . Furthermore , the reconstruction task can be used to indirectly control feature representation learning by asking the model to reconstruct a masked version of the input given the full input x̂masked = g ( f ( x ) ) and minimizing Lrecon masked = 1m ∑m j=1 ||x ( j ) masked − x̂ ( j ) masked||2 , which we used as a replacement for Lrecon . Gradmask Loss : Gradmask is a recently proposed Simpson et al . ( 2019b ) method for controlling which regions of the input are desirable for determining the class label using saliency maps . Saliency maps , or “ input feature attribution ” , can be calculated as ∂|ŷi|∂x for each input x ( Zeiler & Fergus , 2013 ; Simonyan et al. , 2014 ; Lo et al. , 2015 ) . In these experiments we minimize the ’ contrast ’ saliency between healthy and non-healthy classes ( labels y0 and y1 respectively ) , as we expect that input variance which increases the distinction between the two classes leads to overfitting and is what we want to regularize . Therefore , we minimize Lgrad = ∑ x∈D Lclf + ∣∣∣∣∂ |ŷ1 − ŷ0|∂x · ( 1− xseg ) ∣∣∣∣ 2 , ( 2 ) where ŷ0 and ŷ1 are the predicted outputs for our two classes and ( 1 − xseg ) is a binary mask that covers everything outside the defined regions of interest . Control Experiments : For many experiments , we evaluated the effect of simply training a model using masked data ( and evaluating it using unmasked data ) . We call these experiments Classify Masked . We present this as a naive alternative to actdiff . Additionally , we evaluated what would happen if our auto encoder models were asked to reconstruct a masked version of the input ( Lrecon masked above ) . These experiments are reported as Reconstruct Masked . 4 SYNTHETIC DATASET . Method : To evaluate the proposed methods for combating overfitting in the presence of a distractor variable , we generated a dataset following the description provided earlier ( Figure 1 ) with 500 training , 128 validation , and 128 test examples respectively . The position of the distractors was perfectly correlated with class label in the training set and the logic governing this relationship was inverted for the validation and test sets . In cases where the model relies on the distractor to make a class prediction , we expected 0.0 AUC for the validation and test sets . To evaluate the effect of the actdiff loss , gradmask loss , and the reconstruction penalty during training , we constructed a simple CNN architecture that optionally deconvolved the final layer to generate a reconstruction , or optionally did so in the style of a UNet ( Ronneberger et al. , 2015 ) , see Figure 2 . We additionally tested all non-reconstructing approaches using a simple 18-layer ResNet model ( He et al. , 2016 ) . As control experiments , we also evaluated classifier performance when simply trained using masked versions of x , but evaluated on unmasked examples . All models were trained using Adam for 500 epochs with a learning rate of 10e−4 , batch size of 32 , with the weights of our regularizers in the loss term ( denoted as λ ) being λact = 10 , λgrad = 10 , and λrecon = 10 when applicable . The classification weight was set to 1 . Before masking , masks were blurred using a Gaussian filter using a σ = 0.5 , in order for some context to be included around the masked area . For reconstruction , the binary cross entropy loss was used as the input images were binary . Results : The results of all experiments are shown in Table 1 , with the architectures that successfully avoid overfitting in bold . All results are the average of 10 random model initilizations and data splits . To determine the effect of actdiff on feature representations in the network , we display saliency maps on the validation set during the final epoch of training in Figure 3 . First we demonstrate a CNN overfitting on Dtrain , using a simple CNN architecture trained using only the cross entropy loss Lclf = − ∑N i=1 y log ŷi . This model achieves 1.0 AUC on Dtrain and 0.0 AUC on Dvalid . Note that the model is attributing all saliency to the distractor . When the model is trained using masked inputs from Dtrain and trained using Lclf , the model performs well on Dtrain but is unsure how to handle the distractor in Dvalid , leading to performance similar to chance . A CNN trained using both the classification and actdiff loss fails to learn any useful representations , scoring an AUC of 0.5 during train and validation ( see ” CNN Actdiff ” from Table 1 ) . If this model instead a Convolutional Autoencoder ( Conv AE ) or UNet , and is additionally trained using a reconstruction loss , it can successfully learns to ignore the distractor when classifying the image x ( ” Conv AE Actdiff ” and ” UNet Actdiff ” ) . Note in Figure 3 that the UNet reconstruction , taken from the validation set , places the distractor in the location seen in the training set . This implies that the actdiff loss guided the model to learn to ignore the distractor before the classification layer , but retained some representation of it in an early layer of the network . Training using reconstruction without Actdiff does not prevent overfitting ( ” Conv AE Classify ” and ” UNet Classify ” ) . Experiments with an 18-layer ResNet demonstrate good performance using classification and actdiff alone ( ” ResNet Actdiff ” ) . Gradmask never produces a model with good generalizaion performance , with generally unstable training ( Figure 7 ) . We suspect this is because the saliency map is always non-zero everywhere on the input , leading to a constant source of noise in the loss function . The best performing models took many more epochs to reach the optimal solution than would be expected for such a simple dataset . Note the ” Best Epoch ” for overfit models is misleading as the best performance is chance . We found that models using a reconstruction loss more slowly approach the vicinity of their optimum than a ResNet ( Figure 7 ) , and the ResNet model takes longer to each its best epoch ( see Table 1 ) .

Summary: This paper attempts to tackle the overfitting problem due to the focusing on the distracting information. By utilizing the dataset with masks, the authors propose a simple method to ignore the distracting features. By designing actdiff loss and reconstruction loss, the authors demonstrate that classifiers are likely to predict using features unrelated to the task and their losses can mitigate this problem.