When less is more: Simplifying inputs aids neural network understanding

1 INTRODUCTION . A better understanding of the information deep networks learn can lead to new scientific discoveries ( Raghu & Schmidt , 2020 ) , inform our understanding of differences between human and model behaviors ( Makino et al. , 2020 ) and can serve as powerful auditing tools ( Geirhos et al. , 2020 ) . Removing information from the input manually is one way to understand what information content is relevant for learning . For example , researchers have occluded parts of the input or removed specific frequency ranges from the input to see which input regions and frequency ranges are relevant for the network ’ s prediction ( Zintgraf et al. , 2017 ; Makino et al. , 2020 ; Banerjee et al. , 2021 ) . These ablation techniques use simple heuristics such as removing at random ( Hooker et al. , 2019 ) or exploit domain knowledge about interpretable aspects of the input ( input regions , frequency range ) to create simpler versions of the input and analyze the network ’ s prediction on these simpler inputs . What if , instead of using heuristics , one would learn how to simplify inputs that contain predictionrelevant information ? This way , one could synthesize simpler inputs and gain intuition into model behavior without relying on domain knowledge about what information may be relevant for the network . For this , one needs to define the precise meaning of “ simplify an input ” , including useful metrics for the simplification of the inputs and for the retention of task-relevant information . In this work , we propose SimpleBits – an information-reduction method that learns to synthesize simplified inputs which contain less information but still remain informative for the task . To measure simplicity , we use a finding initially reported as a problem for density-based anomaly detection – generative image models tend to assign higher probability densities and hence lower bits to visually simpler inputs ( Kirichenko et al. , 2020 ; Schirrmeister et al. , 2020 ) . Here , we use this to our advantage and minimize the encoding bit size given by a generative network trained on a general image distribution to simplify inputs . At the same time , we optimize that the simplified inputs still contain the task-relevant information . We then investigate what information is retained in the simplified inputs and how the simplification affects network behavior in a variety of settings . We apply SimpleBits both in a per-instance setting , where each image is processed to be a simplified version of itself , and the size of training set remains unchanged , as well as in a condensation setting , where the dataset is compressed to only a few samples per class , with the condensed samples simplified at the same time . Applied during training , SimpleBits can be used to investigate the trade-off between the information content and the task performance . After training , SimpleBits can act as an analysis tool to understand what information a trained model uses for its decision making . Figure 1 summarizes tasks covered in this paper . Our evaluation provides the following insights in the investigated scenarios : 1 . Per-instance simplification during training . SimpleBits successfully removes superfluous information for tasks with injected distractors . On natural image datasets , SimpleBits highlights plausible task-relevant information ( shape , color , texture ) . Increasing simplification leads to accuracy decreases and we report the trade-off between simplifying inputs and task level performance for different datasets . 2 . Dataset simplification with condensation . We evaluate SimpleBits applied to a condensation setting that processes the training data into a much smaller set of synthetic images . SimpleBits simplifies these images to drastically reduce the encoding size without substantial task performance decrease . On a chest radiograph dataset ( Johnson et al. , 2019a ; b ) , SimpleBits can uncover known radiologic features for pleural effusion and gender . 3 . Post-training simplification . For a trained model , SimpleBits provides intuition into the prediction-relevant information in an image . For example , by visualizing simplifications of mispredicted images , we can form hypotheses of why these images were mispredicted . 2 MEASURING AND REDUCING INSTANCE COMPLEXITY . How to define simplicity ? We use the fact that generative image models tend to assign lower encoding bit sizes to visually simpler inputs ( Kirichenko et al. , 2020 ; Schirrmeister et al. , 2020 ) . Concretely , the complexity of an image x can be quantified as the negative log probability mass given by a pretrained generative model with tractable likelihood , G , i.e . − log pG ( x ) . This log probabil- ity mass can be interpreted as the image encoding size in bits per dimension ( bpd ) via Shannon ’ s theorem ( Shannon , 1948 ) : bpd ( x ) = − log2 pG ( x ) /d where d is the dimension of the flattened x . The simplification loss for an input x , given a pre-trained generative model G , is as follows : Lsim ( x ) = − log pG ( x ) ( 1 ) Figure 2 visualizes images and their corresponding bits-per-dimension ( bpd ) values given by a Glow network ( Kingma & Dhariwal , 2018 ) trained on 80 Million Tiny Images ( Torralba et al. , 2008 ) ( see supp . sec . S1 for other models ) . This is the generative network used across all our experiments . A visual inspection of Figure 2 suggests that lower bpd corresponds with simpler inputs , as also noted in prior work ( Serrà et al. , 2020 ) . The goal of our approach , SimpleBits , is to minimize bpd of input images while preserving task-relevant information . We now explore how SimpleBits affects network behavior in a variety of scenarios . In each scenario , we explain the method to optimize Lsim and the retention of task-relevant information , the experimental setup , and results . All code is at https : //tinyurl.com/simple-bits . 3 PER-INSTANCE SIMPLIFICATION DURING TRAINING . When plugged into the training of a classifier f , SimpleBits simplifies each image such that f can still learn the original classification task from the simplified batches . We apply backpropagation through training steps : given a batch of input images Xorig , before updating the classifier f , an image-toimage simplifier network generates a corresponding batch of images Xsim such that : ( a ) images in Xsim have low bpd as measured per Lsim in Equation ( 1 ) , and ( b ) training on the simplified images leads to a reduction of the classification loss on the original images . We optimize ( b ) by unrolling one training step of the classifier . So for a single batch Xorig , y , we first compute an updated classifier f ′ as follows : Xsim = simplifier ( Xorig ) ( 2 ) f ′ = train step ( f , l ( f ( Xsim ) , y ) ) ( 3 ) where simplifier is an image-to-image network , l is the classification loss function , i.e. , the crossentropy loss , and f ′ is the classifier f after one optimization step using the classification loss l ( f ( Xsim ) , y ) on the simplified data . To train the simplifier network , we optimize for both ( a ) ( Equation ( 1 ) ) and for the classification loss on the batch of original images with the updated classifier after the unrolled training step l ( f ′ ( Xorig ) , y ) using backpropagation through training ( Maclaurin et al. , 2015 ; Finn et al. , 2017 ) . Note that it would not be enough to instead optimize performance on the simplified images as the simplifier could then to collapse all simplified images of one class to one representative example . Adding the classification losses on the simplified data both before and after the unrolled training step ensures the training is not influenced by predictions on the simplified data that are very different from the classification target and we found that to improve training stability , leading to : Lcls = l ( f ′ ( Xorig ) , y ) + l ( f ( Xsim ) , y ) + l ( f ′ ( Xsim ) , y ) ( 4 ) Further details to stabilize the training are described in S4 . The total loss for the simplifier is L = λsim · Lsim + Lcls , ( 5 ) where λsim is a hyperparameter to control the trade-off between simplification and task completion . In subsequent experiments , we vary λsim to flexibly control the extent of simplification . Implementation Our training architecture is a normalizer-free classifier architecture to avoid interpretation difficulties that may arise from normalization layers , such as image values being downscaled by the simplifier and then renormalized again . We use Wide Residual Networks and adapt them according to the steps outlined in ( Brock et al. , 2021 ) ; additional details are included in the Supp . Section S2 . The normalizer-free architecture reaches 94.0 % on CIFAR10 in our experiments , however we opt for a smaller variant for faster experiments that reaches 91.2 % , see Supp . Section S2 . For the simplifier network , we use a UNet architecture ( Ronneberger et al. , 2015 ) that we modify to be residual , more details see Supp . Section S3 . For both simplifier and classifier networks , we use AdamW ( Loshchilov & Hutter , 2019 ) with lr = 5 · 10−4 and weight decay = 10−5 . 3.1 SimpleBits REMOVES INJECTED DISTRACTORS We first evaluate whether our per-instance simplification during training successfully removes superfluous information for tasks with injected distractors . To that end , we construct datasets to contain both useful ( ground truth ) and redundant ( distractor ) information for task learning . We create four composite datasets derived from three conventional datasets : MNIST ( LeCun & Cortes , 2010 ) , FashionMNIST ( Xiao et al. , 2017 ) and CIFAR10 ( Krizhevsky , 2009 ) . Their construction , samples and results are described below , and shown in Figure 3 . Side-by-Side MNISTS constructs each image by concatenating , left and right , one sample from Fashion-MNIST and another from MNIST . Each sample is rescaled to 16x32 , so the concatenated image size remains 32x32 ; the order of concatenation is random . The ground truth target is MNIST labels , and therefore FashionMNIST acts like a distractor as it is irrelevant for the classification task . As seen in Figure 3 , the simplifier effectively removes the clothes side of the image . MNIST with Uniform Noise adds uniform noise to the MNIST digits , preserving the MNIST digit as the classification target . Hence the noise is the distractor and is expected to be removed . And indeed the noise is no longer visible in the simplified outputs shown in Figure 3 . Interpolated MNIST and CIFAR10 is constructed by interpolating between MNIST and CIFAR10 images . MNIST digits are the classification target . The expectation is that the simplified images should no longer contain any of the CIFAR10 image information . The result shows that most of the CIFAR10 background is removed , leaving only slight traces of colors . CIFAR10 with Stripes overlays either horizontal or vertical stripes onto CIFAR10 images , with the binary classification label 0 for horizontal and 1 for vertical stripes . With this dataset we observe the most drastic and effective information removal , where only the tip of vertical strips is retained , which by itself is sufficient to solve this binary classification task .

Summary: The paper proposes to minimize the image per-instance complexity to interpret the essential ingredients for neural network learning. The proposed method, SimpleBits, works in three different settings _i)_ Simplification during training _ii)_ Simplification after training _iii)_ Simplification with dataset condensation.   In _i)_, SimpleBits is able to differentiate task-relevant information on simulated composite datasets. In _ii)_, SimpleBits is able to explain the possible reason for misclassification. In _iii)_, SimpleBits reduces the image complexity in dataset condensation even further.  The major contributions can be summarized as follows.  - Simplifying images to interpret the neural models is not new, while this paper performs the novel simplification on image complexity. - The paper proposes a way to simplify image complexity while retaining classifier accuracy. - The paper shows that SimpleBits helps interpret the model learning in three different cases.