Year: 2021

Abstract:
How do neural network classifiers behave if you make their inputs simpler? In this work, we propose SimpleBits, a method to synthesize simplified inputs by reducing information content, and measure the effect of such simplification on learning. To measure simplicity, we use the finding that the encoding bit size given by a pretrained generative image model correlates well with the visual complexity of the image. Hence, we minimize the encoding bit size to simplify inputs and investigate the effect of such simplification on neural network behavior in several scenarios: conventional training, dataset condensation and post-hoc explanations. In all settings, we optimize for simplified inputs that still contain the information to learn the classification task. We show that SimpleBits successfully removes superfluous information for tasks with injected distractors and investigate the tradeoff between input simplicity and task performance on real-world datasets. For dataset condensation, we find that inputs can be simplified with only minimal accuracy degradation. Finally, applied post-hoc to normally trained classifiers, SimpleBits provides intuition into reasons for misclassifications.

heading: 1 INTRODUCTION
text: A better understanding of the information deep networks learn can lead to new scientific discoveries (Raghu & Schmidt, 2020), inform our understanding of differences between human and model behaviors (Makino et al., 2020) and can serve as powerful auditing tools (Geirhos et al., 2020).
Removing information from the input manually is one way to understand what information content is relevant for learning. For example, researchers have occluded parts of the input or removed specific frequency ranges from the input to see which input regions and frequency ranges are relevant for the network’s prediction (Zintgraf et al., 2017; Makino et al., 2020; Banerjee et al., 2021). These ablation techniques use simple heuristics such as removing at random (Hooker et al., 2019) or exploit domain knowledge about interpretable aspects of the input (input regions, frequency range) to create simpler versions of the input and analyze the network’s prediction on these simpler inputs.
What if, instead of using heuristics, one would learn how to simplify inputs that contain predictionrelevant information? This way, one could synthesize simpler inputs and gain intuition into model behavior without relying on domain knowledge about what information may be relevant for the network. For this, one needs to define the precise meaning of “simplify an input”, including useful metrics for the simplification of the inputs and for the retention of task-relevant information.
In this work, we propose SimpleBits – an information-reduction method that learns to synthesize simplified inputs which contain less information but still remain informative for the task. To measure simplicity, we use a finding initially reported as a problem for density-based anomaly detection – generative image models tend to assign higher probability densities and hence lower bits to visually simpler inputs (Kirichenko et al., 2020; Schirrmeister et al., 2020). Here, we use this to our advantage and minimize the encoding bit size given by a generative network trained on a general image distribution to simplify inputs. At the same time, we optimize that the simplified inputs still contain the task-relevant information.
We then investigate what information is retained in the simplified inputs and how the simplification affects network behavior in a variety of settings. We apply SimpleBits both in a per-instance setting, where each image is processed to be a simplified version of itself, and the size of training set remains
unchanged, as well as in a condensation setting, where the dataset is compressed to only a few samples per class, with the condensed samples simplified at the same time. Applied during training, SimpleBits can be used to investigate the trade-off between the information content and the task performance. After training, SimpleBits can act as an analysis tool to understand what information a trained model uses for its decision making. Figure 1 summarizes tasks covered in this paper.
Our evaluation provides the following insights in the investigated scenarios:
1. Per-instance simplification during training. SimpleBits successfully removes superfluous information for tasks with injected distractors. On natural image datasets, SimpleBits highlights plausible task-relevant information (shape, color, texture). Increasing simplification leads to accuracy decreases and we report the trade-off between simplifying inputs and task level performance for different datasets.
2. Dataset simplification with condensation. We evaluate SimpleBits applied to a condensation setting that processes the training data into a much smaller set of synthetic images. SimpleBits simplifies these images to drastically reduce the encoding size without substantial task performance decrease. On a chest radiograph dataset (Johnson et al., 2019a;b), SimpleBits can uncover known radiologic features for pleural effusion and gender.
3. Post-training simplification. For a trained model, SimpleBits provides intuition into the prediction-relevant information in an image. For example, by visualizing simplifications of mispredicted images, we can form hypotheses of why these images were mispredicted.

heading: 2 MEASURING AND REDUCING INSTANCE COMPLEXITY
text: How to define simplicity? We use the fact that generative image models tend to assign lower encoding bit sizes to visually simpler inputs (Kirichenko et al., 2020; Schirrmeister et al., 2020). Concretely, the complexity of an image x can be quantified as the negative log probability mass given by a pretrained generative model with tractable likelihood, G, i.e. − log pG(x). This log probabil-
ity mass can be interpreted as the image encoding size in bits per dimension (bpd) via Shannon’s theorem (Shannon, 1948): bpd(x) = − log2 pG(x)/d where d is the dimension of the flattened x. The simplification loss for an input x, given a pre-trained generative model G, is as follows:
Lsim (x) = − log pG(x) (1)
Figure 2 visualizes images and their corresponding bits-per-dimension (bpd) values given by a Glow network (Kingma & Dhariwal, 2018) trained on 80 Million Tiny Images (Torralba et al., 2008) (see supp. sec. S1 for other models). This is the generative network used across all our experiments. A visual inspection of Figure 2 suggests that lower bpd corresponds with simpler inputs, as also noted in prior work (Serrà et al., 2020). The goal of our approach, SimpleBits, is to minimize bpd of input images while preserving task-relevant information.
We now explore how SimpleBits affects network behavior in a variety of scenarios. In each scenario, we explain the method to optimize Lsim and the retention of task-relevant information, the experimental setup, and results. All code is at https://tinyurl.com/simple-bits.

heading: 3 PER-INSTANCE SIMPLIFICATION DURING TRAINING
text: When plugged into the training of a classifier f , SimpleBits simplifies each image such that f can still learn the original classification task from the simplified batches. We apply backpropagation through training steps: given a batch of input images Xorig, before updating the classifier f , an image-toimage simplifier network generates a corresponding batch of images Xsim such that: (a) images in Xsim have low bpd as measured per Lsim in Equation (1), and (b) training on the simplified images leads to a reduction of the classification loss on the original images.
We optimize (b) by unrolling one training step of the classifier. So for a single batch Xorig,y, we first compute an updated classifier f ′ as follows:
Xsim = simplifier(Xorig) (2) f ′ = train step(f, l(f(Xsim),y)) (3)
where simplifier is an image-to-image network, l is the classification loss function, i.e., the crossentropy loss, and f ′ is the classifier f after one optimization step using the classification loss l(f(Xsim),y) on the simplified data.
To train the simplifier network, we optimize for both (a) (Equation (1)) and for the classification loss on the batch of original images with the updated classifier after the unrolled training step l(f ′(Xorig),y) using backpropagation through training (Maclaurin et al., 2015; Finn et al., 2017). Note that it would not be enough to instead optimize performance on the simplified images as the simplifier could then to collapse all simplified images of one class to one representative example.
Adding the classification losses on the simplified data both before and after the unrolled training step ensures the training is not influenced by predictions on the simplified data that are very different from the classification target and we found that to improve training stability, leading to:
Lcls = l(f ′(Xorig),y) + l(f(Xsim),y) + l(f ′(Xsim),y) (4)
Further details to stabilize the training are described in S4. The total loss for the simplifier is
L = λsim · Lsim + Lcls, (5) where λsim is a hyperparameter to control the trade-off between simplification and task completion. In subsequent experiments, we vary λsim to flexibly control the extent of simplification.
Implementation Our training architecture is a normalizer-free classifier architecture to avoid interpretation difficulties that may arise from normalization layers, such as image values being downscaled by the simplifier and then renormalized again. We use Wide Residual Networks and adapt them according to the steps outlined in (Brock et al., 2021); additional details are included in the Supp. Section S2. The normalizer-free architecture reaches 94.0% on CIFAR10 in our experiments, however we opt for a smaller variant for faster experiments that reaches 91.2%, see Supp. Section S2. For the simplifier network, we use a UNet architecture (Ronneberger et al., 2015) that we modify to be residual, more details see Supp. Section S3. For both simplifier and classifier networks, we use AdamW (Loshchilov & Hutter, 2019) with lr = 5 · 10−4 and weight decay = 10−5.
3.1 SimpleBits REMOVES INJECTED DISTRACTORS
We first evaluate whether our per-instance simplification during training successfully removes superfluous information for tasks with injected distractors. To that end, we construct datasets to contain both useful (ground truth) and redundant (distractor) information for task learning. We create four composite datasets derived from three conventional datasets: MNIST (LeCun & Cortes, 2010), FashionMNIST (Xiao et al., 2017) and CIFAR10 (Krizhevsky, 2009). Their construction, samples and results are described below, and shown in Figure 3.
Side-by-Side MNISTS constructs each image by concatenating, left and right, one sample from Fashion-MNIST and another from MNIST. Each sample is rescaled to 16x32, so the concatenated image size remains 32x32; the order of concatenation is random. The ground truth target is MNIST labels, and therefore FashionMNIST acts like a distractor as it is irrelevant for the classification task. As seen in Figure 3, the simplifier effectively removes the clothes side of the image.
MNIST with Uniform Noise adds uniform noise to the MNIST digits, preserving the MNIST digit as the classification target. Hence the noise is the distractor and is expected to be removed. And indeed the noise is no longer visible in the simplified outputs shown in Figure 3.
Interpolated MNIST and CIFAR10 is constructed by interpolating between MNIST and CIFAR10 images. MNIST digits are the classification target. The expectation is that the simplified images should no longer contain any of the CIFAR10 image information. The result shows that most of the CIFAR10 background is removed, leaving only slight traces of colors.
CIFAR10 with Stripes overlays either horizontal or vertical stripes onto CIFAR10 images, with the binary classification label 0 for horizontal and 1 for vertical stripes. With this dataset we observe the most drastic and effective information removal, where only the tip of vertical strips is retained, which by itself is sufficient to solve this binary classification task.

heading: 3.2 TRADE-OFF CURVES ON CONVENTIONAL DATASETS
text: Sec 3.1 evaluates the ability of SimpleBits to remove redundant information where ground truth is known. Now, we evaluate the trade-off between task performance and input simplification on real-world datasets. We perform instance-level simplification on MNIST, Fashion-MNIST, SVHN (Netzer et al., 2011) and CIFAR10, comparing results with varying weight λsim for the simplification loss, including λsim = 0, so without using any simplification loss. At an extreme, when we have simplified to the black image seen in Figure 2, we expect to see significant erosion of performance. Understanding the trade-offs inbetween the original image and this extreme can help understand how sensitive the training is to the removal of more complex features.
In Figure 5, there is a pronounced decay in performance at high levels of simplify (where final bpd is less than 2). We also observe this decay curve to be more pronounced for relatively more complex datasets such as CIFAR10, suggesting the classifiers need to be trained with more complex features present for highest task performance. Baselines for our simplifier can be found in S8.
The visible retention of task-relevant information in some images together with the decreased accuracies suggests another factor: The simplified inputs may lack noise-features that the classifier has to learn to be invariant to. This is analogous to how removing data augmentation can decrease accuracies even though the augmentations themselves do not contain discriminative information. This view implies SimpleBits can still be used to visualize task-relevant information and at the same motivates the adaptation of SimpleBits to analyze regularly trained classifiers after training in Section 5.

heading: 4 DATASET SIMPLIFICATION WITH CONDENSATION
text: Now we investigate how SimpleBits affects training on a small synthetic condensed dataset. Multiple methods have been proposed to achieve dataset condensation (Zhao et al., 2021; Zhao & Bilen, 2021; Wang et al., 2018; Maclaurin et al., 2015), via backpropagation through training (Wang et al., 2018; Maclaurin et al., 2015), gradient matching (Zhao & Bilen, 2021), or kernel based meta-learning (Nguyen et al., 2021). Due to its small size, one can visualize the full condensed dataset to understand what information is preserved for learning. Our aim here is to combine SimpleBits with dataset condensation to see if we could obtain a both smaller and simpler training dataset than the original.
In this setting, we jointly condense our training dataset to a smaller number of synthetic training inputs and simplify the synthetic inputs according to our simplification loss (Equation (1)). Concretely, we add the simplification loss Lsim to the gradient matching loss proposed by Zhao & Bilen (2021). The gradient matching loss computes the layerwise cosine distance between the gradient of the classification loss wrt. to the classifier parameters θ produced by a batch of original images Xorig and a batch of synthetic images Xsyn:
Lmatch(Xorig,Xsyn) = D(∇θl(f(Xorig),y),∇θl(f(Xsyn),y)). (6)
where D is the layerwise cosine distance. The matching loss is computed separately per class.
Overall, with our simplification loss, we get:
Lsyn(Xorig,Xsyn) = Lmatch(Xorig,Xsyn) + ∑
xsyn∈Xsyn
− log pG(xsyn) (7)
We perform dataset condensation on MNIST, Fashion-MNIST, SVHN and CIFAR10 with varying λsim for the simplification loss. We also apply dataset condensation to the chest radiograph dataset MIMIC-CXR-JPG (Johnson et al., 2019a;b) for predicting pleural effusion and gender. We use the networks from (Zhao et al., 2021), but use Adam (Kingma & Ba, 2015) for optimization.
4.1 SimpleBits RETAINS CONDENSATION PERFORMANCE WHILE GREATLY SIMPLIFYING DATA
In Figure 6, we examine the accuracy for each condensed-and-simplified dataset. We observe that for the natural image datasets, accuracies are mostly retained when decreasing the number of bits per image. Note that the setting with highest bpd is a reimplementation of Zhao et al. (2021) and therefore a baseline without simplification loss. We visualize examples in Figure 6 and observe that the jointly condensed and simplified images look visually smoother, indicating that higher frequency patterns visible in the original images are not needed to reach the same accuracy. These visualizations are also noticeably more smooth than the results for per-instance simplification Figure 3, which suggests that data condensation may already favor features that are less complex.
Evaluation of a medical chest radiograph dataset We also evaluate jointly condensing and simplifying for a dataset of chest radiograph images (Johnson et al., 2019a;b). This dataset has known radiologic features for the presence of pleural effusion (Jany & Welte, 2019; Raasch et al., 1982) and difference in gender (Bellemare et al., 2003). In Figure 7, we visualize both the condensed (top row) and the jointly condensed and simplified dataset (bottom row). The overlayed shows that a visible difference between presence of feature. For pleural effusion, a larger white region on the bottom of the lung occurs in the simplified pathological image, while for gender, lungs appear slightly smaller for the simplified female image.

heading: 5 POST-TRAINING SIMPLIFICATION
text: Per-instance SimpleBits reduces accuracies when applied during training, but can it be used to interpret trained classifiers with high accuracies after training? A trained classifier’s prediction may be influenced by a lot of information it has seen during training. Here, we use SimpleBits to visualize some of the information that would help the classifier remember what it has learned for that specific prediction.
Algorithm 1 Simplification loss function after training 1: given generative network G, input x, simplified input xsim, classifier f , parameter scaling factors s < 1 2: fscd ← ScaleParameters(f, s) . Scale parameters of classifier down by s to simulate ”forgetting” 3: h = f(x),hsim = f(xsim) . Predict original and simplified with unscaled classifier 4: hscd = fscd(x),hsim,scd = fscd(xsim) . Predict original and simplified with scaled classifier 5: Lgrad = D ( ∇sDKL(h‖hscd),∇sDKL(h‖hsim,scd) ) . Compute distance of gradients on scaling factors
6: Lpred = DKL(h‖hsim) +DKL(hscd‖hsim,scd) . Compute prediction differences 7: Lsim = − log pG(xsim)) . Compute needed bits for simplified input 8: return Lgrad + Lpred + λsim · Lsim
For synthesizing the prediction-relevant information, we simulate that the classifier forgets knowledge and then synthesize a simplified input that allows the classifier to relearn the relevant knowledge for the prediction of the original input. To simulate forgetting, we scale down all parameters of the trained classifier f by multiplying them with a gating value φscaled,i = φi · s, s < 1. This scaling removes information from the model by bringing the learned parameter values closer to zero, is identical to weight decay and also makes the network simpler in terms of model encoding size (Hinton & van Camp, 1993). To synthesize a simplified input that helps learning to restore the parameter values that are important for a specific input, we compare gradients between the original and simplified input.
Given an input, we can compute the gradients of the KL-divergence between the rescaled network’s prediction fscd(x) and the original network’s prediction f(x). At each iteration, the layerwise cosine distance between these two sets of gradients (one for original datapoint and the other for the simplified) is the basis of the loss used in SimpleBits. We compute this distance only considering the gradients that are negative for the original input. Combined with the simplification loss Eq. 1, this amounts to asking what input information is needed to recover parts of the original network to restore the original prediction? To ensure that the network is trained towards minimizing the same prediction difference on the simplified and original data, we also add a prediction difference loss Lpred. Further details about the implementation are included in Alg. 1 and Section S5.
In Figure 8, we visualize both the misclassified images according to the original network f and produce the corresponding simplified versions. We observe that simplified images may provide some intuition into the reason for misclassification, highlighting a variety of different features for different images. We imagine a possible practitioner workflow, where the practitioner derives a set of possible hypotheses for the misclassification from SimpleBits and tests them on the real data. We show further post-hoc simplified examples in supplementary Section S13.

heading: 6 RELATED WORK
text: A different approach to reduce input bits while retaining classification performance is to train a compressor that only keeps information that is invariant to predefined label-preserving augmentations. Dubois et al. (2021) implement this elegant approach in two ways. In their first variant, by training a VAE to reconstruct an unaugmented input from augmented (e.g. rotated, translated, sheared) versions. In their second variant, building on the CLIP (Radford et al., 2021) model, they take image-text pairs and view all images with the same text as augmented versions of each other. This allows to use compressed CLIP encodings for classification and achieves up to 1000x compression on Imagenet without decreasing classification accuracy. Their approach focuses on achieving maximum compression while our approach is focused on interpretability. Their approach requires access to predefined label-preserving augmentations and has reduced classification performance in input space compared to latent/encoding space.
Our approach simplifying individual training images builds on Raghu et al. (2021), where they learn to inject information into the classifier training. Per-instance simplification during training can be seen as a instance of their framework combined with the idea of input simplification. In difference to their methods, SimpleBits explicitly aims for interpretability through input simplification.
Other interpretability approaches that synthesize inputs include generating counterfactual inputs (Hvilshøj et al., 2021; Dombrowski et al., 2021; Goyal et al., 2019) or inputs with exaggerated features (Singla et al., 2020). SimpleBits differs in explicitly optimizing the inputs to be simpler.
Generative models have often been used in various ways for interpretability such as generating realistic-looking inputs (Montavon et al., 2018) and by directly training generative classifiers (Hvilshøj et al., 2021; Dombrowski et al., 2021), but we are not aware of any work except (Dubois et al., 2021) (discussed above) to explicitly generate simpler inputs.

heading: 7 CONCLUSION
text: We propose SimpleBits, an information-based method to synthesize simplified inputs. Crucially, SimpleBits does not require any domain-specific knowledge to constrain which input components should be removed; instead SimpleBits itself learns to remove the components of inputs which are least relevant for a given task.
As an interpretability tool, we show that SimpleBits is able to remove injected distractors, suggest plausible reasons for misclassification, and recover known radiologic features from condensed datasets. When combined with data condensation, SimpleBits retains accuracy while greatly reducing the complexity of condensed images.
Our simplification approach sheds light on the information required for a deep network classifier to learn its task. We find that the tradeoff between task performance and input simplification varies by dataset and setting - it is more pronounced for more complex datasets.
REPRODUCIBILITY STATEMENT
We provide the following information to ensure reproducibility. Main concepts, algorithms and basic architectures are described in sections 3, 5 and 4. Further network architecture details are in supp. sections S2 and S3, further optimization details in supp. sections S4 and S5. Finally, the code is available under https://tinyurl.com/simple-bits.

heading: S1 BPDS OF OTHER GENERATIVE MODELS
text: Figure S1 shows that the bits per dimension produced by other generative models than Glow also correlate well with visual complexity, validating our measure. This is consistent with prior work that found bpds of generative models trained on natural image datasets are strongly influenced by general natural image characteristics independent of any specific dataset (Kirichenko et al., 2020; Schirrmeister et al., 2020; Havtorn et al., 2021).

heading: S2 CLASSIFIER NETWORK DETAILS
text: Our classification network built on the Wide ResNet architecture (Zagoruyko & Komodakis, 2016). We used a version with relatively few parameters with depth = 16 and widen factor = 2 to allow for fast iteration on experimentation. We used ELU instead of ReLU nonlinearities.
Additionally, we removed batch normalization to avoid interference of normalization layers with the simplification process. We followed the method from Brock et al. (2021) to create a normalizer-free Wide ResNet. We reparameterize the convolutional layers using Scaled Weight Standardization:
Ŵij = Wij − µi√
Nσi , (S1) where µi = (1/N) ∑ jWij , σ 2 i = (1/N) ∑ j(Wij − µi)2, and N denotes the fan-in. Further as in Brock et al. (2021), ”activation functions are also scaled by a non-linearity specific scalar gain γ, which ensures that the combination of the γ-scaled activation function and a Scaled Weight Standardized layer is variance preserving.” Finally, the output of the residual branch is downscaled by 0.2, so the function to compute the output becomes hi+1 = hi + 0.2 · fi(hi), where hi denotes the inputs to the ith residual block, and fi denotes the function computed by the ith residual branch. Unlike Brock et al. (2021), we did not multiply scalars βi with the input of the residual branch or learned zero-initialized scalars to multiply with the output of the residual branch, as we did not find these two parts helpful in our setting. We also did not attempt to use Stochastic Depth (Huang et al., 2016), which may further improve upon the accuracies reported here. Due to our small batch sizes (32), we also did not use adaptive gradient clipping.

heading: S3 SIMPLIFIER NETWORK DETAILS
text: For the simplifier, we adapted a publicly available implementation of UNet 1. We used num down = 5 downsampling steps, ELU nonlinearities ngf = 64 filters in the last conv layer and a simple affine transformation layer instead of a normalization layer. Furthermore, we made the simplifier residual and ensured the output is within [0, 1] by adding the output of the UNet to the inverse-sigmoid-transformed input and then reapplying the sigmoid function.

heading: S4 OPTIMIZATION DETAILS PER-INSTANCE SIMPLIFICATION DURING TRAINING
text: First, we note that the single train step helps ensure a correspondence between simplified and original images, and is a technique others have used in meta-learning settings (Pham et al., 2021).
For stabilizing the optimization of the per-instance simplification during training, we found two further steps helpful. First, we modify:
Lcls = l(f ′(Xorig),y) + l(f(Xsim),y) + l(f ′(Xsim),y) (S2) to
Lcls = 10 · l(f ′(Xorig),y) + l(f(Xsim),y) + l(f ′(Xsim),y) (S3) as (a) the gradient magnitudes are much smaller from the losses after unrolling and (b) we want to prioritize the classification loss on the original data. Additionally, we dynamically turn off Lsim during training, for any example x where l(f ′(xorig),y) > 0.1, which we also found to further stabilize training.

heading: S5 OPTIMIZATION DETAILS PER-INSTANCE SIMPLIFICATION AFTER TRAINING
text: We found it beneficial to optimize the simplified inputs in the latent space of the pre-trained Glow network and to apply our loss functions to all interpolated inputs on the path between original and simple input in latent space instead of only to the simplified input itself. The points on the path include information from the original input and prevent that the optimization is unable to recover some relevant information from the original input. Additionally, we also ensure that the predictions are the same for the original and interpolated inputs for both the original and the scaled model. The complete loss function can be found in Alg. 2, during training we called it with scaling factors sampled from s U(0.8, 0.95) as these values mostly led to similar but more uniformly distributed predictions than the unperturbed network.
1https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix
Algorithm 2 Simplification loss function after training full algorithm 1: given generative invertible networkG, input x and its latent code z (fromG), simplified input’s latent code
zsim, classification network f , parameter scaling factors s < 1 2: fscaled ← ScaleParameters(f, s) . Scale parameters of classifier down by s to simulate ”forgetting” 3: α ∼ U(0, 1) . Sample interpolation factor uniformly between 0 and 1. 4: zmixed = α · zsim + (1− α) · z . Interpolate simple and original input in latent space 5: xmix = invert(G,zmix) . Invert zmix using invertible network 6: h = f(x),hmix = f(xmix) . Predict original and mixed with classifier 7: hscaled = fscaled(x),hscaled,mix = fscaled(xmix) . Predict original and mixed with scaled classifier 8: Lgrad = d ( ∇sDKL(h‖hscaled),∇sDKL(h‖hmix,scaled) ) . Compute distance between gradients on
scaling factors. 9: Lpred = DKL(h‖hmix) +DKL(hscaled‖hmix,scaled) . Compute prediction differences 10: xsim = invert(G,zsim) . Invert zsim using invertible network 11: Lsimplification = − log pG(xsim)) . Compute needed bits for simplified input 12: return Lgrad + Lpred + λsim · Lsimplification
4.04.55.05.56.0 PNG-compressed File Size in BPDs
70 80 90
Te st
A cc
ur ac
y
End of Joint Training Retraining Orig CIFAR10
Figure S2: Tradeoff between PNG storage space and accuracies. Note that the PNG bpd file sizes do not show the maximally possible savings, these can be seen from the bpd values in the main manuscript.

heading: S6 PNG-COMPRESSED FILE SIZES OF SIMPLIFIED IMAGES
text: Figure S2 shows the tradeoff between PNG storage space of the simplified images and the accuracies achieved when retraining.

heading: S7 LEARNING CURVES FOR RETRAINING
text: Figure S3 shows learning curves during retraining on the simplified images on CIFAR10. There are no noticeable differences in training speed for more or less simplified images.
0 20 40 60 80 100 Training Epoch
25
50
75
Te st
A cc
ur ac
y [% ] Retraining Learning Curves CIFAR10 3.57 BPD 3.56 BPD 3.52 BPD 2.95 BPD 2.81 BPD 2.66 BPD 2.50 BPD 2.26 BPD 1.86 BPD 1.76 BPD
Figure S3: Learning curves for retraining on simplified images on CIFAR10.
123 Bits per dimension
25
50 75 Ac cu ra cy [%
] CIFAR10
123 Bits per dimension
50
75
SVHN
12 Bits per dimension
50
75
FASHIONMNIST
0.51.01.5 Bits per dimension
50
100 MNIST
Per-Instance Simplification During Training, Retraining Results
SimpleBits Minimize MSE and BPD Gaussian Blurring JPEG Compress
Figure S4: Comparison between SimpleBits and two simpler baselines: In the first one, the simplifier network is trained to simultaneously reduce bpd of the simplified image and the mean squared error between the simplified and the original image. In the second one, gaussian blurring is applied to the input images, different runs vary in the standard deviation used to create the gaussian blurring kernel. In the third one, we use JPEG compression with varying quality levels. Tradeoff curves are worse for the baselines than for SimpleBits .

heading: S8 SIMPLIFIER BASELINES
text: We implemented three simpler baselines to check whether the losses used in SimpleBits during training help retain task-relevant information. In the first baseline, we train the simplifier to simultaneously reduce bpd of the simplified image and the mean squared error between the simplified and the original image. Afterwards we train the classifier on the simplified images and evaluate on the original images the same way as during retraining of SimpleBits. In the second baseline, we blur the original images with a gaussian kernel, which also reduces their bpd. We vary the sigma/standard deviation for the gaussian kernel to trade off smoothness and task-informativeness. In the third baseline, we use lossy JPEG compression with varying quality levels. As in SimpleBits and the other baselines, we estimate the bits per dimension of the lossy-JPEG-compressed images through our pretrained Glow network for a fair comparison. The gaussian blurring and JPEG compression each replace the simplifier, so these are fixed simplifier baselines without training a simplifier. While these three baselines also retain some task-relevant information allowing the classifier to retain above-chance accuracies (see Figure S4), the tradeoff between bpd and accuracy is worse than for SimpleBits. This shows the losses used in SimpleBits help retain more task-relevant information compared to these baselines.

heading: S9 MORE IMAGES SIMPLIFIED DURING TRAINING
text: We show a larger number of images that were simplified on CIFAR10 during training with the largest simplification loss weight λsim = 2.0 in Figures S5, S6 and S7.

heading: S10 POTENTIALLY SPURIOUS FEATURES UNCOVERED BY SIMPLEBITS
text: SimpleBits may have the potential to reveal spurious correlations present in the dataset. We show some simplified images that reveal potentially spurious features in Figure S8. These observations can be used as a starting point to further investigate whether these features also affect regularly trained classifiers.

heading: S11 MORE CONDENSED DATASETS
text: We also show some interesting condensed datasets that resulted when we varied the architecture (Figure S10) or the condensation loss (from gradient matching to either negative gradient product or single-training-step unrolling, Figure S11).
Figure S5: Uncurated set of simplified images with λsim = 2.0, 6 per class.

heading: S12 EVALUATION OF CONDENSED DATASETS FOR CONTINUAL LEARNING
text: We also evaluate the simplified condensed datasets in a continual learning setting, following (Zhao & Bilen, 2021). In this task-incremental continual learning setting, the model is trained on different classification datasets sequentially. When training on a new dataset, the model is additionally trained on the condensed versions of the previous datasets.
The continual learning experiment reproduces the setting from (Zhao & Bilen, 2021) to first train on SVHN, then on MNIST and finally on USPS (Hull, 1994), using the average accuracy across all three datasets of the classifier at the end of training as the final accuracy (see (Zhao & Bilen, 2021) for details).
We created a simpler and faster continual training pipeline that achieves comparable results to Zhao & Bilen (2021). First, we train 3 times for 50 epochs on SVHN, with a cosine annealing learning rate schedule (Loshchilov & Hutter, 2017) that is restarted at each time with lr = 0.1. Then for each MNIST and USPS, we train one cosine annealing cycle of 50 epochs for lr = 0.1.
We first verified that we can reproduce the prior continual learning results with our simpler training pipeline and find that our training pipeline indeed even slightly outperforms the reported final results (96.0% vs. 95.2% with, and 95.4% vs 93.0% without knowledge distillation) despite slightly inferior performance in the first training stage (before any continual learning, 93.6% vs 94.1%), see following subsection. When using different SVHN and MNIST condensed datasets, we find that we can retain the original continual learning accuracies even with condensed datasets with substantially less (∼9x less) bits per dimension (see Fig.S12).
Figure S6: Uncurated set of correctly predicted simplified images with λsim = 2.0, 6 per class.
S12.1 WITHOUT CONDENSED DATASET
Our training pipeline still exhibits forgetting when not using any condensed datasets of previously trained-on datasets. As Figure S13 shows, the accuracies are far lower than with just regular sequential training. We performed this ablation to ensure forgetting still occurs in our training pipeline.

heading: S13 MORE IMAGES SIMPLIFIED AFTER TRAINING
text: We show further examples of post-hoc-simplified images for misclassified original images in Figure S14. We also show the output of SimpleBits when applied to the control images of Figure 8 in Figure S15. We also show an uncurated set of incorrectly predicted and correctly predicted images in Figures S16 and S17.
Figure S7: Correctly predicted simplified images with strongest color with λsim = 2.0, 6 per class.
Red Brown texture
Blue color
Car
Predicted as
Dark bottom half, bright top half
Spurious Feature
Red texture
Plane
Correctly Predicted
Incorrectly Predicted
Horse
Ship
Figure S8: Selected simplified images that highlight potentially spurious features. Two leftmost images are correctly predicted, two rightmost images are incorrectly predicted.
Figure S9: Dataset condensation results with varying simplification loss weight. Top: Individual dots represent accuracies for setting with different simplification loss weights. Accuracies can be retained even with substantially reduced bits per dimension. For 1 image per class, arrows highlight the settings that are visualized below. Below: Condensed datasets with varying simplification loss weight. Per dataset, showing condensed datasets with high (top row) and low (bottom row) bits per dimension. Lower bits per dimension datasets are visually simpler and smoother while mostly retaining accuracies.
Instance norm ReLU
Instance norm ELU
No norm ReLU
No norm ELU
22.5% Acc. 4.35 BPD
22.7% Acc. 4.31 BPD
27.3% Acc. 4.35 BPD
28.7% Acc. 4.35 BPD
Figure S10: Dataset condensation on CIFAR10 with varying architecture.
Grad Distance, 1 Outer step
Grad Product
1-step Unrolling
Grad Product, ReLU, 4 Outer Steps
26.4% Acc. 5.24 BPD
26.9% Acc. 0.37 BPD
29.1% Acc. 6.47 BPD
28.7% Acc. 4.35 BPD
Grad Product, 4 Outer Steps
27.6% Acc. 1.6 BPD
Figure S11: Dataset condensation on CIFAR10 with varying condensation loss and varying outer loop steps, i.e. how many steps the classifier is trained at each training epoch (default 1 in the 1 image per class setting), after each step the condensation loss is again optimized.
SVHN SVHN+ MNIST
SVHN+ MNIST+
USPS Training Stage
92 93 94 95 96 Ac cu ra cy [% ]
(a ve
ra ge
a cr
os s d
at as
et s
se en
d ur
in g
tra in
in g) Continual Learning Results
ours w. KD SVHN bpd: 11.9, MNIST BPD: 4.3 ours w.o. KD ours w. KD SVHN bpd: 3.9, MNIST BPD: 2.5 ours w.o. KD ours w. KD SVHN bpd: 1.0, MNIST BPD: 0.8 ours w.o. KD Orig w. KD Orig w.o. KD
Figure S12: Continual Learning Results. Results for first training on SVHN, then MNIST and then USPS for condensed datasets with varying bits per dimension. Solid lines are with and dashed lines without knowledge distillation. Note that continual learning accuracies remain similar also for substantially reduced bits per dimension. Ablations show that accuracies degrade without any condensed dataset, see supplementary.
SVHN SVHN+MNIST SVHN+MNIST+USPS Training Stage
75
80
85
90
95
Ac cu
ra cy
[% ]
(a ve
ra ge
a cr
os s d
at as
et s
se en
d ur
in g
tra in
in g)
Continual Learning Results ours w. KD SVHN bpd: 11.9, MNIST BPD: 4.3 ours w.o. KD ours w. KD SVHN bpd: 3.9, MNIST BPD: 2.5 ours w.o. KD ours w. KD SVHN bpd: 1.0, MNIST BPD: 0.8 ours w.o. KD no condensed dataset Orig w. KD Orig w.o. KD
Figure S13: Continual Learning Results without Condensed Dataset (regular sequential training). Conventions as in Figure S12. Accuracies substantially worse without any condensed dataset.
Original Images
Predicted as Ship Plane
Simplified
Hypothesis for one cause of misclassification
Ground + plane looks like ship
Control Image
Predicted as Plane Ship
Frog
Higher Freq Texture
Bird
Deer
Color and antler -like shapes
Cat
Car
Car and wheelshape
Deer
Black dots
(true class) (true class) (true class) (true class) (true class)
Original Images
Predicted as Deer Frog
Simplified
Hypothesis for one cause of misclassification
Tree as antler
Control Image
Predicted as Horse Cat
Car
Brighter parts form car contours
Cat
Deer
Color, ear/tail antler-like
Dog
Dog
Black-white dog face
Bird
Color
(true class) (true class) (true class) (true class) (true class)
Figure S14: Further examples of post-hoc simplifications of originally misclassified images.
Black dots on ship body
Tree top resembles antler Black pads of paw, red color at bottom High frequency texture
Circle resembles shape of car
Ship
Control Image (predicted correctly)
Hypothesis for one cause of misclassification
SimpleBits output
Predicted as
Original Image
Green color
True class Horse Cat Bird Deer Cat
Plane Deer Dog Frog Car Frog
SimpleBits output on control
Figure S15: Post-hoc simplifications of control images.
Figure S16: Uncurated post-hoc simplifications of incorrectly predicted images.
Figure S17: Uncurated post-hoc simplifications of correctly predicted images.

Review summaries:
In this paper, the authors proposed SimpleBits, which is a method to create simplified input images that retain the most relevant parts for classification while removing irrelevant details. The core of SimpleBits is the simplification loss, which is the negative log likelihood of seeing an image, according to an already trained generative model (as well as a related concept known as bits per dimension, which is simply the simplification loss divided by the dimension of a flattened image). The authors showed empirically that more complex images generally have higher bits per dimension/simplification loss.  The authors applied SimpleBits in three settings: (1) during training, (2) after training, and (3) dataset condensation/summarization. In the first setting, the authors trained both an image classifier and a simplifier network in tandem -- in particular, the simplifier network is trained by minimizing both the classification loss and the simplification loss. In the second setting, the authors tried to create simplified versions of input images, especially input images that are wrongly classified, to hypothesize (posthoc) about how an already trained network made predictions. In the third setting, the authors combined simplification loss with the gradient matching loss (Zhao and Bilen, 2021) to create a smaller dataset with simplified images, and showed that simplification does not have significant impact on accuracy.

- Draft proposes a simplification method for visual recognition task where the information less relevant for the classification task is removed from the samples. In an iterative fashion the proposed approach (with no domain knowledge) synthesizes the simplified samples. Then the draft attempts to (i) measure the effect of the simplification via evaluating the models trained on the simplified samples, (ii) understand the models' behavior via posthoc interpretations, and (iii) simplify the synthetic data learned through dataset condensation.

The paper proposes to minimize the image per-instance complexity to interpret the essential ingredients for neural network learning. The proposed method, SimpleBits, works in three different settings _i)_ Simplification during training _ii)_ Simplification after training _iii)_ Simplification with dataset condensation.   In _i)_, SimpleBits is able to differentiate task-relevant information on simulated composite datasets. In _ii)_, SimpleBits is able to explain the possible reason for misclassification. In _iii)_, SimpleBits reduces the image complexity in dataset condensation even further.  The major contributions can be summarized as follows.  - Simplifying images to interpret the neural models is not new, while this paper performs the novel simplification on image complexity. - The paper proposes a way to simplify image complexity while retaining classifier accuracy. - The paper shows that SimpleBits helps interpret the model learning in three different cases.