W HEN LESS IS MORE : S IMPLIFYING INPUTS AIDS NEURAL NETWORK UNDERSTANDING
Anonymous authors
Paper under double-blind review
A BSTRACT
How do neural network classiﬁers behave if you make their inputs simpler?
In this work, we propose SimpleBits, a method to synthesize simpliﬁed inputs by reducing information content, and measure the effect of such simpliﬁcation on learning.
To measure simplicity, we use the ﬁnding that the encoding bit size given by a pretrained generative image model correlates well with the visual complexity of the image.
Hence, we minimize the encoding bit size to simplify inputs and investigate the effect of such simpliﬁcation on neural network behavior in several scenarios: conventional training, dataset condensation and post-hoc explanations.
In all settings, we optimize for simpliﬁed inputs that still contain the information to learn the classiﬁcation task.
We show that SimpleBits successfully removes superﬂuous information for tasks with injected distractors and investigate the tradeoff between input simplicity and task performance on real-world datasets.
For dataset condensation, we ﬁnd that inputs can be simpliﬁed with only minimal accuracy degradation.
Finally, applied post-hoc to normally trained classiﬁers, SimpleBits provides intuition into reasons for misclassiﬁcations.
I NTRODUCTION
A better understanding of the information deep networks learn can lead to new scientiﬁc discoveries (Raghu & Schmidt, 2020), inform our understanding of differences between human and model behaviors (Makino et al., 2020) and can serve as powerful auditing tools (Geirhos et al., 2020).
Removing information from the input manually is one way to understand what information content is relevant for learning.
For example, researchers have occluded parts of the input or removed speciﬁc frequency ranges from the input to see which input regions and frequency ranges are relevant for the network’s prediction (Zintgraf et al., 2017; Makino et al., 2020; Banerjee et al., 2021).
These ablation techniques use simple heuristics such as removing at random (Hooker et al., 2019) or exploit domain knowledge about interpretable aspects of the input (input regions, frequency range) to create simpler versions of the input and analyze the network’s prediction on these simpler inputs.
What if, instead of using heuristics, one would learn how to simplify inputs that contain predictionrelevant information?
This way, one could synthesize simpler inputs and gain intuition into model behavior without relying on domain knowledge about what information may be relevant for the network.
For this, one needs to deﬁne the precise meaning of “simplify an input”, including useful metrics for the simpliﬁcation of the inputs and for the retention of task-relevant information.
In this work, we propose SimpleBits – an information-reduction method that learns to synthesize simpliﬁed inputs which contain less information but still remain informative for the task.
To measure simplicity, we use a ﬁnding initially reported as a problem for density-based anomaly detection – generative image models tend to assign higher probability densities and hence lower bits to visually simpler inputs (Kirichenko et al., 2020; Schirrmeister et al., 2020).
Here, we use this to our advantage and minimize the encoding bit size given by a generative network trained on a general image distribution to simplify inputs.
At the same time, we optimize that the simpliﬁed inputs still contain the task-relevant information.
We then investigate what information is retained in the simpliﬁed inputs and how the simpliﬁcation affects network behavior in a variety of settings.
We apply SimpleBits both in a per-instance setting, where each image is processed to be a simpliﬁed version of itself, and the size of training set remains
[Figure]
Data Simplification
[Figure]
during Training
[Figure]
Training
[Figure]
Data Simplification with Condensation
[Figure]
unchanged, as well as in a condensation setting, where the dataset is compressed to only a few samples per class, with the condensed samples simpliﬁed at the same time.
Applied during training,
SimpleBits can be used to investigate the trade-off between the information content and the task performance.
After training, SimpleBits can act as an analysis tool to understand what information a trained model uses for its decision making.
Figure 1 summarizes tasks covered in this paper.
Our evaluation provides the following insights in the investigated scenarios:
Per-instance simpliﬁcation during training.
SimpleBits successfully removes superﬂuous information for tasks with injected distractors.
On natural image datasets, SimpleBits highlights plausible task-relevant information (shape, color, texture).
Increasing simpliﬁcation leads to accuracy decreases
and we report the trade-off between simplifying inputs and task level performance for different datasets.
Dataset simpliﬁcation with condensation.
We evaluate SimpleBits applied to a condensation setting that processes the training data into a much smaller set of synthetic images.
SimpleBits simpliﬁes these images to drastically reduce the encoding size without substantial task performance decrease.
On a chest radiograph dataset (Johnson et al., 2019a;b), SimpleBits can uncover known radiologic features for pleural effusion and gender.
Post-training simpliﬁcation.
For a trained model, SimpleBits provides intuition into the prediction-relevant information in an image.
For example, by visualizing simpliﬁcations of mispredicted images, we can form hypotheses of why these images were mispredicted.
M EASURING AND R EDUCING I NSTANCE C OMPLEXITY
How to deﬁne simplicity?
We use the fact that generative image models tend to assign lower encoding bit sizes to visually simpler inputs (Kirichenko et al., 2020; Schirrmeister et al., 2020).
Concretely, the complexity of an image x can be quantiﬁed as the negative log probability mass given by a pretrained generative model with tractable likelihood, G , i.e. − log p G ( x ) .
This log probabil-
[Figure]
ity mass can be interpreted as the image encoding size in bits per dimension (bpd) via Shannon’s theorem (Shannon, 1948): bpd ( x )
= − log 2 p G ( x ) /d where d is the dimension of the ﬂattened x .
The simpliﬁcation loss for an input x , given a pre-trained generative model G , is as follows:
[Equation]
Figure 2 visualizes images and their corresponding bits-per-dimension ( bpd ) values given by a Glow network (Kingma & Dhariwal, 2018) trained on 80 Million Tiny Images (Torralba et al., 2008)
(see supp.
sec. S1 for other models).
This is the generative network used across all our experiments.
A visual inspection of Figure 2 suggests that lower bpd corresponds with simpler inputs, as also noted in prior work (Serr`a et al., 2020).
The goal of our approach, SimpleBits , is to minimize bpd of input images while preserving task-relevant information.
We now explore how SimpleBits affects network behavior in a variety of scenarios.
In each scenario, we explain the method to optimize L sim and the retention of task-relevant information, the experimental setup, and results.
All code is at https://tinyurl.com/simple-bits .
P ER - INSTANCE SIMPLIFICATION DURING TRAINING
When plugged into the training of a classiﬁer f , SimpleBits simpliﬁes each image such that f can still learn the original classiﬁcation task from the simpliﬁed batches.
We apply backpropagation through training steps: given a batch of input images X orig , before updating the classiﬁer f , an image-toimage simpliﬁer network generates a corresponding batch of images
X sim such that: (a) images in
X sim have low bpd as measured per L sim in Equation (1), and (b) training on the simpliﬁed images leads to a reduction of the classiﬁcation loss on the original images.
We optimize (b) by unrolling one training step of the classiﬁer.
So for a single batch X orig , y , we ﬁrst compute an updated classiﬁer f (cid:48) as follows:
[Figure]
Results for Instance Simplification on CIFAR
[Equation]
where simpliﬁer is an image-to-image network, l is the classiﬁcation loss function, i.e., the crossentropy loss, and f (cid:48) is the classiﬁer f after one optimization step using the classiﬁcation loss l ( f ( X sim ) , y ) on the simpliﬁed data.
To train the simpliﬁer network, we optimize for both (a) (Equation (1)) and for the classiﬁcation loss on the batch of original images with the updated classiﬁer after the unrolled training step l ( f (cid:48)
( X orig ) , y ) using backpropagation through training (Maclaurin et al., 2015; Finn et al., 2017).
Note that it would not be enough to instead optimize performance on the simpliﬁed images as the simpliﬁer could then to collapse all simpliﬁed images of one class to one representative example.
Adding the classiﬁcation losses on the simpliﬁed data both before and after the unrolled training step ensures the training is not inﬂuenced by predictions on the simpliﬁed data that are very different from the classiﬁcation target and we found that to improve training stability, leading to:
[Equation]
Further details to stabilize the training are described in S4.
The total loss for the simpliﬁer is
[Equation]
where λ sim is a hyperparameter to control the trade-off between simpliﬁcation and task completion.
In subsequent experiments, we vary λ sim to ﬂexibly control the extent of simpliﬁcation.
Implementation Our training architecture is a normalizer-free classiﬁer architecture to avoid interpretation difﬁculties that may arise from normalization layers, such as image values being downscaled by the simpliﬁer and then renormalized again.
We use Wide Residual Networks and adapt them according to the steps outlined in (Brock et al., 2021); additional details are included in the
Supp.
Section S2.
The normalizer-free architecture reaches 94.0% on CIFAR10 in our experiments, however we opt for a smaller variant for faster experiments that reaches 91.2%, see Supp.
Section
S2.
For the simpliﬁer network, we use a UNet architecture (Ronneberger et al., 2015) that we modify to be residual, more details see Supp.
Section S3.
For both simpliﬁer and classiﬁer networks, we use AdamW (Loshchilov & Hutter, 2019) with lr = 5 · 10 − 4 and weight decay = 10 − 5 .
[Figure]
End of Joint Training
[Figure]
on Simplified
[Figure]
3. SimpleBits R EMOVES I NJECTED D ISTRACTORS
We ﬁrst evaluate whether our per-instance simpliﬁcation during training successfully removes superﬂuous information for tasks with injected distractors.
To that end, we construct datasets to contain both useful (ground truth) and redundant (distractor) information for task learning.
We create four composite datasets derived from three conventional datasets: MNIST (LeCun & Cortes, 2010),
FashionMNIST (Xiao et al., 2017) and CIFAR10 (Krizhevsky, 2009).
Their construction, samples and results are described below, and shown in Figure 3.
Side-by-Side MNISTS constructs each image by concatenating, left and right, one sample from
Fashion-MNIST and another from MNIST.
Each sample is rescaled to 16x32, so the concatenated image size remains 32x32; the order of concatenation is random.
The ground truth target is MNIST labels, and therefore FashionMNIST acts like a distractor as it is irrelevant for the classiﬁcation task.
As seen in Figure 3, the simpliﬁer effectively removes the clothes side of the image.
MNIST with Uniform Noise adds uniform noise to the MNIST digits, preserving the MNIST digit as the classiﬁcation target.
Hence the noise is the distractor and is expected to be removed.
And indeed the noise is no longer visible in the simpliﬁed outputs shown in Figure 3.
Interpolated MNIST and CIFAR10 is constructed by interpolating between MNIST and CIFAR images.
MNIST digits are the classiﬁcation target.
The expectation is that the simpliﬁed images should no longer contain any of the CIFAR10 image information.
The result shows that most of the
CIFAR10 background is removed, leaving only slight traces of colors.
CIFAR10 with Stripes overlays either horizontal or vertical stripes onto CIFAR10 images, with the binary classiﬁcation label 0 for horizontal and 1 for vertical stripes.
With this dataset we observe the most drastic and effective information removal, where only the tip of vertical strips is retained, which by itself is sufﬁcient to solve this binary classiﬁcation task.
3. T RADE - OFF C URVES ON C ONVENTIONAL D ATASETS
Sec 3.1 evaluates the ability of SimpleBits to remove redundant information where ground truth is known.
Now, we evaluate the trade-off between task performance and input simpliﬁcation on real-world datasets.
We perform instance-level simpliﬁcation on MNIST, Fashion-MNIST, SVHN (Netzer et al., 2011) and CIFAR10, comparing results with varying weight λ sim for the simpliﬁcation loss, including λ sim = 0 , so without using any simpliﬁcation loss.
At an extreme, when we have simpliﬁed to the black image seen in Figure 2, we expect to see signiﬁcant erosion of performance.
Understanding the trade-offs inbetween the original image and this extreme can help understand how sensitive the training is to the removal of more complex features.
In Figure 5, there is a pronounced decay in performance at high levels of simplify (where ﬁnal bpd is less than 2).
We also observe this decay curve to be more pronounced for relatively more complex datasets such as CIFAR10, suggesting the classiﬁers need to be trained with more complex features present for highest task performance.
The visible retention of task-relevant information in some images together with the decreased accuracies suggests another factor: The simpliﬁed inputs may lack noise-features that the classiﬁer has to learn to be invariant to.
This is analogous to how removing data augmentation can decrease accuracies even though the augmentations themselves do not contain discriminative information.
This view implies SimpleBits can still be used to visualize task-relevant information and at the same motivates the adaptation of SimpleBits to analyze regularly trained classiﬁers after training in Section 5.
D ATASET SIMPLIFICATION WITH C ONDENSATION
Now we investigate how SimpleBits affects training on a small synthetic condensed dataset.
Multiple methods have been proposed to achieve dataset condensation (Zhao et al., 2021; Zhao & Bilen, 2021;
Wang et al., 2018; Maclaurin et al., 2015), via backpropagation through training (Wang et al., 2018;
Maclaurin et al., 2015), gradient matching (Zhao & Bilen, 2021), or kernel based meta-learning (Nguyen et al., 2021).
Due to its small size, one can visualize the full condensed dataset to understand what information is preserved for learning.
Our aim here is to combine SimpleBits with dataset condensation to see if we could obtain a both smaller and simpler training dataset than the original.
In this setting, we jointly condense our training dataset to a smaller number of synthetic training inputs and simplify the synthetic inputs according to our simpliﬁcation loss (Equation (1)).
Concretely, we add the simpliﬁcation loss L sim to the gradient matching loss proposed by Zhao & Bilen (2021).
The gradient matching loss computes the layerwise cosine distance between the gradient of the classiﬁcation loss wrt.
to the classiﬁer parameters θ produced by a batch of original images
X orig and a batch of synthetic images X syn :
[Equation]
where D is the layerwise cosine distance.
The matching loss is computed separately per class.
Overall, with our simpliﬁcation loss, we get:
[Equation]
We perform dataset condensation on MNIST, Fashion-MNIST, SVHN and CIFAR10 with varying λ sim for the simpliﬁcation loss.
We also apply dataset condensation to the chest radiograph dataset
MIMIC-CXR-JPG (Johnson et al., 2019a;b) for predicting pleural effusion and gender.
We use the networks from (Zhao et al., 2021), but use Adam (Kingma & Ba, 2015) for optimization.
4. SimpleBits RETAINS CONDENSATION PERFORMANCE WHILE GREATLY SIMPLIFYING DATA
In Figure 6, we examine the accuracy for each condensed-and-simpliﬁed dataset.
We observe that for the natural image datasets, accuracies are mostly retained when decreasing the number of bits per image.
Note that the setting with highest bpd is a reimplementation of Zhao et al.
(2021) and therefore a baseline without simpliﬁcation loss.
We visualize examples in Figure 6 and observe that the jointly condensed and simpliﬁed images look visually smoother, indicating that higher frequency patterns visible in the original images are not needed to reach the same accuracy.
These visualizations are also noticeably more smooth than the results for per-instance simpliﬁcation Figure 3, which suggests that data condensation may already favor features that are less complex.
Evaluation of a medical chest radiograph dataset We also evaluate jointly condensing and simplifying for a dataset of chest radiograph images (Johnson et al., 2019a;b).
This dataset has known radiologic features for the presence of pleural effusion (Jany & Welte, 2019;
Raasch et al., 1982) and difference in gender (Bellemare et al., 2003).
In Figure 7, we visualize both the condensed (top row) and the jointly condensed and simpliﬁed dataset (bottom row) .
The overlayed shows that a visible difference between presence of feature.
For pleural effusion, a larger white region on the bottom of the lung occurs in the simpliﬁed pathological image, while for gender, lungs appear slightly smaller for the simpliﬁed female image.
[Figure]
CIFAR
[Figure]
P OST - TRAINING SIMPLIFICATION
Per-instance SimpleBits reduces accuracies when applied during training, but can it be used to interpret trained classiﬁers with high accuracies after training?
A trained classiﬁer’s prediction may be inﬂuenced by a lot of information it has seen during training.
Here, we use SimpleBits to visualize some of the information that would help the classiﬁer remember what it has learned for that speciﬁc prediction.
[Table]
For synthesizing the prediction-relevant information, we simulate that the classiﬁer forgets knowledge and then synthesize a simpliﬁed input that allows the classiﬁer to relearn the relevant knowledge for the prediction of the original input.
To simulate forgetting, we scale down all parameters of the trained classiﬁer f by multiplying them with a gating value φ scaled ,i = φ i · s, s < 1 .
This scaling removes information from the model by bringing the learned parameter values closer to zero, is identical to weight decay and also makes the network simpler in terms of model encoding size (Hinton & van Camp, 1993).
To synthesize a simpliﬁed input that helps learning to restore the parameter values that are important for a speciﬁc input, we compare gradients between the original and simpliﬁed input.
Given an input, we can compute the gradients of the KL-divergence between the rescaled network’s prediction f scd ( x ) and the original network’s prediction f ( x ) .
At each iteration, the layerwise cosine distance between these two sets of gradients (one for original datapoint and the other for the simpliﬁed) is the basis of the loss used in SimpleBits .
We compute this distance only considering the gradients that are negative for the original input.
Combined with the simpliﬁcation loss Eq.
1, this amounts to asking what input information is needed to recover parts of the original network to restore the original prediction?
To ensure that the network is trained towards minimizing the same prediction difference on the simpliﬁed and original data, we also add a prediction difference loss
L pred .
Further details about the implementation are included in Alg.
1 and Section S5.
In Figure 8, we visualize both the misclassiﬁed images according to the original network f and produce the corresponding simpliﬁed versions.
We observe that simpliﬁed images may provide some intuition into the reason for misclassiﬁcation, highlighting a variety of different features for different images.
We imagine a possible practitioner workﬂow, where the practitioner derives a set of possible hypotheses for the misclassiﬁcation from SimpleBits and tests them on the real data.
We show further post-hoc simpliﬁed examples in supplementary Section S13.
R ELATED W ORK
A different approach to reduce input bits while retaining classiﬁcation performance is to train a compressor that only keeps information that is invariant to predeﬁned label-preserving augmentations.
Dubois et al.
(2021) implement this elegant approach in two ways.
In their ﬁrst variant, by training a VAE to reconstruct an unaugmented input from augmented (e.g. rotated, translated, sheared) versions.
In their second variant, building on the CLIP (Radford et al., 2021) model, they take image-text pairs and view all images with the same text as augmented versions of each other.
This allows to use compressed CLIP encodings for classiﬁcation and achieves up to 1000x compression on Imagenet without decreasing classiﬁcation accuracy.
Their approach focuses on achieving maximum compression while our approach is focused on interpretability.
Their approach requires access to predeﬁned label-preserving augmentations and has reduced classiﬁcation performance in input space compared to latent/encoding space.
Our approach simplifying individual training images builds on Raghu et al.
(2021), where they learn to inject information into the classiﬁer training.
Per-instance simpliﬁcation during training can be seen as a instance of their framework combined with the idea of input simpliﬁcation.
In difference to their methods, SimpleBits explicitly aims for interpretability through input simpliﬁcation.
Other interpretability approaches that synthesize inputs include generating counterfactual inputs (Hvilshøj et al., 2021;
Dombrowski et al., 2021; Goyal et al., 2019) or inputs with exaggerated features (Singla et al., 2020).
SimpleBits differs in explicitly optimizing the inputs to be simpler.
[Figure]
Hypothesis
[Figure]
cause
[Figure]
Predicted
[Figure]
Generative models have often been used in various ways for interpretability such as generating realistic-looking inputs (Montavon et al., 2018) and by directly training generative classiﬁers (Hvilshøj et al., 2021; Dombrowski et al., 2021), but we are not aware of any work except (Dubois et al., 2021)
(discussed above) to explicitly generate simpler inputs.
C ONCLUSION
We propose SimpleBits , an information-based method to synthesize simpliﬁed inputs.
Crucially,
SimpleBits does not require any domain-speciﬁc knowledge to constrain which input components should be removed; instead SimpleBits itself learns to remove the components of inputs which are least relevant for a given task.
As an interpretability tool, we show that SimpleBits is able to remove injected distractors, suggest plausible reasons for misclassiﬁcation, and recover known radiologic features from condensed datasets.
When combined with data condensation, SimpleBits retains accuracy while greatly reducing the complexity of condensed images.
Our simpliﬁcation approach sheds light on the information required for a deep network classiﬁer to learn its task.
We ﬁnd that the tradeoff between task performance and input simpliﬁcation varies by dataset and setting - it is more pronounced for more complex datasets.
R EPRODUCIBILITY S TATEMENT
We provide the following information to ensure reproducibility.
Main concepts, algorithms and basic architectures are described in sections 3, 5 and 4.
Further network architecture details are in supp.
sections S2 and S3, further optimization details in supp.
sections S4 and S5.
Finally, the code is available under https://tinyurl.com/simple-bits .
S UPPLEMENTARY OUTLINE
This document completes the presentation of the main paper with the following:
• Bits per dimensions for selected images of other generative models in S • Details about the classiﬁer and simpliﬁer architectures in sections S2 and S • Details about the optimization of per-instance simpliﬁcation during and after training in sections S4 and S • Files sizes of the simpliﬁed images when compressed with PNG in section S • Learning curves during retraining on the simpliﬁed images in section S • Baselines for the per-instance simpliﬁcation during training in section S •
Potentially spurious features revealed by SimpleBits in section S • More images simpliﬁed during and after training in sections S9 and S13, including posthoc- SimpleBits output on the control images • More condensed and simpliﬁed datasets under varying settings in section S • Results for the continual learning setting in section S
S BPD S OF OTHER GENERATIVE MODELS
Figure S1 shows that the bits per dimension produced by other generative models than Glow also correlate well with visual complexity, validating our measure.
This is consistent with prior work that found bpds of generative models trained on natural image datasets are strongly inﬂuenced by general natural image characteristics independent of any speciﬁc dataset (Kirichenko et al., 2020;
Schirrmeister et al., 2020; Havtorn et al., 2021).
S C LASSIFIER N ETWORK D ETAILS
Our classiﬁcation network built on the Wide ResNet architecture (Zagoruyko & Komodakis, 2016).
We used a version with relatively few parameters with depth = 16 and widen factor = 2 to allow for fast iteration on experimentation.
We used ELU instead of ReLU nonlinearities.
Additionally, we removed batch normalization to avoid interference of normalization layers with the simpliﬁcation process.
We followed the method from Brock et al.
(2021) to create a normalizer-free
Wide ResNet.
We reparameterize the convolutional layers using Scaled Weight Standardization:
[Equation]
where µ i = (1 /N )
(cid:80) j W ij , σ 2 i
=
(1 /N )
(cid:80) j
( W ij − µ i ) 2 , and N denotes the fan-in.
Further as in Brock et al.
(2021), ”activation functions are also scaled by a non-linearity speciﬁc scalar gain γ , which ensures that the combination of the γ -scaled activation function and a Scaled Weight
Standardized layer is variance preserving.”
Finally, the output of the residual branch is downscaled by 0 . 2 , so the function to compute the output becomes h
i +
=
h
i + 0 .
2 · f i ( h i ) , where h i denotes the inputs to the i th residual block, and f i denotes the function computed by the i th residual branch.
Unlike Brock et al.
(2021), we did not multiply scalars
β i with the input of the residual branch or learned zero-initialized scalars to multiply with the output of the residual branch, as we did not ﬁnd
these two parts helpful in our setting.
We also did not attempt to use Stochastic Depth (Huang et al., 2016), which may further improve upon the accuracies reported here.
Due to our small batch sizes (32), we also did not use adaptive gradient clipping.
S S IMPLIFIER N ETWORK D ETAILS
For the simpliﬁer, we adapted a publicly available implementation of UNet 1 .
We used num down = 5 downsampling steps, ELU nonlinearities ngf = 64 ﬁlters in the last conv layer and a simple afﬁne transformation layer instead of a normalization layer.
Furthermore, we made the simpliﬁer residual and ensured the output is within [0 , 1] by adding the output of the UNet to the inverse-sigmoid-transformed input and then reapplying the sigmoid function.
S O PTIMIZATION D ETAILS PER -I NSTANCE SIMPLIFICATION DURING TRAINING
First, we note that the single train step helps ensure a correspondence between simpliﬁed and original images, and is a technique others have used in meta-learning settings (Pham et al., 2021).
For stabilizing the optimization of the per-instance simpliﬁcation during training, we found two further steps helpful.
First, we modify:
[Equation]
to
[Equation]
as (a) the gradient magnitudes are much smaller from the losses after unrolling and (b) we want to prioritize the classiﬁcation loss on the original data.
Additionally, we dynamically turn off L sim during training, for any example x where l ( f (cid:48)
( x orig ) , y ) > 0 . 1 , which we also found to further stabilize training.
S O PTIMIZATION D ETAILS P ER -I NSTANCE SIMPLIFICATION AFTER TRAINING
We found it beneﬁcial to optimize the simpliﬁed inputs in the latent space of the pre-trained Glow network and to apply our loss functions to all interpolated inputs on the path between original and simple input in latent space instead of only to the simpliﬁed input itself.
The points on the path include information from the original input and prevent that the optimization is unable to recover some relevant information from the original input.
Additionally, we also ensure that the predictions are the same for the original and interpolated inputs for both the original and the scaled model.
The complete loss function can be found in Alg. 2, during training we called it with scaling factors sampled from s U (0 . 8 , 0 .
95) as these values mostly led to similar but more uniformly distributed predictions than the unperturbed network.
[Table] [Table]
[Figure] [Figure]
c y
End of Joint Training
Retraining
Orig CIFAR
S PNG-C OMPRESSED F ILE S IZES OF S IMPLIFIED I MAGES
Figure S2 shows the tradeoff between PNG storage space of the simpliﬁed images and the accuracies achieved when retraining.
S L EARNING C URVES FOR R ETRAINING
Figure S3 shows learning curves during retraining on the simpliﬁed images on CIFAR10.
There are no noticeable differences in training speed for more or less simpliﬁed images.
[Figure] [Figure]
a c y [ % ]
Retraining Learning Curves CIFAR 3.57 BPD 3.56 BPD 3.52 BPD 2.95 BPD 2.81 BPD 2.66 BPD 2.50 BPD 2.26 BPD 1.86 BPD 1.76 BPD
[Figure]
S S IMPLIFIER B ASELINES
We implemented two simpler baselines to check whether the losses used in SimpleBits during training help retain task-relevant information.
In the ﬁrst baseline, we train the simpliﬁer to simultaneously reduce bpd of the simpliﬁed image and the mean squared error between the simpliﬁed and the original image.
Afterwards we train the classiﬁer on the simpliﬁed images and evaluate on the original images the same way as during retraining of SimpleBits .
In the second baseline, we blur the original images with a gaussian kernel, which also reduces their bpd.
The gaussian blurring replaces the simpliﬁer, so this is a ﬁxed simpliﬁer baseline without training a simpliﬁer.
We vary the sigma/standard deviation for the gaussian kernel to trade off smoothness and task-informativeness.
While these baseline also retain some task-relevant information allowing the classiﬁer to retain above-chance accuracies (see Figure S4), the tradeoff between bpd and accuracy is worse than for SimpleBits .
This shows the losses used in SimpleBits help retain more task-relevant information compared to these baselines.
S M ORE I MAGES S IMPLIFIED D URING T RAINING
We show a larger number of images that were simpliﬁed on CIFAR10 during training with the largest simpliﬁcation loss weight λ sim = 2 .
in Figures S5, S6 and S7.
S P OTENTIALLY SPURIOUS FEATURES UNCOVERED BY S IMPLE B ITS
SimpleBits may have the potential to reveal spurious correlations present in the dataset.
We show some simpliﬁed images that reveal potentially spurious features in Figure S8.
These observations can be used as a starting point to further investigate whether these features also affect regularly trained classiﬁers.
S M ORE CONDENSED DATASETS
We also show some interesting condensed datasets that resulted when we varied the architecture (Figure S10) or the condensation loss (from gradient matching to either negative gradient product or single-training-step unrolling, Figure S11)
. S E VALUATION OF C ONDENSED D ATASETS FOR C ONTINUAL L EARNING
We also evaluate the simpliﬁed condensed datasets in a continual learning setting, following (Zhao & Bilen, 2021).
In this task-incremental continual learning setting, the model is trained on different classiﬁcation datasets sequentially.
When training on a new dataset, the model is additionally trained on the condensed versions of the previous datasets.
The continual learning experiment reproduces the setting from (Zhao & Bilen, 2021) to ﬁrst train on SVHN, then on MNIST and ﬁnally on USPS (Hull, 1994), using the average accuracy across all three datasets of the classiﬁer at the end of training as the ﬁnal accuracy (see (Zhao & Bilen, 2021) for details).
We created a simpler and faster continual training pipeline that achieves comparable results to Zhao & Bilen (2021).
First, we train 3 times for 50 epochs on SVHN, with a cosine annealing learning rate schedule (Loshchilov & Hutter, 2017) that is restarted at each time with lr = 0 .
Then for each
MNIST and USPS, we train one cosine annealing cycle of 50 epochs for lr = 0 . 1 .
We ﬁrst veriﬁed that we can reproduce the prior continual learning results with our simpler training pipeline and ﬁnd that our training pipeline indeed even slightly outperforms the reported ﬁnal results (96.0% vs. 95.2% with, and 95.4% vs 93.0% without knowledge distillation) despite slightly inferior performance in the ﬁrst training stage (before any continual learning, 93.6% vs 94.1%), see following subsection.
When using different SVHN and MNIST condensed datasets, we ﬁnd that we can retain the original continual learning accuracies even with condensed datasets with substantially less ( ∼ 9x less) bits per dimension (see Fig.
S12).
S12. W ITHOUT C ONDENSED D ATASET
Our training pipeline still exhibits forgetting when not using any condensed datasets of previously trained-on datasets.
As Figure S13 shows, the accuracies are far lower than with just regular sequential training.
We performed this ablation to ensure forgetting still occurs in our training pipeline.
S M ORE IMAGES S IMPLIFIED A FTER T RAINING
We show further examples of post-hoc-simpliﬁed images for misclassiﬁed original images in Figure S14.
We also show the output of SimpleBits when applied to the control images of Figure 8 in
Figure S15.
We also show an uncurated set of incorrectly predicted and correctly predicted images in Figures S16 and S17.
[Figure]
InstancenormReLU
InstancenormELU
[Figure] [Figure]
[Figure] [Figure]
[Figure] [Figure]
v
[Figure]
r a
[Figure]
e a c r o ss d a t a s e t s s ee n d u r i n g t
r
a
i n i n g )
[Figure]
[Figure]
A cc u r a c
y
[ % ] ( a v e r a g e a c r o ss d a t a s e t s s ee n d u r i n g t
r
a
i n i n g )
[Figure]
Conventions as in Figure S12.
Accuracies substantially worse without any condensed dataset.
Original Images
Predicted as
Ship
Plane
Simplified
Hypothesis for one cause of misclassification
Ground + plane looks like ship
Control Image
Predicted as
Plane
Ship
Frog
Higher Freq Texture
Bird
Deer
Color and antler -like shapes
Cat
Car
Car and wheelshape
Deer
Black dots (true class) (true class) (true class) (true class) (true class)
Original Images
Predicted as
Deer
Frog
Simplified
Hypothesis for one cause of misclassification
Tree as antler
Control Image
Predicted as
Horse
Cat
Car
Brighter parts form car contours
Cat
Deer
Color, ear/tail antler-like
Dog Dog
Black-white dog face
Bird
Color (true class) (true class) (true class) (true class) (true class)
[Figure]