Underwhelming Generalization Improvements From Controlling Feature Attribution
Authors: Joseph D Viviano
Becks Simpson
Francis Dutil
Yoshua Bengio
Joseph Paul Cohen
Venue:  (2020)
Keywords: interpretability
medical
generalization
saliency

Abstract: Overfitting is a common issue in machine learning, which can arise when the model learns to predict class membership using convenient but spuriously-correlated image features instead of the true image features that denote a class. These are typically visualized using saliency maps. In some object classification tasks such as for medical images, one may have some images with masks, indicating a region of interest, i.e., which part of the image contains the most relevant information for the classification. We describe a simple method for taking advantage of such auxiliary labels, by training networks to ignore the distracting features which may be extracted outside of the region of interest, on the training images for which such masks are available. This mask information is only used during training and has an impact on generalization accuracy in a dataset-dependent way. We observe an underwhelming relationship between controlling saliency maps and improving generalization performance.

Under review at ICLR 2020
UNDERWHELMING GENERALIZATION IMPROVEMENTS FROM CONTROLLING FEATURE ATTRIBUTION
Anonymous
ABSTRACT
Overfitting is common issue in machine learning, which can arise when the model learns predict class membership using convenient but spuriousty- correlated image features instead of the true image features that denote a class. These are typically visualized using saliency maps. In some object classification tasks such as for medical images, one may have some images with masks, indi- cating region of interest, i.e., which part of the image contains the most relevant information for the classification. We describe simple method for taking advan- tage such auxiliary labels, by training networks to ignore the distracting features which may be extracted outside of the region of interest, the training images for which such masks are available. This mask information only used during train- ing and has impact generalization accuracy in dataset-depende way. We observe an underwhelming relationship between controlling saliency maps and improving generalization performance.
INTRODUCTION
Overfitting common problem in machine learning, particularly when one uses powerful function approximators such as deep neural networks. When training these models with backpropagation. the network will evolve from modelling simple to more complicated functions until it finds salient discriminative features in the data. Once the model has found these, the gradients of the loss do not encourage the model to find other discriminative features in the data, even if they exist (Reed) & Marks 1999). In the classification case, this can be problematic if there exists some distractor feature za in the data that correlated with one of the output classes. This a common issue in industry data (e.g., medical) where datasets are typically small and there are many confounding variables.
Consider the extreme in binary classification problem where in the training distribution there exists a confounding distractor element zy the input data such that for Dtrain- p(y = =1 1, while in the validation distribution Doelid, ply 0) (Figure D In this scenario, predicting using F. is easier than predicting using the true features that denote class membership and classifier trained on Dtrain with traditional classification loss would predict the incorrect class with 100% probability on Dvatid- This is a textbook example of overfitting (Goodfellow et al.) [016 Reed & Marks 1999). The existence these overfit features is the motivation behind methods seeking to leamn domain- invarininvariant representations (Ganin & Lempitsky 2014 Fernando et al. 2014)) and is
common problem with real-world data (Badgelev al. (0099] Zhao al. 2019 Young al. 2019) Train Samples
Valid Samples
Class 0
Class
Class 0
Class 1
+
Figure 1: Example images from Dtrain and Dealtd from both classes. In both distributions, cross size can vary between samples. In Dtrain. two crosses (denoting class 0) are always accompanied by box za the bottom right-hand corner, while a single cross (denoting class 1) always ac- companied by a distractor in the bottom left-hand corner. In Dealid. the relationship between classes and crosses remains the same, but the logic governing the location of the distractor is reversed. The distractor is indicated with red arrow.

---

Under review at ICLR 2020
In this paper, explore the utility of various methods that allow one to use mask the input data to guide the network to avoid predicting from the defined region and penalize the network for attributing prediction to distractor. We present synthetic dataset that encourages all models tested to overfit to an easy to represent distractor instead of a more complicated counting task. We present novel "activation difference" (actdiff) regularizer which mitigates this behaviour directly. We also present a method where we train an autoencoder/UNet to reconstruct a masked version of the input, indirectly controlling feature representations used for classification. We compare these methods with the recently- recently-proposed proposed gradmask (Simpson et al. 2019b), and present an expanded analysis of this algorithm's ' behaviour. All code for this paper, and this dataset, are available here: https://gi thub. http:///iggeeelii/////eeeeeeeeee
We compare the real-life performance of these methods on open medical datasets with traditional classifiers, and demonstrate the differences in their feature attributions using saliency maps. Finally, we describe a medical dataset curated from two openly-available X-ray databases, and describe how samples can be drawn from each to generate dataset biased by a site-diagnosis correlation inspired by previous work (Zhao et al. 2019). We demonstrate that, similarly to our synthetic datasets, classifiers are likely predict using features unrelated to the task, and demonstrate that the proposed methods do mitigate this and often successfully refine the saliency maps to focus on the correct anatomy. However they do not consistently prevent overfitting.
RELATED WORK
It is well- well-documented phenomenon that convolutional neural networks (CNNs), instead of building object-level representations of the input data, tend to find convenient surface-level statisties in the training data that are predictive of class (Jo & Bengio, 2017). Previous work has attempted to reduce the model's proclivity to use distractor features by randomly masking out regions of the input (DeVries& & Taylor 2017). By randomly removing information from the inputs to the network, this method helped the network learn representations that aren't 't dependent on single feature types in the image. However, this regularization approach gives no control over the kinds of representations learned by the model.
Recently, the Gradmask (Simpson et al. 2019b) and CARE methods (Zhuang al. 2019 both proposed to control feature representations by penalizing the model for utilizing gradients outside of regions of interest. CARE was additionally designed to deal with class imbalances by increasing the impact of the gradients inside region of interest of the under-represented class.
In contrast these two methods, we propose new method which does not work with saliency map, which can be noisy due to the ReLUU activations allowing irrelevant features to pass through the activation function (Kim et al. 2019). Instead, this method operates directly on the activations themselves, encouraging the model to produce similar activation patterns in the presence of, and absence of, information outside of the region of interest.
METHODS
Actdiff Loss: To mitigate the effect of xa, we propose to explicitly regularize the network to ignore va at test time by minimizing the distance between the network activations of the model when presented with full input image and one where the information outside of some mask the image has been corrupted. The model is only directly trained on unmasked examples. This encourages the network to build features which appear inside the masked regions even though it always sees the full image during training. The method requires having access to masks drawn by an expert who can distinguish between interesting and non-interesting discriminatve features, as is often the case in medical imaging. The actdiff regularization term is
Lact Σ (loakea))) ox(x)112,
(I)
where (x) are the pre-activation outputs for layer of the n-layer encoder f(x) when the network is presented with the original data x, and 0y (Xmmakke))) are the pre-activations outputs for layer / when presented with masked data Xmasked- We call this the actdiff penalty. Xmasked should be constructed by randomizing the indices of all pixels that fall outside of the mask, destroying any spatial information available in those regions of the image, but retaining the distribution of intensities
2

---

Under review at ICLR 2020
L mask
Reconstruction loss
aly'l
I mask dz
Activdiff loss
Skip connections
Gradmask loss
-Oy Classification loss
Figure 2: Schematic of the model used in all experiments (alongside an 18-layer ResNet). The actdiff penalty was only applied the encoder path of the model. The reconstruction path (post classification) was optionally used when reconstruction was requested of the model. Skip connec- tions were optionally employed in the style of UNet. Both of these optional paths are denoted using alternating dashed lines. All losses are denoted using standard dashed lines.
found in the data. Furthermore retain important context around masked region, we always dilate the mask by set number of pixels. This loss has previously been used in style transfer to transform random noise image into one containing the same layer-wise activations some target image Gatys et al. (2015).
represent aspects of the input data can be balanced with reconstruction term that encourages the model to represent the data. Con- cretely, we can employ an auto-encoder architecture
Reconstruction Loss: The pressure from Last
(((())). where is a convolutional encoder and is a deconvolutional decoder, and require the model to minimize the reconstruc- - ||x(] 110 where indexes the pixel-wise difference between the input æ and reconstruction ã. In our experiments a reconstruction term is helpful for guiding the network toward building useful representations of the data while employing Leet- All auto- encoder models presented in this paper employed this loss term. Furthermore, the reconstruction task can be used to indirectly control feature representation learning by asking the model re- construct a masked version of the input given the full input Mmakee (xf(x) and minimizing Crecon masked m Σド [+( x(0) *maskedlle la, which we used replacement for Crecon.
tion loss Crecon
mesked
Gradmask Loss: Gradmask is recently proposed (Simpson al. (2019b) method for controlling
which regions of the input are desirable for determining the class label using saliency maps. Saliency
maps, "input feature attribution". can be calculated 0|a:l for each input (Zeiler & Fergus 2013 Simonyan et al., 2014 Lo et al. 2015). these experiments we minimize the "contrast saliency between healthy and non-healthy classes (labels yo and /1 respectively),
8x
expect that input variance which increases the distinction between the two classes leads to overfitting and is what want regularize, Therefore, minimize
Egrad Σ Las +
vo Ox
Xxeg)
XED
2
where do and ú1 the predicted outputs for two classes and (1 Xseg) is a binary mask that covers everything outside the defined regions of interest.
Control Experiments: For many experiments, evaluated the effect of simply training model using masked data (and evaluating using unmasked data). We call these experiments Clas- sify Masked. We present this naive alternative to actdiff. Additionally, would happen our auto encoder models were asked to reconstruct masked version of the input (Crccon masked above). These experiments reported Reconstruct Masked.
evaluated what
3

---

Under review at ICLR 2020
4 SYNTHETIC DATASET
Method: To evaluate the proposed methods for combating overfitting in the presence of a distractor variable, we generated a dataset following the description provided earlier (Figure[]) with 500 train- ing, 128 validation, and 128 test examples respectively. The position of the distractors was perfectly correlated with class label in the training set and the logic governing this relationship was inverted for the validation and test sets. In cases where the model relies on the distractor to make class prediction, we expected 0.0 AUC for the validation and test sets.
To evaluate the effect of the actdiff loss, gradmask loss, and the reconstruction penalty during train- ing, we constructed a simple CNN architecture that optionally deconvol the final layer to generate reconstruction, optionally did so in the style of UNet (Ronneberger al., 2015), see Figure We additionally tested all non-reconstructing approaches using a simple 18-layer ResNet model (He al., 2016). As control experiments, we also evaluated classifier performance when simply trained using masked versions of x, but evaluated on unmasked examples. All models were trained using Adam for 500 epochs with learning rate of 10e batch size of 32, with the weights of our regularizers in the loss term (denoted as X) being Auct 10, >grad = 10, and Areeon = 10 when applicable. The classification weight was set to 1. Before masking, masks were blurred using a Gaussian filter using a = 0.5, in order for some context to be included around the masked area. For reconstruction, the binary cross entropy loss was used as the input images were binary.
2
The results of all experiments are shown in Table with the architectures that successfully avoid overfitting in bold. All results are the average of 10 random model initilizations and data splits. To determine the effect of actdiff feature representations in the network, we display saliency maps on the validation set during the final epoch of training in Figure
Results:
Train AUC
Valid AUC
Best Epoch (/500)
Experiment Name
Conv AE Classify
1.00 + 0.00
0.06 + 0.03
0.00 0.00
Conv AE Actdiff
0.80 + 0.26
0.80 = 0.26
140.10 + 156. 84
Conv AE Gradmask
0.50 + 0.00
0.50 0.00
44.70 + 102.68
Conv AE Actdiff & Gradmask
0.50 + 0.00
0.50 0.00
0.00 + 0.00
Conv AE Classify Masked
1.00 0.00
0.53 0.01
127.00 + 132.24
Conv AE Reconstruct Masked
1.00 0.00
0.06 0.05
0.00 0.00
CNN Classify
1.00 0.00 0.00
0.00
38.10 1 120.13
CNN Actdiff
0.50 0.00
0.50
7.30 23.08
CNN Gradmask
0.50+:0.00
0.50
0.00 0.00
CNN Actdiff & Gradmask
0.50 + 0.00
0.50 0.00
1.00
CNN Classify Masked
1.00 : 0.00
0.55 0.02
14.10 7.36
ResNet Classify
1.00
0.00 = 0.000
0.000 = 0.00
ResNet Actdiff
1.00+0.00
1.00 0.00
147.60 148.59
ResNet Gradmask
0.55 + 0.06
0.54 + 0.073
243.10 + 218.66
ResNet Actdiff & Gradmask
0.53 1 0.04
0.52 4.03
143.60 + 190.57
ResNet Classify Masked
1.00 0.00
0.73 + 0.21
169.80 + 178.95
UNet Classify
+.00
0.05 0.02
0.00 0.00
UNet Actdiff'
1.00 - 0.00
0.90 0.31
43.50 + 30.06
UNet Gradmask
0.50 + 0.00
0.50
107.10 + 142.43
UNet Actdiff & Gradmask
0.63 + 0.14
0.00
348.00 + 160.69
UNet Classify Masked
1.00 0.00
0.54
51.40 + 96.29
UNet Reconstruct Masked
1.00 + 0.00
0.01 0.01
0.00 0.00
Table 1: Synthetic Dataset Test AUC after 500 epochs, averaged over 10 seeds. Mean and standard deviation presented.
First we demonstrate CNN overfitting on Derasn. using simple CNN architecture trained using only the cross entropy loss Letf - -. y log Di- This model achieves 1.0 AUC on Dtrain and 0.0 AUC on Dyalid- Note that the model is attributing all saliency to the distractor. When the model is trained using masked inputs from Derain and trained using Lat,, the model performs well on D Praan but is unsure how to handle the distractor in Dratids leading to performance similar to chance.
4

---

Under review at ICLR 2020
Saliency: Nonhealthy djyl/dx
Image
CNN Classincation
UNet Acteitf
UNet Reconstructin
+
+
Mask
CNN Classify Masked
ResNet18 Actdiff
RasNet18 Gradmasi
Figure 3: Saliency maps showing the different major behaviours on Deatid observed across the mod- els tested on the input image (top left; distractor indicated with red arrow) with the dialated mask (bottom left). These images correspond to the rows of Table [ CNN classification demonstrates overfitting, where the saliency is concentrated Masked demonstrates that the model has not learned to ignore the distractor because never saw one during training (center bottom left). UNet actdiff (center top right) and ResNet 18 actdiff (center bottom right) demonstrate that the model has successfully learned to ignore the distractor. Note that the network learned reconstruct the distractor in the location observed during Dtrain (top right, red arrow). The gradmask model fails to ignore the distractor and does not pay equal attention to the two features interest (bottom right).
the distractor (center top left). CNN Classify
A CNN trained using both the classification and actdiff loss fails to learn any useful representations, scoring an AUC 0.5 during train and validation (see "CNN Actdiff" from Table TT. If this model instead a Convolutional Autoencoder (Conv AE) UNet, and is additionally trained using recon- struction loss, it can successfully learns to ignore the distractor when classifying the image x ("Conv AE Actdiff" and "UNet Actdiff"). Note in Figure that that the UNet reconstruction, taken from the val- idation set, places the distractor the location seen the training set. This implies that the actdiff loss guided the model learn ignore the distractor before the classification layer, but retained some representation of it in an early layer of the network. Training using reconstruction without Actdiff does not prevent overfitting ("Conv AE Classify" and "UNet Classify"), Experiments with an 18-layer ResNet demonstrate good performance using classification and actdiff alone ("ResNet Actdiff").
Gradmask never produces model with good generalizaion performance, with generally unstable training (Figure Z We suspect this because saliency map always non-zero everywhere on the input, leading to constant source of noise in the loss function.
The best performing models took many more epochs to reach the optimal solution than would be expected for such a simple dataset. Note the "Best Epoch" for overfit models is misleading as the best performance chance. We found that models using a reconstruction loss more slowly approach the vicinity of their optimum than a ResNet (Figure 7) and the ResNet model takes longer to each its best epoch (see Table
SINGLE-SITE MEDICAL DATASET WITH SEGMENTATIONS
Method: We applied all previous methods to three medical imaging datasets from the Medical Segmentation Decathlon (MSD) where one task is to detect the presence of an organ in the image, and for each organ we have a segmentation. Classifiers were previously shown to use the incorrect organ when performing this task (Simpson et al. 2019a). We tested our approaches on the liver detection task in CT, cardiac left atrium detection task in MRI, and pancreas detection task in CT. All results are the average of 10 independent seeds, which also lead independent splits of the data. For each seed 128 training samples, 256 valid samples, and 256 test samples were randomly selected. A sample
2D image containing the segmentation. The mask blur factor
16
5

---

Under review at ICLR 2020
Synthetic Test AUC
Liver Test AUC
Cardiac Test AUC
Pancreas Test AUC
Experiment Name
CNN Classify
0.35 + 0.20
0.87 0.02
0.86 + 0.13
0.82 + 0.02
CNN Actdiff'
0.50 + 0.00
0.77 = 0.12
0.60 = 0.11
0.66 - 0.12
CNN Gradmask
0.50 1: 0.00
0.85 0.01
0.75 1 0.20
0.82 1: 0.02
CNN Actdiff & Gradmask
0.50 + 0.00
0.10
0.50 = 0.00
0.66 + 0.05
CNN Classify Masked
0.74 + 0.11
0.05
0.75 + 0.07
0.50 + 0.01
ResNet Classity
0.50 + 0.00
0.00
0.90 = 0.03
0.81 = 0.02 0.02
ResNet Actdiff
0.98 1 0.01
0.86 0.002
0.88 1 0.03
0.81 1 0.02
ResNet Gradmask
0.71 +: 0.05
0.09
0.93 1 0.01
0.82 0.03
ResNet Actdiff & Gradmask
0.67 + 0.04
0.00
0.92 0.03
484+0000
ResNet Classify Masked
0.96 + 0.03
0.03
0.77 0.08
0.57 = 0.05
Conv AE Classify
0.50 + 0.00
0.78
0.76 0.17
0.77 0.01
Conv AE Actdiff
0.78 + 0.24
0.80
0.84 0.08
0.76 0.02
Conv AE Gradmask
0.52 + 0.05
0.00
0.89 + 0.10
0.77 + 0.02
Conv AE Actdiff & Gradmask
0.50 + 0.00
0.00
0.78
Conv AE Classify Masked
0.79 1 0.11
0.59
0.69 0.10
0.51 1
Conv AE Reconstruct Masked
0.50 + 0.00
0.84 0.02
0.82+ 0.17
0.81 + 0.02
UNet Classify
0.50 1 0.00
0.81
0.83 0.17
0.77 1: 0.09
UNet Actdiff'
0.95 + 0.12
0.02
0.88 0.03
0.78 0.03
UNet Gradmask
0.55 + 0.07
0.02
0.82 0.13
0.79 0.02
UNet Actdiff & Gradmask
0.63 0.10
0.78 0.04
0.78 0.09
0.76 + 0.02
UNet Classify Masked
0.74 1: 0.10
0.58 0.09
0.56 1 0.10
0.53 1 0.05
UNet Reconstruct Masked
0.50 0.00
0.87 0.01
0.75 + 0.20
0.81 + 0.03
Table 2: Test Results (Best Valid Epoch over 500 epochs) all MSD Datasets. Results
averaged over 10 seeds, and we present the standard deviation.
for the Liver and Pancreas datasets, and o 8 for the cardiac dataset. All images were resized to 100 x 100 pixels. These blur values were selected as they provided the best gradmask performance, and are higher for the medical datasets when compared with the synthetic dataset experiments as the images used here are much larger (512² vs 24 All models were trained with an Adam optimizer with learning rate of 10 0e for the Pancreas and Liver datasets, and 4e-3 for the Cardiac dataset, which were found to the optimal learning rates using a hyperparameter search. All models were trained with batch size of 32 and batch shuffling. The weights of regularizers in the loss term (denoted as X) were set to Pact = 1, Agrad 1, and Arecon = 1, when applicable. The classification loss weight always set to 1. The reconstruction loss was the mean squared error.
Results: We present all test AUCs for the best valid AUC over 500 epochs of training in Table 2 alongside the experiments from the previous section. For each model, the best-performing (or otherwise notable) configurations are in bold.
For CNN-based models, classification alone gave best performance. In contrast, the ResNet model generally benefited from the addition of gradmask, in contrast to the synthetic dataset results. The one notable exception was for the pancreas dataset, where the combination of classification, actdiff, and gradmask gave the best performance.
The best-performing model baseline the ResNet model, which is unsurprising given its perior expressive power over the simple CNN architecture we used for all other experiments. For the Liver and Cardiac datasets, classification with gradmask outperformed the bascline and all other models. In the case of the Pancreas dataset, classification with both actdiff and gradmask performed best. In all cases, classification with actdiff alone performed as well as, or worse than, the baseline.
Surprisingly, the auto-encoding eecodding models showed the best performance when trained to reconstruct masked version of the input for the Liver and Pancreas dataset. For the Cardiac dataset, the best performing method was classification with gradmask for the Convolutional AutoEncoder and with actdiff for the UNet, and each achieve similar performance. This is likely because actdiff is too strong of a regularizer for CNN models, the skip connections in the UNet allow the model to greatly reduce the actdiff penalty in the deeper layers of the encoder. The variance of model performance across seeds is higher if trained with gradmask than actdiff. Again, training with actdiff and gradmask outperforms either approach alone in the Liver and Pancreas datasets.
Individual saliency map examples for the Liver, Cardiac, and Pancreas datasets be found in Fig- ure 9 and the mean saliency over 100 test-set examples for all datasets can be found in Figure 4 In both the the I Liver and Cardiac dataset, actdiff encourages the ResNet model to focus the correct

---

Under review at ICLR 2020
anatomy, but this does not lead to increase in test AUC performance baseline. In contrast, gradmask less consistently encourages the model to focus its attention on the specified anatomy, but results in consistent test AUC performance improvements relative to bascline. The combination of the two methods (" ("Actgrad") also produces improved feature attribution in the absence of improved generalization performance. In the Pancreas dataset, the saliency maps of the model broadly across the anatomy. The gradmask and combination actdiff and gradmask models improved over baseline, but there no clear reason why this would be true from the saliency maps. We therefore conclude an inconsistent relationship between improved gen- eralization performance and refined saliency maps, where improved saliency does nessicarily lead to improved performance.
both actdiff and gradmask both focus
Image
Pathclogy
Resnet Bascline
Resnet ActDiff
Resnet Gradmask
Image
Pathclogy
Resnet Baseline
Resnet ActDiff
Resnet Gradmask
Image
Pathclogy
Resnet Baseline
Figure 4: The average saliency map from 100 pandomly-selected test-set images in the Cardiac (top), Liver (middle), and Pancreas (bottom) dataset. The pathology column shows the mean non-blurred segmentation for each dataset. All three datasets show clear and consistent changes in the location of the strongest saliencies when using both Actdiff and Gradmask when compared with a normal classifier.
6 MULTI-SITE X-RAY DATASET
Method: In an attempt to replicate the results of the synthetic dataset in a real world application, constructed X-Ray dataset using combination of the PadChest (Bustos al. 2019) dataset and the NIH Chestx-Ray8 (Wang et al. 2017) dataset. A site-driven overfitting signal has previously been reported when combining these datasets around the border of the images (Zech et al. 2018). We also observed this in regions far from the lungs (see the mean X-ray from each dataset in Figure 8), and therefore hypothesized we could improve overfitting performance by masking out the edges of the image using a circular mask. We constructed a joint dataset that allowed us to define patholog correlation the training set, and then produce validation and test set where the reverse relationship is true. In the training set, 90% of the unhealthy patients were drawn from the PadChest dataset and the remaining 10% of the unhealthy patients were drawn from the NIH dataset, and the reverse logic was followed for the validation and test sets. In all splits the classes and site distributions were always balanced, making it tempting for the classifier to use a site- site-specific feature when predicting the class in the presence of site- sathopath correlation. We chose emphysema detection as the detection task, resulting in 998 samples for training, 498 samples for validation and 504 samples for test. All images resized to 1 128 128 pixels. All experiments trained 18-layer

---

Under review at ICLR 2020
Experiment Name
AUC
Classify No SPC
0.70 0.02
Classify w/ SPC
0.44 0.08
Actdiff w/ SPC
0.53 0.04
Gradmask w/ SPC
0.48 0.03
All Masked w SPC
0.53 0.03
(b)
(a)
Figure 5: Results on the chest X-ray task. (a) Saliency maps of the different models with different methods to prevent incorrect feature attribution. The task is to predict Emphysema (a lung con- dition). Two different images from the test set are shown with the masks that were used during training. The top image is negative example and the bottom positive. (b) Test Results (Best Valid Epoch) using a ResNet on the Chest X-ray task. SPC=site-patholog correlation.
ResNet model using an Adam optimizer with a learning rate of 10e for 100 epochs. All results were averaged over 10 seeds. We trained classifier the same dataset with site-pathology correlation as a baseline, and compare these results that with the same classifier in the face of site-pathology correlation of 90%. We train models using Lact and Lyrad with masks for all input images in the training set. We also trained normal classifier with no Loet and gred penalty, but where the masked region was set to zero the train, valid, and test splits, obtain an upper bound on the expected performance of our model.
See Table 5b A ResNet trained on the dataset mixed, without site-pathology bias scores a test AUC of 0.7, while one trained in the presence of a strong site aite-pathology bias scores below chance on the test set (0.44). Both actdiff and gradmask improve performance of the model, but only actdiff scores above chance, and performs similarly to model trained with the areas outside of the mask completely removed. However, the saliency maps the ResNet trained with actdiff shows strong feature attribution from outside of the lungs. The model appears to be paying attention to the brightest regions of the image, which might be predictive the scanner. Since a region of high intensity is available in the center of each X-Ray, actdiff when trained with these masks cannot handle this case of overfitting. Models trained with gradmask appropriately attribute more saliency to the lungs, but score below-chance level the test set. Generally, the poor performance of all models in the presence of a site-pathology bias suggests that there is no regional source of the site- bias. This bias likely exists in almost every pixel of the dataset and therefore methods such as actdiff or gradmask are not well-suited to controlling overfitting in these scenarios.
Results:
7 CONCLUSION
We hypothesized that poor generalization performance could be partially attributable to classifiers exploiting apatially-distinee distractor features, and proposed the actdiff regularizer that prevents this behaviour on synthetic dataset. We compare the performance of this method against previously- proposed methods operating on saliency maps and demonstrate that the methods influence feature construction and generalization performance in dataset- dataset-deepnddet manner. We conclude that while our methods successfully control the features constructed from the data, and solve the overfitting problem in synthetic setting where the distracting feature is spatially distinct from the discrimi- native features, in real data we found no evidence of a spatially removed to mitigate overfitting. We now doubt the validity of using saliency maps for diagnosing whether a model is overfit because improving them does consistently improve generalization. Im- proved generalization performance observed in saliency-map based approaches may be due to the fact that these approaches add useful noise into the updates, similarly to cutout (DeVries & Taylor 2017). We leave this conjecture to future work.
signal that can be reliably
REFERENCES
Marcus Badgeley, John R Zech, Luke Oakden- Rayner, Benjamin Glicksberg, Manway Liu, William Gale, Michael V McConnell, Bethany Percha, Thomas M Snyder, and Joel T Dudley.
8

---

Under review at ICLR 2020
Deep learning predicts hip fracture using confounding patient and healthcare variables. npj Digital Medicine, 2019.
Aurelia Bustos, Antonio Pertusa, Jose-Maria Salinas, and Maria de la Iglesia-Vayy PadChest: A large chest x-ray image dataset with multi-label annotated reports. arXiv e-prints, 2019 art. 1.1.:1101..07411.
Terrance DeVrsse and Graham W. Taylor. 2017 Improved Regularization of Convolutional Neural Networks with Cutout.
Basura Fernando, Amaury Habrard, Mare Sebban, and Tinne Tuytelaars. 2014 2014 Subspace Alignment For Domain Adaptation.
Yaroslav Ganin and Victor Lempitsky. Unsupervised Domain Adaptation by Backpropagation. 2014 In International Conference on Machine Learning.
Leon A Gatys, Alexander S Ecker, and Matthias Bethge. A neural algorithm of artistic style. arXiv preprint arXiv:1508.0m 150.0066668... 2015.
Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press, 2016.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition, In Conference on Computer Vision and Pattern Recognition, 2016. doi: 10.1 109/ CVPR.2016.90.
Jason Jo and Yoshua Bengio. Measuring the tendency of cnns learn surface statistical regularities. CoRR, 2017.
Beomsu Kim, Junghoon Seo, SeungHyun Jeon, Jamyoung Koo, Jeongyeol Choe, and Taegyun Jeon. Why are saliency maps noisy? cause of and solution to noisy saliency maps. arXiv preprint arXiv: 1902. 04893, 2019.
Henry Z. Lo, Joseph Paul Cohen, and Wei Ding. Prediction gradients for feature extraction and analysis from convolutional neural networks. 2015 In International Conference Automatic Face and Gesture Recognition. doi: 10.1109/FG20015.76666
Russell D. Reed and Robert J. Marks. Neural smithing supervised learning in feedoooor artificial neural networks. MIT Press, 1999.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-Net: cal Image Segmentation. 2015 In Medical Image Computing and Computer Assisted Intervention.
Convolutional Networks for Biomedi-
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps. In International Conference Learning Representations (ICLR), 2014.
Amber L. Simpson, Michela Antonelli, Spyridon Bakas, Michel Bilello, Keyvan Farahani, Bram van Ginneken, Annette Kopp-Schneider, Bennett A. Landman, Geert Litjens, Bjoern Menze, Olaf Ronneberger, Ronald M. Summers, Patrick Bilic, Patrick F. Christ, Richard K. G. Do, Marc Gollub, Jennifer Colia-Pernice Stephan H. Heckers, William R. Jarnagin, Maureen K. McHugo, Sandy Napel. Eugene Vorontsov, Lena Maier-Hein and M. Jorge Cardoso. A large annotated medical image dataset for the development and evaluation of segmentation algorithms, 2019a.
Becks Simpson. Francis Dutil, Yoshua Bengio, and Joseph Paul Cohen. GradMask: Reduce Over- fitting by Regularizing Saliency. In Medical Imaging with Deep Learning Workshop, 2019h.
Xiaosong Wang, Yifan Peng, Le Lu, Zhiyong Lu, Mohammadhadi Bagheri, and Ronald M Sum- mers. Chestx-ray8: Hospital-scale chest x-ray database and benchmarks weakly-supervised classification and localization of common thorax diseases. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2017.
Kyle Young, Gareth Booth, Becks Simpson, Reuben Dutton, and Sally Shrapnel. Deep neural network dermatologist? arXiv preprint arXiv: /tt.::/900000000 2019.

---

Under review at ICLR 2020
John R. Zech, Mareus A. Badgeley, Manway Liu, Anthony B. Costa, Joseph J. Titano, and Eric K. Oermann. Confounding variables can degrade generalization performance radiological deep learning models. CoRR, 2018.
Matthew D. Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. CoRR, 2013.
Qingyu Zhao, Ehsan Adeli, Adolf Pfefferbaum, Edith V Sullivan, and Kilian M Pohl. Confounder- Aware Visualization of ConvNets. Technical report, 2019.
Jiaxin Zhuang. Jiabin Cai, Ruixuan Wang. Jianguo Zhang, and Weishi Zheng. {CARE}: Class attention to regions of lesion for classification on imbalanced data. In International Conference on Medical Imaging with Deep Learning - Full Paper Track, 2019 London, United Kingdom.
10

---

Under review at ICLR 2020
APPENDIX
A
MASK REQUIREMENT
Actdiff's requirement for hand-drawn masks can be a detriment in practice they are costly to acquire from human experts. To determine whether actdiff has applicability in the setting where only a subset of the training set has masks, we repeated our experiments detailed above, retaining either 20%, 40%, 60%, 80%, or 100% of the masks in the training set. We analyzed the resulting final test AUC (Figure ba) and the number of epochs (Figure (6b) required to reach this level of performance on the two best-performing actdiff models: the UNet and the ResNet18 model, averaging across 5 seeds. general, the ResNet18 model appears to be more robust to missing masks, although across datasets, there does not scem to be a direct correlation between more masks and better performance. In fact, the addition of more masks can decrease performance, suggesting that the quality of the masks used is more important than the quantity (Figure 6ap. There number of masks used during training and the number of epochs required to reach the best epoch. We suspect having a small set of very precise masks is sufficient guide the model toward developing good representations of the anatomy given that more compute time is available.
consistent effect of the
B ARCHITECTURE OF THE CNN, AUTOENCODER, AND UNET MODEL
The encoder of the AutoEncoder and UNet model was shared with the CNN model and was layers deep, with each layer consisting of a double convolution (kernel size of and stride of 1). All predictions were made off of the deepest layer of the network. The number of input channels was 16 for the synthetic dataset and 64 for the medical datasets, and doubled for each subsequent layer. All activations for actdiff were saved before applying the ReLU activation during the forward pass. During reconstruction, sigmoid activation was optionally applied to the output to assist in the binary output case (for the synthetic dataset). In the decoder path of the autoencoding models, upsampling was applied using bilinear interpolation before each double convolution.
11

---

Under review at ICLR 2020
Synth
Liver
0.9
Best Text AUC
ResNet Gradmask
UNet Actdlff
UNet Gradmask
0.6
Cardiac
100
0.9
Best Test AUC
0.7
0.6
% Masks Training Examples
Masks Training Examples
(a) Mcan and standard deviation of the test AUC (as selected by the valid AUC) across 10 seeds for all periments, under the condition that only some percentage of the masks are available. Only shown for the two best-performing Actdiff models (UNet and ResNet18).
Synth
Liver
Best Epoch
300
ResNet Actdiff
Gradmask
UNet Actdiff
UNet Gradmask
0-
Cardiac
Pancreas
500
Best Epoch
300
200
0.0
0.2
0.4
0.6
0.8
10
0.0
0.2
0.4
0.6
0.8
100
% Masks Training Examples
Masks Training Examples
(b) Mean and standard deviation the best epoch (out 500) based on the valid AUC across seeds for all experiments, under the condition that only some percentage of the masks are available. Only shown for the two hest-performing Actdiff models (UNet and ResNet18). 18).
Figure 6: Results of the maximum masks experiments. (a) Best Test AUC for the best Valid AUC for each of the maximum masks conditions. (b) Best valid Epoch for each of the maximum masks conditions.
12

---

Under review at ICLR 2020
UNet Experiments
Autoencoder Experiments
Casssty
AUC
06
Actgrad
Classify
0.
Gradmask
Reconstruct Masked
ResNet 18 Experiments
CNN Experiments
Valid AUC
0.2
100
Epoch
Epoch
Figure Line plots showing the Valid AUC for each the 500 epochs during training for all models on the synthetic dataset. We can see that training models with masked data has no substantial benefit on validation set, models simply trained to classify (or, in addition, reconstruct a masked version of the input) overfit early in training, gradmask models fails train, and actdiff surpasses the performance in all cases where it is effective (i.c., not the CNN model).
25
25
50
50
100
100
125
125
150
150
175
175
200
200
50
100
150
200
50
100
150
200
Figure 8: Mean resized X-Ray from the NIH dataset (left) and PadChest dataset (right). There clear differences in the site distributions that are obvious around the edges of the image.
13

---

Under review at ICLR 2020
image
Pothology Mask (blur=15)
Ur et Baseline Recor
ActGuad
Unit Rassline
Lnii A2t1211
Unel Gracimane
Unit ActiG
Unat Masked
(blur-5)
Unet Daseline Retor
Boand Bascline
Resnet Actbil
Rosnet Gradmask
Resnet ActGrad
Und Rusked Rocon
Pathology
Resnet Saseline
Resnet ActDi!
Resnet Gradinakk
Resnet ActGrad
Unet Kasked Recon
Figure 9: Saliency maps showing where the model attributes areas of the visual input space to the prediction made by the network for the Liver detection (top), Caridac Left Atrium detection (middle), and Pancreas (bottom) datasets. The top 10% of gradients are shown each image for visualization. The top left image shows the raw input, and to its right is the anatomy segmentation before and after blurring. From left to right along the bottom, the ResNet model outputs are shown in the second row and the UNet results shown in the third row, for the baseline classification model, ActDiff, Gradmask. and ActDiff & Gradmask ("ActGrad"). The rightmost column shows outputs specific the UNet reconstructions: the top image shows the standard reconstruction, right middle image shows the output of the Reconstruct Masked task, and bottom image shows the feature attribution of the Reconstruct Masked model.
14


Reviews:
[{"review_id": "rygobQ69FS", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper attempts to tackle the overfitting problem due to the focusing on the distracting information. By utilizing the dataset with masks, the authors propose a simple method to ignore the distracting features. By designing actdiff loss and reconstruction loss, the authors demonstrate that classifiers are likely to predict using features unrelated to the task and their losses can mitigate this problem.\n\nPros:\nThis kind of study can make us understand deeply what is going on in deep learning and thus to make it better, which shows this work’s high significance.\n\nCons:\n1.\tThe presentation of this paper is not clear. For example, in Table 1, the experiment name is so confusing. The authors mention that “Conv AE Actdiff” is the one using both actdiff loss and reconstruction loss. Then what does “Conv AE Reconstruct Masked” mean? Does Conv AE means they implicitly contain reconstruction loss? The authors should elaborate them clearly. The corresponding double quotation marks should be revised. Figure 3 is also confusing. At first, I thought different lines represent different input data (one without mask and one with mask) and different columns represent different methods. However, this is not the case. Furthermore, the fourth column is more confusing. I do not see any red circles in the fourth column. Why the reconstruction of distractor is far away from the original one. Does the right top picture show the advantages of UNet Reconstruction or its drawbacks?\n2.\tIn fact, the training data with masking is not sufficient (may only available in some medical images). It is hard to utilize these data to generalize to various other tasks including large amounts of images without masking.\n3.\tFrom the experiments on real tasks, I barely see improvements by the proposed method, which makes the conclusion unconvincing.\n\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}, {"review_id": "rkl72wslcH", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\nSummary\n---\n\n(motivation)\nCNN image classifiers tend to overfit to distractor patterns.\nPerhaps these patterns are spatially local, such that in most images signal is at one\nlocation while the noise models tend to overfit to is somewhere else.\nIf so, then generalization should improve if models are given additional\nsupervision (i.e., a mask identifying salient regions) that specifies where the signal is and is not.\nThis paper designs the Actdiff loss to realize this intuition.\n\n(approach)\nActdiff:\n1) The Actdiff loss requires a mask that highlights areas of the input\nimage which have signal and not distractor regions. It extracts features from\nthe original input image and its masked version then encourages the two features\nto be similar at every layer of the CNN using an L2 loss.\n\nActdiff is compared to 5 other methods including a reconstruction loss and Gradmask (previous work).\n\n(evaluation - synthetic dataset)\nA synthetic dataset is constructed for a simple binary classification task based on the presence of simple shapes.\nTwo patterns can predict the correct class at train time, but one of the patterns is removed at test time so additional information (masks in this case)\nis required to specify which pattern the classifier should use.\nAll the losses except Actdiff achieve at best 50% accuracy on the val set, but Actdiff gets 80% or more accuracy on 3 of 4 tested model variations.\nThis shows that Actdiff can effectively introduce the relevant masking information.\n\n(evaluation - Medical Segmentation Decathalon)\nThis dataset provides 3 segmentation tasks (Liver, Cardiac, Pancreas), including ground truth masks for those images.\nAll 6 methods outperform all the others at least some of the time.\nThe conclusion is that adding mask information using Actdiff doesn't improve segmentation performance.\n\n(evaluation - Multi-Site dataset)\nA final task tries to construct another synthetic dataset out of real X-ray images\ncollected at two different places.\nThe train set is largely from one place and the test set is mostly from the other place, and masks are constructed so Actdiff can try to eliminate this bias.\nActdiff causes a negligible increase in performance.\n\nThe paper concludes that signals CNNs tend to fit to in this type of data are not very spatially distinct.\n\n\nStrengths\n---\n\nThere is some novelty in the approach. The actdiff loss makes sense and masking + activation mapping have not been tried together before to my knowledge.\n\nExperiments follow a logical progression, starting by verifying the idea on a synthetic dataset, then moving to real data, and then evaluating on a half-synthetic dataset designed to debug the approach.\n\nExperiments average over many random initializations.\n\nThe paper embraces its negative result.\n\n\nWeaknesses\n---\n\nThe approach is not very compelling to me:\n* Implicit in this paper is that any information outside the mask is a distractor and any information inside is not a distractor. Why should the particular masks chosen for the experiments have this property? How can an expert know which features a model will find useful?\n\nThe paper's novelty is somewhat limited. The idea of regularizing using saliency maps has been explored and even applied to medical data like the MSD used here in Gradmask (one of the strong baselines this paper compares to). Activation matching is also common (e.g. [1]), though it has not been combined with masking before.\n\nWhile the weights applied to the various losses are provided, it's not clear how they were tuned. In this case there may be lots of competing losses, so it's important to tune the weights somehow to ensure the tradeoff between losses is optimal.\n\nThe results (and the conclusions) suggest Actdiff is not very effective at increasing generalization. Table 2 reports test results on all datasets. In that table, each loss outperforms all the other losses in at least two cases (a case is a model-dataset pair).\n\n[1]: Gatys, Leon A. et al. “A Neural Algorithm of Artistic Style.” ArXiv abs/1508.06576 (2015): n. pag.\n\n\nMissing experiments:\n\n* In the Multi-Site experiment, compare to what happens when the circular mask is applied to all images and not just those from one site. This is a necessary control to be sure that any benefits from masking are due to domain transfer and not other regularization effects. Either conclusion could be useful, but it would be nice to know.\n\nPresentation weaknesses:\n\n* In the synthetic dataset the model cannot tell the difference between correct and incorrect signals at train time. Therefore, I think there's no way for some of the baselines (plain classifier, autoencoder) to generalize correctly. Is that right? If so, it should be clear that comparisons to these baselines are not fair when discussing the synthetic evaluation in section 4.\n\nMissing details / points of clarification:\n\n* What is the Conv AE? I assume it is a CNN based autoencoder of some sort. A detailed description of the non-standard architectures would be useful for reproducibility, though probably only in the appendix.\n\n* What are the lambda hyperparameters? I assume these are weights on the corresponding loss terms, but this is never made explicit.\n\n* How does f(.) relate to the function o_l(.)? Is o_l(.) an intermediate step in f(.)?\n\n* The MSD dataset is not clearly described. Is this a classification dataset where classes are different diseases? What do the ground truth masks capture?\n\n* I think only Conv AE and UNet contain reconstruction losses. This presentation is a bit confusing since reconstruction loss was presented as another loss and it shows up in the tables implicitly based on the architecture being compared.\n\n\nSuggestions\n---\n\n* This paper would be a bit more convincing if it started with a concrete example of the problem illustrated on some dataset (e.g., maybe an example from Gradmask). That may also help drive intuitions later on in the paper.\n\n\nPreliminary Evaluation\n---\n\nClarity: The paper is fairly clear.\nQuality: Quality is mixed. Lots of relevant experiments are reported but they don't support clear conclusions and I'm not sure how well the models were tuned.\nOriginality: There is some novelty, but it is limited as discussed above.\nSignificance: I see limited significance.\n\nFor me this paper requires special scruitiny because it presents a negative result. Here are some factors that come to mind when thinking about whether to publish a negative result:\n* Is the approach compelling? - This approach is not very compelling (e.g., comments about limited novelty and lack of concrete examples to boost intuition).\n* Are the experiments thorough? - The experiments could be significantly more thorough (e.g., comments about tuning lambda).\n* Will readers learn something useful? - This paper may help researchers trying to leverage similar intuitions, but it won't be very useful outside this audience.\n* Does the paper present experiments that promote deeper understanding of why the approach failed? - This paper makes significant reasonable steps in that direction with sections 4 and 6, but I was still a bit dissapointed with the conclusions of these sections.\n* Does the paper discuss alternative approaches that were investigated? - Many alterantive approaches were considered and their performance reported.\n\nOverall I think this paper is close but fails to meet the bar because it does a bit worse than expected on most criteria above.\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "N/A", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}, {"review_id": "r1lEgf-2YB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper considers how we can train image classification models so that they can ignore task irrelevant features.\nFor this purpose, the authors considered a situation where task relevant parts of the images are annotated as masks by the human experts.\nThe authors then proposed using Actdiff loss, Reconstruction loss, and Gradmask loss that are designed to suppress the effect of irrelevant features. \n\nI think the problem considered in this paper is interesting and important.\nAs the authors pointed out, the medical images taken from different hospitals may contain hospital-specific features which is irrelevant to the targeting task.\nThus, we need a way to train image classifiers that can ignore such irrelevant features.\n\nMy major concern on this study is the experiments.\nThe authors mention that part of their methods sometimes did not work (see below).\nThis makes the effectiveness of the proposed losses a bit questionable.\nIf I understad correctly, in practice, the users need to tune the weights of several losses carefully until they can obtain a good model.\nIf this is the case, I am not very sure if the proposed losses are essential, or tuning a right weight can occasionally provide good models.\n\n* (Sec4) \"Gradmask proves to be too powerful a regularizer for this task, and never produces a model with good generalization performance.\"\n\n* (Sec5) \"For the Cardiac dataset, the best performing method was classification with gradmask for the Convolutional AutoEncoder and with actdiff for the UNet, and each achieve similar performance.\"\n\n* (Sec5) \"The gradmask and combination actdiff and gradmask models improved over baseline, but there is no clear reason why this would be true from the saliency maps.\"\n\n* (Sec6) \"Both actdiff and gradmask improve performance of the model, but only actdiff scores above chance, and performs similarly to a model trained with the areas outside of the mask completely removed.\"\n\n### Updated after author response ###\nThe authors have tried to address my concern by re-running the experiments, which I greatly appreciate.\nHowever, the effectiveness of the proposed approach is not convincing enough.\nI expect the authors to design the more effective experiments in the future version.\nIt would be great if the authors can clarify and demonstrate which regularization is helpful under which circumstances and why, and when it is not.", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "N/A", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper studies the effect of training image classifier with masked images to exclude distraction regions in the image and avoid formation of spurious correlation between them and predicted labels. The paper proposes actdiff regularizer and demonstrates that it prevents such overfitting phenomenon on synthetic data.  However, there was no success on real data. This is important as it shows that the improvement reported in some saliency-map based approaches in the literature may be due to other regularization effects such as cutout.

This was a unique submission in my batch, as it embraces its negative results. Among our internal discussions, all reviewers that and we all believe that negative results are important and should be encouraged. However, in order for the negative results to be sufficiently insightful for the entire community, they need to be examined under well-organized experiments. This is the aspect that the reviewers think the paper needs to improve on.  In particular, R2 believes the paper could consider a larger set of possible regularizations as well as a broader range of  applications. The insights in such setting may then lead to solid insights on why the current approaches are not very helpful, and in which direction the follow-up researches should focus on.
Decision: Reject