Does enhanced shape bias improve neural network robustness to common corruptions?
Authors: Chaithanya Kumar Mummadi
Ranjitha Subramaniam
Robin Hutmacher
Julien Vitay
Volker Fischer
Jan Hendrik Metzen
Venue: ICLR 2021 Poster (2021)
Keywords: neural network robustness
shape bias
corruptions
distribution shift

Abstract: Convolutional neural networks (CNNs) learn to extract representations of complex features, such as object shapes and textures to solve image recognition tasks. Recent work indicates that CNNs trained on ImageNet are biased towards features that encode textures and that these alone are sufficient to generalize to unseen test data from the same distribution as the training data but often fail to generalize to out-of-distribution data. It has been shown that augmenting the training data with different image styles decreases this texture bias in favor of increased shape bias while at the same time improving robustness to common corruptions, such as noise and blur. Commonly, this is interpreted as shape bias increasing corruption robustness. However, this relationship is only hypothesized. We perform a systematic study of different ways of composing inputs based on natural images, explicit edge information, and stylization. While stylization is essential for achieving high corruption robustness, we do not find a clear correlation between shape bias and robustness. We conclude that the data augmentation caused by style-variation  accounts for the improved corruption robustness and increased shape bias is only a byproduct.

Published as a conference paper at ICLR 2021
DOES ENHANCED SHAPE BIAS IMPROVE NEURAL NETWORK ROBUSTNESS TO COMMON CORRUPTIONS?
Chaithanya Kumar Mummadi
Ranjitha Subramaniam
University of Freiburg
Department Computer Science
Bosch Center for Artificial Intelligence Cha thanyaKumar Mummad's @ @bosch. com
TU Chemnitz
ran weviviiiiiiiin 00 @gma i .om com
Robin Hutmacher
Julien Vitay
Bosch Center for Artificial Intelligence Robin. Hutmacher bosch. com
Department of Computer Science TU Chemnitz julien. www.evvvaaaiiaaaaaii.... www.:attni...ttcctttttteeeetet de
Volker Fischer
Jan Hendrik Metzen
Bosch Center for Artificial Intelligence Volker. Folker.Fiscee Fww.cerrdceeeee bos ch. com
Bosch Center for Artificial Intelligence
JanHendrik www.Meteeeeeee.mm bosch. com
ABSTRACT
Convolutional neural networks (CNNs) learn to extract representations of complex features, such object shapes and textures to solve image recognition tasks. Re- cent work indicates that CNNs trained on ImageNet are biased towards features that encode textures and that these alone are sufficient to generalize unseen test data from the same distribution as the training data but often fail to gener- alize -iiit--diittitttion data. It has been shown that augmenting the training data with different image styles decreases this texture bias in favor of increased shape bias while at the same time improving robustness to common corruptions, such noise and blur. Commonly, this is interpreted as shape bias increasing corruption robustness. However, this relationship only hypothesized. We per- form a systematic study of different ways of composing inputs based on natural images, explicit edge information, and stylization. While stylization essential for achieving high corruption robustness, do not find clear correlation be- tween shape bias and robustness. We conclude that the data augmentation caused by style-variation accounts for the improved corruption robustness and increased shape bias is only byproduct.
INTRODUCTION
As deep learning is increasingly applied to open-world perception problems in safety-critical do- mains such as robotics and autonomous driving, its robustness properties become of paramount im- portance. Generally, lack of robustness against adversarial examples has been observed (Szegedy) et al. 2014 Goodfellow et al. (2015). making physical-wort
adversarial attacks on perception sys- 2017 Eykholt et al. 2018 Lee & Kolter) 2019). In this work, we
tems feasible (Kurakin focus on a different kind of robustness: namely, robustness against naturally occurring common im- age corruptions. Robustness of image classifiers against such corruptions can be evaluated using the ImageNet-C benchmark (Hendrycks & Dietterich, 2019). in which corruptions such as noise, blur, weather effects, and digital image transformations are simulated. [Hendrycks Dietterich) (2019) observed that recent advances in neural architectures increased performance undistorted data without significant increase in relative corruption robustness.
One hypothesis for the lack of robustness is over-reliance non-robust features that generalize well within the distribution used for training but fail to generalize to out-of-distribution data. [lyas
Equual contribution.

---

Published as a conference paper at ICLR 2021
Higher the better
Loweri the better
Higher the better
Higher 1 the better
Trained on standard images
92 88 87
Increase in Reasons for shape the gap?
shape based evaluation
46
previous work
Accuracy
34
34
23
standard styliized images
11
validation data
patch shuffled 4x4
texture-shape cue conflict
gaussian blur (severity level 4)
ours
standard Images + edge maps
Figure Illustration of the effect of different training augmentations. While both style-based (Geirhos et al.] (2019) and edge-based augmentation (this paper) reach the same validation accuracy, edge-based augmentation shows a stronger increase in shape bias as evidenced by lower accuracy on patch-shuffled images and higher rate of classifying according to the shape category for texture- shape cue conflicts. Nevertheless, only style- based augmentation shows a considerable improvement against common corruptions such as Gaussian blur. This challenges the hypothesis that increased shape bias causes improved robustness to corruption.
ct al. 2019) provide evidence for this hypothesis on adversarial examples. Similarly, it has been hypothesized that models which rely strongly texture information are more vulnerable to com- mon corruptions than models based on features encoding shape information (Geirhos et al.] 2019] Hendrycks & Dietterich] 2019). Alternative methods for increasing corruption robustness not mo- tivated by enhancing shape bias use more (potentially unlabeled) training data (Xie et al.) (219)) or use stronger data augmentation (Lopes et al. [019 Hendrycks* et al. (2020). Note that meaning of "shape" & "texture" is built on the definitions by Geirhos et al. (2019).
In this paper, we re-examine the question of whether increasing the shape bias of model actually helps in terms of corruption robustness. While prior work has found that there are training methods Diet- that increase both shape bias and corruption robustness (Geirhos al. 2019 Hendrycks terich (201)), this only establishes a correlation and not causal relationship. To increase the shape bias, Geirhos et al.) (2019) "stylize" images by imposing the style of a painting onto the image, leaving the shape-related structure of the image mostly unchanged while modifying texture cues so that they get largely uninformative of the class. Note that image stylization can be interpreted as specific form of data augmentation, providing alternative hypothesis for increased corruption robustness which would leave the enhanced shape bias as a mostly unrelated byproduct.
In this work, we investigate the role of the shape bias for corruption robustness in more detail. We propose two novel methods for increasing the shape bias:
auxiliary dataset which encourages learning shape features. In contrast to Geirhos et al. (2019) that use stylized images, this dataset consists of the edge maps for the training images that generated using the pre-trained neural network of Liu et al.] (2017) for edge detection. This method maintains global object shapes but removes eeettte-rllted information, thereby encouraging learning shape-based representations. In addition to pre-training on edge maps, we also propose style randomization to further enhance the shape bias. Style randomization is based upon sampling parameters of the affine transforma- tions of normalization layers for each input from a uniform distribution.
Similar to Geirhos et al. 22019), we pre-train the CNN
Our key finding is summarized in Figure [ While pre-training on stylized images increases both shape bias and corruption robustness, these two quantities are not necessarily correlated: pre-training on edge maps increases the shape bias without consistently helping in terms of corruption robustness. In order to explain this finding, we conduct systematic study in which we create inputs based on natural images, explicit edge information, and different ways of stylization (see Figure 2 for an illustration). We find that the shape bias gets maximized when combining edge information with stylization without including any texture information (Stylized Edges). However, for maximal corruption robustness, superimposing the image (and thus its textures) on these stylized edges is required. This, however, strongly reduces shape bias. In summary, corruption robustness seems to benefit most from style variation in the vicinity of the image manifold, while shape bias is mostly
2

---

Published as a conference paper at ICLR 2021
Intra Stylized [I-SIN]
Intra Stylized Edge Superposition (I:SE)
Natural (IN)
Edge
Stylized (SIN)
Stylized Edge (SE)
0.5 SE
Overview of content and stylization variants used in this paper: Content is natural image (IN) or an edge map (E). Content is stylized in three different ways: No stylization, style transfer with Painter by Numbers as style source as proposed in Geirhos al. (2019) (SIN and SE), style transfer with different in-distribution image as style source (I-SIN and I-SE). Additionally, we show a superposition (SE+IN) between natural (IN) and stylized edge image (SE).
Figure 2:
unrelated. Thus, image stylization is best interpreted strong data augmentation technique that encourages robust representations, regardless whether these representations are shape-hased or not.
Moreover, present results for setting where we fine-tune only parameters of the affine transfor- mation of a normalization layer on the target distribution (stylized or corrupted images, respectively) for CNN trained on regular images. Surprisingly, this is already sufficient for increasing the shape bias/corruption robustness considerably. We conclude that CNNs trained on normal images do learn shape-basse based features and features robust corruptions but assign little weight them. It may thus be sufficient to perform augmentation in feature space (extending) Nam et al. ;: Li et al. (2020) so that higher weights are assigned features that are robust to relevant domain shifts.
2 RELATED WORK
Texture-vs-Shapp Bias Geirhos al. (2019) and Baker et al. (2018) hypothesized that CNNs tend to be biased towards textural cues rather than shape cues. This line of research is further supported by Brendel & Bethge 2019), where the authors show that BagNets, Deep Neural Networks (DNN) trained and evaluated only on small restricted local image patches, already perform reasonably well on ImageNet. Similarly, Yin al.) (2019) and Jo Bengio (2017) showed using a Fourier space analysis that DNNs rely surface statistical regularities and high-frequency components. The eexture-v-sshape bias can be quantified by evaluating a network either on images with texture-shape images which were patch-wise shuffled (Luo et al. 2019).
cue conflict (Geirhos al.) 2019)
Robustness Against Common Corruptions Common corruptions are potentially stochastic image transformations motivated by real-world effects that can be used for evaluating model robustness. Hendrycks & Dietterich (2019) proposed the ImageNet-C dataset that contains simulated corrup- tions such as noise, blur, weather effects and digital image transformations. Geirhos al. (2018) showed that humans are more robust to image corruptions than CNNs.
Approaches to improve corruption robustness include data augmentation (Lopes al. 2019 Yun et al. 2019 Hendrycks* et al., 22200 Cubuk et al. 2019), self self-training with more training data Xie
al. 2019 novel architectures and building blocks (Zhang 2019 Hu al. 22018), and changes in
the training procedure (Hendrycks et al. 2019 Rusak et al. 2020 Wang et al. 2019). Motivated by
the texture-vs-shape hypothesis, Geirhos al. 2019) and MndMaecceelin
al. (2019) train their network
on stylized version of ImageNet. The idea is that style transfer removes textural cues and models trained on stylized data thus have rely more on shape information. The observed increase in corruption robustness on this stylized data was attributed to the shape bias. In this work, we provide evidence that contradicts this claim.
Similar to training on stylized images, Style Blending (Nam et al., 2019) employs style transfer in latent space by interpolating between feature statistics of different samples in batch. [ et al. (2020) extend this idea and use feature space blending along with label interpolation. Hendryeks et al. (2019) considers sepe-supe training with the prediction of image rotations as an auxiliary task. The authors argue that predicting rotation requires shape information and thus improves robustness. Similarly. (Shi et al. (2020) proposes Dropout-like algorithm to reduce the texture bias and thereby increase the shape bias improve model robustness. However, the authors also discuss that "sweet
3

---

Published as a conference paper at ICLR 2021
spot" between shape and texture is needed for the model to be robust for domain generalization. With Patch-wise Adversarial Regularization, Wang et al.] (2019) try to penalize reliance on local predictive representations in early layers and encourage the network to learn global concepts. Other augmentation techniques that aim to improve common corruption robustness are PatchGaussian (oppes et al.] 2019), CutMix (Yun et al.) 2019), AugMix (Hendryeks* at. 2020), and RandAugment Cubuk et al. 2019). At this point, it remains unclear whether the increase in robustness caused by these augmentations is due to learning fundamentally different representations such as more shape- biased ones or to more incremental improvements in feature quality.
Edge-based Representations A classical method for extracting edge maps is the Canny edge ex- tractor Canny (1986). More recent approaches use DNNs (Xie & Tu. 2015; Liu et al. (017)) (see Figure Alp. Geirhos et al. (2019) evaluate their shape-biaseds models on edge maps obtained with a Canny edge detector. ImageNet-Sketch (Wang et al., (019) is newly collected sketch-like dataset matching the ImageNet validation dataset in shape and size. It is used to evaluate generalization to domain shifts. In contrast to these works, we generate the edge-based representations with an edge detector using Richer Convolutional Features (RCF) (Liu et al. (017) (see Figure and use them explicitly for training. We provide evidence that edge-based representations enhance the shape bias, through an evaluation on images with induced texture- shape cue conflitt and match-shuffler images.
LEARNING SHAPE-BASED REPRESENTATIONS
Similar to Geirhos et al. (2019), we aim to enhance the shape bias of a network so that it bases its decision more on shape details than the style of objects encoded in textures. While Geirhos et al. (219) augment training data with different styles (stylization), thereby making texture cues less predictive, we extract edge information (edge maps) from the training images to maintain explicit shape details and remove texture-related information completely. Here, we consider grayscale in- tensity edge maps rather than separate edge maps for each color channel. We propose to train CNNs using the edge maps in addition to the standard training data to learn shape-based representations for more effective shape-based decision-making
Besides training on the dataset with explicit shape cues, high capacity networks learn different fea- ture representations when trained jointly on datasets from different distributions. Despite edge maps encouraging CNNs to learn shape-based representations, we observe that the network learns to en- code features with texture details when introduced the standard image data during training. We propose here to further restrain the network from learning texture details on standard image data. We diseuss below the extraction of edge details from images to create the edge map dataset and explain the technique to reduce the texture bias of the CNN.
Edge dataset Given standard image dataset, we construct a new dataset with edge maps (named the Edge dataset) by extracting the edge details of each image. The edge details are extracted by the
CNN-based edge detector using richer convolutional features (RCF) proposed in Liu et al. (017
RCF network produces single-channel edge map that contains the pixel values between [0, 255]. We convert the non-bine -on-binary edge map into a binary map with values in 255) using a threshold of 128 and transform it into 3-channel RGB edge map by duplicating the channels, we use the edge maps as a direct input to train the CNNs. The edge maps from the Edge dataset are used as input and can be independently used train or evaluate CNNs without necessarily being combined with the standard image data. Please refer to Section A. for the details of RCF network.
Style Randomization (SR) While using dataset with explicit shape cues enhances shape- representations, we propose to further reduce the texture bias of the network when training on stan- dard images. It shown in the literature of style transfer (Dumoulin et al. 2016 Huang & Belongie, 2017) that the statistics of feature maps (c.g., mean and standard deviation) of a CNN effectively capture the style of an image and changing these statistics would correspond to a change the style of an image. SIN dataset is generated using such style transfer technique and shown to reduce the texture bias of the networks. Inspired by this observation, we propose a simple technique to effec- tively reduce the texture bias using the feature statisties when being trained on standard training data. We modify the style of an image in the feature space so that the network becomes style-invariant. In particular, we randomize the style details, i.e. feature statistics, of an image during training such that the network can not rely on the texture cues. A similar approach named Style Blending (SB) is proposed in Nam et al. (2019) which randomizes the style information by interpolating the feature
4

---

Published as a conference paper at ICLR 2021
statistics between different samples in nitii-btch.. We propose here slightly different approach to make the network invariant to style information. Instead of interpolating the statistics of similar distribution of data ie, training samples, completely randomize the feature statistics (mean and standard deviation) by randomly sampling them from an uniform distribution. Considering X; as the tth feature map of an intermediate layer in CNN, and μt; & og as the feature statistics of X;, the style randomized feature map X; is defined as:
Xi - Mi + t/s X; = o,
(1)
d:
where o, Uniform(0.1,1) and i. Uniform(-1, 1). These specific choices of sampling for 5; and P. were found to perform best on our evaluations. The style transfer technique described in Huang & Belongie (2017) replaces the feature statistics of content image with the statistics of a desired style image to change the style. Similarly, we replace the statistics of content image with random statistics change the style information. Training the network with SR reduces the texture bias and improves shape-based decision making. An advantage of SR over SB is that the feature statistics are sampled from a different distribution than the training data, that encourages learning representations to generalize better to out- of-distributton data. We show in Section that SR outperforms SB and aids the network induce stronger shape-basee based representations.
EXPERIMENTAL SETTINGS
Dataset We use a subset of 20 classes from ImageNet dataset (ImageNet20, or IN) that are chosen randomly, to study the role of shape bias towards corruption robustness; the main reason being that extensive experiments on this dataset are feasible with limited computation. Details about this dataset can be found Section A.2 The Edge dataset of IN (referred to as E) is generated as described in Section 3
Stylization variants In addition to enhancing the shape bias using the edge maps, we further study the contribution of different factors of Stylized ImageNet (SIN) (Geirhos et al.) (019)] to gain insights on its improved performance on corruptions. We break down SIN into different factors to understand corruption robustness. We segregate the factors that jointly generate the stylized images and the factors that are hypothesized to improve corruption robustness. These include i) shape bias of the network. ii) styles that are transferred from paintings and iii) statistics of natural images from IN. The role of shape bias is studied using the Edge dataset (E) proposed Section 3 Other variants study the role of the remaining factors and are explained below:
their influence
Role of stylization We create Stylized Edges (SE, see Figure 2 for which the styles from the paint- ings are transferred to the edge maps of Edge ImageNet20 (E). Here, we study the significance of stylization without the presence of the statistics (texture details) of natural images.
Role of out-of-distribution styles SIN is generated by transferring the styles from -uiti--dit images, namely paintings. We create its variant called Intra-Stylized IN (I-SIN, see Figure 2 for which in-distribution images from IN are chosen randomly to transfer the styles. We also generate Intra-Stylized Edges (I-SE) where the image styles of IN are transferred to the Edge dataset E.
Role of natural images statistics The above variants of E SE test the role of shape and stylization without retaining texture cues of natural images. We create another variant called Superposition (SE+IN, see Figure 2) that interpolates images ISE from SE with images INN from IN to embed the statistics (texture details) from natural images: /SEAIN == (1 - a:) ISE + JIN. We set 0.5.
These different stylized variants including E allow insights into the interplay between shape bias and corruption robustness. For simplicity, we term the networks that are trained on certain dataset using the name of that dataset. For example, network trained on Stylized Edge (SE) is referred to as SE. The evaluation of SIN and I-SIN reveals the significance of the choice of styles and evaluation of Edge (E) indicates the role of the shape bias for corruption robustness. SE explains the importance of stylization and finally SE+IN allows to understand the importance of natural image statistics that are preserved in SIN and I-SIN but are missing from SE. Table 3 provides an overview of the input image compositions of different variants that are described above.
5

---

Published as a conference paper at ICLR 2021
shuffled image patches x 4 acc(%%
shape based cue conflict #400
No styling
style blending
style randomization
No styling
style blending
style randomization
Network
IN
67.22
51.34
41.97
63
82
86
SIN
38.46
36.96
34.95
144
155
156
E
34.11
33.95
28.43
155
166
193
Table 1: Comparison of different feature space style augmentation methods on 4 4 shuffled image patches and number of shape based predictions in texture-shape cue conflict images. Evaluation of shuffled patches is conducted on 598 correctly classified validation images by all the networks.
Network details We employ a R architecture with group normalization (Wu & He, 2018) and weight standardization (Qiao et al. 2019). We include SR described Section in the architec- ture. ResNet18 contains 4 stages of series of residual blocks and SR is inserted before every stage. We train R8 on different datasets and their variants described above. IN and SIN are consid- ered as baselines. We show that E possesses more global shape details of the objects whereas SIN demonstrates little or no texture bias for decision making. Both these datasets are complementary to each other and further enhance shape- based predictions when combined (termed as E-SIN). Note that SR is used to reduce texture bias and IN contains by far the strongest texture cues. Hence, SR applied only the training samples of IN but on other dataset variants. Nevertheless, SR applied on other dataset variants found no differences in the results.
Training details Network on differnt dataset variants except IN are trained in stages. The first stage begins with training the network on the respective dataset variant (e.g: E) for a total of 75 epochs starting with learning rate of 0.1, which dropped at the 60th epoch by factor 10. In the second stage, the networks are then fine-tume tunet on the respective dataset along with IN (e.g: E & IN) for another round of 75 epochs starting with learning rate of 0.01, later reduced 0.001 the 60th epoch. On the other hand, the network on IN is trained for 100 epochs with a learning rate of 0.1, reduced to 0.01 and 0.001 the 60th and 90th epochs, respectively. samples with the SGD optimizer and weight decay 10-1
use a batch size 128
During the fine-tuning stage, we freeze the first convolutional layer and the first normalization layer's affine parameters. We observed that freezing these two layers demonstrate more global shape bias than fine-tuning all the layers in the network. During fine-tuning, the networks receive an equal number of training samples from both datasets (e.g: 128 samples from E and 128 samples from IN in a different than the distribution of images from other datasets. Fine-tuning the network on inputs with different distributions results in degradation of the performance. In other words, the datasets E, SE and I-SE do not preserve natural image statistics and degrade task performance when finetuning along with clean images. Hence, we weigh the loss of training samples of edge maps from E, SE and I-SE when fine-tuning along with IN. The loss between training samples is weighted as follows: style variants SIN, I-SIN that better
Note that the data distribution of edge maps from the datasets E, SE and I-SE
Loss L LIN + eedggempps with > 0.01. Finetuning preserve natural image statistics does not affect classification performance significantly, hence is not used. Larger À preserves the shape bias but affects the clean accuracy while smaller À reduces shape bias of the network. In case of E-SIN, We fine-tune the network that is pre-trained on the first stage of training with SIN and IN in the second stage and show that such setup further improves shape-based predictions. All ResNet18 models have validation accuracy of about 87% on IN.
5 EVALUATION OF SHAPE BIAS
In this section, we evaluate different methods in terms of their shape bias using two different evalu- ation criteria Shuffled image patches and Texture-shape shape cue conftict that are described below.
Shuffled image patches: Following Luo al. (2019), manipulate images by perturbing the shape details while preserving the local texture of the objects. We divide an image into different patches of size n with (2,4,8) 4,8) and randomly shuffle the patches as shown in Figure

---

Published as a conference paper at ICLR 2021
shuffled image patches acc( %)
texture-shape cue conffict results
Network
2 x 2
4 x 4
8 x 8
shape #400
shape #100
texture #100
IN
78.57
41.93
31.21
86
18
20
SIN
75.78
35.56
18.48
156
32
2
E
73.29
28.42
11.18
193
46
15
SE 66.77
28.73
12.89
224
55
6
E-SIN
71.12
23.76
10.25
234
58
6
Table Comparison of models trained different datasets on shuffled image patches and number of texture-shape cue conflict predictions based on shape and texture labels. Evaluation of shuffled image patches conducted on 644 validation images that are correctly classified by all the networks.
Larger n corresponds to more distorted shapes. The performance of networks that rely more on shape is expected deteriorate more strongly as the number of patches increases. We conduct this evaluation only on the ImageNet20 validation images that were correctly classified by all the networks that are selected for comparison.
A2a
Texture-shape cue conflict: The cue conflict image dataset proposed by Geirhos et al. (2019) con- sists of images where the shape of an object carries the texture of different object. For example, the object cat holds the texture of e elephant as shown in Figure A26 Each image in the dataset contains two class labels: labels with respect to shape and texture. The evaluation is carried out to test the network's bias towards shape or texture. Networks with strong shape bias will exhibit higher accuracy according to the shape label while networks with texture bias will have higher accuracy for texture-based label. The original dataset contains total of 1280 cue conflict images designed for the evaluation of the networks trained on the entire ImageNet dataset. 400 of these images have classes (shape labels) present in ImageNet20. A subset of 100 instances (20 instances from 5 dif- ferent categories) from the selected images also has a texture label that belongs to ImageNer20 (see Figure A2b bottom). The remaining 300 images with texture labels that do not belong the classes of Im2geNet20 are not considered for texture-based classification.
Results The results Table [ compare style blending (SB) (Nam et al., 2019), style randomization (SR) (Section 3) and no styling in feature space for networks trained on IN, SIN and E. In terms of performance 4 shuffled patches, SB performs worse than styling, and SR performs even worse than SB. This indicates increasing shape bias from no styling over SB to SR. This finding is re- inforced by an increasing number of images classified according to the shape label for texture-shan shape cue conflict images from no styling over SB to SR. Similarly, when comparing different training datasets, SIN results in stronger shape bias than IN, and E exhibits stronger shape bias than SIN.
In Table 2 we compare additional networks, all with SR enabled. Here, we again see a consistent trend of increasing shape bias from IN over SIN to E. Moreover, stylized edges (SE) further increase shape bias than E. Lastly, E-SIN improves shape bias even slightly beyond SE. In summary, we can see a clear increase in shape bias for the methods proposed this paper over IN or SIN. Next, we investigate to which extent this also results in an increased corruption robustness.
INFLUENCE OF SHAPE BIAS ON COMMON CORRUPTIONS
We compare different networks in terms of their corruption robustness. Figure shows the accuracy of different networks for two types corruptions: Gaussian noise and frost (refer Tigure/AS) for all corruptions). Table presents the corruption accuracy averaged over 15 ImageNet-C corruptions along with shape and texture results on the texture-shape cue conflict dataset. Generally, a CNN trained on IN performs poorly in terms of corruption robustness while SIN is relatively robust. On the other hand, E performs considerably worse than SIN and not consistently better than IN despite having stronger shape bias than SIN. Networks SE and E-SIN further increase shape bias but

---

Published as a conference paper at ICLR 2021
gaussian
100
100
90
90
Accuaayy5)
Accuracy55)
Intra serra-ttyliee CNN (1-SIN)
Intro-sty intre-stylized CNN (I-SIN)
Edge
30
Stylized edge CNN (SE)
30
Stylized edge CNN (SE)
strl--ttyizeedeooo edge
Intra-stylized
20
Edge Stylized CNN (E SIN)
20
Edge Stylized CNN (E-SIN)
10
CNN (SIN) (Previous work)
Stylized CNN (Previous work)
Superposition (SE+IN)
Superposition (SE+IN)
Severity
Severity
Figure 3: Classification accuracy of different networks on corruptions across severity levels. Severity 0 represents accuracy on clean validation data of IN. Severity levels 5 follow the corrup- tion parameters from Hendrycks & Dietterich (219)) and represent increasingly strong corruptions.
Input image composition
Natural image
Edge map
Style transfer
Shape #100
Texture #100
Mean corruption acc( (%)
Network
IN
X
X
39
64.69
SIN
/
X
/
34
2
77.64
E
X
/
X
46
15
62.01
SE
X
6
71.81
E-SIN
62
5
71.55
SE+IN
22
13
78.96
Table 3: Mean corruption accuracy (mCA) and texture/shape results texture-shape conflict dataset of different networks. mCA is the mean accuracy over 15 ImageNet-C corruption and sever- ities ranging from 1 to 5. Networks trained with style transfer augmentation perform better than those without and network trained on superpositioned images (SE+IN) yield best mCA.
still perform considerably worse than SIN in terms of corruption robustness. These results contradict the hypothesis that stronger shape bias results in increased corruption robustness.
The only method that slightly surpasses SIN in terms of corruption robustness is the superposition of SE with natural images (SE+IN). However, this method has relatively small shape bias. A common theme of SIN and SE+IN is that both exhibit properties a natural image but are strongly distorted by stylization (see Figure D.. We hypothesize that these methods correspond to strong augmentation methods that stay close enough to the data manifold while inducing high diversity in appearance and thereby encourage learning robust representations, which need not necessarily We extend these findings to larger datasets with 200 classes of ImageNet, deeper architectures like ResNet50, DenseNet121. MobileNerV2 and different normalization layers like BatchNorm Sec- tion A.5 Lastly, as can be seen from Figure 3 intra-stylization is nearly as effective as stylization based on paintings, implying that style need not necessarily be disstribuittnno for being useful.
shape-based.
ON THE ADAPTABILITY OF LEARNED REPRESENTATIONS
As seen in the previous section, style augmentation on natural images is important for the network to be able to generalize to different domains such common corruptions. We now study how easily
8

---

Published as a conference paper at ICLR 2021
Corruptions acc( %)
Cue Conffict
Speckle noise
Gaussian blur
SIN val ace(
shape #400
texture
Network
Frost
Pixelate
#100
IN
61.28
42.96
66.62
78.54
42.0
63
39
IN (fine-tuned)
82.7
77.3
81.02
87.02
68.0
130
13
E 67.76
44.48
61.04
70.94
62.3
193
15
E (fine-tuned)
80.18
71.74
73.7
74.78
72.4
222
Table 4: Mean corruption accuracy, SIN and cue conflict results of networks with & without ad- ditional fine-tuning of the affine parameters of normalization layers the respective comuptions. Fine-tuned networks perform significantly better, despite only the normalization layers are updated.
a pre-trained network can be adapted to a different distribution such as corruptions. Importantly, this uses the "unknown" distortion during training: this experiment is not meant practical procedure for the ImageNet-C benchmark but rather for understanding internal representations of network.
Chang et al. (2019) showed that domain-specific affine parameters in normalization layers es- sential when training network on different input data distributions jointly. We conduct a similar experiment with the key difference that our network is already pre-trained on IN/E and only the affine parameters of normalization layers are fine-tuned to fit the distribution of the respective target domain. First, we fine-tune affine parameters of the network on several ImageNet-C corruptions separately and evaluate the mean corruption accuracy on the same corruption across different sever- ity levels. As shown in Table 4 (left), performance on the corruptions can be greatly improved even with fixed convolutional parameters trained on IN/E by just tuning the affine parameters. Similarly, we also fine-tune the affine parameters of pre-trained CNN on SIN. Results in Table 4 (right) show not only improvement SIN validation accuracy but also improved shape-based classification results on sexppre-shape cue conflict images. These results suggest that the standard CNN encodes robust representations that can be leveraged when adapting affine parameters target domain.
CONCLUSION
We performed systematic empirical evaluation of the hypothesis that enhanced shape bias of neu- ral network is predictive for increased corruption robustness. Our evidence suggests that this is not the case and increased shape bias is mostly unrelated byproduct. Increased corruption robustness by image stylization is better explained as a strong form of augmentation which encourages robust representations regardless whether those are shape-based based on other corruption robustness is the main objective, one should not primarily focus on increasing the shape bias of learned representations. Potential future research directions will focus on understanding whether shape- biised representations offer advantages in other domains than corruption robustness (Heddyycks et al., 2020). Moreover, one could try devising stronger augmentation procedures in image or feature space based on our findings. Lastly, gaining better understanding of which kind of features (if not shape-basee based ones) are essential for corruption robustness an important direction.
We conclude that if
REFERENCES
Pablo Arbelaez, Michael Maire, Charless Fowlkes, and Jitendra Malik. Contour detection and hi- erarchical image segmentation. IEEE transactions pattern analysis and machine intelligence, 33(5):898-916, 2010.
Nicholas Baker, Hongjing Lu, Gennady Erlikhman, and Philip Kellman. Deep convolutional networks do not classify based global object shape. PLoS computational biology. 14(12): e1006613, 2018.
W. Brendel and M. Bethge. Approximating enns with bag-of local-features models works surpris- ingly well on imagenet. May 2019.

---

Published as a conference paper at ICLR 2021
John Canny. A computational approach edge detection. IEEE Transactions on pattern analysis and machine intelligence, (6):679-698) 1986.
Woong. Gi Chang, Tackgeun You, Seonguk Seo, Suha Kwak, and Bohyung Han. Domain-specifie batch normalization for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pp. 7354-7362, 2019.
Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V Le. Randaugment: Practical automated data augmentation with reduced search space. arXiv preprint (.Kiv:/999.137/111 2019.
Vincent Dumoulin, Jonathon Shlens, and Manjunath Kudlur. A learned representation for artistic style, arXiv preprint arXiv:1610.07629, 2016
Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Florian Tramer, Atul Prakash, Tadayoshi Kohno, and Dawn Song. Physical Adversarial Examples for Object Detectors. In 12th USENIX Workshop on Offensive Technologies, July 2018. URL httt:://atxii/// org/
abs/ abs/18077 0te//1807.07777 07769
Robert Geirhos, Carlos RM Temme, Jonas Rauber, Heiko H Schütt, Matthias Bethge, and Felix A Wichmann. Generalisation in humans and deep neural networks. In Advances in Neural Informa- fion Processing Systems, pp. 7538-7550, 2018
Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel. Imagenet-trained CNNs are biased towards texture; increasing shape bias im- proves accuracy and robustness. In International Conference on Learning Representations, 2019.
URL BRRLhtttp:///penner/// http://ppeeeeeeeeeiee.eeoooooooo
ftrrmformmmmmmmm By-
Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and Harnessing Adversarial Examples. In International Conference on Learning Representations (ICLR), 2015.
Dan Hendrycks and Thomas G. Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. International Conference on Learning Representations (ICLR), 2019.
wab//////aaa//ooeeeeeeoee/eeeeee 1903. 199./12201111111
Dan Hendrycks, Mantas Mazeika, Saurav Kadavath, and Dawn Song. Using self-supervised learning can improve model robustness and uncertainty. Advances Neural Information Processing Systems, pp. 15637-15648, 2019.
Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang. Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. The many faces of robustness: A critical analysis of out-of-disttibution generalization. arXiv preprint 2xXiv22006.16244 2020.
Dan Hendrycks*' Norman Mu Ekin Dogus Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshminarayanan. Augmix: A simple method to improve robustness and uncertainty under data shift. In International Conference on Learning Representations, 2020. URL https: ///ogereeviee..ntt/foororiioeeee
Jie Hu, Li Shen, and Gang Sun. Squeeze-and-exciaatin networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7132-7141, 2018
Xun Huang and Serge Belongie. Arbitrary style transfer in real-time with adaptive instance nor- malization. In Proceedings of the IEEE International Conference on Computer Vision, pp. 1501- 1510, 2017.
Andrew Ilas,, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Alcksander Madry. Adversarial examples are not bugs, they are features. In Advances in Neural Information Processing Systems, pp. 136-136, 2019.
Jason Jo and Yoshua Bengio. Measuring the tendency of cnns to learn surface statistical regularities. arXiv preprint arXiv: /. 1556, 2017.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world. International Conference Learning Representations (Workshop). April 2017.
10

---

Published as a conference paper at ICLR 2021
Mark Lee and J. Zico Kolter. On physical adversarial patches for object detection. Interna- tional Conference on Machine Learning (Workshop), 2019.
abs/
1906.11897
Boyi Li, Felix Wu, Ser-Nam Lim, Serge Belongie, and Kilian Q. Weinberger. On feature normal- ization and data augmentation, 2020.
Yun Liu, Ming-Ming Cheng, Xiaowei Hu, Kai Wang, and Xiang Bai. Richer convolutional fea- tures for edge detection. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3000-3009, 2017.
Raphael Gontijo Lopes, Dong Yin, Ben Poole, Justin Gilmer, and Ekin D Cubuk. Improv- ing robustness without sacrificing accuracy with patch gaussian augmentation. arXiv preprint DrXiv:1906.026/1 2019.
Tiange Luo, Tianle Cai, Mengxiao Zhang, Siyu Chen, Di He, and Liwei Wang. Defective convolu- tional layers learn robust cnns. arXiv preprint 19Xiv:9111008433 2019.
Claudio Michaelis, Benjamin Mitzkus, Robert Geirhos, Evgenia Rusak, Oliver Bringmann, Alexan- der Ecker, Matthias Bethge, and Wieland Brendel. Benchmarking robustness object detec- tion: Autonomous driving when winter is coming. arXiv preprint arXiv: 2019.
Hyeonseob Nam, HyumJae Lee, Jongchan Park, Wonjun Yoon, and Donggeun Yoo. Reducing do- main gap via style-agnostic networks. arXiv preprint arXiv:19100 1645, 2019.
Siyuan Qiao, Huiyu Wang, Chenxi Liu, Wei Shen, and Alan Yuille. Weight standardization. arXiv preprint arXiv: 10520, 2019.
E. Rusak, L. Schott, R. Zimmermann, J. Bitterwolf, O. Bringmann, M. Bethge, and W. Brendel. Increasing the robustness of dans against image corruptions by playing the game of noise. arXiv, Jan 2020. Uttt:::::://aaae www.eiii.orrogo.eeeeee wttpoorggaabb/ooooosoe 2001. 06057]
Baifeng Shi, Dinghuai Zhang, Qi Dai, Zhanxing Zhu, Yadong Mu, and Jingdong Wang. Informative dropout for robust representation learning: A shape-bias perspective. In International Conference on Machine Learning, pp. 8828-8839. PMLR, 2020.
Christian Szegedy. Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfel- low, and Rob Fergus. Intriguing properties of neural networks. In International Conference on Learning Representations (ICLR), 2014.
Haohan Wang, Songwei Ge, Zachary Lipton, and Eric P Xing. Learning robust global representa- tions by penalizing local predictive power. In Advances in Neural Information Processing Sys- tems, pp. 10506-10511 2019.
Yuxin Wu and Kaiming He. Group normalization. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 3-19, 2018.
Qizhe Xie, Eduard Hovy, Minh-Thang Luong, and Quoe V Le. Self-training with noisy student improves imagenet classification. arXiv preprint arXiv: 1.1.:229 2019.
Saining Xie and Zhuowen Tu. Holistically-nestedd edge detection. In Proceedings of the IEEE international conference computer vision, pp. 1303, 2015.
Dong Yin, Raphael Gontijo Lopes, Jon Shlens, Ekin Dogus Cubuk, and Justin Gilmer. A fourier per- spective on model robustness in computer vision. In Advances in Neural Information Processing Systems, pp. 13255-13265. 2019.
Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regularization strategy to train strong classifiers with localizable features. In Proceed- ings of the IEEE International Conference on Computer Vision, pp. 6023-6032, 2019.
Richard Zhang. Making convolutional networks shift- invariaat again. ICML, 2019.


Reviews:
[{"review_id": "nDxdwEoxdN", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper disproves the hypothesis that addressing shape bias improves robustness to corruptions of neural networks, which has been stated by the previous studies [1, 2]. The paper demonstrates that the degree of shape bias of a model is not correlated with classification accuracy on corrupted images via experiments. For the experiments, this paper presents two novel methods to encourage CNNs to be shape-biased: 1) edge dataset and 2) style randomization (SR). In the experiments, the authors train CNNs to be shape-biased to various degrees based on the proposed methods. Additionally, they compare test accuracies of the models to evaluate shape bias and robustness to corruption. In addition, this paper shows that through fine-tuning the affine parameters of the normalization layers, a CNN trained on original images can achieve comparable, if not better, performance than a CNN trained with data augmentation.\n\nPros:\n\n- The paper clearly demonstrates that shape bias is not correlated with robustness to corrupted images. The test accuracy on texture-shape cue conflict images effectively indicates the degree of a model’s shape bias. Also, the authors visualize the results that explicitly compare the models’ accuracy on corrupted images.\n\n- The authors present an interesting analysis towards Stylized ImageNet (SIN) dataset by separating the dataset into different factors that are used to generate SIN. According to this paper, the newly made datasets based on each factor leads a CNN to be shape-biased to various degrees. This provides insightful perspectives on shape bias and corruption robustness. Specifically, it is intriguing that a model trained on a stylized dataset using styles from in-distribution images achieves comparable performance compared to what uses styles from out-of-distribution images, such as paintings.\n\nCons:\n\n- The novelty of the two proposed methods, edge dataset and SR, is limited. First, the edge dataset is created based on an existing model. The authors simply convert the non-binary edge map into a binary one. Furthermore, there are not enough experiments or evidence that validate the effectiveness of this method compared to the original method. Additionally, SR is also similar to existing methods, except that they change the target distribution from training samples’ distribution to a uniform distribution.\n\n- According to this paper, a CNN trained on edge dataset is more shape-biased but achieves lower accuracy on corrupted images compared to a CNN trained on SIN. The authors state that the result indicates that shape bias is not correlated to corruption robustness.\nHowever, it seems to be unfair to compare edge dataset and SIN to disprove the correlation between shape bias and corruption robustness.\nEdge dataset, unlike SIN, does not contain any information except edges. Therefore, a CNN trained on edge dataset ‘learns only shape information, while a SIN-trained CNN learns other information as well, but ‘focuses more on the shape’. Since the edge dataset provides much less information of the original images, it is trivial that a CNN trained on edge dataset achieves lower performance compared to a SIN-trained CNN. \nThe previous approaches [1, 2] mentioned in this paper also imply that shape bias is basically related to how much the model focuses on shape information, not to how much shape information the model is given.\nTherefore, it would be more reasonable to compare a SIN-trained CNN with other CNNs that are trained on datasets containing a similar amount of information, but have different levels of concentration on shape information.\n\n[1] Geirhos et al., “ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness.” ICLR’19\n\n[2] Michaelis et al., “Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming.” ICLR’20\n\n\n--------------------------\nAfter rebuttal:\n\nThank you for the additional explanation.\n\nThe comments by the authors effectively address my concerns. Although the edge dataset and SR are quite similar to the existing methods, the authors clearly present the usefulness of them as well as their additional contributions. Also, additional training details support the adequacy of the comparison in the experiments. Therefore, I would like to increase my final score to 7: Good paper, accept.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "f4VyLoRijaG", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary of the paper:\nThis paper tries to study whether increasing shape-bias of a neural network trained with imagined will make it more robust to common corruptions such as gaussian noises.\nThe paper falsified this point by producing a data augmentation method which leads to more shape biased network yet less susceptive to common corruptios.\nThe paper further hypothesize that it’s the stylization augmentation that leads to increase robustness of the network by ablation studies.\n\nStrength:\nThe paper does provide a counter example to the common hypothesis that increase shape bias can lead to more robust network. This provides insight to future researches on understanding how shape-bias and texture-bias affects on neural network robustness. If the results of the counter example is reproducible and significance, then the claim is convincing and the paper did a good job verifying such hypothesis.\n\nWeakness:\n1. The experiment results seemed to be limited in this dataset. In order to make such general claim, I would expect results for different datasets (e.g. CIFAR), large scale datasets (e.g. the whole imagined), and different network and training procedure (e.g. not just resent).\n2. This might be some minor things, but it would be nice if there are statistic significance test for the results (or at least show the variance of couple runs). When I looked at the difference of the number, it would be nice that one can make sure such results is statistically significant with respect with difference runs.\n3. Most of the paper is very empirical, and there is little insight or theory or principal ways that organize the results. \n\n\nJustifications:\nWhile I do like the paper that provide insight and show negative results falsifying some prevalent claim, but the paper provides rather limited evaluation without theoretical insights. Since I’m not an expert in this field, I will recommend borderline scores to hear about the authors’ response.\n\n-------------------\nUPdate: the authors' reply address my concerns well, so I raise my rating to the acceptance side.", "extra_scores": {}, "extra_text": {}}, {"review_id": "amqbqo5Knsu", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 9, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "##Updated Review##\n\nI'd like to thank the reviewers for their responses, and for updating to the much clearer naming scheme for the different methodologies!  Much easier to follow with those names.\n\nI maintain that this is high quality novel work that contradicts a widely held belief within the field and as such is a clear accept.\n\n# Main Idea\nThe main idea is that training to reduce texture bias in convnets in favor of shape bias is often thought to cause the concomitant increase in corruption robustness, but this has not been tested directly.  The authors propose a systemic study of this relationship and conclude that both shape bias and increase corruption robustness are both byproducts of style-variation during training, that is they share a common causal mechanism, but that shape bias does not itself cause corruption robustness.\n\nThey accomplish this by creating a new augmented dataset which encourages learning shape features (created from the edge maps of the training images) but does not also induce robustness against common corruptions.\n\n#### Additional findings from the study:\nShape bias gets maximized when edge information and stylization are combined without including any texture information.\n\nCorruption robustness is maximized by superimposing the image (and its textures) on the above stylized edges.\n\nThey propose that corruption robustness seems to benefit most from style variation in the vicinity of the image manifold.\n\n# General Strengths\nI think the paper makes a compelling case.\n\nThe Geirhos result, while elegant and interesting, has always struck me as a roundabout way to get to the shape bias question.  These authors more directly attack it, and show that the relationship doesn’t hold, while showing the real. Value of the Geirhos result was in showing that manifold-local variation increased robustness (and also consequently shape bias).\n\nIn addition to the elegance of the result, the authors use of “texture randomization via texture feature randomization” is a an elegant (and much faster) implementation than regenerating an entire dataset with many different textures.   The insight is good, the image itself doesn’t need to be rerendered, but the networks interpretation of its texture needs to be scrambled.  This doesn’t restrict you to the statistics of any database on which you might extract textures\n\n# Weaknesses\nDoesn’t report I-SIN results across ImageNetC corruption dataset.\nWould have preferred to see the specific corruption type results broken out for more than just the two shown and the average robustness (the variation and performance on different types of corruption could be useful information).\n\nAlso make sure the names are consistent across graphs and paragraphs.  Not the best naming scheme (too many very similar acronyms which aren’t immediately evocative of the underlying point).", "extra_scores": {}, "extra_text": {}}, {"review_id": "SNoi8KcW7Q5", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper delves deeper into understanding shape-based representation of CNNs in an empirical way. Based on the stylized images, it proposes to use edge maps to more explicitly feed shape information to learning models. Besides, the common way to let models learn the shape-based representation is to train on the dataset contained the shape information while the texture information is severely distorted. The paper takes the point of changing the statistics of feature maps would result in style changes and proposed style-randomization to help CNNs better focus on shape information. Also, it connects the biasing degree on shape information of models with the defensive performance against common corruptions, like Gaussian additive noise, blur, and etc. An intuitive conclusion was drawn that there is no clear correlation between shape bias and robustness against common corruptions, and justified by extensive experiments. \n\nThe studied direction is important for understanding the learned representation of CNNs. The proposed points, using edge maps and style randomization, are simple and effective for making CNNs focus on shape information. Also, the discussion with the common corruptions is intriguing and helps us better understand the relationships between the shape-based representation and the robustness. \n\nThe experiments are technically sound and results are sufficient to support their arguments, though more analysis would add support to the claims. The experimental setups are described in detail that makes it easy to replicate the experiments. Overall this paper is easy to follow. The reviewer would encourage the author to share their codes and datasets. \n\n-------------------------------------------------------------------\nSome concerns and comments are listed below:\n\nSince most of the experiments are conducted on ImageNet-20 and ResNet-18 (appendix lists few results on ImageNet-200 and ResNet-50), the reviewer doubts if the conclusions are still valid on different datasets and architectures. Especially for the aspects of the architecture, different architectures have different inductive biases and may result in different phenomena. \n\nAccording to Table 2, 3 and discussion on the bottom of the 6th page, the Stylized-Edge data would increase shape bias larger than Edge. This is a little bit counter-intuitive due to Stylized-edge only contain more uncorrelated texture information. The reviewer wonders if the quality of the Edge data is not very high and result in the phenomenon. To verify this, the author better to do experiments on the high-quality edge data with human annotations or performing edge algs on simple data.\n\nTable 3 and A5 list the results of E-SIN and SE+IN. The reviewer wonders if there are any \"intermediate\" results such as E-IN, E+IN to better compare. Also, the reviewer is curious about the useful of superposition. Will the concat or mix data bring similar benefits, compared to superposition?\n\nCurrently, to test the degree of focusing on shape information, the author runs the random-shuffled images and tests on stylized-imagenet. This is good but the reviewer wonders if we directly test on the edge maps of validation sets.\n\nTo sum up, the reviewer thinks this paper would bring some new understandings to the community. \n\n--------------after rebuttal-------------------\n\nI've read all reviews and the rebuttal, and thank the authors for their efforts and extra experiments. I think it is ok to be accepted due to provide further understandings about the learning representation with solid experimental results.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This work investigates the recently proposed hypothesis that enhanced shape bias improves neural network robustness to common corruptions. Several interesting experiments are performed to better understand the contributing factors that lead to improved robustness of models trained with texture randomization. Of particular note, the authors design a data augmentation strategy that verifiably increases the shape bias of model, but for which corruption robustness is not improved. Reviewers agreed that this is an interesting counter-example to the shape-bias hypothesis and improves our understanding of why stylization improves robustness. Given the carefully designed experiments investigating an important topic I recommend accept.
Decision: Accept (Poster)