Original: I MAGE N ET - TRAINED CNN S ARE BIASED TOWARDS TEXTURE ; INCREASING SHAPE BIAS IMPROVES ACCURACY AND ROBUSTNESS.
Revised: I MAGE N ET - TRAINED CNN S ARE BIASED TOWARDS TEXTURE ; INCREASING SHAPE BIAS IMPROVES ACCURACY AND ROBUSTNESS

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Convolutional Neural Networks (CNNs) are commonly thought to recognise objects by learning increasingly complex representations of object shapes.
Revised: Convolutional Neural Networks (CNNs) are commonly thought to recognise objects by learning increasingly complex representations of object shapes.

Original: Some recent studies hint to a more important role of image textures.
Revised: Some recent studies hint to a more important role of image textures.

Original: We here put these conﬂicting hypotheses to a quantitative test by evaluating CNNs and human observers on images with a texture-shape cue conﬂict.
Revised: We here put these conﬂicting hypotheses to a quantitative test by evaluating CNNs and human observers on images with a texture-shape cue conﬂict.

Original: We show that ImageNettrained CNNs are strongly biased towards recognising textures rather than shapes, which is in stark contrast to human behavioural evidence and reveals fundamentally different classiﬁcation strategies.
Revised: We show that ImageNettrained CNNs are strongly biased towards recognising textures rather than shapes, which is in stark contrast to human behavioural evidence and reveals fundamentally different classiﬁcation strategies.

Original: We then demonstrate that the same standard architecture (ResNet-50) that learns a texture-based representation on ImageNet is able to learn a shape-based representation instead when trained on ‘StylizedImageNet’, a stylized version of ImageNet.
Revised: We then demonstrate that the same standard architecture (ResNet-50) that learns a texture-based representation on ImageNet is able to learn a shape-based representation instead when trained on ‘StylizedImageNet’, a stylized version of ImageNet.

Original: This provides a much better ﬁt for human behavioural performance in our well-controlled psychophysical lab setting (nine experiments totalling 48,560 psychophysical trials across 97 observers) and comes with a number of unexpected emergent beneﬁts such as improved object detection performance and previously unseen robustness towards a wide range of image distortions, highlighting advantages of a shape-based representation.
Revised: This provides a much better ﬁt for human behavioural performance in our well-controlled psychophysical lab setting (nine experiments totalling 48,560 psychophysical trials across 97 observers) and comes with a number of unexpected emergent beneﬁts such as improved object detection performance and previously unseen robustness towards a wide range of image distortions, highlighting advantages of a shape-based representation.

Original: [Figure]
Revised: [Figure]

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: How are Convolutional Neural Networks (CNNs) able to reach impressive performance on complex perceptual tasks such as object recognition (Krizhevsky et al., 2012) and semantic segmentation (Long et al., 2015)?
Revised: How are Convolutional Neural Networks (CNNs) able to reach impressive performance on complex perceptual tasks such as object recognition (Krizhevsky et al., 2012) and semantic segmentation (Long et al., 2015)?

Original: One widely accepted intuition is that CNNs combine low-level features (e.g. edges) to increasingly complex shapes (such as wheels, car windows) until the object (e.g. car) can be readily classiﬁed.
Revised: One widely accepted intuition is that CNNs combine low-level features (e.g. edges) to increasingly complex shapes (such as wheels, car windows) until the object (e.g. car) can be readily classiﬁed.

Original: As Kriegeskorte (2015) puts it, “the network acquires complex knowledge about the kinds of shapes associated with each category.
Revised: As Kriegeskorte (2015) puts it, “the network acquires complex knowledge about the kinds of shapes associated with each category.

Original: [...] High-level units appear to learn representations of shapes occurring in natural images” (p. 429).
Revised: [...] High-level units appear to learn representations of shapes occurring in natural images” (p. 429).

Original: This notion also appears in other explanations, such as in LeCun et al.
Revised: This notion also appears in other explanations, such as in LeCun et al.

Original: (2015): Intermediate CNN layers recognise “parts of familiar objects, and subsequent layers [...] detect objects as combinations of these parts” (p. 436).
Revised: (2015): Intermediate CNN layers recognise “parts of familiar objects, and subsequent layers [...] detect objects as combinations of these parts” (p. 436).

Original: We term this explanation the shape hypothesis .
Revised: We term this explanation the shape hypothesis .

Original: This hypothesis is supported by a number of empirical ﬁndings.
Revised: This hypothesis is supported by a number of empirical ﬁndings.

Original: Visualisation techniques like Deconvolutional Networks (Zeiler & Fergus, 2014) often highlight object parts in high-level CNN features.
Revised: Visualisation techniques like Deconvolutional Networks (Zeiler & Fergus, 2014) often highlight object parts in high-level CNN features.

Original: 1 Moreover, CNNs have been proposed as computational models of human shape perception by Kubilius et al.
Revised: 1 Moreover, CNNs have been proposed as computational models of human shape perception by Kubilius et al.

Original: (2016), who conducted an impressive number of experiments comparing human and CNN shape representations and concluded that CNNs “implicitly learn representations of shape that reﬂect human shape perception” (p. 15).
Revised: (2016), who conducted an impressive number of experiments comparing human and CNN shape representations and concluded that CNNs “implicitly learn representations of shape that reﬂect human shape perception” (p. 15).

Original: Ritter et al.
Revised: Ritter et al.

Original: (2017) discovered that CNNs develop a so-called “shape bias” just like children, i.e. that object shape is more important than colour for object classiﬁcation.
Revised: (2017) discovered that CNNs develop a so-called “shape bias” just like children, i.e. that object shape is more important than colour for object classiﬁcation.

Original: Furthermore, CNNs are currently the most predictive models for human ventral stream object recognition (e.g. Cadieu et al., 2014; Yamins et al., 2014); and it is well-known that object shape is the single most important cue for human object recognition (Landau et al., 1988), much more than other cues like size or texture (which may explain the ease at which humans recognise line drawings or millennia-old cave paintings).
Revised: Furthermore, CNNs are currently the most predictive models for human ventral stream object recognition (e.g. Cadieu et al., 2014; Yamins et al., 2014); and it is well-known that object shape is the single most important cue for human object recognition (Landau et al., 1988), much more than other cues like size or texture (which may explain the ease at which humans recognise line drawings or millennia-old cave paintings).

Original: On the other hand, some rather disconnected ﬁndings point to an important role of object textures for CNN object recognition.
Revised: On the other hand, some rather disconnected ﬁndings point to an important role of object textures for CNN object recognition.

Original: CNNs can still classify texturised images perfectly well, even if the global shape structure is completely destroyed (Gatys et al., 2017; Anonymous, 2018).
Revised: CNNs can still classify texturised images perfectly well, even if the global shape structure is completely destroyed (Gatys et al., 2017; Anonymous, 2018).

Original: Conversely, standard CNNs are bad at recognising object sketches where object shapes are preserved yet all texture cues are missing (Ballester & de Ara´ujo, 2016).
Revised: Conversely, standard CNNs are bad at recognising object sketches where object shapes are preserved yet all texture cues are missing (Ballester & de Ara´ujo, 2016).

Original: Additionally, two studies suggest that local information such as textures may actually be sufﬁcient to “solve” ImageNet object recognition:
Revised: Additionally, two studies suggest that local information such as textures may actually be sufﬁcient to “solve” ImageNet object recognition:

Original: Gatys et al.
Revised: Gatys et al.

Original: (2015) discovered that a linear classiﬁer on top of a CNN’s texture representation (Gram matrix) achieves hardly any classiﬁcation performance loss compared to original network performance.
Revised: (2015) discovered that a linear classiﬁer on top of a CNN’s texture representation (Gram matrix) achieves hardly any classiﬁcation performance loss compared to original network performance.

Original: More recently, Anonymous (2018) demonstrated that CNNs with explicitly constrained receptive ﬁeld sizes throughout all layers are able to reach surprisingly high accuracies on ImageNet, even though this effectively limits a model to recognising small local patches rather than integrating object parts for shape recognition.
Revised: More recently, Anonymous (2018) demonstrated that CNNs with explicitly constrained receptive ﬁeld sizes throughout all layers are able to reach surprisingly high accuracies on ImageNet, even though this effectively limits a model to recognising small local patches rather than integrating object parts for shape recognition.

Original: Taken together, it seems that local textures indeed provide sufﬁcient information about object classes—ImageNet object recognition could , in principle, be achieved through texture recognition alone.
Revised: Taken together, it seems that local textures indeed provide sufﬁcient information about object classes—ImageNet object recognition could , in principle, be achieved through texture recognition alone.

Original: In the light of these ﬁndings, we believe that it is time to consider a second explanation, which we term the texture hypothesis : in contrast to the common assumption, object textures are more important than global object shapes for CNN object recognition.
Revised: In the light of these ﬁndings, we believe that it is time to consider a second explanation, which we term the texture hypothesis : in contrast to the common assumption, object textures are more important than global object shapes for CNN object recognition.

Original: Resolving these two contradictory hypotheses is important both for the deep learning community (to increase our understanding of neural network decisions) as well as for the human vision and neuroscience communities (where CNNs are being used as computational models of human object recognition and shape perception).
Revised: Resolving these two contradictory hypotheses is important both for the deep learning community (to increase our understanding of neural network decisions) as well as for the human vision and neuroscience communities (where CNNs are being used as computational models of human object recognition and shape perception).

Original: In this work we aim to shed light on this debate with a number of carefully designed yet relatively straightforward experiments.
Revised: In this work we aim to shed light on this debate with a number of carefully designed yet relatively straightforward experiments.

Original: Utilising style transfer (Gatys et al., 2016),  created images with a texture-shape cue conﬂict such as the cat shape with elephant texture depicted in Figure 1c.
Revised: Utilising style transfer (Gatys et al., 2016), we created images with a texture-shape cue conﬂict such as the cat shape with elephant texture depicted in Figure 1c.

Original: This enables us to quantify texture and shape biases in both humans and CNNs.
Revised: This enables us to quantify texture and shape biases in both humans and CNNs.

Original: To this end, we perform nine comprehensive and careful psychophysical experiments comparing humans against CNNs on exactly the same images, totalling 48,560 psychophysical trials across 97 observers.
Revised: To this end, we perform nine comprehensive and careful psychophysical experiments comparing humans against CNNs on exactly the same images, totalling 48,560 psychophysical trials across 97 observers.

Original: These experiments provide behavioural evidence in favour of the texture hypothesis: A cat with an elephant texture is an elephant to CNNs, and still a cat to humans.
Revised: These experiments provide behavioural evidence in favour of the texture hypothesis: A cat with an elephant texture is an elephant to CNNs, and still a cat to humans.

Original: Beyond quantifying existing biases, we subsequently present results for our two other main contributions: changing biases, and discovering emergent beneﬁts of changed biases.
Revised: Beyond quantifying existing biases, we subsequently present results for our two other main contributions: changing biases, and discovering emergent beneﬁts of changed biases.

Original: We show that the texture biasin standard CNNs can be overcome and changed towards a shape bias if trained on a suitable data set. Remarkably, we then show that networks with a higher shape bias are inherently more robust to many different image distortions (for some even reaching or surpassing human performance, despite never being trained on any of them ) and reach higher performance on classiﬁcation and object recognition tasks.  M ETHODS In this section we outline the core elements of paradigm and procedure.
Revised: We show that the texture bias

Original: Extensive details to facilitate replication are provided in the Appendix.

Original: [Table]
Revised: [Table]

Original: oogLe

Original: N e t
Revised: N e t

Original: G oogLe
Revised: G oogLe

Original: N e t
Revised: N e t

Original: G oogLe
Revised: G oogLe

Original: N e t
Revised: N e t

Original: G oogLe
Revised: G oogLe

Original: N e t
Revised: N e t

Original: G oogLe
Revised: G oogLe

Original: N e t
Revised: N e t

Original: G oogLe
Revised: G oogLe

Original: N e t
Revised: N e t

Original: G oogLe
Revised: G oogLe

Original: N e t
Revised: N e t

Original: G oogLe
Revised: G oogLe

Original: N e t
Revised: N e t

Original: G oogLe
Revised: G oogLe

Original: N e t
Revised: N e t

Original: G oogLe
Revised: G oogLe

Original: N e t
Revised: N e t

Original: G oogLe
Revised: G oogLe

Original: N e t
Revised: N e t

Original: G oogLe
Revised: G oogLe

Original: N e t
Revised: N e t

Original: G oogLe
Revised: G oogLe

Original: N e t
Revised: N e t

Original: G oogLe
Revised: G oogLe

Original: N e t
Revised: N e t

Original: G oogLe
Revised: G oogLe

Original: N e t
Revised: N e t

Original: G oogLe
Revised: G oogLe

Original: N e t
Revised: N e t

Original: G oogLe
Revised: G oogLe

Original: N e t
Revised: N e t

Original: G oogLe
Revised: G oogLe

Original: N e t
Revised: N e t

Original: G oogLe
Revised: G oogLe

Original: N e t
Revised: N e t

Original: G oogLe
Revised: G oogLe

Original: N e t
Revised: N e t

Original: G oogLe
Revised: G oogLe

Original: N e t
Revised: N e t

Original: G oogLe
Revised: G oogLe

Original: N e t
Revised: N e t

Original: G oogLe
Revised: G oogLe

Original: N e t
Revised: N e t

Original: G oogLe
Revised: G oogLe

Original: N e t
Revised: N e t

Original: G oogLe
Revised: G oogLe

Original: N e t
Revised: N e t

Original: V GG − 
Revised: V GG − V GG − V GG − V GG −

Revised: V GG −

Revised: V GG −

Revised: V GG −

Revised: V GG −

Revised: V GG −

Revised: V GG −

Revised: V GG −

Revised: V GG −

Revised: V GG −

Revised: V GG −

Revised: V GG −

Revised: V GG −

Revised: V GG −

Revised: V GG −

Revised: V GG −

Revised: V GG −

Revised: V GG −

Revised: V GG −

Revised: V GG −

Revised: V GG −

Revised: V GG −

Revised: V GG −

Revised: V GG −

Revised: V GG −

Revised: V GG −

Revised: V GG −

Revised: V GG −

Revised: V GG −

Revised: V GG −

Revised: V GG −

Original: V GG − 
Revised: V GG − V GG − V GG − V GG −

Original: V GG − V GG −
Revised: V GG − V GG −

Original: V GG −V GG − V GG − V GG −
Revised: V GG −

Original: V GG −V GG − V GG − V GG −
Revised: V GG −

Original: V GG −V GG − V GG − V GG −
Revised: V GG −

Original: V GG − V GG −
Revised: V GG − V GG −

Original: V GG − V GG − V GG − V GG −
Revised: R e s N e t −

Original: V GG − V GG − V GG −
Revised: R e s N e t − R e s N e t −

Original: V GG − V GG − V GG −
Revised: R e s N e t − R e s N e t −

Original: V GG − V GG − V GG −
Revised: R e s N e t − R e s N e t −

Original: V GG − V GG − V GG −
Revised: R e s N e t − R e s N e t −

Original: V GG − V GG − V GG −
Revised: R e s N e t − R e s N e t −

Original: V GG − V GG − V GG − V GG −
Revised: R e s N e t −

Original: V GG − V GG − V GG − V GG −
Revised: R e s N e t −

Original: R e s N e t − R e s N e t −
Revised: R e s N e t − R e s N e t −

Original: R e s N e t − R e s N e t −R e s N e t −
Revised: R e s N e t − R e s N e t −

Original: R e s N e t − R e s N e t −R e s N e t −
Revised: R e s N e t − R e s N e t −

Original: R e s N e t − R e s N e t −R e s N e t −
Revised: R e s N e t − R e s N e t −

Original: R e s N e t − R e s N e t −R e s N e t −
Revised: R e s N e t − R e s N e t −

Original: R e s N e t − R e s N e t −R e s N e t −
Revised: R e s N e t − R e s N e t −

Original: R e s N e t − R e s N e t −R e s N e t −
Revised: R e s N e t − R e s N e t −

Original: R e s N e t − R e s N e t −R e s N e t −
Revised: R e s N e t − R e s N e t −

Original: R e s N e t − R e s N e t −R e s N e t −
Revised: R e s N e t − R e s N e t −

Original: R e s N e t − R e s N e t −R e s N e t −
Revised: R e s N e t − R e s N e t −

Original: R e s N e t − R e s N e t −R e s N e t −
Revised: R e s N e t − R e s N e t −

Original: R e s N e t − R e s N e t −R e s N e t −
Revised: R e s N e t − R e s N e t −

Original: R e s N e t − R e s N e t −R e s N e t −
Revised: R e s N e t − R e s N e t −

Original: R e s N e t − R e s N e t −R e s N e t −
Revised: R e s N e t − R e s N e t −

Original: R e s N e t − R e s N e t −
Revised: R e s N e t − R e s N e t −

Original: R e s N e t − R e s N e t −
Revised: R e s N e t − R e s N e t −

Original: H u m an s H u m an s
Revised: H u m an s H u m an s

Original: H u m an s H u m an s 
Revised: H u m an s H u m an s H u m an s

Original: H u m an s H u m an s 
Revised: H u m an s H u m an s H u m an s

Original: H u m an s H u m an s
Revised: H u m an s H u m an s

Original: H u m an s H u m an s
Revised: H u m an s H u m an s

Original: H u m an s H u m an s
Revised: H u m an s H u m an s

Original: H u m an s H u m an s
Revised: H u m an s H u m an s

Original: H u m an s H u m an s
Revised: H u m an s H u m an s

Original: H u m an s H u m an s
Revised: H u m an s H u m an s

Original: H u m an s H u m an s
Revised: H u m an s H u m an s

Original: H u m an s H u m an s
Revised: H u m an s H u m an s

Original: H u m an s H u m an s
Revised: H u m an s H u m an s

Original: H u m an s H u m an s
Revised: H u m an s H u m an s

Original: H u m an s H u m an s
Revised: H u m an s H u m an s

Original: H u m an s H u m an s
Revised: H u m an s H u m an s

Original: H u m an s H u m an s
Revised: H u m an s H u m an s

Original: H u m an s H u m an s
Revised: H u m an s H u m an s

Original: H u m an s H u m an s
Revised: H u m an s H u m an s

Original: H u m an s H u m an s
Revised: H u m an s H u m an s

Original: H u m an s H u m an s
Revised: H u m an s H u m an s

Original: H u m an s H u m an s
Revised: H u m an s H u m an s

Original: H u m an s 
Revised: H u m an s 97 99 10010044 49 48 40 28 24 100100100100original greyscale silhouette edges texture in standard CNNs can be overcome and changed towards a shape bias if trained on a suitable data set. Remarkably, networks with a higher shape bias are inherently more robust to many different image distortions (for some even reaching or surpassing human performance, despite never being trained on any of them ) and reach higher performance on classiﬁcation and object recognition tasks.

Original:  H u m an s
Revised:  M ETHODS

Original: H u m an s 97 99 10010044 49 48 40 28 24 100100100100original greyscale silhouette edges texture
Revised: In this section we outline the core elements of paradigm and procedure. Extensive details to facilitate replication are provided in the Appendix. Data, code and materials are available from this repository: https://github.com/rgeirhos/texture-vs-shape

Original: 2. P SYCHOPHYSICAL EXPERIMENTS
Revised: 2. P SYCHOPHYSICAL EXPERIMENTS

Original: All psychophysical experiments were conducted in a well-controlled psychophysical lab setting and follow the paradigm of Geirhos et al.
Revised: All psychophysical experiments were conducted in a well-controlled psychophysical lab setting and follow the paradigm of Geirhos et al.

Original: (2018), which allows for direct comparisons between human and CNN classiﬁcation performance on exactly the same images.
Revised: (2018), which allows for direct comparisons between human and CNN classiﬁcation performance on exactly the same images.

Original: Brieﬂy, in each trial participants were presented a ﬁxation square for 300 ms, followed by a 300 ms presentation of the stimulus image.
Revised: Brieﬂy, in each trial participants were presented a ﬁxation square for 300 ms, followed by a 300 ms presentation of the stimulus image.

Original: After the stimulus image we presented a full-contrast pink noise mask (1/ f spectral shape) for 200 ms to minimise feedback processing in the human visual system and to thereby make the comparison to feedforward CNNs as fair as possible.
Revised: After the stimulus image we presented a full-contrast pink noise mask (1/ f spectral shape) for 200 ms to minimise feedback processing in the human visual system and to thereby make the comparison to feedforward CNNs as fair as possible.

Original: Subsequently, participants had to choose one of 16 entry-level categories by clicking on a response screen shown for 1500 ms.
Revised: Subsequently, participants had to choose one of 16 entry-level categories by clicking on a response screen shown for 1500 ms.

Original: On this screen, icons of all 16 categories were arranged in a 4 × 4 grid.
Revised: On this screen, icons of all 16 categories were arranged in a 4 × 4 grid.

Original: Those categories were airplane , bear , bicycle , bird , boat , bottle , car , cat , chair , clock , dog , elephant , keyboard , knife , oven and truck .
Revised: Those categories were airplane , bear , bicycle , bird , boat , bottle , car , cat , chair , clock , dog , elephant , keyboard , knife , oven and truck .

Original: Those are the so-called “16-class-ImageNet” categories introduced in
Revised: Those are the so-called “16-class-ImageNet” categories introduced in

Original: Geirhos et al.
Revised: Geirhos et al.

Original: The same images were fed to four CNNs pre-trained on standard ImageNet, namely AlexNet (Krizhevsky et al., 2012), GoogLeNet (Szegedy et al., 2015), VGG-16 (Simonyan & Zisserman, 2015) and ResNet-50 (He et al., 2015).
Revised: The same images were fed to four CNNs pre-trained on standard ImageNet, namely AlexNet (Krizhevsky et al., 2012), GoogLeNet (Szegedy et al., 2015), VGG-16 (Simonyan & Zisserman, 2015) and ResNet-50 (He et al., 2015).

Original: The 1,000 ImageNet class predictions were mapped to the 16 categories using the WordNet hierarchy (Miller, 1995)—e.g.
Revised: The 1,000 ImageNet class predictions were mapped to the 16 categories using the WordNet hierarchy (Miller, 1995)—e.g.

Original: ImageNet category tabby cat would be mapped to cat .
Revised: ImageNet category tabby cat would be mapped to cat .

Original: In total, the results presented in this study are based on 48,560 psychophysical trials and 97 participants.
Revised: In total, the results presented in this study are based on 48,560 psychophysical trials and 97 participants.

Original: 2. DATA SETS (P SYCHOPHYSICS )
Revised: 2. DATA SETS (P SYCHOPHYSICS )

Original: In order to assess texture and shape biases, we conducted six major experiments along with three control experiments, which are described in the Appendix.
Revised: In order to assess texture and shape biases, we conducted six major experiments along with three control experiments, which are described in the Appendix.

Original: The ﬁrst ﬁve experiments (samples visualised in Figure 2) are simple object recognition tasks with the only difference being the image features available to the participant:
Revised: The ﬁrst ﬁve experiments (samples visualised in Figure 2) are simple object recognition tasks with the only difference being the image features available to the participant:

Original: Original 160 natural colour images of objects (10 per category) with white background.
Revised: Original 160 natural colour images of objects (10 per category) with white background.

Original: Greyscale Images from Original data set converted to greyscale using skimage.color.rgb2gray.
Revised: Greyscale Images from Original data set converted to greyscale using skimage.color.rgb2gray

Original: For CNNs, greyscale images were stacked along the colour channel.
Revised: For CNNs, greyscale images were stacked along the colour channel.

Original: Silhouette Images from Original data set converted to silhouette images showing an entirely black object on a white background (see Appendix A.6 for procedure).
Revised: Silhouette Images from Original data set converted to silhouette images showing an entirely black object on a white background (see Appendix A.6 for procedure).

Original: Edges Images from Original data set converted to an edge-based representation using Canny edge extractor implemented in MATLAB.
Revised: Edges Images from Original data set converted to an edge-based representation using Canny edge extractor implemented in MATLAB.

Original: Texture 48 natural colour images of textures (3 per category).
Revised: Texture 48 natural colour images of textures (3 per category).

Original: Typically the textures consist of full-width patches of an animal (e.g. skin or fur) or, in particular for man-made objects, of images with many repetitions of the same objects (e.g many bottles next to each other, see Figure 7 in the Appendix).
Revised: Typically the textures consist of full-width patches of an animal (e.g. skin or fur) or, in particular for man-made objects, of images with many repetitions of the same objects (e.g. many bottles next to each other, see Figure 7 in the Appendix).

Original: It is important to note that we only selected object and texture images that were correctly classiﬁed by all four networks.
Revised: It is important to note that we only selected object and texture images that were correctly classiﬁed by all four networks.

Original: This was made to ensure that our results in the sixth experiment on cue conﬂicts, which is most decisive in terms of the shape vs texture hypothesis, are fully interpretable.
Revised: This was made to ensure that our results in the sixth experiment on cue conﬂicts, which is most decisive in terms of the shape vs texture hypothesis, are fully interpretable.

Original: In the cue conﬂict experiment we present images with contradictory features (see Figure 1) but still ask the participant to assign a single class.
Revised: In the cue conﬂict experiment we present images with contradictory features (see Figure 1) but still ask the participant to assign a single class.

Original: Note that the instructions to human observers were entirely neutral w.r.t.
Revised: Note that the instructions to human observers were entirely neutral w.r.t.

Original: shape or texture (“click on the object category that you see in the presented image; guess if unsure.
Revised: shape or texture (“click on the object category that you see in the presented image; guess if unsure.

Original: There is no right or wrong answer, we are interested in your subjective impression”).
Revised: There is no right or wrong answer, we are interested in your subjective impression”).

Original: Cue conﬂict Images generated using iterative style transfer (Gatys et al., 2016) between an image of the Texture data set (as style) and an image from the Original data set (as content).
Revised: Cue conﬂict Images generated using iterative style transfer (Gatys et al., 2016) between an image of the Texture data set (as style) and an image from the Original data set (as content).

Original: We generated a total of 1280 cue conﬂict images (80 per category), which allows for presentation to human observers within a single experimental session.
Revised: We generated a total of 1280 cue conﬂict images (80 per category), which allows for presentation to human observers within a single experimental session.

Original: We deﬁne “silhouette” as the bounding contour of an object in 2D (i.e., the outline of object segmentation).
Revised: We deﬁne “silhouette” as the bounding contour of an object in 2D (i.e., the outline of object segmentation).

Original: When mentioning “object shape”, we use a deﬁnition that is broader than just the silhouette of an object: we refer to the set of contours that describe the 3D form of an object, i.e. including those contours that are not part of the silhouette.
Revised: When mentioning “object shape”, we use a deﬁnition that is broader than just the silhouette of an object: we refer to the set of contours that describe the 3D form of an object, i.e. including those contours that are not part of the silhouette.

Original: 2. S TYLIZED -I MAGE N ET
Revised: 2. S TYLIZED -I MAGE N ET

Original: Starting from ImageNet we constructed a new data set (termed Stylized-ImageNet or SIN) by stripping every single image of its original texture and replacing it with the style of a randomly selected painting through AdaIN style transfer (Huang & Belongie, 2017)
Revised: Starting from ImageNet we constructed a new data set (termed Stylized-ImageNet or SIN) by stripping every single image of its original texture and replacing it with the style of a randomly selected painting through AdaIN style transfer (Huang & Belongie, 2017)

Original: (see examples in Figure 3) with a stylization coefﬁcient of α = 1 . 0 .
Revised: (see examples in Figure 3) with a stylization coefﬁcient of α = 1 . 0 .

Original: We used Kaggle’s Painter by Numbers data set 2 as a style source due to its large style variety and size (79,434 paintings).
Revised: We used Kaggle’s Painter by Numbers data set 2 as a style source due to its large style variety and size (79,434 paintings).

Original: We used AdaIN fast style transfer rather than iterative stylization (e.g. Gatys et al., 2016) for two reasons:
Revised: We used AdaIN fast style transfer rather than iterative stylization (e.g. Gatys et al., 2016) for two reasons:

Original: Firstly, to ensure that training on SIN and testing on cue conﬂict stimuli is done using different stylization techniques, suchthat the results do not rely on a single stylization method.
Revised: Firstly, to ensure that training on SIN and testing on cue conﬂict stimuli is done using different stylization techniques, such

Revised: Within the responses that corresponded to either the correct texture or correct shape category, the fractions of texture and shape decisions are depicted in the main plot (averages visualised by vertical lines).

Original: Secondly, to enable stylizing entire
Revised: On the right side, small barplots display the proportion of correct decisions (either texture or shape correctly recognised) as a fraction of all trials. Similar results for ResNet-152, DenseNet-121 and Squeezenet1 1 are reported in the Appendix

Revised: Figure 13.

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: lll lllllllllllll l l l l l l l ll l l l ll l l that the results do not rely on a single stylization method.

Original:  ImageNet, which would take prohibitively long with an iterative approach. 
Revised: Secondly, to enable stylizing entire ImageNet, which would take prohibitively long with an iterative approach. We provide code to create Stylized-ImageNet here: [Equation]

Original:  R ESULTS 3. T EXTURE VS SHAPE BIAS IN HUMANS AND I MAGE N ET - TRAINED CNN S
Revised:  R ESULTS 3. T EXTURE VS SHAPE BIAS IN HUMANS AND I MAGE N ET - TRAINED CNN S

Original: Almost all object and texture images ( Original and Texture data set) were recognised correctly by both CNNs and humans (Figure 2).
Revised: Almost all object and texture images ( Original and Texture data set) were recognised correctly by both CNNs and humans (Figure 2).

Original: Greyscale versions of the objects, which still contain both shape and texture, were recognised equally well.
Revised: Greyscale versions of the objects, which still contain both shape and texture, were recognised equally well.

Original: When object outlines were ﬁlled in with black colour to

Original: Within the responses that corresponded to either the correct texture or correct shape category, the fractions of texture and shape decisions are depicted in the main plot (averages visualised by vertical lines).

Original: On the right side, small barplots display the proportion of correct decisions (either texture or shape correctly recognised) as a fraction of all trials.

Original: [Figure]

Original: [Table] [Figure] [Figure] lll lllllllllllll l l l l l l l ll l l l ll l l generate a silhouette, CNN recognition accuracies were much lower than human accuracies.
Revised: When object outlines were ﬁlled in with black colour to generate a silhouette, CNN recognition accuracies were much lower than human accuracies.

Original: This was even more pronounced for edge stimuli, indicating that human observers cope much better with images that have little to no texture information.
Revised: This was even more pronounced for edge stimuli, indicating that human observers cope much better with images that have little to no texture information.

Original: One confound in these experiments is that CNNs tend not to cope well with domain shifts, i.e. the large change in image statistics from natural images (on which the networks have been trained) to sketches (which the networks have never seen before).
Revised: One confound in these experiments is that CNNs tend not to cope well with domain shifts, i.e. the large change in image statistics from natural images (on which the networks have been trained) to sketches (which the networks have never seen before).

Original: We thus devised a cue conﬂict experiment that is based on images with a natural statistic but contradicting texture and shape evidence (see Methods).
Revised: We thus devised a cue conﬂict experiment that is based on images with a natural statistic but contradicting texture and shape evidence (see Methods).

Original: Participants and CNNs have to classify the images based on the features (shape or texture) that they most rely on.
Revised: Participants and CNNs have to classify the images based on the features (shape or texture) that they most rely on.

Original: The results of this experiment are visualised in Figure 4.
Revised: The results of this experiment are visualised in Figure 4.

Original: Human observers show a striking bias towards responding with the shape category (95.9% of correct decisions).
Revised: Human observers show a striking bias towards responding with the shape category (95.9% of correct decisions).

Original: This pattern is reversed for CNNs, which show a clear bias towards responding with the texture category (VGG-16: 17.2% shape vs. 82.8% texture; 
Revised: This pattern is reversed for CNNs, which show a clear bias towards responding with the texture category (VGG-16: 17.2% shape vs. 82.8% texture; [Table] [Table] % vs

Revised: [Table]

Original: GoogLeNet: 31.2% vs. 68.8%; AlexNet: 42.9% vs. 57.1%; ResNet-50: 22.1% vs. 77.9%).
Revised: 68.8%; AlexNet: [Table] 9% vs [Table]

Revised: [Table]

Original: 3. O VERCOMING THE TEXTURE BIAS OF CNN S
Revised: 1%; ResNet-50: [Table] % vs [Table]

Revised: [Table]

Revised: 3. O VERCOMING THE TEXTURE BIAS OF CNN S

Original: The psychophysical experiments suggest that ImageNet-trained CNNs, but not humans, exhibit a strong texture bias.
Revised: The psychophysical experiments suggest that ImageNet-trained CNNs, but not humans, exhibit a strong texture bias.

Original: One reason might be the training task itself: from Anonymous (2018)
Revised: One reason might be the training task itself: from Anonymous (2018)

Original: we know that ImageNet can be solved to high accuracy using only local information.
Revised: we know that ImageNet can be solved to high accuracy using only local information.

Original: In other words, it might simply sufﬁce to integrate evidence from many local texture features rather than going through the process of integrating and classifying global shapes.
Revised: In other words, it might simply sufﬁce to integrate evidence from many local texture features rather than going through the process of integrating and classifying global shapes.

Original: In order to test this hypothesis we train a
Revised: In order to test this hypothesis we train a

Original: ResNet-50 on our Stylized-ImageNet (SIN) data set in which we replaced the object-related local texture information with the uninformative style of randomly selected artistic paintings.
Revised: ResNet-50 on our Stylized-ImageNet (SIN) data set in which we replaced the object-related local texture information with the uninformative style of randomly selected artistic paintings.

Original: A standard ResNet-50 trained and evaluated on Stylized ImageNet (SIN) achieves 79.0% topaccuracy (see Table 1).
Revised: A standard ResNet-50 trained and evaluated on Stylized-ImageNet (SIN) achieves 79.0% top-5 accuracy (see Table 1).

Original: In comparison, the same architecture trained and evaluated on ImageNet (IN) achieves 92.9% top-5 accuracy.
Revised: In comparison, the same architecture trained and evaluated on ImageNet (IN) achieves 92.9% top-5 accuracy.

Original: This performance difference indicates that SIN is a much harder task than IN since textures are no longer predictive, but instead a nuisance factor (as desired).reported Appendix [Figure]
Revised: This performance difference indicates that SIN is a much harder task than IN since textures are no longer predictive, but instead a nuisance factor (as desired).

Original: lll lllllllllllll Intriguingly, ImageNet features generalise poorly to SIN (only 16.4% top-5 accuracy); yet features learned on SIN generalise very well to ImageNet (82.6% top-5 accuracy without any ﬁne-tuning).
Revised: Intriguingly, ImageNet features generalise poorly to SIN (only 16.4% top-5 accuracy); yet features learned on SIN generalise very well to ImageNet (82.6% top-5 accuracy without any ﬁne-tuning).

Original: In order to test wheter local texture features are still sufﬁcient to “solve” SIN we evaluate the performance of so-called BagNets .
Revised: In order to test wheter local texture features are still sufﬁcient to “solve” SIN we evaluate the performance of so-called BagNets .

Original: Introduced recently by Anonymous (2018), BagNets have a ResNetarchitecture but their maximum receptive ﬁeld size is limited to 9 × 9 , 17 × 17 or 33 × 33 pixels.
Revised: Introduced recently by Anonymous (2018), BagNets have a ResNetarchitecture but their maximum receptive ﬁeld size is limited to 9 × 9 , 17 × 17 or 33 × 33 pixels.

Original: This precludes BagNets from learning or using any long-range spatial relationships for classiﬁcation.
Revised: This precludes BagNets from learning or using any long-range spatial relationships for classiﬁcation.

Original: While these restricted networks can reach high accuracies on ImageNet, they are unable to achieve the same on SIN, showing dramatically reduced performance with smaller receptive ﬁeld sizes (such as 10.0% top-5 accuracy on SIN compared to 70.0% on ImageNet for a BagNet with receptive ﬁeld size of 9 × 9 pixels).
Revised: While these restricted networks can reach high accuracies on ImageNet, they are unable to achieve the same on SIN, showing dramatically reduced performance with smaller receptive ﬁeld sizes (such as 10.0% top-5 accuracy on SIN compared to 70.0% on ImageNet for a BagNet with receptive ﬁeld size of 9 × 9 pixels).

Original: This is a clear indication that the SIN data set we propose does actually remove local texture cues, forcing a network to integrate long-range spatial information.
Revised: This is a clear indication that the SIN data set we propose does actually remove local texture cues, forcing a network to integrate long-range spatial information.

Original: Most importantly, the SIN-trained ResNet-50 shows a much stronger shape bias in our cue conﬂict experiment (Figure 5), which increases from 22% for a IN-trained model to 81%.
Revised: Most importantly, the SIN-trained ResNet-50 shows a much stronger shape bias in our cue conﬂict experiment (Figure 5), which increases from 22% for a IN-trained model to 81%.

Original: In many categories the shape bias is almost as strong as for humans
Revised: In many categories the shape bias is almost as strong as for humans.

Original: [Table]

Original: [Table]

Original: 3. R OBUSTNESS AND ACCURACY OF SHAPE - BASED REPRESENTATIONS
Revised: 3. R OBUSTNESS AND ACCURACY OF SHAPE - BASED REPRESENTATIONS

Original: Does the increased shape bias, and thus the shifted representations, also affect the performance or robustness of CNNs?
Revised: Does the increased shape bias, and thus the shifted representations, also affect the performance or robustness of CNNs?

Original: In addition to the IN- and SIN-trained ResNet-50 architecture we here additionally analyse two joint training schemes:
Revised: In addition to the IN- and SIN-trained ResNet-50 architecture we here additionally analyse two joint training schemes:

Original: • Training jointly on SIN and IN.
Revised: • Training jointly on SIN and IN.

Original: • Training jointly on SIN and IN with ﬁne-tuning on IN.
Revised: • Training jointly on SIN and IN with ﬁne-tuning on IN.

Original: We refer to this model as Shape-ResNet .
Revised: We refer to this model as Shape-ResNet .

Original: We then compared these models with a vanilla ResNet-50 on three experiments: (1) classiﬁcation performance on IN, (2) transfer to Pascal VOC 2007 and (3) robustness against image perturbations.
Revised: We then compared these models with a vanilla ResNet-50 on three experiments: (1) classiﬁcation performance on IN, (2) transfer to Pascal VOC 2007 and (3) robustness against image perturbations.

Original: Classiﬁcation performance Shape-ResNet surpasses the vanilla ResNet in terms of top-1 and top5 ImageNet validation accuracy as reported in Table 2.
Revised: Classiﬁcation performance Shape-ResNet surpasses the vanilla ResNet in terms of top-1 and top5 ImageNet validation accuracy as reported in Table 2.

Original: This indicates that SIN may be a useful data augmentation on ImageNet that can improve model performance without any architectural changes.
Revised: This indicates that SIN may be a useful data augmentation on ImageNet that can improve model performance without any architectural changes [Table] other networks are reported in the Appendix, Figure

Revised: [Table]

Original: Transfer learning We tested the representations of each model as backbone features for Faster
Revised: [Figure] [Table] [Figure] [Table]

Revised: lll lllllllllllll

Revised: [Table]

Revised: Transfer learning We tested the representations of each model as backbone features for Faster

Original: R-CNN (Ren et al., 2017) on Pascal VOC 2007.
Revised: R-CNN (Ren et al., 2017) on Pascal VOC 2007.

Original: Incorporating SIN in the training data substantially improves object detection performance from 70.7 to 75.1 mAP50 as shown in Table 2.
Revised: Incorporating SIN in the training data substantially improves object detection performance from 70.7 to 75.1 mAP50 as shown in Table 2.

Original: This is in line with the intuition that for object detection, a shape-based representation is more beneﬁcial than a texture-based representation, since the ground truth rectangles encompassing an object are by design aligned with global object shape.
Revised: This is in line with the intuition that for object detection, a shape-based representation is more beneﬁcial than a texture-based representation, since the ground truth rectangles encompassing an object are by design aligned with global object shape.

Original: Robustness against distortions We systematically tested how model accuracies degrade if images are distorted by uniform or phase noise, contrast changes, high- and low-pass ﬁltering or eidolon perturbations.
Revised: Robustness against distortions We systematically tested how model accuracies degrade if images are distorted by uniform or phase noise, contrast changes, high- and low-pass ﬁltering or eidolon perturbations.

Original: The results of this comparison, including human data for reference, are visualised in Figure 6.
Revised: The results of this comparison, including human data for reference, are visualised in Figure 6.

Original: While lacking a few percent accuracy on undistorted images, the SIN-trained network outperforms the IN-trained CNN on almost all image manipulations.
Revised: While lacking a few percent accuracy on undistorted images, the SIN-trained network outperforms the IN-trained CNN on almost all image manipulations.

Original: (Low-pass ﬁltering / blurring is the only distortion type on which SIN-trained networks are more susceptible, which might be due to the over-representation of high frequency signals in SIN through paintings and the reliance on sharp edges.)
Revised: (Low-pass ﬁltering / blurring is the only distortion type on which SIN-trained networks are more susceptible, which might be due to the over-representation of high frequency signals in SIN through paintings and the reliance on sharp edges.)

Original: The SIN-trained ResNet-50 approaches human-level distortion robustness— despite never seeing any of the distortions during training . 
Revised: The SIN-trained ResNet-50 approaches human-level distortion robustness— despite never seeing any of the distortions during training . [Table] [Figure]

Original:  D ISCUSSION
Revised:  D ISCUSSION

Original: As noted in the Introduction, there seems to be a large discrepancy between the common assumption that CNNs use increasingly complex shape features to recognise objects and recent empirical ﬁndings which suggest a crucial role of object textures instead.
Revised: As noted in the Introduction, there seems to be a large discrepancy between the common assumption that CNNs use increasingly complex shape features to recognise objects and recent empirical ﬁndings which suggest a crucial role of object textures instead.

Original: In order to explicitly probe this question, we utilised style transfer (Gatys et al., 2016) to generate images with conﬂicting shape and texture information.
Revised: In order to explicitly probe this question, we utilised style transfer (Gatys et al., 2016) to generate images with conﬂicting shape and texture information.

Original: On the basis of extensive experiments on both CNNs and human observers in a controlled psychophysical lab setting, we provide evidence that unlike humans, ImageNet-trained
Revised: On the basis of extensive experiments on both CNNs and human observers in a controlled psychophysical lab setting, we provide evidence that unlike humans, ImageNet-trained

Original: CNNs tend to classify objects according to local textures instead of global object shapes.
Revised: CNNs tend to classify objects according to local textures instead of global object shapes.

Original: In combination with previous work which showed that changing other major object dimensions such as

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure]

Original: [Table]

Original: [Figure] [Table] [Figure] colour (Geirhos et al., 2018) and object size relative to the context (Eckstein et al., 2017) do not have a strong detrimental impact on CNN recognition performance, this highlights the special role that local cues such as textures seem to play in CNN object recognition.
Revised: In combination with previous work which showed that changing other major object dimensions such as colour (Geirhos et al., 2018) and object size relative to the context (Eckstein et al., 2017) do not have a strong detrimental impact on CNN recognition performance, this highlights the special role that local cues such as textures seem to play in CNN object recognition.

Original: Intriguingly, this offers an explanation for a number of rather disconnected ﬁndings: CNNs match texture appearance for humans (Wallis et al., 2017), and their predictive power for neural responses along the human ventral stream appears to be largely due to human-like texture representations, but not human-like contour representations (Laskar et al., 2018; Long & Konkle, 2018).
Revised: Intriguingly, this offers an explanation for a number of rather disconnected ﬁndings: CNNs match texture appearance for humans (Wallis et al., 2017), and their predictive power for neural responses along the human ventral stream appears to be largely due to human-like texture representations, but not human-like contour representations (Laskar et al., 2018; Long & Konkle, 2018).

Original: Furthermore, texture-based generative modelling approaches such as style transfer (Gatys et al., 2016), single image super-resolution (Gondal et al., 2018) as well as static and dynamic texture synthesis (Gatys et al., 2015; Funke et al., 2017) all produce excellent results using standard CNNs, while CNNbased shape transfer seems to be very difﬁcult (Gokaslan et al., 2018).
Revised: Furthermore, texture-based generative modelling approaches such as style transfer (Gatys et al., 2016), single image super-resolution (Gondal et al., 2018) as well as static and dynamic texture synthesis (Gatys et al., 2015; Funke et al., 2017) all produce excellent results using standard CNNs, while CNNbased shape transfer seems to be very difﬁcult (Gokaslan et al., 2018).

Original: CNNs can still recognise images with scrambled shapes (Gatys et al., 2017; Anonymous, 2018), but they have much more difﬁculties recognising objects with missing texture information (Ballester & de Ara ´ ujo, 2016; Yu et al., 2017).
Revised: CNNs can still recognise images with scrambled shapes (Gatys et al., 2017; Anonymous, 2018), but they have much more difﬁculties recognising objects with missing texture information (Ballester & de Ara´ujo, 2016; Yu et al., 2017).

Original: Our hypothesis might also explain why an image segmentation model trained on a database of synthetic texture images transfers to natural images and videos (Ustyuzhaninov et al., 2018).
Revised: Our hypothesis might also explain why an image segmentation model trained on a database of synthetic texture images transfers to natural images and videos (Ustyuzhaninov et al., 2018).

Original: Beyond that, our results show marked behavioural differences between ImageNet-trained
Revised: Beyond that, our results show marked behavioural differences between ImageNet-trained

Original: CNNs and human observers.
Revised: CNNs and human observers.

Original: While both human and machine vision systems achieve similarly high accuracies on standard images (Geirhos et al., 2018), our ﬁndings suggest that the underlying classiﬁcation strategies might actually be very different.
Revised: While both human and machine vision systems achieve similarly high accuracies on standard images (Geirhos et al., 2018), our ﬁndings suggest that the underlying classiﬁcation strategies might actually be very different.

Original: This is problematic, since CNNs are being used as computational models for human object recognition (e.g. Cadieu et al., 2014; Yamins et al., 2014).
Revised: This is problematic, since CNNs are being used as computational models for human object recognition (e.g. Cadieu et al., 2014; Yamins et al., 2014).

Original: In order to reduce the texture bias of CNNs we introduced Stylized-ImageNet (SIN), a data set that removes local cues through style transfer and thereby forces networks to go beyond texture recognition.
Revised: In order to reduce the texture bias of CNNs we introduced Stylized-ImageNet (SIN), a data set that removes local cues through style transfer and thereby forces networks to go beyond texture recognition.

Original: Using this data set, we demonstrated that a ResNet-50 architecture can indeed learn to recognise objects based on object shape, revealing that the texture bias in current CNNs is not by design but induced by ImageNet training data.
Revised: Using this data set, we demonstrated that a ResNet-50 architecture can indeed learn to recognise objects based on object shape, revealing that the texture bias in current CNNs is not by design but induced by ImageNet training data.

Original: This indicates that standard ImageNet-trained models may be taking a “shortcut” by focusing on local textures, which could be seen as a version of Occam’s razor:
Revised: This indicates that standard ImageNet-trained models may be taking a “shortcut” by focusing on local textures, which could be seen as a version of Occam’s razor:

Original: If textures are sufﬁcient, why should a CNN learn much else?
Revised: If textures are sufﬁcient, why should a CNN learn much else?

Original: While texture classiﬁcation may be easier than shape recognition, we found that shape-based features trained on
Revised: While texture classiﬁcation may be easier than shape recognition, we found that shape-based features trained on

Original: SIN generalise well to natural images.
Revised: SIN generalise well to natural images.

Original: Our results indicate that a more shape-based representation can be beneﬁcial for recognition tasks that rely on pre-trained ImageNet CNNs.
Revised: Our results indicate that a more shape-based representation can be beneﬁcial for recognition tasks that rely on pre-trained ImageNet CNNs.

Original: Furthermore, while ImageNet-trained CNNs generalise poorly towards a wide range of image distortions (e.g. Dodge & Karam, 2017; Geirhos et al., 2017; 2018), our ResNet-50 trained on Stylized-ImageNet often reaches or even surpasses human-level robustness (without ever being trained on the speciﬁc image degradations).
Revised: Furthermore, while ImageNet-trained CNNs generalise poorly towards a wide range of image distortions (e.g. Dodge & Karam, 2017; Geirhos et al., 2017; 2018), our ResNet-50 trained on Stylized-ImageNet often reaches or even surpasses human-level robustness (without ever being trained on the speciﬁc image degradations).

Original: This is exciting because (Geirhos et al., 2018) showed that networks trained on speciﬁc distortions in general do not acquire robustness against other unseen image manipulations.
Revised: This is exciting because (Geirhos et al., 2018) showed that networks trained on speciﬁc distortions in general do not acquire robustness against other unseen image manipulations.

Original: This emergent behaviour highlights the usefulness of a shape-based representation: While local textures are easily distorted by all sorts of noise (including those in the real world, such as rain and snow), the object shape remains relatively stable.
Revised: This emergent behaviour highlights the usefulness of a shape-based representation: While local textures are easily distorted by all sorts of noise (including those in the real world, such as rain and snow), the object shape remains relatively stable.

Original: Furthermore, this ﬁnding offers a compellingly simple explanation for the incredible robustness of humans when coping with distortions: a shape-based representation.
Revised: Furthermore, this ﬁnding offers a compellingly simple explanation for the incredible robustness of humans when coping with distortions: a shape-based representation.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In summary, we provided evidence that machine recognition today overly relies on object textures rather than global object shapes as commonly assumed.
Revised: In summary, we provided evidence that machine recognition today overly relies on object textures rather than global object shapes as commonly assumed.

Original: We demonstrated the advantages of a shapebased representation for robust inference (using our Stylized-ImageNet data set  to induce such a representation in neural networks).
Revised: We demonstrated the advantages of a shapebased representation for robust inference (using our Stylized-ImageNet data set 5 to induce such a representation in neural networks).

Original: We envision our ﬁndings as well as our openly available model weights,training code and behavioural data set (49K trials across 97 observers)  to achieve three goals: Firstly, an improved understanding of CNN representations and biases.
Revised: We envision our ﬁndings as well as our openly available model weights, code and behavioural data set (49K trials across 97 observers) 6 to achieve three goals: Firstly, an improved understanding of CNN representations and biases.

Original: Secondly, a step towards more plausible models of human visual object recognition.
Revised: Secondly, a step towards more plausible models of human visual object recognition.

Original: Thirdly, a useful starting point for future undertakings where domain knowledge suggests that a shape-based representation may be more beneﬁcial than a texture-based one.
Revised: Thirdly, a useful starting point for future undertakings where domain knowledge suggests that a shape-based representation may be more beneﬁcial than a texture-based one.

Original: A A PPENDIX A. R EPRODUCIBILITY & A CCESS TO C ODE / M ODELS / D ATA
Revised: A CKNOWLEDGMENTS This work has been funded, in part, by the German Federal Ministry of Education and Research (BMBF) through the Bernstein Computational Neuroscience Program T¨ubingen (FKZ 01GQ1002) and the T ¨ ubingen AI Center (FKZ 01IS18039A) as well as the German Research Foundation (DFG; Sachbeihilfe Wi 2103/4-1 and SFB 1233 on “Robust Vision”).

Original: All trained model weights reported in this paper will be released after the end of the anonymous review period.
Revised: The authors thank the International Max Planck Research School for Intelligent Systems (IMPRS-IS) for supporting R.G. and C.M.; M.B. acknowledges support by the Centre for Integrative Neuroscience T ¨ ubingen (EXC 307) and by the Intelligence Advanced Research Projects Activity (IARPA) via Department of Interior/Interior

Original: We will furthermore make our human behavioural data set (48,560 psychophysical trials across 97 observers) openly available.
Revised: Business Center (DoI/IBC) contract number D16PC00003. Furthermore, we would like to express our gratitude towards Alexander Ecker, Leon Gatys, Tina Gauger, Silke Gramer, Heike K ¨ onig, Jonas Rauber, Steffen Schneider, Heiko Sch ¨ utt, Tom Wallis and Uli Wannek for support and/or useful discussions. A A PPENDIX A. R EPRODUCIBILITY & A CCESS TO C ODE / M ODELS / D ATA

Original: In this Appendix, we report experimental details for human and CNN experiments.
Revised: In this Appendix, we report experimental details for human and CNN experiments.

Revised: All trained model weights reported in this paper as well as our human behavioural data set (48,560 psychophysical trials across 97 observers) are openly available from this repository: https://github.com/rgeirhos/texture-vs-shape

Original: A. P ROCEDURE
Revised: A. P ROCEDURE

Original: We followed the paradigm of Geirhos et al.
Revised: We followed the paradigm of Geirhos et al.

Original: (2018) for maximal comparability.
Revised: (2018) for maximal comparability.

Original: A trial consisted of 300 ms presentation of a ﬁxation square and a 200 ms presentation of the stimulus image, which was followed by a full-contrast pink noise mask (1/ f spectral shape) of the same size lasting forms.
Revised: A trial consisted of 300 ms presentation of a ﬁxation square and a 200 ms presentation of the stimulus image, which was followed by a full-contrast pink noise mask (1/ f spectral shape) of the same size lasting forms.

Original: Participants had to choose one of 16 entry-level categories by clicking on a response screen shown for 1500 ms.
Revised: Participants had to choose one of 16 entry-level categories by clicking on a response screen shown for 1500 ms.

Original: On this screen, icons of all 16 categories were arranged in a 4 × 4 grid.
Revised: On this screen, icons of all 16 categories were arranged in a 4 × 4 grid.

Original: The experiments were not self-paced and therefore one trial always lasted 2200 ms (300 ms + 200 ms + 200 ms + 1500 ms = 2200 ms).
Revised: The experiments were not self-paced and therefore one trial always lasted 2200 ms (300 ms + 200 ms + 200 ms + 1500 ms = 2200 ms).

Original: The necessary time to complete an experiment with 1280 stimuli was 47 minutes, for 160 stimuli six minutes, and for 48 stimuli two minutes.
Revised: The necessary time to complete an experiment with 1280 stimuli was 47 minutes, for 160 stimuli six minutes, and for 48 stimuli two minutes.

Original: In the experiments with 1280 trials, observers were given the possibility of taking a brief break after every block oftrials (ﬁve blocks in total).
Revised: In the experiments with 1280 trials, observers were given the possibility of taking a brief break after every block oftrials (ﬁve blocks in total).

Original: As preparation, participants were shown the response screen prior to an experiment and were asked to name all 16 categories in order to get an overview over the possible stimuli categories and to make sure that all categories were clear from the beginning.
Revised: As preparation, participants were shown the response screen prior to an experiment and were asked to name all 16 categories in order to get an overview over the possible stimuli categories and to make sure that all categories were clear from the beginning.

Original: They were instructed to click on the category they believed was presented.
Revised: They were instructed to click on the category they believed was presented.

Original: Responses through clicking on a response screen could be changed within the 1500 ms response interval, only the last entered response was counted as the answer.
Revised: Responses through clicking on a response screen could be changed within the 1500 ms response interval, only the last entered response was counted as the answer.

Original: Prior to the real experiment a practice session was performed for the participants to get used to the time course of the experiment and the position of category items on the response screen.
Revised: Prior to the real experiment a practice session was performed for the participants to get used to the time course of the experiment and the position of category items on the response screen.

Original: This screen was shown for an additional 300 ms in order to provide feedback and indicate whether the entered answer was incorrect.
Revised: This screen was shown for an additional 300 ms in order to provide feedback and indicate whether the entered answer was incorrect.

Original: In that case, a short low beep sound occurred and the correct category was highlighted by setting its background to white.
Revised: In that case, a short low beep sound occurred and the correct category was highlighted by setting its background to white.

Original: The practice session consisted of 320 trials.
Revised: The practice session consisted of 320 trials.

Original: After 160 trials the participants had the chance to take a short break.
Revised: After 160 trials the participants had the chance to take a short break.

Original: In the break, their performance of the ﬁrst block was shown on the screen along the percentage of trials where no answer was entered.
Revised: In the break, their performance of the ﬁrst block was shown on the screen along the percentage of trials where no answer was entered.

Original: After the practice blocks, observers were shown an example image of the manipulation (not used in the experiment) to minimise surprise.
Revised: After the practice blocks, observers were shown an example image of the manipulation (not used in the experiment) to minimise surprise.

Original: Images used in the practice session were natural images from 16-class-ImageNet (Geirhos et al., 2018), hence there was no overlap with images or manipulations used in the experiments.[Table]
Revised: Images used in the practice session were natural images from 16-class-ImageNet (Geirhos et al., 2018), hence there was no overlap with images or manipulations used in the experiments.

Original: A. A PPARATUS
Revised: A. A PPARATUS

Original: Observers were shown the 224 × 224 pixels stimuli in a dark cabin on a 22”, 120 Hz VIEWPixx LCD monitor (VPixx Technologies, Saint-Bruno, Canada).
Revised: Observers were shown the 224 × 224 pixels stimuli in a dark cabin on a 22”, 120 Hz VIEWPixx LCD monitor (VPixx Technologies, Saint-Bruno, Canada).

Original: The screen of size 484 × 302 mm corresponds to 1920 × 1200 pixels, although stimuli were only presented foveally at the center of the screen ( 3 × 3 degrees of visual angle at a viewing distance of 107 cm) while the background was set to a grey value of 0.7614 in the [0 , 1] range, the average greyscale value of all stimuli used in the original experiment.
Revised: The screen of size 484 × 302 mm corresponds to 1920 × 1200 pixels, although stimuli were only presented foveally at the center of the screen ( 3 × 3 degrees of visual angle at a viewing distance of 107 cm) while the background was set to a grey value of 0.7614 in the [0 , 1] range, the average greyscale value of all stimuli used in the original experiment.

Original: Participants used a chin rest to keep their head position static during an experiment.
Revised: Participants used a chin rest to keep their head position static during an experiment.

Original: Stimulus presentation was conducted with the Psychophysics Toolbox (version 3.0.12) in MATLAB (Release 2016a, The MathWorks, Inc., Natick, Massachusetts, United States) using a 12-core desktop computer (AMD HD7970 graphics card “Tahiti” by AMD, Sunnyvale, California,
Revised: Stimulus presentation was conducted with the Psychophysics Toolbox (version 3.0.12) in MATLAB (Release 2016a, The MathWorks, Inc., Natick, Massachusetts, United States) using a 12-core desktop computer (AMD HD7970 graphics card “Tahiti” by AMD, Sunnyvale, California,

Original: United States) running Kubuntu 14.04 LTS.
Revised: United States) running Kubuntu 14.04 LTS.

Original: Participants clicked on a response screen, showing an iconic representation of all of the 16 object categories as reported in Geirhos et al.
Revised: Participants clicked on a response screen, showing an iconic representation of all of the 16 object categories as reported in Geirhos et al.

Original: (2018), with a normal computer mouse.
Revised: (2018), with a normal computer mouse.

Original: A. P ARTICIPANTS
Revised: A. P ARTICIPANTS

Original: In total, 97 human observers participated in the study.
Revised: In total, 97 human observers participated in the study.

Original: For a detailed overview about how they were distributed across experiments see Table 3.
Revised: For a detailed overview about how they were distributed across experiments see Table 3.

Original: No observer participated in more than one experiment, and all participants reported normal or corrected-to-normal vision.
Revised: No observer participated in more than one experiment, and all participants reported normal or corrected-to-normal vision.

Original: Observers participating in  experiments with a cue conﬂict manipulation were paid e 10 per hour or gained course credit.
Revised: Observers participating in [Table] experiments with a cue conﬂict manipulation were paid e 10 per hour or gained course credit.

Original: Observers measured in all other experiments (with a clear ground truth category) were able to earn an additional bonus up to e 5 or equivalent further course credit based on their performance.
Revised: Observers measured in all other experiments (with a clear ground truth category) were able to earn an additional bonus up to e 5 or equivalent further course credit based on their performance.

Original: This motivation scheme was applied to ensure reliable answer rates, and explained to observers in advance.
Revised: This motivation scheme was applied to ensure reliable answer rates, and explained to observers in advance.

Original: Participant bonus, in these cases, was calculated as follows:
Revised: Participant bonus, in these cases, was calculated as follows:

Original: The base level with a bonus of e 0 was set to 50% accuracy.
Revised: The base level with a bonus of e 0 was set to 50% accuracy.

Original: For every additional 5% of accuracy, participants gained a e 0.50 bonus.
Revised: For every additional 5% of accuracy, participants gained a e 0.50 bonus.

Original: This means that with a performance above 95%, an observer was able to gain the full bonus of e 5 or equivalent course credit.
Revised: This means that with a performance above 95%, an observer was able to gain the full bonus of e 5 or equivalent course credit.

Original: Overall, we took the following steps to prevent low quality human data: 1., using a controlled lab environment instead of an online crowdsourcing platform; 2.
Revised: Overall, we took the following steps to prevent low quality human data: 1., using a controlled lab environment instead of an online crowdsourcing platform; 2.

Original: the payment motivation scheme as explained above; 3. displaying observer performance on the screen at regular intervals during the practice session; and 4.
Revised: the payment motivation scheme as explained above; 3. displaying observer performance on the screen at regular intervals during the practice session; and 4.

Original: splitting longer experiments into ﬁve blocks, where participants could take a break in between blocks.
Revised: splitting longer experiments into ﬁve blocks, where participants could take a break in between blocks.

Original: A. CNN MODELS & TRAINING DETAILS
Revised: A. CNN MODELS & TRAINING DETAILS

Original: ResNet-50 We used a standard ResNet-50 architecture from PyTorch (Paszke et al., 2017), the torchvision.models.resnet50 implementation.
Revised: ResNet-50 We used a standard ResNet-50 architecture from PyTorch (Paszke et al., 2017), the torchvision.models.resnet50 implementation.

Original: For the comparison against BagNets reported in Table 1, results for IN training correspond to a ResNet-50 pre-trained on ImageNet without any modiﬁcations (model weights from torchvision.models ).
Revised: For the comparison against BagNets reported in Table 1, results for IN training correspond to a ResNet-50 pre-trained on ImageNet without any modiﬁcations (model weights from torchvision.models ).

Original: Reported results for SIN training correspond to the same architecture trained on SIN for 60 epochs with Stochastic Gradient
Revised: Reported results for SIN training correspond to the same architecture trained on SIN for 60 epochs with Stochastic Gradient

Original: Descent ( torch.optim.
Revised: Descent ( torch.optim.

Original: SGD ) using a momentum term of 0.9, weight decay (1e-4) and a learning rate of 0.1 which was multiplied by a factor of 0.1 after 20 and 40 epochs of training.
Revised: SGD ) using a momentum term of 0.9, weight decay (1e-4) and a learning rate of 0.1 which was multiplied by a factor of 0.1 after 20 and 40 epochs of training.

Original: We used a batch size of 256.
Revised: We used a batch size of 256.

Original: This SIN-trained model is the same model that is reported in Figures 5 and 6 as well as in Table 2.
Revised: This SIN-trained model is the same model that is reported in Figures 5 and 6 as well as in Table 2.

Original: In the latter, this corresponds to the second row (training performed on SIN, no ﬁne-tuning on ImageNet).
Revised: In the latter, this corresponds to the second row (training performed on SIN, no ﬁne-tuning on ImageNet).

Original: For the model reported in the third row, training was jointly performed on
Revised: For the model reported in the third row, training was jointly performed on

Original: SIN and on IN.
Revised: SIN and on IN.

Original: This means that both training data sets were treated as one big data set (exactly twice the size of the IN training data set), on which training was performed for 45 epochs with identical hyperparameters as described above, except that the initial learning rate of 0.1 was multiplied by 0.
Revised: This means that both training data sets were treated as one big data set (exactly twice the size of the IN training data set), on which training was performed for 45 epochs with identical hyperparameters as described above, except that the initial learning rate of 0.1 was multiplied by 0.

Original: after 15 and 30 epochs.
Revised: after 15 and 30 epochs.

Original: The weights of this model were then used to initialise the model reported in the fourth row of Table 2, which was ﬁne-tuned for 60 epochs on ImageNet (identical hyperparameters except that the initial learning rate of 0.01 was multiplied by 0.1 after 30 epochs).
Revised: The weights of this model were then used to initialise the model reported in the fourth row of Table 2, which was ﬁne-tuned for 60 epochs on ImageNet (identical hyperparameters except that the initial learning rate of 0.01 was multiplied by 0.1 after 30 epochs).

Original: We compared training models from scratch versus starting from an ImageNet-pretrained model.
Revised: We compared training models from scratch versus starting from an ImageNet-pretrained model.

Original: Empirically, using features pre-trained on ImageNet led to better results across experiments, which is why we used ImageNet pre-training throughout experiments and models (for both ResNet-50 and restricted ResNet-50 models).
Revised: Empirically, using features pre-trained on ImageNet led to better results across experiments, which is why we used ImageNet pre-training throughout experiments and models (for both ResNet-50 and restricted ResNet-50 models).

Original: All model weights will be released along with trainig code after the anonymous review period.

Original: BagNets Model weights (pre-trained on ImageNet) and architectures for BagNets (results reported in Table 1) were kindly provided by Anonymous (2018).
Revised: BagNets Model weights (pre-trained on ImageNet) and architectures for BagNets (results reported in Table 1) were kindly provided by Anonymous (2018).

Original: For SIN training, identical settings as for the SIN-trained ResNet-50 were used to ensure comparability (training for 60 epochs with SGD and identical hyperparameters as reported above).
Revised: For SIN training, identical settings as for the SIN-trained ResNet-50 were used to ensure comparability (training for 60 epochs with SGD and identical hyperparameters as reported above).

Original: Faster R-CNN We used the Faster R-CNN implementation from https://github.com/ jwyang/faster-rcnn.pytorch (commit 21f289867d09f410631f1b9b7b46ce8c1e81ae07) with all hyperparameters kept at default.
Revised: Faster R-CNN We used the Faster R-CNN implementation from https://github.com/ jwyang/faster-rcnn.pytorch (commit 21f289867d09f410631f1b9b7b46ce8c1e81ae07) with all hyperparameters kept at default.

Original: The only change we made to the model is replacing the encoder with a ResNet-50 and applying custom input whitening.
Revised: The only change we made to the model is replacing the encoder with a ResNet-50 and applying custom input whitening.

Original: We trained this model on Pascal
Revised: We trained this model on Pascal

Original: VOC 2007 for 7 epochs with a batch size of 1, a learning rate of 0.001 and a learning rate decay step after epoch 5.
Revised: VOC 2007 for 7 epochs with a batch size of 1, a learning rate of 0.001 and a learning rate decay step after epoch 5.

Original: Pre-trained AlexNet, GoogLeNet, VGG-16 We used AlexNet (Krizhevsky et al., 2012),
Revised: Pre-trained AlexNet, GoogLeNet, VGG-16 We used AlexNet (Krizhevsky et al., 2012),

Original: GoogLeNet (Szegedy et al., 2015) and VGG-16 (Simonyan & Zisserman, 2015) for the evaluation reported in Figure 4.
Revised: GoogLeNet (Szegedy et al., 2015) and VGG-16 (Simonyan & Zisserman, 2015) for the evaluation reported in Figure 4.

Original: Evaluation was performed using Caffe (Jia et al., 2014).
Revised: Evaluation was performed using Caffe (Jia et al., 2014).

Original: Network weights (training on ImageNet) were obtained from https://github.com/BVLC/caffe/ wiki/Model-Zoo (AlexNet & GoogLeNet) and http://www.robots.ox.ac.uk/ (VGG16).
Revised: Network weights (training on ImageNet) were obtained from https://github.com/BVLC/caffe/ wiki/Model-Zoo (AlexNet & GoogLeNet) and http://www.robots.ox.ac.uk/ (VGG16).

Original: ResNet-101 pre-trained on Open Images V2 For our comparison of biases in ImageNet vs.
Revised: ResNet-101 pre-trained on Open Images V2 For our comparison of biases in ImageNet vs.

Original: OpenImages (Figure 13 right) the ResNet-101 pretrained on Open Images V2 (Krasin et al., 2017) was used.
Revised: OpenImages (Figure 13 right) the ResNet-101 pretrained on Open Images V2 (Krasin et al., 2017) was used.

Original: It was obtained from https://github.com/openimages/dataset/blob/ master/READMEV2.md along with the inference code provided by the authors.
Revised: It was obtained from https://github.com/openimages/dataset/blob/ master/READMEV2.md along with the inference code provided by the authors.

Original: In order to map predictions to the 16 classes, we used the parameters top k = 100000 and score threshold = 0 .to obtain as all predictions, and then mapped the responses to our 16 classes using the provided label map.
Revised: In order to map predictions to the 16 classes, we used the parameters top k = 100000 and score threshold = 0 .to obtain as all predictions, and then mapped the responses to our 16 classes using the provided label map.

Original: 15 out of our 16 classes are classes in Open Images as well; the remaining class keyboard was mapped to Open Images class computer keyboard (in this case, Open Images makes a ﬁner distinction to separate musical keyboards from computer keyboards).
Revised: 15 out of our 16 classes are classes in Open Images as well; the remaining class keyboard was mapped to Open Images class computer keyboard (in this case, Open Images makes a ﬁner distinction to separate musical keyboards from computer keyboards).

Original: ResNet-101, ResNet-152, DenseNet-121, SqueezeNet1 1 For the comparison to other models pre-trained on ImageNet (Figure 13 left), we evaluated the pre-trained networks provided by torchvision.models .
Revised: ResNet-101, ResNet-152, DenseNet-121, SqueezeNet1 1 For the comparison to other models pre-trained on ImageNet (Figure 13 left), we evaluated the pre-trained networks provided by torchvision.models .

Original: Training AlexNet, VGG-16 on SIN For the evaluation of model biases after training on SIN (Figure 11), we obtained the model architectures from torchvision.models and trained the networks under identical circumstances as ResNet-50.
Revised: Training AlexNet, VGG-16 on SIN For the evaluation of model biases after training on SIN (Figure 11), we obtained the model architectures from torchvision.models and trained the networks under identical circumstances as ResNet-50.

Original: This includes identical hyperparameter settings, except for the learning rate.
Revised: This includes identical hyperparameter settings, except for the learning rate.

Original: The learning rate for AlexNet was set to 0.001 and for VGGto 0.01 initially; both learning rates were multiplied by 0.1 after 20 and 40 epochs of training ( epochs in total).
Revised: The learning rate for AlexNet was set to 0.001 and for VGGto 0.01 initially; both learning rates were multiplied by 0.1 after 20 and 40 epochs of training ( epochs in total).

Original: A. I MAGE MANIPULATIONS AND IMAGE DATABASE
Revised: A. I MAGE MANIPULATIONS AND IMAGE DATABASE

Original: In total, we conducted nine different experiments.
Revised: In total, we conducted nine different experiments.

Original: Here is an overview of the images and / or image manipulations for all of them.
Revised: Here is an overview of the images and / or image manipulations for all of them.

Original: All images were saved in the png format and had a size of 224 ×pixels.
Revised: All images were saved in the png format and had a size of 224 ×pixels.

Original: Original, texture and cue conﬂict images are visualised in Figure 7.
Revised: Original, texture and cue conﬂict images are visualised in Figure 7.

Original: Original experiment This experiment consisted of 160 coloured images, 10 per category.
Revised: Original experiment This experiment consisted of 160 coloured images, 10 per category.

Original: All of them had a single, unmanipulated object (belonging to one category) in front of a white background.
Revised: All of them had a single, unmanipulated object (belonging to one category) in front of a white background.

Original: This white background was especially important since these stimuli were being used as content images for style transfer, and we thus made sure that the background was neutral to produce better style transfer results.
Revised: This white background was especially important since these stimuli were being used as content images for style transfer, and we thus made sure that the background was neutral to produce better style transfer results.

Original: The images for this experiment as well as for the texture experiment described below were carefully selected using Google advanced image search with the criteria “labelled for noncommercial reuse with modiﬁcation (free to use, share and modify)” and the search term “ < entity > white background” (original) or “ < entity > texture” (texture).
Revised: The images for this experiment as well as for the texture experiment described below were carefully selected using Google advanced image search with the criteria “labelled for noncommercial reuse with modiﬁcation (free to use, share and modify)” and the search term “ < entity > white background” (original) or “ < entity > texture” (texture).

Original: In some cases where this did not lead to sufﬁcient results, we used images from the ImageNet validation data set which were manually modiﬁed to have a white background if necessary.
Revised: In some cases where this did not lead to sufﬁcient results, we used images from the ImageNet validation data set which were manually modiﬁed to have a white background if necessary.

Original: We made sure that both the images from this experiment as well as the texture images were all correctly recognised by all four pre-trained CNNs (if an image was not correctly recognised, we replaced it by another one).
Revised: We made sure that both the images from this experiment as well as the texture images were all correctly recognised by all four pre-trained CNNs (if an image was not correctly recognised, we replaced it by another one).

Original: This was used to ensure that our results for cue conﬂict experiments are fully interpretable: if, e.g., a texture image was not correctly recognised by CNNs, there would be no point in using it as a texture (style) source for style transfer.
Revised: This was used to ensure that our results for cue conﬂict experiments are fully interpretable: if, e.g., a texture image was not correctly recognised by CNNs, there would be no point in using it as a texture (style) source for style transfer.

Original: Greyscale experiment This experiment used the same images as the original experiment with the difference that they were converted to greyscale using skimage.color.rgb2gray .
Revised: Greyscale experiment This experiment used the same images as the original experiment with the difference that they were converted to greyscale using skimage.color.rgb2gray .

Original: For CNNs, greyscale images were stacked three times along the colour channel.
Revised: For CNNs, greyscale images were stacked three times along the colour channel.

Original: Silhouette experiment The images from the original experiment were transformed into silhouette images showing an entirely black object on a white background.
Revised: Silhouette experiment The images from the original experiment were transformed into silhouette images showing an entirely black object on a white background.

Original: We used the following transformation procedure: First, images were converted to bmp using command line utility ( convert ).
Revised: We used the following transformation procedure: First, images were converted to bmp using command line utility ( convert ).

Original: They were then converted to svg using potrace , and then to png using convert again.
Revised: They were then converted to svg using potrace , and then to png using convert again.

Original: Since an entirely automatic binarization pipeline is not feasible (it takes domain knowledge to understand that a car wheel should, but a doughnut should not be ﬁlled with black colour), we then manually checked every single image and adapted the silhouette using GIMP if necessary.
Revised: Since an entirely automatic binarization pipeline is not feasible (it takes domain knowledge to understand that a car wheel should, but a doughnut should not be ﬁlled with black colour), we then manually checked every single image and adapted the silhouette using GIMP if necessary.

Original: Edge experiment The stimuli shown in this condition were generated by applying the “Canny” edge extractor implemented in MATLAB (Release 2016a, The MathWorks, Inc., Natick, Massachusetts, United States) to the images used in the original experiment.
Revised: Edge experiment The stimuli shown in this condition were generated by applying the “Canny” edge extractor implemented in MATLAB (Release 2016a, The MathWorks, Inc., Natick, Massachusetts, United States) to the images used in the original experiment.

Original: No further manipulations were performed on this data set.
Revised: No further manipulations were performed on this data set.

Original: This line of code was used to detect edges and generate the stimuli used in this experiment: imwrite(1-edge(imgaussfilt(rgb2gray(imread(filename)), 2), ’Canny’), targetFilename);
Revised: This line of code was used to detect edges and generate the stimuli used in this experiment: imwrite(1-edge(imgaussfilt(rgb2gray(imread(filename)), 2), ’Canny’), targetFilename);

Original: Texture experiment Images were selected using the procedure outlined above for the original experiment.
Revised: Texture experiment Images were selected using the procedure outlined above for the original experiment.

Original: Some objects have a fairly stationary texture (e.g. animals), which makes it easy to ﬁnd texture images for them.
Revised: Some objects have a fairly stationary texture (e.g. animals), which makes it easy to ﬁnd texture images for them.

Original: For the more difﬁcult case (e.g. man-made objects), we made use of the fact that every object can become a texture if it is used not in isolation, but rather in a clutter of many objects of the same kind (e.g. Gatys et al., 2017).
Revised: For the more difﬁcult case (e.g. man-made objects), we made use of the fact that every object can become a texture if it is used not in isolation, but rather in a clutter of many objects of the same kind (e.g. Gatys et al., 2017).

Original: That is, for a bottle texture we used images with many bottles next to each other (as visualised in Figure 7).
Revised: That is, for a bottle texture we used images with many bottles next to each other (as visualised in Figure 7).

Original: Cue conﬂict experiment This experiment used images with a texture-shape cue conﬂict.
Revised: Cue conﬂict experiment This experiment used images with a texture-shape cue conﬂict.

Original: They were generated using iterative style transfer (Gatys et al., 2016) between a texture image (from the texture experiment described above) and a content image (from the original experiment) each.
Revised: They were generated using iterative style transfer (Gatys et al., 2016) between a texture image (from the texture experiment described above) and a content image (from the original experiment) each.

Original: While 48 texture images and 160 content images would allow for a total of 48 × 160 = 7680 cue conﬂict images (480 per category), we used a balanced subset of 1280 images instead (80 per category), which allows for presentation to human observers within a single experimental session.
Revised: While 48 texture images and 160 content images would allow for a total of 48 × 160 = 7680 cue conﬂict images (480 per category), we used a balanced subset of 1280 images instead (80 per category), which allows for presentation to human observers within a single experimental session.

Original: The procedure for selecting the style and content images was done as follows.
Revised: The procedure for selecting the style and content images was done as follows.

Original: For all possible 16 ×combinations of style and texture categories, exactly ﬁve cue conﬂict images were generated by randomly sampling style and content images from their respective categories.Sampling was per- o r i g i n a l c on t e n t
Revised: For all possible 16 ×combinations of style and texture categories, exactly ﬁve cue conﬂict images were generated by randomly sampling style and content images from their respective categories.

Original: i m a g e s o r

Original: i g i n a l t e x t u r e i m a g e s c u e c on ﬂ i c t ( ﬁ ll e d s il hou e tt e s ) c u e c on ﬂ i c t ( s t y l e t r a n s f e r ) formed without replacement for as long as possible, and then without replacement for the remaining images.
Revised: Sampling was performed without replacement for as long as possible, and then without replacement for the remaining images.

Original: The same stimuli acquired with this method were used for the cue conﬂict control experiments, where participants saw exactly these images but with different instructions biased towards shape and towards texture (results described later).
Revised: The same stimuli acquired with this method were used for the cue conﬂict control experiments, where participants saw exactly these images but with different instructions biased towards shape and towards texture (results described later).

Original: For our analysis of texture vs. shape biases (Figure 4), we excluded trials for which no cue conﬂict was present (i.e., those trials where a bicycle content image was fused with a bicycle texture image, hence no texture-shape cue conﬂict present).
Revised: For our analysis of texture vs. shape biases (Figure 4), we excluded trials for which no cue conﬂict was present (i.e., those trials where a bicycle content image was fused with a bicycle texture image, hence no texture-shape cue conﬂict present).

Original: Filled silhouette experiment Style transfer is not the only possibility to generate a texture-shape cue conﬂict, and we here aimed at testing one other method to generate such stimuli: cropping texture images with a shape mask, such that the silhouette of an object and its texture constitute a cue conﬂict (visualised in Figure 7).
Revised: Filled silhouette experiment Style transfer is not the only possibility to generate a texture-shape cue conﬂict, and we here aimed at testing one other method to generate such stimuli: cropping texture images with a shape mask, such that the silhouette of an object and its texture constitute a cue conﬂict (visualised in Figure 7).

Original: Stimuli were generated by using the silhouette images from the silhouette experiment as a mask for texture images.
Revised: Stimuli were generated by using the silhouette images from the silhouette experiment as a mask for texture images.

Original: If the silhouette image at a certain location has a black pixel, the texture was used at this location, and for white pixels the resulting target image pixel was white.
Revised: If the silhouette image at a certain location has a black pixel, the texture was used at this location, and for white pixels the resulting target image pixel was white.

Original: In order to have a larger variety of textures than the 48 textures used in the texture experiment, the texture database was augmented by rotating all textures with ten different previously chosen angles uniformly distributed between 0 and 360 degrees, resulting in a texture database of 480 images.
Revised: In order to have a larger variety of textures than the 48 textures used in the texture experiment, the texture database was augmented by rotating all textures with ten different previously chosen angles uniformly distributed between 0 and 360 degrees, resulting in a texture database of 480 images.

Original: Results for this control experiment, not part of the main paper, are reported later.
Revised: Results for this control experiment, not part of the main paper, are reported later.

Original: We ensured that no silhouette was seen more than once per observer.
Revised: We ensured that no silhouette was seen more than once per observer.

Original: Robustness experiment (distorted images)
Revised: Robustness experiment (distorted images)

Original: For this experiment, human accuracies for reference were provided by Geirhos et al.
Revised: For this experiment, human accuracies for reference were provided by Geirhos et al.

Original: Human ‘error bars’ indicate the full range of results for human observers.
Revised: Human ‘error bars’ indicate the full range of results for human observers.

Original: CNNs were then evaluated on different image manipulations applied to natural images as outlined in the paper.
Revised: CNNs were then evaluated on different image manipulations applied to natural o r i g i n a l c on t e n t i m a g e s o r i g i n a l t e x t u r e

Original:  For maximal comparability, we also used the same images.
Revised: i m a g e s c u e c on ﬂ i c t ( ﬁ ll e d s il hou e tt e s ) c u e c on ﬂ i c t ( s t y l e t r a n s f e r ) images as outlined in the paper. For maximal comparability, we also used the same images.

Original: For a description of the parametric distortion we kindly refer the reader to Geirhos et al.
Revised: For a description of the parametric distortion we kindly refer the reader to Geirhos et al.

Original: In
Revised: In

Original: Figure 8, we plot one example image across manipulations.
Revised: Figure 8, we plot one example image across manipulations.

Original: A. S TYLIZED -I MAGE N ET (SIN)
Revised: A. S TYLIZED -I MAGE N ET (SIN)

Original: We used AdaIN style transfer (Huang & Belongie, 2017) to generate Stylized-ImageNet.
Revised: We used AdaIN style transfer (Huang & Belongie, 2017) to generate Stylized-ImageNet.

Original: More speciﬁcally, the AdaIN implementation from https://github.com/naoto0804/ pytorch-AdaIN (commit 31e769c159d4c8639019f7db7e035a7f938a6a46) was employed to stylize the entire ImageNet training and validation data sets.
Revised: More speciﬁcally, the AdaIN implementation from https://github.com/naoto0804/ pytorch-AdaIN (commit 31e769c159d4c8639019f7db7e035a7f938a6a46) was employed to stylize the entire ImageNet training and validation data sets.

Original: Style transfer was performed once per ImageNet image.
Revised: Style transfer was performed once per ImageNet image.

Original: As a style source, we used images from Kaggle’s Painter by Numbers data set ( https://www.kaggle.com/c/painter-by-numbers/ , accessed on March 1, 2018).
Revised: As a style source, we used images from Kaggle’s Painter by Numbers data set ( https://www.kaggle.com/c/painter-by-numbers/ , accessed on March 1, 2018).

Original: Style selection was performed randomly with replacement.
Revised: Style selection was performed randomly with replacement.

Original: Every ImageNet image was stylized once and only once.
Revised: Every ImageNet image was stylized once and only once.

Original: Paintings from the Kaggle data set were used if at least 224 × 
Revised: Paintings from the Kaggle data set were used if at least 224 × 224 pixels in size; the largest possible square crop was then downsampled to this size prior to using it as a style image.

Original: [Figure] 
Revised: All accuracies are reported on the respective validation data sets. Code to generate StylizedImageNet from ImageNet (and the Kaggle paintings) is available on github in this repository: https://github.com/rgeirhos/Stylized-ImageNet A.

Original: −
Revised: R ESULTS : CUE CONFLICT CONTROL EXPERIMENTS ( DIFFERENT INSTRUCTIONS ) We investigated the effect of different instructions to human observers. The results presented in the main paper for cue conﬂict stimuli correspond all to a neutral instruction, not biased w.r.t.

Original: [Figure]
Revised: texture or shape. In two separate experiments, participants were explicitly instructed to ignore the textures and click on the shape category of cue conﬂict stimuli, and vice versa. The results, presented in Figure 10, indicate that for a shape bias instruction, human data are almost exactly the same as for the neutral instruction reported earlier (indicating that human observers are indeed using shapes per default); and if they are instructed to ignore the shapes and click on the texture category, they still show a substantial shape bias (indicating that even if they seek to ignore shapes, they ﬁnd it extremely difﬁcult to do so).

Original: −
Revised: This experiment was conducted as a control experiment to make sure that the strong differences between humans and CNNs when presented with cue conﬂict images are not merely an artefact of the particular setup that we employed. Stimuli are visualised in Figure 7; results in Figure 12. In a

Original: [Figure]
Revised: [Figure]

Original: −
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Revised: H

Original: [Figure]
Revised: [Figure]

Revised: [Table]

Original:  Hu
Revised: s H

Original: [Table]
Revised: [Table]

Original: an s
Revised: an s

Revised: H

Revised: [Table]

Original: H u m an s 
Revised: m an s H u

Revised: [Table]

Original: H u m an s 
Revised: an s H

Revised: [Table]

Original: H u m an s 
Revised: an s H

Original: H u m an s
Revised: [Table] an s

Original: H
Revised: H

Original: [Figure]
Revised: [Table]

Original: m an s
Revised: m an s

Revised: H u m an s

Original: H u m an s H u m an s 
Revised: H u m an s H u m an s H

Original: H u m an s
Revised: [Table] an s

Original: H
Revised: H

Original: [Table]
Revised: [Table]

Original: m an s
Revised: an s

Original: H
Revised: H

Original: [Table]
Revised: [Table]

Original: m an s
Revised: an s

Original: H
Revised: H

Original: [Figure]
Revised: [Table]

Original: m an s
Revised: m an s

Revised: H u m an s

Original: H 
Revised: H u

Original: [Table]
Revised: [Table]

Original: an s
Revised: an s

Original: H u m an s

Original: H u m an s
Revised: H u m an s

Original: H
Revised: H

Original: [Table]
Revised: [Table]

Original: an s H u [Table]
Revised: s H u m an s

Original: an s H 
Revised: H u

Original: [Figure]
Revised: [Table]

Original: an s
Revised: an s

Original: H u m an sH u m an s H u m an s
Revised: H u m an s

Original: H u m an s
Revised: H u m an s

Original: H
Revised: H

Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: s
Revised: s

Original: H
Revised: H

Original: [Figure]

Original: an s

Original: [Figure]

Original: [Table]
Revised: [Table]

Original: s
Revised: s

Revised: H

Original: [Figure]
Revised: [Figure]

Original: an s

Original: [Figure]
Revised: [Table]

Original: s
Revised: s

Original: [Figure]
Revised: [Figure]

Original: s
Revised: s

Original: [Figure]
Revised: [Figure]

Original: an
Revised: an

Original: [Figure]
Revised: [Figure]

Revised: [Table]

Original: an s
Revised: an s

Original: H u m an s H u m an s
Revised: H u m an s H u m an s

Original: H u m an s H u m an s
Revised: H u m an s H u m an s

Original: H u m an s H u m an s
Revised: H u m an s H u m an s

Original: H u m an s H u m an s
Revised: H u m an s H u m an s

Original: H u m an s H u m an s
Revised: H u m an s H u m an s

Original: H u m an s H u m an s
Revised: H u m an s H u m an s

Original: H u m an s H u m an s
Revised: H u m an s H u m an s

Original: H u m an s 100       100 original{+
 [Figure]+} greyscale silhouetteedges texture pixels in size; the largest possible square crop was then downsampled to this size prior to using it as a style image.
Revised: H u m an s 100       {+
 [Figure]+} greyscale silhouette

Original: All accuracies are reported on the respective validation data sets. Code to generate Stylized-ImageNet from ImageNet (and the Kaggle paintings) will be made available on github after the anonymous review period. A. R ESULTS : CUE CONFLICT CONTROL EXPERIMENTS ( DIFFERENT INSTRUCTIONS )
Revised: [Figure]

Original: We investigated the effect of different instructions to human observers.
Revised: [Figure]

Original: The results presented in the main paper for cue conﬂict stimuli correspond all to a neutral instruction, not biased w.r.t.
Revised: S hape c a t ego r i e s lll l llllllllllll ll l

Original: texture or shape.
Revised: [Figure]

Original: In two separate experiments, participants were explicitly instructed to ignore the textures and click on the shape category of cue conﬂict stimuli, and vice versa. The results, presented in
Revised: lll

Original: Figure 10, indicate that for a shape bias instruction, human data are almost exactly the same as for the neutral instruction reported earlier (indicating that human observers are indeed using shapes per default); and if they are instructed to ignore the shapes and click on the texture category, they still show a substantial shape bias (indicating that even if they seek to ignore shapes, they ﬁnd it extremely difﬁcult to do so)
Revised: [Figure]

Original: . A. R ESULTS : FILLED SILHOUETTE EXPERIMENT
Revised: llllllllllll l l l l l l l l l l l l l l l l

Original: This experiment was conducted as a control experiment to make sure that the strong differences between humans and CNNs when presented with  cue conﬂict images are not merely an artefact of the particular setup that we employed.
Revised: [Figure] grey squares) on stimuli with a texture-shape cue conﬂict generated with style transfer, and biased rather than neutral instructions to human observers. Plotting conventions and CNN data as in Figure 4.

Original: Stimuli are visualised in Figure 7; results in Figure 12.
Revised: [Figure] S hape c a t ego r i e s l l ll l l l l l llll l l l

Original: In a nutshell, we also ﬁnd a shape bias in humans when stimuli are not generated via style transfer but instead through cropping texture images with a shape mask, such that the silhouette of an object and its texture constitute a cue conﬂict.
Revised: Plotting conventions as in Figures 4 and since network not provide nutshell, we also ﬁnd a shape bias in humans when stimuli are not generated via style transfer but instead through cropping texture images with a shape mask, such that the silhouette of an object and its texture constitute a cue conﬂict.

Original: CNNs have a less pronounced texture bias in these experiments;
Revised: CNNs have a less pronounced texture bias in these experiments;

Original: ResNet-50 trained on SIN still responds with the shape category more than ResNet-50 trained on
Revised: ResNet-50 trained on SIN still responds with the shape category more than ResNet-50 trained on

Original: IN.
Revised: IN.

Original: Overall, these results are much more difﬁcult to interpret since the texture-silhouette cue conﬂict stimuli, visualised in Figure 7, do not have a clear-cut texture-shape distinction like the cue conﬂict stimuli generated via style transfer.
Revised: Overall, these results are much more difﬁcult to interpret since the texture-silhouette cue conﬂict stimuli, visualised in Figure 7, do not have a clear-cut texture-shape distinction like the cue conﬂict stimuli generated via style transfer.

Original: Still, they are largely in accord with the style transfer results presented in the main paper.
Revised: Still, they are largely in accord with the style transfer results presented in the main paper.

Original: A. I MAGE RIGHTS & ATTRIBUTION
Revised: A. I MAGE RIGHTS & ATTRIBUTION

Original: The images presented in Figure 7 were collected from different origins.
Revised: The images presented in Figure 7 were collected from different origins.

Original: We here indicate their URL, creator and license terms (if applicable).
Revised: We here indicate their URL, creator and license terms (if applicable).

Original: Some of the images presented in Figure 7 also appear in Figures 1, 2 and 9; the terms below apply accordingly.
Revised: Some of the images presented in Figure 7 also appear in Figures 1, 2 and 9; the terms below apply accordingly.

Original: Top row, cat image: https://pixabay.com/p-964343/ , released under the CC0 creative commons license as indicated on the website.
Revised: image: https://pixabay.com/p-964343/ , released under the CC0 creative commons license as indicated on the website.

Original: The CC0 creative commons license is accessible from https://creativecommons.org/publicdomain/zero/1.0/legalcode .
Revised: The CC0 creative commons license is accessible from https://creativecommons.org/publicdomain/zero/1.0/legalcode .

Original: Car image: https://pixabay.com/p-1930237/ , released under the CC0 creative commons license as indicated on the website.
Revised: Car image: https://pixabay.com/p-1930237/ , released under the CC0 creative commons license as indicated on the website.

Original: Bear image: ImageNet image n02132136871.JPEG , manually modiﬁedto have a white background.
Revised: image n02132136 manually modiﬁed

Original: Secondrow, elephant texture cropped from https://www.flickr.com/photos/flowcomm/5089601226 ,released under 
Revised: Second from https://www.flickr.com/photos/flowcomm/5089601226 , under BY 0 license by user flowcomm as indicated on website license is accessible from https://creativecommons.org/licenses/by/2.0/legalcode . Clock texture: https://

Original: [Figure]
Revised: .wikimedia org/

Original: S hape c a t ego r i e s l l l ll l l l llllll l l l l l l l l l l l ll l l l l l
Revised: HK_Sheung_Wan_%E4%B8%AD%E6%BA%90%E4%B8%AD%E5%BF%83_Midland_Plaza_ shop_Japan_Home_City_clocks_displayed_for_sale_April org/licenses/ sa/ Bottle texture: cropped from

Original: [Figure]
Revised: [Figure]

Original: l l ll l l l l l llll l l l l l l l l l l l l l l l l l l l

Original: [Figure]
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: llllllllllllllll

Original: Plotting conventions as in Figures 4 and 5.

Original: Plot shows biases for AlexNet (purple diamonds), VGG16 (blue triangles) and human observers (red circles) for comparison.

Original: For GoogLeNet, no data is available since network training was performed in PyTorch and torchvision.models unfortunately does not provide a GoogLeNet (inception v1) architecture.

Original: Fraction of 'texture' decisions
Revised: Fraction of 'texture' decisions

Original: Fraction of 'shape' decisions
Revised: Fraction of 'shape' decisions

Original: S hape c a t ego r i e s l l ll l l l l l llll l l l l l ll l l l l l l l l l l l l[-0 0.1 0.2 0.3 0.4 0.5 0.
-]
Revised: S hape c a t ego r i e s l l ll l l l l l llll l l l l l ll l l l l l l l l l l l l[-0 0.1 0.2 0.3 0.4 0.5 0.
-]

Original:  Fraction of 'texture' decisions
Revised: [Figure] Fraction of 'texture [Figure] decisions

Original: Fraction of 'shape' decisions
Revised: Fraction of 'shape' decisions

Original: S hape c a t ego r i e s l l l ll l l l l llll l l l  the CC BY 2.0 license by user flowcomm as indicated on the website.The license is accessible from https://creativecommons.org/licenses/by/2.0/legalcode .
Revised: S hape c a t ego r i e s l l l ll l l l l llll l l l //commons.wikimedia.org/wiki/File:Liquor_bottles.jpg , released under the CC BY 2.0 license by user scottfeldstein as indicated on the website.

Original: Clock texture: cropped from https://commons.wikimedia.org/wiki/File:

Original: HK_Sheung_Wan_%E4%B8%AD%E6%BA%90%E4%B8%AD%E5%BF%83_Midland_Plaza_ shop_Japan_Home_City_clocks_displayed_for_sale_April- jpg licenses/ Bottle texture: cropped from //commons.wikimedia.org/wiki/File:Liquor_bottles jpg released
Revised: The CC BY 2.0 license is accessible from https://creativecommons.org/licenses/by/2.0/legalcode .

Original: [Figure]
Revised: [Figure]

Original: S hape c a t ego r i e s llllllllllllll ll
Revised: S hape c a t ego r i e s llllllllllllll ll

Original: [Figure]
Revised: [Figure]

Original: ll
Revised: ll