What a difference a pixel makes: An empirical examination of features used by CNNs for categorisation

1 INTRODUCTION . Object recognition in humans is largely a function of analyzing shape ( Biederman , 1987 ; Hummel , 2013 ) . A wealth of data from psychological experiments show that shape plays a privileged role in object recognition compared to other diagnostic features such as size , colour , luminance or texture . For example , Biederman & Ju ( 1988 ) showed that error rates and reaction times are virtually identical in a recognition task when full coloured photographs of objects are replaced by their line drawings even when colour was a diagnostic feature . This indicates that shape-based representations mediate recognition . Similarly , Mapelli & Behrmann ( 1997 ) found that , for patients with an object recognition deficit ( visual agnosia ) , surface colour played minimal role in aiding object recognition unless the shape of the object was ambiguous , indicating that shape is instrumental to recognition , whereas surface characteristics such as colour and texture play only a secondary role . More recently , Baker & Kellman ( 2018 ) have shown that participants extract shape information automatically from arrays of dot patterns within the first 100ms of stimulus onset , even for tasks where extracting this information may be detrimental to performance on a task . Experiments from developmental psychology show that this privileged status of shape starts early in life and becomes stronger with age . For example , Landau et al . ( 1988 ) found that 2-3-year-old children as well as adults weight shape more heavily than size or texture when generalising the name of a learnt object to novel instances . They also found that the weight placed on shape increases in strength and generality from early childhood to adulthood . Following Landau et al . ( 1988 ) , we will call this privileged status of shape in performing recognition a “ shape bias ” . By contrast , it is unclear whether shape plays a privileged role in how convolutional neural networks ( CNNs ) categorise objects . It is often claimed that CNNs learn representations of objects that are similar to the representations that monkeys and humans use when identifying objects ( Rajalingham et al. , 2018 ) , and that CNNs largely rely on learning shape representations in order to categorise objects ( Kubilius et al. , 2016 ; Ritter et al. , 2017 ; Jozwik et al. , 2017 ) . On the other hand , there are a growing number of studies that show that CNNs often categorise images on the basis on nonshape attributes of images . This is demonstrated by the existence of adversarial images that are confidently classified as a familiar category despite the lack of any shape information in the input ( Nguyen et al. , 2015 ) , adversarial images that contain the correct shape but altered colours that are confidently misclassified ( e.g. , categorizing an image of an airplane as a dog when only the colour of the plane has been manipulated ) , and large reductions in performance when trained coloured images are converted to greyscale ( Geirhos et al. , 2017 ) or the colours are inverted ( Hosseini et al. , 2017 ) . In addition , there are demonstrations that CNNs can easily learn to categorise random patterns of pixels that have no shape ( Zhang et al. , 2016 ) . All of these findings suggest that shape may not play a privileged role in CNN ’ s object categorisation , or that the relevant role of shape and non-shape features depends on the specific model or training conditions . Here we systematically explore the impact of non-shape features in the categorisation performance of convolutional neural networks on CIFAR-10 images . We introduced non-shape features to images by adding informative noise-like masks to the training set . We tried several types of masks and an extreme version where the non-shape feature consisted of just a single pixel with a location correlated to the image category ( see Figure 1 and Appendix B ) . We show that CNNs often learn and depend on non-shape features that are highly diagnostic of object categories and often fails to learn anything about shape under these conditions . This highlights that CNNs simply picks up whatever statistical structure is most relevant to learning the training set , with shape playing no special role . Note that this does not imply that CNNs do not encode shape information under any circumstance , but that shape does not seem to be weighted more than other diagnostic features , even when these features are noise-like masks or the luminance of a single pixel . Importantly , this behaviour contrasts with humans , for whom shape plays a privileged role in performing recognition , even in the presence of much more salient diagnostic features such as size , colour or texture . If the models are to more closely capture human performance , these results suggest that additional machinery needs to be added to networks in order to prioritize the role of learning shape-based representations while performing the object categorisations . This might also reduce the model ’ s susceptibility to being fooled by non-shape features of images , and being more robust to various forms of non-shape noise that currently reduce performance . 2 EXPERIMENT PREMISE & DESIGN . We modified the CIFAR-10 dataset ( see Appendix A ) so that each image contained not only features that pertain to the shape ( e.g . object outlines ) but also features without any shape information . As non-shape features we used noise-like masks that were combined with the original image . Two different types of masks were used : the salt-and-pepper noise mask turned a certain proportion of image pixels to either black or white , while a additive uniform noise mask added a value sampled from a uniform distribution to each pixel of an image . We also tested an extreme form of the salt-andpepper noise mask where only one pixel was turned to a particular colour . In this case the location and colour of the pixel were different for different categories but correlated for images within a category . Masks were independently sampled for each category but were either fixed for all images in a category ( in which case the mask predicted the category ) or sampled from a distribution with category-dependent parameters ( in which case these parameters predicted the category ) . So these modified images concurrently contained features that were related to shape and features without shape information . Both types of features were predictive of the category of an image . Examples of modified images are given in Appendix B . We trained the model on these modified sets of images and tested it under three conditions . During the ‘ Same ’ condition , the test set was modified in exactly the same manner – i.e. , either images in each category were generated by using the same mask as that for the training images of that category ( when the mask was fixed ) or they were generated by using the same parameters as the parameters used to generate noise masks for training images of that category ( when the mask was variable ) . In contrast , during the ‘ Diff ’ condition , the noise masks ( or their parameters ) for each category were swapped with another category . The premise here was that if the model based it ’ s decisions on shape-related features , then it would ignore the noise mask and the performance during ‘ Same ’ and ‘ Diff ’ condition should be similar . On the other hand , if the model relied on properties of the ( nonshape ) mask , then it ’ s performance would be worse in the ‘ Diff ’ condition compared to the ‘ Same ’ condition . Finally , we used a third , ‘ NoPix ’ , condition to estimate the extent to which the network relied on features of the noise mask . In this condition , we presented the network with a version of the image without any mask , with the premise that the difference between the performance in ‘ Same ’ and ‘ NoPix ’ condition should quantify the relative extent to which the network relied on shape-based and non-shape features . We ran all of the simulations using the well-known VGG-16 network ( Simonyan & Zisserman , 2014 ) and checked that our main results replicate for a deeper network , ResNet-101 ( He et al. , 2016 ) . To give the model the best chance to recognise shape-based features , all simulations were carried out on CNNs that had previously been trained on ImageNet categories and replaced only the fully-connected layers to perform the new classification task . We then turned the learning rate to a small value and trained these networks on the new classification task ( see Appendix C for simulations were learning was completely frozen in convolutional layers ) . 3 RESULTS . In the first set of experiments , all images in a category had the same noise mask . For salt-and-pepper mask , this meant that noise masks were sampled independently for each category , but the same set of pixels in each image were modified for all images in a category . Similarly , for the additive uniform noise mask , the same mask was added to each image in a category . For the single pixel noise , the location and colour of the added pixel were independently sampled for each category , but kept constant for all images in a category . The results of these experiments are shown in Figure 2 . We obtain the same pattern of results for all three cases : when noise mask in the test images matches the noise mask in training images , the model classifies images nearly perfectly ; when noise masks are swapped , the accuracy drops to zero ; when the mask is completely removed , the categorisation accuracy is at chance . Furthermore , we get the same pattern of results on both VGG and ResNet networks . These results clearly indicate that the model learns to completely rely on features of the noise-like mask , rather than any shaperelated information present in the images . Even in the extreme case , where only one pixel amongst 50176 was diagnostic of the category , the model prefers to classify based on this feature over other shape-related features present in each image . One response to this preference for these non-shape , but statistically relevant , features is that it can be addressed by regularisation methods that prevent overfitting to idiosyncratic features in the training dataset . The network is simply failing to select the more general feature – shape – choosing instead to memorise the noise-like mask . If this was the case , then using some of the well-known regularisation methods , such as Batch Normalization , Weight Decay or Dropout , may diminish this preference for the non-shape feature . However , we found that regularisation did not make any significant difference to the results above ( Figure 3 ) . Even when a dropout of 50 % was added after every convolution layer , the pattern of results remained the same . Another possible reason why humans prefer to rely on shape-related features to categorise objects while CNNs do not is that humans are guided by past experience and bring this past knowledge to new categorisation tasks . So when a human sees an object with superimposed noise , they generalise from past experience and look for shape-based information , paying less attention to non-shape related features such as the noise-like mask in above images . In order to test whether the network will similarly generalise from experience on other tasks , we conducted two further experiments . In the first experiment , we divided the training set into two subsets . The first subset ( ‘ with pix ’ ) contained three randomly chosen categories from CIFAR-10 and , like above , contained a categorycorrelated pixel in all images of these categories . The second subset ( ‘ unaltered ’ ) contained the remaining seven categories from CIFAR-10 and was left unaltered – i.e . we did not add the categorycorrelated pixel to images of this subset . The network was then trained on all ten categories at the same time . We were interested in finding out whether the network generalised from one subset to another and started using the features used to categorise images in the ‘ unaltered ’ subset to images of the ‘ with pix ’ subset . The results from this experiment are shown in Figure 4a . The model learnt to predict the images in the ‘ unaltered ’ subset with nearly 90 % accuracy . However the performance on the ‘ with pix ’ subset still completely depended on the location and colour of the added pixel : accuracy was nearly 100 % when test images contained the pixel in the same location , but dropped below chance when this pixel was removed . Thus , the network did not seem to generalise the features learnt in the ‘ unaltered ’ categories to the categories containing the diagnostic pixel . The above experiment required the model to simultaneously learn on categories with and without a diagnostic pixel . In a second experiment we tested what happens when the network is first trained on images that did not contain such a pixel ( a ‘ before ’ phase ) followed by a second ( ‘ after ’ ) phase in which such a pixel was inserted in the training set . This setup more closely resembles the situation faced by humans , where they bring their knowledge from previous categorisation tasks and use this knowledge to select the shape-based features to a new task . In the first phase , a pre-trained VGG network was trained on an unaltered CIFAR-10 training set . Once the network had learnt this task , we trained it on the modified set of images in a second phase , introducing a predictive pixel in each category . So all that changes between the ‘ before ’ and ‘ after ’ phases is the insertion of a single category-correlated pixel to each image . We observed that ( Figure 4b ) , instead of relying on past experience with these images , the model learnt to completely rely on the predictive pixel to perform categorisation – accuracy dropped from nearly 100 % to 0 % between ‘ Same ’ and ‘ Diff ’ conditions . Crucially , the model completely forgot about how to perform categorisation when the predictive pixel was removed – accuracy was close to chance in the ‘ NoPix ’ condition during the ‘ after ’ phase . Thus learning about the diagnostic feature seemed to be accompanied by unlearning previously learnt representations . This , catastrophic forgetting , is a well-known problem in neural networks ( McCloskey & Cohen , 1989 ) and contrasts with how humans transfer their knowledge from one task to another .

Summary: Humans leverage shape information to recognize objects. Shape prior information helps human object recognition ability to generalize well to different scenarios. This paper aims to highlight the fact that CNNs will not necessarily learn to recognize objects based on their shape. Authors modified training images by changing a value of a pixel where its location is correlated with object category or by adding noise-like (additive or Salt-and-pepper) masks to training images. Parameters of such noise-like masks are correlated with object category. In other words if one learns noise parameters or location of altered pixel for each object category, they can categorize all images in the training set. This paper shows that CNNs will overfeat to these noise based features and fail to correctly classify images at test time when these noise based features are changed or not added to the test images.