Title: CAUSAL IMPORTANCE OF ORIENTATION SELECTIVITY FOR GENERALIZATION IN IMAGE RECOGNITION Anonymous

Abstract: Although both our brain and deep neural networks (DNNs) can perform high-level sensory-perception tasks such as image or speech recognition, the inner mechanism of these hierarchical information-processing systems is poorly understood in both neuroscience and machine learning. Recently, Morcos et al. (2018) examined the effect of class-selective units in DNNs, i.e., units with high-level selectivity, on network generalization, concluding that hidden units that are selectively activated by specific input patterns may harm the network's performance. In this study, we revisit their hypothesis, considering units with selectivity for lowerlevel features, and argue that selective units are not always harmful to the network performance. Specifically, by using DNNs trained for image classification (7-layer CNNs and VGG16 trained on CIFAR-10 and ImageNet, respectively), we analyzed the orientation selectivity of individual units. Orientation selectivity is a low-level selectivity widely studied in visual neuroscience, in which, when images of bars with several orientations are presented to the eye, many neurons in the visual cortex respond selectively to a specific orientation. We found that orientation-selective units exist in both lower and higher layers of these DNNs, as in our brain. In particular, units in the lower layers become more orientationselective as the generalization performance improves during the course of training of the DNNs. Consistently, networks that generalize better are more orientationselective in the lower layers. We finally reveal that ablating these selective units in the lower layers substantially degrades the generalization performance, at least by disrupting the shift-invariance of the higher layers. These results suggest to the machine-learning community that, contrary to the triviality of units with highlevel selectivity, lower-layer units with selectivity for low-level features can be indispensable for generalization, and for neuroscientists, orientation selectivity can play a causally important role in object recognition.

1. INTRODUCTION:

Recognizing the natural world is a fundamental competency for animals and artificial intelligence. Although our cerebral cortex and recent deep neural networks (DNNs) both achieve high accuracy in sensory perception (Krizhevsky et al., 2012; Hinton et al., 2012) , especially object recognition through natural vision, the rationale for this performance is not well understood, either in visual neuroscience or machine learning.

In machine learning, numerous studies have been conducted on why DNNs have good generalization ability (summarized in section 4.2). One recent interesting hypothesis proposed in Morcos et al. (2018) is that networks that rely on single directions1 may generalize poorly. The authors performed ablation experiments to argue that networks that generalize poorly are sensitive to unit ablations and that selectively activated units are not important for generalization. However, the authors examined only high-level single directions; they performed ablation experiments on higher layers and analyzed the class selectivity of individual units. Their hypothesis must be evaluated for other directions, especially low-level directions (or "features"), in order to conclude that it is correct.

In visual neuroscience, the most popular experimental setting is to present bar or grating images (Fig. A1 ) at several orientations to animals (e.g., cats, mice, or humans) while simultaneously recording the visually evoked neuronal responses. This paradigm is used to analyze the orientationselective feature of many neurons in the visual cortex. This feature causes the responses evoked by a bar or grating image presented to an animal to be tuned to the orientation of the image (Hubel & Wiesel, 1959) (examples of this "orientation tuning curve" are shown in Fig. 1 ). Numerous neuroscience researchers have thus far analyzed orientation selectivity to investigate the functions of the visual cortex, especially primary visual cortex (V1), such as how neurons with similar properties are interconnected (Ko et al., 2011; Wertz et al., 2015) , how visual discrimination tasks modulate neural coding (Schoups et al., 2001; Dragoi et al., 2002; Jehee et al., 2012) , and how neural selectivity emerges and matures during development (Chapman & Stryker, 1993; White et al., 2001) . However, there is a large gap between object recognition in a natural environment and orientation selectivity. Specifically, no researchers have thus far investigated whether orientation selectivity does have an important role in object recognition, or is merely a superficial byproduct of object recognition.

In this study, we address these two issues simultaneously by performing neuroscience-inspired experiments on DNNs trained for image classification. The use of DNNs to model the visual cortex is supported by several studies that suggested the hierarchical similarity of feature representations between DNN and the visual cortex (Yamins et al., 2014; Khaligh-Razavi & Kriegeskorte, 2014; Cadieu et al., 2014; Güc ¸lü & van Gerven, 2015; Horikawa & Kamitani, 2017) . Just as in neuroscience experiments, we inputted grating images to the trained networks and analyzed the orientation selectivity of the units in the hidden layers to explore whether orientation selectivity contributes to the generalization performance in object recognition. The main findings in this paper can be summarized into four aspects.

• Orientation-selective units exist in all the layers of a DNN trained for image classification.

• During training, units in the lower layers become more orientation-selective in sync with the generalization performance.

• Networks that generalize better are more orientation-selective in the lower layers.

• Ablation experiments revealed that orientation-selective units in the lower layers play a causal role2 in generalization, at least by introducing shift-invariance of the higher layers.

The major implications derived from these empirical analyses are:

• From the neuroscience perspective, orientation selectivity in the lower layers of hierarchical visual systems may be a causally crucial component and not be a trivial byproduct.

• From the machine-learning perspective, our results, in conjunction with very recent studies that argued the importance of class selectivity from different viewpoints (Liu et al., 2018; Zhou et al., 2018) , suggest that the uselessness of single directions proposed in Morcos et al. (2018) is overstated. In addition, the significance of the orientation-selective units in the lower layers directly supports the assertion that Gabor feature representations in the lower layers are indispensable for the generalization of the DNNs.

2. METHODS:

All analyses were performed on Keras (2.0.8) and Tensorflow (1.3.0). We primarily analyzed a 7-layer CNN (six convolutional (Conv) layers plus one fully connected (FC) layer) trained on the CIFAR-10 dataset (Krizhevsky & Hinton, 2009) (see section A.1 for hyperparameters and training details). To check the reproducibility of our results, we also analyzed i) another 7-layer CNN trained on CIFAR-10 with different initializations and ii) a VGG16 network (Simonyan & Zisserman, 2015) , which is a 20-layer CNN trained on the ImageNet dataset (Russakovsky et al., 2015) . We fed grating images to these trained networks and analyzed the orientation selectivity of the hidden units3 in exactly the same way as in visual neuroscience studies.

2.1. GRATING IMAGES:

A grating image is a two-dimensional gray-scaled sinusoidal wave G(x, y), formulated as follows.

EQUATION

where k 0 is the spatial frequency (SPF), τ is the phase, θ is the orientation, and (x 0 , y 0 ) is the center coordinate of the image. For the 7-layer CNNs, we prepared a total of 432 grating images using nine SPFs, 12 orientations (15 • apart), and four phases (90 • apart). For the 20-layer CNN, we prepared 2,736 grating images with 57 SPFs using the same 12 orientations and four phases.

2.2. ORIENTATION SELECTIVITY:

Similar to the neuroscience studies that present grating images to animals while recording their neuronal responses, the unit activations with respect to each grating image were collected. The activation matrix of each unit has a shape of N ori × N SP F × N phase , where its (k, l, m) th element represents the activation with respect to the grating of the k th orientation, l th SPF, and m th phase.

After taking the maximum along the phase dimension, we extracted the activations on the SPF that yielded the highest activation, generating a vector of length N ori . Using this vector, orientation selectivity was quantified by using the global orientation selectivity index (gOSI), which is a popular metric in visual neuroscience (Wörgötter & Eysel, 1987) . The gOSI of the i th unit is formulated as

EQUATION

where R i k is the activation of the i th unit with respect to the grating image of the k th orientation and θ k is the degree of orientation (0-180 • ). This metric is between 0 and 1, and a higher value indicates higher orientation selectivity. Note that this metric is equivalent to 1 -"circular variance."

3. RESULTS:



3.1. ORIENTATION SELECTIVITY IN DNNS:

To investigated whether the units in the DNNs are tuned to the orientations, as is the case in the visual cortex, we fed grating images to the trained networks and analyzed the evoked activations. Activations of 10 example units of a 7-layer CNN along each orientation are shown in Fig. 1 . Most units, both in the lower layer and the higher layer, were selectively activated by specific orientations. We then quantified the degree of orientation selectivity by using the gOSI metric for all units in all layers. Interestingly, units with high gOSI (orientation-selective units4 ) existed in all layers (Fig. 2 ). Furthermore, this orientation selectivity was almost independent of the SPFs of the grating images (section A.2 and Fig. A2 ). Similar results were obtained with another 7-layer CNN with different initializations and a 20-layer CNN trained on ImageNet (Figs. A4 and A3 ), indicating the robust existence of orientation selectivity in all layers of the networks.

One can consider that the high orientation selectivity results from the sparsity of the hidden units. For example, a sparsely active unit, with its activation to the 0 • grating being 1 and otherwise 0, has gOSI = 1.0. To investigate this possibility, we also computed gOSI after the activation matrix of each unit was randomly shuffled. The gOSI calculated in this way was much smaller than the original gOSI for most layers (Fig. 2 ), except for the high layers of the 20-layer CNN (Fig. A4 ), indicating that sparsity cannot fully explain the observed high gOSI.

3.2. MATURATION OF ORIENTATION SELECTIVITY DURING THE COURSE OF TRAINING:

A big challenge in visual neuroscience is to elucidate when and how orientation selectivity emerges and matures during development. To tackle this question, we analyzed the orientation selectivity during the course of training. Fig. 3a shows the average gOSI for each layer. The average gOSI of Conv 2-Conv 5 saturated at around 15 epochs, when validation loss also saturated, while that of Conv 6 and FC 1 saturated much earlier. To understand the relationship between the orientation selectivity and generalization, we then plotted the validation loss versus the average gOSI for each epoch (Fig. 3b ), finding that the higher the average gOSI was, the less the validation loss was for the lower layers (Conv 1-Conv 5) (Fig. 3c ). Similar results were obtained for a network with different initializations (Fig. A5 ). This strong correlation between gOSI and generalization suggests that, from the neuroscience perspective, orientation selectivity in the lower layers matures as the ability of object recognition matures, and from the machine-learning perspective, gOSI is a strong indicator for the generalization performance and is possibly a substitute signal for early stopping. 

3.3. RELATIONSHIP BETWEEN ORIENTATION TUNING AND GENERALIZATION PERFORMANCE:

Section 3.2 triggers an intriguing question, viz., is orientation selectivity a correlate of generalization performance? We studied this question by comparing many networks trained in a different manner. We prepared four different settings of data augmentation (no augmentation; horizontal and vertical shift of the images; horizontal flip of the images; and horizontal shift, vertical shift, and horizontal flip were all incorporated). For each setting, we trained 50 models with independent initializations, resulting in generation of 200 networks with different generalization performances. All the networks had seven layers and were trained on CIFAR-10. When the generalization performance and gOSI of the 200 models were compared (Fig. 4a ), we found a strong correlation between gOSI and generalization performance in the lower layers (Fig. 4b ), except for Conv 1. Similar results were obtained for a set of 200 CNNs with different initializations (Fig. A6a and b ).

A scenario of concern is that since the training loss and the test loss are correlated, orientation selectivity might be important only for the training loss, not for the test loss, and we are observing a side effect. To examine this possibility, we additionally trained six networks on CIFAR-10, whose labels were corrupted at different rates (p) from 0.0 (no labels were corrupted) to 1.0 (all labels were random) (Zhang et al., 2017) . In this experiment, data augmentation was not incorporated so that networks were able to memorize the dataset. The training accuracy was nearly 100% for all models, whereas the test accuracy decreased as the corruption rate increased; test accuracy was approximately 70% for p = 0.0 and 10% for p = 1.0. We then compared the gOSI of the six networks (Fig. 4c ), revealing that networks with lower test accuracy (higher label corruption rates) had lower gOSI in the low layers (especially in Conv 2-Conv 4; note that the y-axis range is different among the layers). Similar results were obtained for a set of CNNs with different initializations (Fig. A6c ). These results indicate that orientation selectivity in the lower layers does correlate with the generalization performance.

3.4. CAUSAL ROLE OF ORIENTATION TUNING FOR GENERALIZATION:

Thus far, we have shown that networks that generalize better are more orientation-selective in the lower layers. We further investigated the causal relationship between orientation selectivity and generalization performance through ablation experiments (Morcos et al., 2018) . The use of ablation experiments was inspired by several neuroscience studies in which the causal role in some behavior of a set of neurons is investigated by lesioning or inactivating them (e.g., using optogenetics). Similar to Morcos et al. (2018) , after ablating a set of units by clamping their activations to zero, we computed the difference between the test losses before and after ablation. As shown in Fig. 5 , for the lower layers (Conv 2 and Conv 3; indicated as asterisks), ablating the units with the top 50% gOSI values caused a more than twofold impact on test loss than ablating the units with the bottom 50% gOSI values. This difference in ablation impact cannot be explained by the possibility that the units with the bottom 50% gOSI values are more silent because the average activity of the units with the top 50% gOSI values in response to the grating images was lower than that of the units with the bottom 50% gOSI values, both for Conv 2 and Conv 3. Similar results were obtained for a network with different initializations and a 20-layer CNN trained on ImageNet5 (Fig. A7 ; Conv 2 and Conv 3 in the 7-layer CNN; and block1 conv2, block1 pool, and block2 conv1 in the 20-layer CNN). These results suggest that orientation-selective units in the lower layers are causally important for the generalization of the networks.

3.5. A POSSIBLE MECHANISM OF ORIENTATION-SELECTIVE UNITS FOR GENERALIZATION:

Finally, we examined how the orientation-selective units in the lower layers contribute to the overall generalization observed thus far. Because in neuroscience, orientation-selective filters ("simple cells" in V1) have been hypothesized to be combined to create a shift-invariant filter ("complex cells") (Movshon et al., 1978) , we hypothesized that orientation-selective units in the lower layers of DNNs contribute to the invariance of the higher layers with respect to parallel shifts of the in-Figure 5 : Importance of orientation-selective units in the lower layers for generalization revealed through ablation experiments. The difference between test losses before and after ablation is shown for each layer. Red: units with the top 50% gOSI values were ablated. Blue: units with the bottom 50% gOSI values were ablated. Asterisks indicate layers for which ablating units with the top 50% gOSI values yielded more than twofold impact than ablating units with the bottom 50% gOSI values. put images. Here, we examined the degree of shift-invariance of each unit6 in all layers using the following approach, which was inspired by Goodfellow et al. (2009) but slightly modified. After collecting activations of the target unit with respect to test images that activate the unit near maximally (> 90%) and their shifted images7 , we computed how much the unit activations are influenced by the shift by using the coefficient of variation (CV) metric, which was afterward averaged along with the test images. We first confirmed that this metric decreases as the layer becomes deeper, indicating that unit activations of higher layers are more shift-invariant as suggested in (Goodfellow et al., 2009) . When CVs computed in this way were compared between the vanilla network and the network where orientation-selective units of Conv 2 or Conv 3 were ablated, as we did in section 3.4, we found that units in the fully connected layer of the ablated network had significantly higher variances than those of the vanilla network (Fig. 6a ), indicating that ablating orientation-selective units in the lower layers significantly disrupts the shift-invariance of the fully connected layer. Similar results were obtained with another 7-layer CNN with different initializations (Fig. 6b ). These observations imply that orientation-selective units in the lower layers produce a part of shift-invariance of the fully connected layer, thereby contributing to the generalization performance. 

4. RELATED WORK AND DISCUSSION:

Our results suggest that, for the several network architectures we explored, the orientation selectivity in the lower layers of DNNs is causally indispensable for object recognition, not a superficial byproduct of object recognition. In addition, these representations are not necessary for memoriza-tion (Fig. 4c ) but are important for generalization, at least by introducing shift-invariance of the fully connected layer.

4.1. DISCUSSION FROM THE NEUROSCIENCE PERSPECTIVE:

In neuroscience, orientation selectivity has been extensively analyzed in numerous papers since Hubel & Wiesel (1959) introduced the concept that neurons in V1 selectively respond to a specific orientation of bars. Nevertheless, whether the orientation-selective property does contribute to object recognition in natural scenes has not been examined thus far, possibly due to experimental limitations. Comparing the performance of object recognition with the degree of orientation selectivity among numerous well-trained animals (as we did in section 3.3) or inactivating orientation-selective neurons alone in the visual cortex (as we did in section 3.4) is experimentally very difficult or impossible with the current biotechnology. In this study, we tackle this issue by using DNN to model the brain. With this approach, we can obtain comprehensive data on neural activity, neural connectivity, and developmental process with infinite signal-to-noise ratio, at single-cell resolution, and chronically.

We also found that orientation-selective units exist in all layers of the DNNs. This is consistent with neuroscience studies where orientation-selective neurons exist not only in V1 but also in higher layers of the visual hierarchy, such as V4 (Desimone & Schein, 1987) and middle temporal (MT) area (Albright, 1984) . However, as with the DNNs shown in this study, these orientation-selective neurons in the higher visual cortex might not encode the orientation per se.

We also analyzed orientation selectivity during the course of training and revealed that orientation selectivity in the lower layers matures as the ability of object recognition saturates. Again, this finding has not been proven in neuroscience, primarily due to experimental difficulty. Corresponding neuroscience experiments might involve the chronic recording of neurons during development and longitudinal comparison between the performance of object recognition and orientation selectivity, which is very difficult with current biotechnology.

4.2. DISCUSSION FROM THE MACHINE-LEARNING PERSPECTIVE:

Generalization of DNN itself is intriguing and has been investigated in many papers, especially after Zhang's work (Zhang et al., 2017) . Several researches have attributed the high generalization ability of DNNs to the convergence into flat minima (Keskar et al., 2017; Neyshabur et al., 2017) . Wilson et al. (2017) proposed that stochastic gradient descent has an advantageous effect, and Ulyanov et al. (2018) proposed that the network structure itself is important. Although the relationship between these theories and orientation selectivity would be part of a future study, in this study we provide empirical evidence on the contribution of orientation-selective units in the lower layers to the overall generalization, partly via producing shift-invariance of the fully connected layer.

The role of orientation-selective units in the lower and higher layers might be different. In the lower layers (e.g. Conv 2), a Gabor filter is the optimal stimulus that activates the unit most strongly (Erhan et al., 2009; Krizhevsky et al., 2012; Zeiler & Fergus, 2014) . This is also verified in neuroscience; Ukita et al. (2018) recently performed unbiased analyses to reveal that Gabor-like images indeed activate V1 neurons most strongly. On the other hand, Gabor filters might be suboptimal stimuli for units in the higher layers considering more elaborated features coded in the higher layers (Le et al., 2012; Simonyan et al., 2014) . This might explain why orientation-selective units in the higher layers do not contribute to generalization as shown in the sections from 3.2 to 3.4. Examining the detailed role of orientation-selective units in the higher layers would be part of a future study.

When orientation selectivity is regarded as one of the low-level single directions, the results are opposite to those of a recent study (Morcos et al., 2018) . Therefore, low-level single directions (e.g., orientation selectivity) and high-level single directions (e.g., class selectivity) might contribute differently to generalization. While selectivity to high-level directions might discourage distributed coding, selectivity to low-level directions might be important for embedding natural images. Further studies should investigate this discrepancy in more detail. 

:

"The activation of a single unit or feature map or some linear combination of units in response to some input"(Morcos et al.,

2018).

Determining causality between phenomena has been a central goal in natural science, including neuroscience: "Scientific work will always be the search for causal interdependence of phenomena"(Born, 1949), "Cracking the neural code-in other words, determining which spatiotemporal patterns of activity [...] causally drive behavior"(Packer et al., 2013).

The feature maps of convolutional layers and units of fully connected layers are both referred to as "unit" in this paper.

In visual neuroscience, neurons with gOSI > 0.33 are often considered to be orientation-selective(Piscopo et al., 2013;Kondo & Ohki, 2015).

Impact on loss was evaluated on the 50,000 validation images of ImageNet Large Scale Visual Recognition Challenge 2014 (ILSVRC2014). http://image-net.org/challenges/LSVRC/2014/download-images-5jj5.php

In this part, "unit" refers to each component of the feature maps of convolutional layers and unit of fully connected layers.

we shifted the images in three ways; 10% vertical shift, 10% horizontal shift, or 10% horizontal and vertical shifts.

Figures and Tables:

Figure 1: Orientation-tuning curve of example units. The gOSI value is shown in each panel.

Figure 2: Orientation-selective units exist in all hidden layers. Histograms of gOSI for each layer are shown in red. Histograms of gOSI, after the activation matrix was shuffled, are shown in blue as a control.

Figure 3: Units in the lower layers become more orientation-selective in sync with generalization performance during the course of training. (a) Evolution of loss (top) and average gOSI (bottom). (b) Relationship between the validation loss and average gOSI during the first 50 epochs. Each point indicates one epoch. (c) Spearman correlation coefficient between the validation loss and average gOSI during the first 50 epochs.

Figure 4: Networks that generalize better are more orientation-selective in the lower layers. (a) Relationship between the test loss and average gOSI. Each dot indicates one of the 200 trained models. (b) Spearman correlation coefficient between the loss and gOSI for each layer. (c) Relationshipbetween the label corruption rate and gOSI. The higher the label corruption rate, the worse is the generalization accuracy(Zhang et al., 2017).

Figure 6: Orientation-selective units are important for the shift-invariance of the fully connected layer. Coefficients of variation of the unit activations with respect to the parallel shift of the test images are shown for each layer (mean ± standard error).

Figure A1: All grating images presented to the 7-layer CNNs trained on CIFAR-10.

Figure A2: Minimal dependence of orientation selectivity on spatial frequencies. The range of preferred orientations among various spatial frequencies was computed for each orientationselective unit (gOSI > 0.33) and their distribution is plotted.

Figure A3: Reproduction of Fig. A2 using (a) a 7-layer CNN with different initializations and (b) a 20-layer CNN trained on ImageNet. The range of preferred orientations among various spatial frequencies was computed for each orientation-selective unit (gOSI > 0.33) and their distribution is plotted.

Figure A4: Reproduction of Fig. 2 using (a) a 7-layer CNN with different initializations and (b) a 20-layer CNN trained on ImageNet. Histograms of gOSI for each layer are shown in red. Histograms of gOSI, after the activation matrix was shuffled, are shown in blue as a control.

Figure A5: Reproduction of Fig. 3 using a 7-layer CNN with different initializations. (a) Relationship between the validation loss and average gOSI during the first 50 epochs. Each point indicates one epoch. (b) Spearman correlation coefficient between the validation loss and average gOSI during the first 50 epochs.

Figure A6: Reproduction of Fig. 4 using 7-layer CNNs with different initializations. (a) Relationship between the test loss and average gOSI. Each dot indicates one of the 200 trained models. (b) Spearman correlation coefficient between the loss and gOSI for each layer. (c) Relationshipbetween the label corruption rate and gOSI. The higher the label corruption rate, the worse is the generalization accuracy(Zhang et al., 2017).

Figure A7: Reproduction of Fig. 5 using (a) a 7-layer CNN with different initializations and (b) a 20-layer CNN trained on ImageNet. The difference between test losses before and after ablation is shown for each layer. Red: units with the top 50% gOSI values were ablated. Blue: units with the bottom 50% gOSI values were ablated. Asterisks indicate layers for which ablating units with the top 50% gOSI values yielded more than twofold impact than ablating units with the bottom 50% gOSI values.

Figure A8: A small number of units is already orientation-selective before training. Orientation selectivity was analyzed on a network with random weights.

Peer review (iclr2019):
 The paper presents a study on orientation selectivity in DNNs for image classification, arguing that this type of selectivity in the lower layers is crucial for generalization. This hypothesis is tested through an ablation study, which the authors interpret as a suggestion of the existence of a causal relation. The authors tackle a very interesting problem that seems to have not received yet enough attention. The paper is quite well-written and clear, making it understandable also to non-experts. The only concern I would have is about the causal claims in Section 3.4. I’m not completely sure the ablation experiments are the correct way to “prove” causality, as opposed to somehow trying to intervene on the orientation selectivity directly. On the other hand, this seems complicated to prove and the approach proposed in the paper seems a pragmatic solution. I would possibly hedge slightly the causal claims. From an outsider point of view, I think the paper provides an interesting contribution to the discussion on orientation selectivity. I particularly appreciated its clarity and reproducibility.