Class selectivity, typically defined as how different a neuron's responses are across different classes of stimuli or data samples, is a common metric used to interpret the function of individual neurons in biological and artificial neural networks . However, it remains an open question whether it is necessary and/or sufficient for deep neural networks (DNNs) to learn class selectivity in individual units. In order to investigate the causal impact of class selectivity on network function , we directly regularize for or against class selectivity. Using this regularizer , we were able to reduce mean class selectivity across units in convolutional neural networks by a factor of 2.5 with no impact on test accuracy, and reduce it nearly to zero with only a small (\sim2\%) change in test accuracy. In contrast, increasing class selectivity beyond the levels naturally learned during training had rapid and disastrous effects on test accuracy  . These results indicate that class selectivity in individual units is neither neither sufficient nor strictly necessary for DNN performance, and more generally encourage caution when focusing on the properties of single units as representative of the mechanisms by which DNNs function.
The properties of individual neurons are often analyzed in order to understand the biological and artificial neural networks in which they're embedded. Class selectivity-typically defined as how different a neuron's responses are across different classes of stimuli or data samples-is commonly used for this purpose . However, it remains an open question whether it is necessary and/or sufficient for deep neural networks (DNNs) to learn class selectivity in individual units. We investigated the causal impact of class selectivity on network function by directly regularizing for or against class selectivity. Using this regularizer to reduce class selectivity across units in convolutional neural networks increased test accuracy by over 2\% in ResNet18 trained on Tiny ImageNet. In ResNet20 trained on CIFAR10 we could reduce class selectivity by a factor of 2.5 with no impact on test accuracy, and reduce it nearly to zero with only a small (\sim2\%) drop in test accuracy. In contrast, regularizing to increase class selectivity had rapid and disastrous effects on test accuracy across all models and datasets . These results indicate that class selectivity in individual units is neither sufficient nor strictly necessary , and can even impair DNN performance. They also encourage caution when focusing on the properties of single units as representative of the mechanisms by which DNNs function.
R
Class selectivity, typically
The properties of individual neurons are often analyzed in order to understand the biological and artificial neural networks in which they're embedded. Class selectivity-typically
R
samples, is a common metric used to interpret the function of individual neurons in biological and artificial neural networks
samples-is commonly used for this purpose
R
In order to investigate
We investigated
R
, we directly regularize
by directly regularizing
R
, we were able to reduce mean
to reduce
R
by
increased test accuracy by over 2\% in ResNet18 trained on Tiny ImageNet. In ResNet20 trained on CIFAR10 we could reduce class selectivity by
R
change
drop
R
increasing class selectivity beyond the levels naturally learned during training
regularizing to increase class selectivity
A
across all models and datasets
D
neither
R
for DNN performance, and more generally
, and can even impair DNN performance. They also