Towards robust vision by multi-task learning on monkey visual cortex
Authors: Shahd Safarani
Arne Nix
Konstantin Friedrich Willeke
Santiago A Cadena
Kelli Restivo
George Denfield
Andreas S. Tolias
Fabian H. Sinz
Venue: NeurIPS 2021 Poster (2021)
Keywords: deep learning
neuroscience
computer vision
robustness

Abstract: Deep neural networks set the state-of-the-art across many tasks in computer vision, but their generalization ability to simple image distortions is surprisingly fragile. In contrast, the mammalian visual system is robust to a wide range of perturbations. Recent work suggests that this generalization ability can be explained by useful inductive biases encoded in the representations of visual stimuli throughout the visual cortex. Here, we successfully leveraged these inductive biases with a multi-task learning approach: we jointly trained a deep network to perform image classification and to predict neural activity in macaque primary visual cortex (V1) in response to the same natural stimuli. We measured the out-of-distribution generalization abilities of our resulting network by testing its robustness to common image distortions. We found that co-training on monkey V1 data indeed leads to increased robustness despite the absence of those distortions during training. Additionally, we showed that our network's robustness is often very close to that of an Oracle network where parts of the architecture are directly trained on noisy images. Our results also demonstrated that the network's representations become more brain-like as their robustness improves. Using a novel constrained reconstruction analysis, we investigated what makes our brain-regularized network more robust. We found that our monkey co-trained network is more sensitive to content than noise when compared to a Baseline network that we trained for image classification alone. Using DeepGaze-predicted saliency maps for ImageNet images, we found that the monkey co-trained network tends to be more sensitive to salient regions in a scene, reminiscent of existing theories on the role of V1 in the detection of object borders and bottom-up saliency. Overall, our work expands the promising research avenue of transferring inductive biases from biological to artificial neural networks on the representational level, and provides a novel analysis of the effects of our transfer.

Towards robust vision by multi-task learning on monkey visual cortex
Shahd Safarani, 1,' Arne Nix, 1-2 Konstantin Willeke, Santiago A. Cadena,
Kelli Restivo, 43 George Denfield, Andreas S. Tolias, Fabian H. Sinz! 00
Institute for Bioinformatics and Medical Informatics, University Tübingen, Germany 2 International Max Planck Research School for Intelligent Systems, Tübingen, Germany
Institute for Computer Science, University of Göttingen, Germany Department for Neuroscience, Baylor College of Medicine, Houston, TX, USA
5 Center for Neuroscience and Artificial Intelligence, Baylor College of Medicine, Houston, TX, USA Columbia University. Department of Psychiatry, New York, USA
shahdsaf @hotaaihttttlll com,
sinzecs www-goottingnn..c.mm
Abstract
Deep neural networks set the state-of-ther oftth--rrttttt across many tasks in computer vision, but their generalization ability to simple image distortions surprisingly fragile. In contrast, the mammalian visual system is robust wide range of perturbations. Recent work suggests that this generalization ability can be explained by useful inductive biases encoded in the representations of visual stimuli throughout the visual cortex. Here, we successfully leveraged these inductive biases with a multi- task learning approach: jointly trained deep network perform image classification and to predict neural activity in macaque primary visual cortex natural stimuli. We measured the out-of-distribttion
(V1) in response the generalization abilities of our resulting network by testing its robustness to common image distortions. We found that co-training on monkey VI data indeed leads to increased robustness despite the absence of those distortions during training. Additionally, we showed that our network's robustness is often very close that of an Oracle network where parts of the architecture are directly trained on noisy images. Our results also demonstrated that the network's 's representations become more brain-like their robustness improves. Using novel constrained reconstruction analysis, we investigated what makes our brain-regularized network more robust. We found that our monkey co-trained network is more sensitive to content than noise when compared to a Baseline network that we trained for image classification alone. Using SrepGaze-predictte saliency maps for ImageNet images, we found that the monkey co- co-trained network tends to be more sensitive salient regions in scene, reminiscent of existing theories on the role of VI in the detection object borders and bottom-up saliency. Overall, our work expands the promising research avenue of transferring inductive biases from biological to artificial neural networks on the representational level, and provides a novel analysis of the effects of transfer.
Introduction
Although machine Icarning algorithms have witnessed enormous progress thanks to the recent success of deep learning methods [I], current state-of-the-ar deep models (244 still fall behind the generalization abilities of biological brains 5 6 This includes lack of robustness to image
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

corruptions as pointed out by Hendrycks and Dietterich 77 who measured network's performance on 15 different image corruptions applied to the ImageNet [ test-set. Studies with similar image corruptions have proven to severely decrease performance in classification networks while having smaller impact on human perception 9 suggesting that the ability to extrapolate is weak these networks compared to the mammalian visual system. This gap in extrapolation has previously been attributed to differences in feature representations [0 [ and internal strategies for decision making
12 between humans and CNNs.
Historically, neuroscience has inspired many innovations in artificial intelligence most of this transfer between neuroscience and machine learning happens on the implementational level [13 [3] However, we currently know too little about the structure of the brain at the level of detail needed to transfer functional generalization properties from biological to artificial systems [5]. To transfer functional inductive biases from the brain to deep neural networks (DNNss, it may thus be better to consider the representational level by capturing biological feature representations in the responses of biological neurons to visual input - abstracting away from the implementational level. In fact, deep neural networks have set new standards in capturing brain activity across multiple areas in the visual cortex [16), becoming the state-of-the-art for neural response prediction the primary visual cortex (area VI. [7]. but also see Marques et al. [8] for biologically inspired models that perform similarly well). Recent work has shown that CNNs, which were fitted to V1 neural data, can generalize well to other neurons, animals, and stimuli [19 20). Vice versa, prior work suggests that enforcing brain-like representations in CNNs via neural data from humans [2N] mice [22, or monkeys 23 can have beneficial effects the generalization abilities of these networks to stimuli outside their training distribution for object recognition.
and
Our work expands on this line of research by exploring the extrapolation capabilities of multi-task learning models (MTL: (24)) trained on image classification and prediction of neural responses from monkey VI. This approach was proposed as the neural co-training hypothesis by Sinz et al. [5], but remained untested to the best of our knowledge. We implement MTL via a shared representation between image classification and neural response prediction (Fig. [. Our hypothesis is that MTL with neural data regularizes the shared representation to inherit good functional inductive biases from neural data, and improves the network's 's generalization abilities to rendering more robust. We empirically test this idea using common corruptions on tiny ImageNet (TIN)'. We show that MTL on monkey V1 has a positive effect on generalization it increases the model's robustness to image distortions, even though it is trained on undistorted images only. We compare model with robust Oracle model quantify what performance improvement can be expected given that only parts of the network are shared during MTL. Subsequently, we develop a constrained reconstruction based method to analyze learned sensitivities and invariances in the feature representation of the different models. We find that the robust models qualitatively exhibit different feature sensitivities than a standard classification model. Finally, we show that the feature sensitivity of the monkey V1 co- trained model is related to salient image features, consistent with existing theories about the role of V1 (25). Overall, our results support the neural co-training hypothesis and further expand the scope of prior results, by exploring the relationship between brain-like representations and robustness. To the best of our knowledge, we are the first to analyze learned feature representations of neurally co-trained models and thus take a step towards semantic understanding of what makes biological vision robust.
images, thus
Class tabel
neuron spike count
Comvekrion
Shared Layors
mas-PPcting
Image Cheos Eizeiion
Ln + C.
Yeu
Goussian Reatout
Nleand Preckition
4/
Figure 1: VGG-19 architecture for MTL on image classification and neural prediction.
2

---

2 Neural multi-task learning
Data The images for the classification task and the neurophysiological experiments were selected from the ImageNet dataset (8). For the classification task, we used a grayscale version of TIN"
The tiny ImageNet dataset (TIN) subset of ImageNet containing 100000 training images of 200 classes (500 for each class) downsized to images of size 64x64. In addition, each class has 50 validation images and 50 test images. For neural prediction, we used neurophysiological recordings of 458 neurons from the primary visual cortices (area VI) of two fixating awake macaque monkeys, recorded with a 32-channel depth electrode during 15 (monkey 1) and 17 (monkey 2) sessions. In each session, approximately 1000 trials of 15 images were presented each image for 120ms. We extracted the spike count from 40ms to 160ms after image onset. The image set presented to the monkey consists of 24075 images from 964 categories 25 images per category. Of those, 24000 were designated to model training and 75 to testing. For each training trial, a new subset of 15 images was randomly sampled from the training set. Test images were displayed in fixed order in 5 test trials consisting of 15 images each. Each of the test trials were randomly interleaved among training trials and repeated 40-50 times per session. All images were converted to gray-scale and presented at x20x420 px, covering 6.7% visual angle for the monkey, resulting in 63 pixels per degree (ppd). For model training, images were downscaled and cropped to 64 pixels, corresponding to 14.0 ppd.
Architecture All our experiments were based on a variant of the VGG-19 architecture [26] with batch normalization layers [27] after every convolutional layer (Figure I. To allow for arbitrary image sizes, we made the network fully convolutional by replacing the fully connected readout by three convolutional layers with dropout of 0.5 after the first two, and final max maxxloooing operation and softmax (28). We predicted neural responses by feeding the output of the convolutional layer conv-3-1, 1. shown by Cadena et al. [17] to be optimal for predicting VI responses, into Gaussian readout 19 yielding spike count prediction per neuron and image.
Models We use a VGG-19, like we described it above, trained grayscale TIN to serve as the Baseline for image classification in our experiments. To prepare our neural co-training, similar to Li et al. 22), we first trained a Monkey Predictor model the image-response pairs of our recorded neural data. We then used that model to predict neural responses for all input images of the TIN classification dataset. These predicted responses served the basis neural dataset we used in our MTL approach. This allowed us to balance the amount of data we have for each task and it removed trial-to-trial variability in the neural data.
Since co-training only affects the shared representation up to layer conv-3-1, we cannot expect the network to be robust network where all layers are trained on data augmented with the image distortions. To explore the limits on robustness resulting from sharing lower layers only, we trained classification model with a 1:1 mixture of clean and distorted images drawn from the pool of 14 ImageNet-C (7) corruptions (cf. Figure 9 for examples). To push the robustness to the frozen part, added a second loss that penalizes the Euclidean distance between the outputs of layer conv-3-1 for the same image augmented with different corruptions similar to Chen et al. 229 We then froze all layers up to conv-3-1, re-initialized the rest, and re-trained the remaining network on clean data only. We refer to this model as the Oracle since it has access to the image distortions during training- unlike our MTL models.
To demonstrate that MTL can in principle transfer robustness properties without showing distorted images in training. we generated neural responses from our Oracle model for all images of the clean TIN dataset by freezing the Oracle model and training a Gaussian readout on top of layer conv-3-1 for 10 epochs predict V1 data. Then, we trained model on the resulting neural responses alongside clean image classification using MTL. We call this model MTL-Oracle. This model also gives us a realistic "upper bound" for our MTL experiments.
For our main experiment, we trained MTL with the neural responses generated from the Monkey Predictor model, and refer to it as M7T.-Monkey. This model has never seen distorted images at any point during training. To demonstrate that MTL-Monney has an effect beyond introducing noise into the training, we perform control experiment which we refer to as MTL-Shuffld. For this, we train a model on the same neural data but with shuffled responses across images for all neurons. An overview of all models used in this study can be found in Table I
https//www..agge..om////// http::/cmmcciiiiiimmmeeeeetvviov
3

---

Table Overview of the different models that we use this study.
Model
Classification
Neural Prediction
Baseline
Clean TIN
Monkey Predictor
Monkey responses
Oracle
Noise augmented TIN
MTL-Oracle
Clean TIN
Oracle model responses
MTLe-Monkey
Clean TIN
Monkey predictor responses
MTLL-Shuffled
Clean TIN
Monkey predictor responses (shuffled)
Training We used cross-entropy loss for single task image classification and Poisson loss for single task neural prediction. For multi-tusk training, the challenge was to find the optimal balance between the two objectives that achieves reasonable performance on each task individually, and allows both tasks to benefit from each other by learning common representations. To put both objectives on the same scale, we used their corresponding negative log-likelihood and learned their balance through trainable observation noise parameters [30]. This yields combined loss of oे CCE(C (0, vb)) + 2 LMSE(O, (0, v),)) + log oe + log o, where are the shared parameters and ub.. de and th., T, are the task-specific parameters for classification and neural prediction, respectively. The classification objective LCe was the standard cross-entropy, analogous the single-task case. For MTL on neural data, we used mean-square error LMSE because the targets are predictions from the network trained on neural data and not the original noisy neural responses. For optimization, we accumulated the gradients over the different losses to optimize the shared parameters 0 in a single combined gradient step. By definition, the two loss components would contribute equally to the learning process.
We standardized all pixel values with the mean and standard deviation of the training set, and augmented the images by random cropping, horizontal flipping, and rotations in a range of 15 for classification. Optimization was performed using stochastic gradient descent with momentum in all classificatio--relteed cases, and Adam for single task neural prediction [31]. We used a batch- size of 128 and weight decay with a factor of 10-4 throughout all our experiments. The initial learning rate was determined for each task individually and reduced by a (task-specific) factor via an adaptive learning rate schedule. The schedule reduces the learning rate depending on the validation performance classification performance in the case of MTL when the rate of improvement is not above 10-4 for 5 consecutive epochs. The training was stopped when we reach either five learning rate reduction steps or a maximum number of epochs, that we defined for each task. This training setup was determined via prior hyper-parameter searches on the validation-set. We repeated every experiment with five different random initializations. Error bars were obtained by bootstrapping (250 repetitions).
3 Results
Our goal test whether co-training on neural data can lead to improved extrapolation abilities. To this end, we evaluated our model's 's robustness on distorted copies of the TIN validation-set settioon--et--- used as test-set our experiments following the corruption paradigm of Hendryeks and Dietterich 7 We
reproduced the distortions with an m-thhe-fly implementation (32), dropped glass blur because is computationally expensive, and refer to our resulting test-set as TIN-TC. We quantified the robustness for each of the remaining 14 noise types and five levels of corruption severity separately, and computed a summary robustness score adopted from Hendrycks and Dietterich 77: 1/2 - 7robuss / Baseline
where A. _/4 55555 >1. At,c,s denotes the mean accuracy corruption across levels i and seeds
MTL can successfully transfer robustness Comparing the robustness of the MTL--rracee model on TIN-TC to the robustness of the single-task Baseline model trained on clean TIN only, we saw clear signs of successful transfer (Fig- and Fig-3/A. (3).B) although the MTL network has never seen the image distortions of TIN-TC. In fact. the MTL-Oracle performed close to the Oracle in most cases.

---

Baseline
MTL. MTT.-Monkey
Shuffee
+
Oracle
Pixelate
Impulse Noise
Contrast
40
40
40
Accurucy [%t
20
20
2
3
4
1
3
4
5
1
2
5
Corruption Severity
Corruption Severity
Corruption Severity
Figure 2: Exemplary classification results on TIN-TC, showing 3 corruption types with the best (left), median (center) and worst (right) robustness scores for MTL-Monkey across 5 inereasing levels of severity each.
Baseline
MTL-Monkey
MTL-Shuffled
MTL-Oracle
Oracle
Robustness Score [%)
150
B
Robustness Score [%]]
Clean Accuraey [%)
125
48
120
100
46
75
100
44
50
Noise
Blur
Weather
Digital
Total
0.25
0.50
Neural Prediction [FEV]
Figure 3: A Robustness scores for each model grouped by corruption category, as defined in Hendrycks and Dietterich [7]. Overall robustness scores for our different models. C Robustness and neural prediction correlate positively for MTL-Monkey models across 12 different batch-ratios and random seeds per model (grey line: linear regression from neural performance to robustness). Neural prediction performance is measured as the fraction of explained variance (FEV), as described in Cadena et al. [17. darker color indicates higher accuracy on the clean TIN test-set.
Co-training with monkey V1 increases robustness. The results of MTL-Oracle show that MTL with neural responses from robust network in response to undistorted images successfully transfers robustness properties. Furthermore, MTL-Monkey generalized better to the TIN-TC image distortions than the Baseline model, similar to MTL-Oracle, despite the fact that MTL-Monkey has not seen distorted images at any stage during the training process. We found increased robustness for 9/14 image corruptions. This improvement is mainly observed across 3 groups of distortions: Noise, Blur and Digital (Fig. (F)g.BA), whereas MTL-Monney did not exceed the Baseline performance for Weather group. The shuffled control did not provide any benefits (Fig. [2 Fig. Tig.3AAB AB,) suggesting that the improved robustness of our monkey network is, in fact, due to the original neural data.
The more "brain-like" the neural network, the better it generalizes to image distortions. If features the neural data affect the robustness, we would expect that the robustness of MTL-Monkey correlates positively with its neural prediction performance on real monkey V1 data. To test this hypothesis, we created a pool MTL-Monkey models with varying neural performance by altering the amount of neural data introduced during co-training. We experiments with different ratios of neural data to image classification that was presented to the network before each backward pass. We ran experiments for ratios 1 15, 10, 1: 7, 5, 4, 4. 3, 2, 1 1,2 1,3 1, 1, and with five seeds each, giving us 60 models to plot in Figure C. We found that both the model's 's test accuracy clean images and its neural performance real monkey VI data improved the network's robustness (Figure CC; < 10 for both neural prediction and clean accuracy
2t-test for both factors in 2-factor linear regression, in which robustness (dependent variable) is predicted from clean test accuracy for image classification and performance V1 prediction (independent variables).
5

---

Analysis for MTL-Shuffled showed slight connection between robustness and neural performance (p 0.034 for neural prediction and p 10 for clean accuracy). When comparing the regression coefficient of neural prediction in the case of MTLL-Monkey and MTLL-Shuffled, we found that the influence of neural performance on robustness two orders of magnitude larger for real neural data Dmonkey 54.72 than for the shuffled version fled = 0.26. Overall, our results are consistent with previous work finding a positive correlation between model robustness and "brain- "eainnlikeness" [33
Analysis
In the previous section, we showed that our MTI. approach can transfer robustness properties and that improved robustness correlates with more brain-like representations learned from monkey V1 data. The aim of this section is to understand the representational differences compared to other models that could be responsible for the increased robustness of our MTL-Monkey model. To this end, we visualize which image features the networks are sensitive to, using novel resource constrained image reconstruction from a given layer across all of the models used in this study. The rationale behind a constrained reconstruction is to put resource limitation the total power of image, thereby force the reconstruction to put contrast in the image wherever it is necessary to recreate the activity of given layer, and thus visualize the sensitivities and invariances of this layer (see Figure Specifically, given a noisy target image Xp. we computed the corresponding activations f (xx) of a particular layer and reconstructed the original image by minimizing the squared loss between the target activations and the activations from the reconstructed image l (Xo, x) [I[(x)) / (xo)]l subject to norm constraint
x = =argmin [lf(x) - (xo)]l s.t.//x|2 <
Note that, if ||xo|| S r, one trivial optimal solution is x = Xg. However, if | > r, the constraint becomes active and the reconstruction has choose where to put power in the image (see Figure 4 This can be seen more formally if we approximate the loss function with a second order Taylor approximation e (xo, x) R (x xo) H (x xo) around the optimal solution Xo. Using the local approximation in the optimization problem and solving for x using Lagrange multipliers
1 max, minx, (x (xx) H (x xx) + 7 2 (1|x r2) s.t. 7 N 0
xo))
yields (H + + where denotes the Lagrange multiplier chosen such that ||x||| r if the constraint is active. To see that this preferentially reconstructs images along directions where the loss in activation space is more sensitive, consider x and Xo in the eigenbasis U of the Hessian H = U-UT, which we denote by U and Vo, respectively. In that space, the solution is
Noise
Robust model
Agnostic model
Constraints
A v (A (A+ or e A, +
+
Resonsttuuttoos
which means that directions with A; >> 0, i.e. high curvature or strong sensitivity, stay as they are while directions with small X; i.e. low curvature or "in- variance", get diminished. How strongly they are diminished depends on the Lagrange multiplier or, cquivalently, the norm constraint.
Oh
Content
Figure 4: An illustration of the reconstruction process. We reconstruct noise corrupted tar- get image (green) from a given model under a resource constraint (gray circles). An agnos- tic model (dark pink) would be, by definition, equally sensitive to: The image content and noise. In contrast, a robust model (orange) would be more sensitive to content and less to noise. Resource constraints that do not allow for full reconstruction force the optimization to put more image power towards directions to which the model is more sensitive.
In our analysis, we optimized the pixels based on the activations of layer conv-3-1- the co-trained layer for all five models MTLL-Mnneey Baseline, Oracle, Oracle and MTL-Shuffld. We found SGD with a learning rate of 5 to be most suitable to reconstruct from all the MTL models, and the Adam optimizer with learning rate of 01.1 best for the Baseline and Oracle models. We always optimized for 8000 steps per image and model for each norm constraint.

---

XTT Minuaayy
MIL
MTL
Spectral cuntaal
MTL Muniayy
MTL
MIL
Spectral
Orsicl
origiad
XTTL Moniloy
AFTIL Siutifled
XELL Ornelo
Specteul centrol
MTL Monloy
MTL Stntflee
Spacttal control
Booelmme
Ornelo
Boseline
Oradle
Oradle
morm
Figure 5: Reconstruction examples from all the 5 models of 4 noisy images (see last row for original images) with Gaussian noise of severity level under 3 norm constraints: and 60. See main text for details on the spectral control.
Reconstructions show qualitative differences between models Reconstructions from test images distorted with Gaussian noise under three different norm constraints qualitatively show that the constraint
models are sensitive to different features (Figure When looking mid-range (norm=15), where we expected the difference between the models to be the largest (see Figure 4 found that the Baseline model seems to be sensitive distortions present in the original image. This manifests itself in a stronger noise component in the background of the image reconstructed from the Baseline model compared to other models. The robust networks (MTL-Monkey, Oracle and MTL-Oracle), on the other hand, were less sensitive to these perturbations and exhibited more content structure for that constraint (see appendix for reconstructions). However, when comparing our MTLL-Monkey model with the other robust networks, we noticed that the Oracle and MTLL-Oracle models generally preserved the original image content as much possible, while reconstructions from the MTL-Monkey model seem to put slightly more emphasis on edges and object boundaries.
MTTL-Monkey's sensitivities cannot be fully explained by frequency filtering One simple mech- anism that would make network robust against noise types with high-frequency perturbations would be to change the frequency sensitivity towards low-pass components. To assess whether this might be the case for the MTL-Monkey network, used simple "spectral control". where transferred the Fourier amplitude spectrum of the reconstructed image to the original noisy image. This enforces the norm constraint on the original image, and exactly matches the frequency content. If the MTL-Monkey model were simply changing the frequency sensitivity, we would expect the reconstruction and the spectral control to match. However, this does not seem spectral control exhibited more noise in the background and showed less edge enhancing compared to the MTL-Monkey reconstruction. Thus, while changed frequency sensitivity might play a role, there might be additional effects that lead to more noise suppression and more edge enhancement.
the case. The
MTL-Monkey exhibits increased sensitivity to salient image regions Our constrained recon- struction analysis exposes image content that is relevant to recreate the responses of particular layer. In this final section, we try to characterize this content in terms of known perceptual mechanisms of mammalian vision. Motivated by the potential role of V1 in bottom-up saliency (25), we specifically
investigated the relation between salient regions of an image and the regions that get emphasized in
7

---

Original
Ground Truth
Binary Mask
Norm = 15
Norm = 60
Figure 6: Examples from the ImageNet images used for testing our neural saliency hypothesis in the MTL-Mooneyy model. In addition to the original image, we show the DeepGaze predicted saliency map and the resulting binarized mask for computing the norm ratio as well as the MTL-MMnney reconstructions with norm constraints (15 and 60).
the reconstructions from different models. Salient regions of an image often include boundaries and shapes of central objects (25] chapter 5]. It could be possible that focusing on more salient regions of an image improves the robustness of a model to corruptions as a correlation between shape-like feature detection and robustness has been reported before 10 32 From our qualitative observation of the reconstruction examples (Figure we noticed that the reconstructions from the MTL-Monkey model seem often enhance the central object in scene by emphasizing edges and boundaries compared to the reconstructions from our Oracle models. To investigate whether the MTL-Monkey model more sensitive to salient regions compared to other models, we collected 70 undistorted images with structured background from ImageNet. We used grayscale versions of these images as targets for the reconstruction from our MTL.-Monkey, Oracle, and Baseline models. Additionally, we used the DeepGaze II model [34] to predict the saliency maps of these images. To define binary saliency mask, we took the predicted density map, sorted all resulting pixel values in a descending order, and selected all pixels up to cumulative sum of 0.7 "salient" (see section A). Afterwards, we resized the target image and the binarized mask to 64x64 pixels, which is the standard size used for our models (see Figure 6). To quantify how much contrast spent the salient region salient(]) compared to the entire image we computed the ratio of the norm of the salient region against the full image's norm:
e(T) |saaient(()]] |
We then computed the difference e (I,) e (Io) between the norm ratio of each reconstructed image I, and the original image Io. To put the values common scale, normalized this difference by its maximally achievable value e (Io) finspired by [2]:
0. 2 (1,) - e(L)
(1)
1 e (T.)
Our results show that the MTL-Monkey is more sensitive salient regions than the Oracle and Baseline models across images under low to mid-range norm constraints (Figure And as the norm approaches the norm the full image, the ratios for both models become more equal to those of the MTL-Monkey model, as expected (right). In comparison to the Baseline and Oracle models, the spectral control seems to closer to the diagonal. However, in most cases the MTL-MMnney model emphasized the salient regions more strongly, supporting previous observation that frequency

---

filtering cannot fully explain the sensitivities of the co- co-trained model (see section We want to stress that our analysis is purely correlational at this point: Robust MTL monkey models seem to be more sensitive to salient image content than other -robust and non-robust- models. However, if the focus on salient features turns out to be causal, it would open up the possibility that the Oracle models and the MTLL-Monkey model are robust due to different reasons.
5 Related work
A number of previous studies also transferred useful inductive biases from biological to artificial neural networks. For instance, Arai et al. (35] utilized neural data -recorded from the superior colliculus (SC) in monkeys- to regularize the network's hidden layers for predicting saccadic eye movements accurately and improving the model's generalization abilities. Moreover, Rezai et al. 36) used the prediction of responses from the primate middle temporal area (MT) to train the lower layers of deep CNN for visual odometry via multi-task learning, which is the main approach in work. Although Arai al. [35] and Rezai et al. [36] leverage the idea of co-training, their main focus is on neuroscientific modeling in contrast to our work. We directly focus on improving machine learning models, which is more in line with other related examples from the literature. Fong et al. 21 used fMRI data of the human brain's 's activity to guide the training of neural networks on a classification task, obtaining general performance improvements for CNNs. Other works used neural data from animals to regularize the training of artificial networks on image classification, such as in Li et al. (2), through representational distance learning (RDL, 37 Li and colleagues used mouse V1 to regularize CNNs towards a more brain-like representation. They used the CIFAR-I0 10 and CIFAR- 100 datasets, and evaluated robustness on Gaussian noise corruptions. Similar to our findings, they observed an increase in robustness when comparing their network to the Baseline while the model regularized by the shuffled neural data did not yield similar benefits. They also observed increased robustness to adversarial attacks. We extend their work by using larger set of 14 distinct types of corruption to evaluate the robustness on tiny ImageNet. Federer al. (23] used neural data from the primary visual cortex of macaque monkeys for regularization, and reported an improvement in test accuracy and an increased robustness to label corruption.
Finally, previous work, summarized by Yamins and DiCarlo [16], found a strong correlation between how well neural network performs
classification task and how well it predicts neural responses. Recent work by Dapello et al. (33) also found that robust networks tend to be better at predicting VI responses in macaque monkeys than non-robust models. They also reported correlation between the brain-likeneds on the representational level of a network and robustness, without explicitly training on neural data, as the model architecture was hand-crafted emulate V1. In this work, we show similar
Monkey vs. Baseline
MTL--Monkey vs. Oracle
MTL.-Mokeyy vs. Spectral Control
Norm
Norm 15
Norm 60
Norm Ratio MMTL-Monkey)
0.5
0.5
0.5
0.0
0.0
0.0
-0.5
0.5
0.5
0.5
0.0
0.5
00.5
0.0
0.5
-0.5
0.0
0.5
Norm Ratio
Norm Ratio
Norm Ratio
Figure 7: Each datapoint corresponds to one ImageNet image, for which we compute the normalized norm ratio (cquation [ of the reconstructed image from the MTL-Monkey model (y-axis) compared to the Oracle, Baseline or spectral control model in blue, black or red respectively (x-axis). The panels show varying norm constraints as the basis of the reconstructions. For all panels, if the datapoint is above the diagonal, means that the salient regions the corresponding image are more emphasized by our MTL-Monkey model than the model associated with the particular marker style.

---

results on the representational level by directly co-training on neural data while using an established image classification architecture (see sections an
and B). Therefore, we believe that our approach of
learning a neurally informed representation from data is more flexible and readily generalizable in comparison to Dapello et al. [3] For example, extending our MTI. approach to higher brain areas straightforward to do, whereas that is not immediately clear with the approach from Dapello et al.
[33. Finally, we think that Dapello et al. [33] supports our results about transferring properties of VI into our network, especially that the similar behavior on the ImageNet-C test set, with "weather" corruptions also having a weaker performance than the rest, supports the validity of our results.
Discussion and conclusion
In this work, we show a successful transfer of robustness properties via multi-task learning on neural data and object classification. Our findings are generally consistent with prior works on inductive bias transfer from the brain, and constitutes first test of the neural co-training hypothesis [5] for improving the robustness of neural networks. Furthermore, we are the first, to the best of our knowledge, to go a step further by introducing novel attribution method to understand what makes our neurally co-trained model robust. Through that analysis, we find that our MTL monkey model is more sensitive to salient regions of image compared to other models. Since V1 has been implicated in bottom-up saliency before [25), it could be that our finding might be connected to this computational property of V1 neurons. In that respect, our results are consistent with the VI saliency hypothesis, which has been already supported by several studies in the literature 38 41
In this work, our aim was to learn robustness from a system that is known to be robust against most corruptions - the mammalian visual system ( Data augmentation with image corruptions during training -as in our Oracle model- is a simple and effective baseline for robust networks 7 42 However, the effectiveness is limited when comes to unseen corruptions [9), although some noise types might generalize when calibrated carefully 42). Thus, to get truly robust network, one would need to anticipate every possible corruption to include them in training. which is obviously intractable. Notably, our MTL approach achieves better generalization without modifying the network's ' input and without the additional overhead of a noise generator, and we hope that further improvements can eventually replace data corruption. Although our models are still far behind the human visual system in terms of generalization, our work is a conceptual step towards bridging the gap between artificial and biological intelligence. This could be deciding factor in helping the reliability and generalization capabilities of computer vision. In addition, our findings might help to get better understanding of the computational role of V1, such its role in bottom-up saliency. A promising future direction is to include higher brain areas for neural co-training, inspired by Kietzmann et al. 43.. They trained a network for object classification with RDL on neural dynamics of multiple visual areas -including higher ones- and showed that recurrent architectures achieve better test classification performance than feedforward architectures with additional self-connections (ramping feedforward architectures). Thus, using higher areas for neural co-training while potentially relying on recurrence will presumably yield stronger robustness against more complex distortions, and when combined with our analyses, it could improve our understanding of the functional role of these areas as well
Our work represents fundamental research into the link between biological and artificial vision. Since this direction is at an early stage, the risk of misuse or unethical use of our results is present but not larger than in other fundamental investigations of the principles of vision. Our data is collected through animal experiments which are currently the only method to get high numbers of single unit responses across brain regions. All data coming from monkeys complied with the approved protocol of local authorities (see appendix). A key ad advantage of the type of data we used is that it is suited for a wide range of analyses beyond this paper. Therefore, our paper simply improves the scientific yield of animal recordings. Furthermore, our approach highlights that it is not strictly necessary to record dozens of new datasets for new tasks the model that we trained on the original neural data was powerful enough to predict responses to unseen images, and these responses. in turn, can be successfully used for multi-task learning. We do believe that a network trained on a larger amount of already existing neuronal data (possibly also from other areas) of the primate visual system can be used for many more conceivable tasks. Thus, we think that our work contributes to reducing the need for invasive animal experiments and rather encourages the use of surrogate models.
3 'The data and code for this work are made public here: https://gthubb.com//iiibbbbeeeee cotraining
10

---

Acknowledgments
We thank Konstantin Lurz, Mohammad Bashiri, Christoph Blessing, Pawel Pierzchlewicz, and Matthias Kümmerer for helpful comments on the manuscript. The authors thank the International Max Planck Research School for Intelligent Systems (IMPRS-IS) for supporting Arne Nix and Konstantin Willeke.
Funding Transparency Statement
Shahd Safarani was funded by the Claussen-Simon Foundation. This work was partially supported by the Cyber Valley Research Fund (CyVy-RF-2019-01), FHS is supported by the Carl-Zeiss- Stiftung and acknowledges the support of the DFG Cluster of Excellence "Machine Learning - New Perspectives for Science", EXC 2064/1, project number 390727645. This work was supported by an AWS Machine Learning research award to FHS. Supported by the Intelligence Advanced Research Projects Activity via Department of Interior/Interior Business Center contract number D D16PC00003, DP1 EY023 176 Pioneer Grant (to A.S.T) T.) and grants from the US Department of Health & Human Services, National Institutes of Health, National Eye Institute (nos. RO1 EY026927 to A.S.T. and T32 EY00252037 and T32 EY07001).
References
[1] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. Nature, 521(7553):436-444 2015. doi: 10. 10.1008/nature 14539. /nttss//:/// org/10/ 10 1038/nature111539
[2] Geoffrey Hinton, Li Deng, Dong Yu, George E Dahl, Abdel-rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara N Sainath, al. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups.
IEEE Signal processing magazine, 29(6):82-977 2012.
[3] Waseem Rawat and Zenghui Wang. Deep convolutional neural networks for image classification: comprchensive review. Neural computation, 29(9):2352-2449, 2017.
[4] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. ImageNet classification with deep convolutional neural networks. In Advances in neural information processing systems, pages 1097-1105, 2012.
[5] Fabian H Sinz, Xaq Pitkow, Jacob Reimer, Matthias Bethge, and Andreas Tolias. Engincering less artificial intelligence. Neuron, 103(6):967-977 2019.
[6] Thomas Serre. Deep learning: the good, the bad, and the ugly. Annual review of vision science, 5:399-426 2019.
[7] Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. Proceedings of the International Conference Learning Representations, 2019.
[8] Deng, W. Dong, Socher, L.-J. Li, K. Li, and L. Fei-Fei. ImageNet: A Large-Scale Hierarchical Image Database. In CVPR09, 2009.
[9] R. Geirhos, C. R. M. Temme, Rauber, H. H. Schütt,, M. Bethge, and F. A. Wichmann. Generalisation in humans and deep neural networks. In Advances in Neural Information Processing Systems 31, 2018.
[10] R. Geirhos, P. Rubisch, C. Michaelis, M. Bethge, F. A. Wichmann, and W. Brendel. ImageNet- trained CNNs are biased towards texture; increasing shape bias improves accuracy and ro- bustness. In International Conference on Learning Representations (ICLR), May 2019. URL httppparweereeeeeeeeoooedddddaaa
[11] W. Brendel and M. Bethge. Approximating CNNs with lag-of-local-featuee models works surprisingly well on ImageNet. International Conference on Learning Representations (ICLR), May 2019.

---

[12] R. Geirhos, K. Meding, and F. A. Wichmann. Beyond accuracy: quantifying tial-by-ttria behaviour of CNNs and humans by measuring error consistency. In Advances in Neural Information Processing Systems 33, 2020.
[13] Demis Hassabis, Dharshan Kumaran, Christopher Summerfield, and Matthew Botvinick. Neuroscience-inspired artificial intelligence. Neuron, 95(2):245-258, 2017.
[14] K Fukushima. Neocognitron: self organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological cybernetics, 36(4):193, 1980.
[15] David Marr. Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. Henry Holt and Co., Inc., New York, NY, USA, 1982. ISBN 0716715678.
[16] Daniel LK Yamins and James J DiCarlo. Using goal-driven deep learning models understand sensory cortex. Nature neuroxcience, 19(3):356-365, 2016.
[17] S. A. Cadena, G. H. Denfield, E. Y. Walker, L. A. Gatys, A. S. Tolias, M. Bethge, and A. Ecker. Deep convolutional models improve predictions of macaque VI responses to natural images. PLoS Computational Biology, 2019. doi: 10. 10.1101/201744.
[18] Tiago Marques, Martin Schrimpf, and James J. DiCarlo. Multi-scale hierarchical neural network models that bridge from single neurons in the primate primary visual cortex to object recognition behavior. March 2021. doi: 10.1101/2001.0.01..43444 URL http:: http:://///oi.oo11.1111/111
2021 03. 01 433495
[19]
Monstantin-Klemenn Lurz, Mohammad Bashiri, Konstantin Willeke, Akshay Jagadish, Eric Wang, Edgar Y. Walker, Santiago Cadena, Taliah Muhammad, Erick Cobos, Andreas S. Tolias, Alexander S Ecker, and Fabian H. Sinz. Generalization in data-driven models of primary visual cortex. International Conference on Learning Representations, 2021. URL attps://openreviess http://etetttttormmmodrpppoaoooa
(20) E Y Walker, F H Sinz, E Cobos, T Muhammad, E Froudarakis, P P Fahey, A S Ecker, Reimer, Pitkow, and Tolias. Inception loops discover what excites neurons most using deep predictive models. Nat. Neurosci,, 2019.
[21] Ruth C. Fong, Walter J. Scheirer, and David D. Cox. Using human brain activity to guide machine learning. Scientific Reports, 8( 8(1):5397, Mar 2018. doi: 10.1088//41598--18-22666 URL https://dolt org/rr////o 10. 1038/ 841598- 01-0-011---26868
[22] Zhe Li, Wieland Brendel, Edgar Walker, Erick Cobos, Taliah Muhammad, Jacob Reimer, Matthias Bethge, Fabian Sinz, Zachary Pitkow, and Andreas Tolias. Learning from brains how to regularize machines. Advances Neural Information Processing Systems, pages 9529-9539, 2019.
[23] Callie Federer, Haoyan Xu, Alona Fyshe, and Joel Zylberberg. Improved object recognition using neural networks trained to mimic the brain't 's statistical properties. Neural Networks, 131: 103-114, 2020.
[24] Richard A. Caruana. Multitask Learning: A Knowledge-Basee Based Source of Inductive Bias. In Machine Learning Proceedings 1993. 1993. doi: 10.1016/b978- 1-55860-307-33500122
[25] Li Zhaoping and Zhaoping Li. Understanding vision: theory, models, and data. Oxford University Press, USA, 2014.
[26] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In Yoshua Bengio and Yann LeCun, editors. 3rd International Conference on Learning Representations, ICIR 2015, San Diego CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
(27] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the 32nd International Conference on International Conference on Machine Learning Volume 37, ICML 15, page 448-456. JMLR.org, 2015.
12

---

[28] John S Bridle. Probabilistic interpretation of feedforward classification network outputs, with relationships to statistical pattern recognition. In Neurocompuuting pages 227-236. Springer, 1990.
[29] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pages 1597-1607. PMLR, 2020.
[30] Alex Kendall, Yarin Gal, and Roberto Cipolla. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7482-7491, 2018.
[31] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun, editors, 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
[32] C. Michaelis, B. Mitzkus, Geirhos, E. Rusak, O. Bringmann, A. S. Ecker, M. Bethge, and W. Brendel. Benchmarking robustness in object detection: Autonomous driving when winter is coming. In Machine Learning for Autonomous Driving Workshop, NeurIPS 2019, volume 190707484, Jul 2019.
[33] Joel Dapello, Tiago Marques, Martin Schrimpf, Franziska Geiger, David Cox, and James J DiCarlo. Simulating a primary visual cortex at the front of CNÑs improves robustness to image perturbations. In Advances in Neural Information Processing Systems (NeurIPS 2020), volume 33, pages 13073-13087. Curran Associates, Inc., 2020.
[34] Matthias Kummerer, Thomas S. A. Wallis, Leon A. Gatys, and Matthias Bethge. Understanding low- and high-level contributions to fixation prediction. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Oct 2017.
[35] Kuniharu Arai, Edward I. Keller, and Jay A Edelman. Two-dimensional neural network model of the primate saccadic system. Neural networks, 7(6-7):1115-11 1135, 1994.
[36] Omid Rezai, Pinar Boyraz Jentsch, and Bryan Tripp. A video-driven model of response statistics in the primate middle temporal area. Neural Networks, 108:424-444 2018.
[37] Patrick McClure and Nikolaus Kriegeskorte. Representational distance learning for deep neural networks. Frontiers in computational neuroscience, 10:131, 2016.
[38] Zhaoping Li. A saliency map in primary visual cortex. Trends in cognitive sciences, 6(1):9-16. 2002.
[39] Li Zhaoping and Li Zhe. Primary visual cortex as a saliency map: a parameter-free prediction and its test by behavioral data. PLOS Comput Biol, (1110):e000373) 2015.
[40] Xilin Zhang. Li Zhaoping, Tiangang Zhou, and Fang Fang. Neural activities in v1 create bottom-up saliency map. Neuron, 73(1):183-192, 2012.
[41] Nobuhiko Wagatsuma, Akinori Hidaka, and Hiroshi Tamura. Correspondence between monkey visual cortices and layers of saliency map model based deep convolutional neural network for representations of natural images. Eneuro, 8(1), 2021.
[42] Evgenia Rusak, Lukas Schott, Roland S Zimmermann, Julian Bitterwolf, Oliver Bringmann, Matthias Bethge, and Wieland Brendel. simple way to make neural networks robust against diverse image corruptions. In European Conference on Computer Vision, pages 53 69, Springer, 2020.
[43] Tim C. Kietzmann, Courtney J. Spoerer, Lynn K. A. Sörensen, Radoslaw M. Cichy, Olaf Hauk, and Nikolaus Kriegeskorte. Recurrence is required to capture the representational dynamics of the human visual system. Proceedings of the National Academy of Sciences, 116 (43):21854 21863, 2019. ISSN 0027-8424. doi: 10.1073/pnas. 00.1073/pmss1195541111 1905544116. URL https // //www. pnas org/coo//oo wwwteett//6///////222
13


Reviews:
[{"review_id": "kdP5zacfgxn", "reviewer": "Reviewer_KL3f", "summary": "This paper uses recordings from monkeys to co-train a classifier with a second head from layer conv-3-1 (of VGG-19) that predicts the cortical responses in V1 to the same pictures. The authors test this model on 14 different distortions of images and find that it is more robust than a baseline model trained only to classify the images. They show an upper-bound on robustness via a network trained on distorted images from the same distributions, and show that training on scrambled neural responses provides no advantages. \n\nFinally, they analyze the network in two ways, qualitatively and quantitatively. First they reconstruct several input images corrupted by gaussian noise from conv-3-1 in their model and the various control networks. They find that the monkey-trained network appears to focus more on the object than the noise compared to a network trained only on the images. This is obviously subjective, so they go on to measure how much the monkey-trained network attends to salient regions (based on Deep-Gaze-2) compared to the other models, and find that it is generally more sensitive to the salient regions.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: While several previous works have used neural recordings (fMRI, monkey cortex) to regularize their models, this is (I believe) the first to co-train a model on categorization of the image as well as regression to the monkey recordings. Furthermore, I don’t believe anyone has measured the robustness to such a broad array of types of distortions in a neurally-constrained model. \n\nQuality: Everything appears technically sound. The results are strong, and there are thorough control networks, each controlling for different things, or testing different aspects. There is a baseline model that is only trained on clean images. Then there is the main model, named “MTL-monkey”, which instantiates the main idea of co-training on clean images and predicting V1 neural recordings. Actually, the “neural recordings” are from a network trained to produce them. This is done to give the MTL network a single target for each image, since the monkey recordings vary across presentations. It is unclear why this was necessary; one wonders what would happen if the MTL-monkey network had been trained directly on the data. However, (see below), this neural prediction network is used to make “pseudo-recordings” on images never shown to monkeys, expanding the dataset.\n\nThere are two “oracle” models as well. The first, simply titled the “Oracle”, is trained on a 50-50 split of clean and distorted images, using both classification and SimCLR-type representation learning between clean and distorted images at the conv-3-1 level. Now that the front end (up to conv-3-1) has been trained to be robust to distortions, the rest of the network is re-trained on clean images. This represents an upper bound on robustness where the front end of the network is what is robust to noise. \n\nThe second oracle network is a multi-task trained network, called MTL_Oracle. It is identical to the MTL-monkey model in structure, but it is trained on neural predictions from the Oracle network. To provide these predictions, the Oracle network is frozen and V1 predictions are trained from the representations at conv-3-1. These neural predictions from the Oracle network are thus trained from a front-end that has been “robustified” to distortions, and provides an upper-bound on the amount of transfer learning possible from the neural signal. Then MTL-Oracle is trained only on clean images, but the targets are from a network sensitive to image distortions. This results in a network that is nearly as good as the Oracle network, and better than the baseline network, demonstrating that transfer learning is possible in principle. \n\nClarity: The paper is written in a pretty confusing manner. It was not easy to extract the above description from the paper. There are a dizzying array of both single-task and MTL models. For example, the “Models” section starts on line 92. But at the end of the paragraph above that, the paper describes a single-task network that is simply trained to map from the images to the neural recordings. This network is then used to predict neural recordings for all of the TIN (Tiny Image Net) images. These predictions are then used in the following models. So, it seems that not all of the neural recordings that the MLT-monkey model is trained on are actually neural recordings, but instead are **predictions** of neural recordings. This makes sense, as it expands the data that can be used to train the neural responses, given the issue that monkey recordings are difficult, and one might not have neural recordings for all images. However, this should be made clearer in the paper. An example of where this leads to confusion is line 167, where you describe your model as being trained on “real neural data.” It isn’t, if I understand correctly.\n\nFurthermore, the Models section doesn’t describe all of the models. Instead, it describes generically what network is used, and how it is modified to be fully-convolutional. \n\nAnother confusing part is the description of the Oracle training. This is reported in the “Results” section - it probably should be back in the “Training” section, or the “Models” section. On line 141, it describes how the Oracle network, trained on 1:1 clean:distorted images, is frozen up to layer conv-3-1 and then the rest of the network is re-trained only on clean images. But then on line 142-144, they mention further training on the frozen part using SimCLR-style training to map the distorted and clean images closer together in layer conv-3-1. If you are doing that, it isn’t frozen! So, I infer that you are describing how the 1:1 network was trained **before** it was frozen. These sentences should be rearranged. Oh, and it isn’t clear that after you have frozen the network up to layer conv-3-1, that you re-initialized the rest of VGG-19 to random weights again, i.e., trained that part from scratch. I assume that’s what you did.\n\nIt would be best to systematically describe each model, and why you created it. First, there’s the neural recording predictor, used to create single targets for each image, and targets for images that were never actually shown to monkeys. This is the “neural data” that you train the MTL model on, as well was what you trained the frozen Oracle model on to make MTL-Oracle. The reason for MTL-Oracle is to demonstrate that transfer learning is possible. It might make the paper clearer to leave out MTL-Oracle altogether, since you demonstrate transfer learning with MTL-Monkey, but that’s up to you. Also, you should describe MTL-shuffled in the methods before the results. Why did you create MTL-shuffled? Just to show that adding on a second head doesn’t help if it isn’t making correct predictions (this seems a bit silly, but ok). Because the reader encounters Figure 3 before the descriptions of all of the different models, many of the names given there are somewhat mysterious. \n\nUnder the Models section, you mention using pooling; I assume you are referring to global average pooling? If so, say so.\n\nIt’s a bit weird to say that you learn to balance the two losses, but then say you can control it with the batch ratio. Then you use a batch ratio of 1:1! It is only later that this is used in Figure 3C - it might be good to say why you have this earlier. In the batch ratio, I assume that in order to implement this, you have instances where you only collect gradients from the neural head when the ratio favors the neural recordings and vice versa? Also, in Figure 3C, you mean the regression is from neural performance to robustness, right? Not the other way around. \n\nIt would be good to have a figure showing a clean image along with the 14 distortions. We shouldn’t have to go back to the Hendrycks and Dietterich paper to find out what they look like. \n\nI notice there are no error bars on the baseline network. \n\nLine 172: What was the range of batch ratios used? \n\nLine 174: You describe this as “two-factor regression” - I read that to mean you had two independent variables. What are the two factors in the regression? There’s an independent variable (neural prediction) and a dependent variable (robustness), right? How can there be a p-value for both?\n\nlines 248-251 (sentence starting “It could be…”) I can’t parse that sentence. Please rewrite it.\n\nLine 260: I don’t know what you mean by all pixels up to a cumulative sum of 0.7. Pixels go from 0-255. What are you summing? Why not just do a median split?\n\nIn the discussion, you mention that it would be interesting to use higher brain areas in training. I think this has already been done in Kietzmann et al,, (2019) Recurrence is required to capture the representational dynamics of the human visual system. PNAS 116(43):21854-21863. There, they used the dynamics of RSA over time to train a network to have those dynamics. \n\nSignificance: The results here are very interesting: Training a network to also predict neural responses makes it more robust. It isn’t obvious that this should be so, so this is an important result. I just wish the paper had been written more clearly. \n\n\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "They mention some limitations, but I think one is that they used neural predictions rather than raw neural data. This is either a bug or a feature, depending on how you look at it. I think it should be promoted as a feature, myself, but it could be seen as a limitation.\n\nI see no societal impacts of this work, and neither do they.\n", "review_text": "", "extra_scores": {"ethical_concerns": "None.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4.75"}, "extra_text": {}}, {"review_id": "OiSOW3gIYtT", "reviewer": "Reviewer_ji1K", "summary": "The paper introduces a new method of regularizing deep networks with neural data. Specifically, the network is co-trained on a main task and regression of neural recordings (or at least deep-network predictions of these recordings for new images). The analysis shows that a network trained on neural responses is sensitive to different features, and encodes gaze-salient image regions more strongly, than control networks. ", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: As the authors point out, the work is closely related to past work using representational similarity from neural recordings to regularize deep networks. The authors also mention that task/neural-data-reconstruction co-training has been suggested before (Sinz et al. 2019), but I think the idea has been in the air for some time before that, see e.g. (particularly the latter): \n  Rezai, O., Jentsch, P. B., & Tripp, B. (2018). A video-driven model of response statistics in the primate middle temporal area. Neural Networks, 108, 424-444.\n  Arai, K., Keller, E. L., & Edelman, J. A. (1994). Two-dimensional neural network model of the primate saccadic system. Neural Networks, 7(6–7), 1115–1135. \nNonetheless, this work seems to be the best example so far of carrying out this idea. \n \nQuality: I think the work is sound and well developed. It could potentially be explored more thoroughly using different tasks, datasets, networks, and other measures of robustness. \n\nClarity: The writing is basically clear. The explanation of methodological details is compact (e.g. Gaussian readout, a recent and non-obvious approach, is only briefly mentioned), so I think it would be helpful to share the code. There are a few small points that could be further clarified, e.g.: \n1) Link, reference, and description of the TINS dataset. Also explain the choice of dataset. \n2) Which network was used to produce the predicted responses? I’m not sure whether line 96 refers to the model that produces predicted responses, or the site of co-training. I think the process is described on line 147, but it would help to expand the explanation around line 96. \n\nSignificance: I have three reservations about the significance: \n1) Given that the approach is closely related to, e.g., Federer et al., I think it would be useful to compare the current approach to that one. Which one works better? Does each have advantages and disadvantages? \n2) The main practical benefit of co-training on neural data seems to be that the network becomes nearly as robust to corruptions as a network that is trained on corrupted images. However, neural data is expensive and corrupted images are cheap, so the practical significance may be minor, or perhaps the authors could discuss it in more detail. \n3) The pipeline for achieving the above benefit included corrupted images. Removing these from the pipeline, the MTL-Monkey model was notably less robust. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "These issues were not addressed, but I don’t think there are problematic impacts. ", "review_text": "", "extra_scores": {"needs_ethics_review": "Yes", "ethics_review_area": ["Responsible Research Practice (e.g., IRB, documentation, research ethics)"], "time_spent_reviewing": "5 hours"}, "extra_text": {"ethical_concerns": "The main ethical issue is the use of animal data. The authors point out that the collected data is useful for a wide range of analyses beyond this one, so the study improves the usefulness of the data. If possible, this point could be expanded with examples. If the method is really effective, it seems that it could conceivably spawn an industry of new data collection for neural data in a wide variety of tasks. It would be positive to set a precedent here and release the data. There is a statement that “data coming from monkeys complied with the approved protocol of local authorities.” \n"}}, {"review_id": "ADlFdDXy5bD", "reviewer": "Reviewer_81tw", "summary": "This study follows a recent trend of results that use similarity to neural representations in early visual areas as a regularizer during training of deep neural networks (DNNs) for object recognition. Here the authors jointly train a DNN for image classification (grayscale version of Tiny ImageNet) and predicting neuronal responses in macaque primary visual cortex (V1), observing that this joint training procedure leads to increased robustness to common image corruptions. Finally, the authors claim that this improvement in robustness is likely to result from an increased sensitivity to salient regions of an image.\n\n[Score updated 5 -> 6]\n", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall the paper is well written, clear and the experiments appear to be properly done. I found the analysis on the saliency maps particularly interesting, deserving further exploration.  However, I have several concerns that negatively affect my score. \n\n\n1) Main results are not entirely novel.\nThe authors state that the neural co-training hypothesis by Sinz et al was untested prior to this study. In my opinion, this statement is not entirely true since similar approaches have been used in the past. Specifically, Li et al NeurIPS 2019 used mouse V1 data to regularize CNNs towards more brain-like representations. This is very similar to the approach used here. While the method for introducing the regularization, the dataset (CIFAR vs Tiny ImageNet), and the species (mouse vs monkey) are different, this study is at most an extension of the existing approach, which is perfectly fine. However, as an extension it fails to make significant novel contributions. Dapello et al NeurIPS 2020 had already shown that introducing a neuroscientific model of monkey V1 leads to improved robustness to image corruptions on color ImageNet, which is a considerably larger dataset and closer to real-world applications than the one used on this study (check model trained without stochasticity for a better comparison). Furthermore, when comparing the improvements of these two studies on a corruption by corruption basis, the results are strikingly similar (improved robustness for noise, blur and digital corruptions and worse robustness for weather effects). So in summary, this study extends Li et al to a larger dataset and to predicting monkey V1, and in the process replicates findings already described in Dapello et al - macaque V1 similarity is correlated with robustness and making the early stages of the model better approximate V1 leads to improved robustness.\n \n2) Clean accuracy missing.\nClean accuracy is only reported for the MTL-Monkey models in Figure 3C. This is problematic since the main comparisons are done using a robustness metric which normalizes accuracy to corrupted images by the clean accuracy. Also, the clean accuracy scores reported for the MTL-Monkey models appear to be surprisingly low (under 50%). If this is true, then the usefulness of this approach for improving models seems to be considerably limited.\n\n3) Great variability in the absolute accuracy scores across corruption types.\nWhile some perturbations show overall very large accuracies throughout the severity ranges (brightness between 30 and 50%), others have extremely low accuracies even at the lowest severity levels (defocus blur < 3%). This is very surprising and not consistent with the ImageNet-C accuracy levels. This suggests that the implementation of these corruptions for the grayscale Tiny ImageNet dataset is not appropriate and makes these results harder to interpret.\n\n4) I am not a fan of the glass blur corruption being left out from the test dataset. Given all the compute used in this study, I fail to see how producing the glass blur perturbation for the grayscale TIN validation-set would represent a substantial increase in compute. This particular corruption affects other defense methods differently from the remaining blur corruptions and its exclusion here should be avoided. \n\nMinor issues:\n\n- What is the range of repetitions per images during the neurophysiological recordings (measured per neuron) for both the training and the test images? This information should be reported in the paper\n\n- The input size of 64x64 with 14.5ppd when matching the V1 recordings means that the model’s input size in physical quantities is ~4.5deg, is this correct? This is smaller than what was considered optimal by Cadena et al 2019. Why the change? \n\n- The authors claim that CNNs are state-of-the-art for neural response prediction of primate V1 (line 47). I would say that this statement is still somewhat under debate since Cadena et al 2019 reported only minor improvements over a very simple classical V1 model and more recently Marques et al 2021 argued that some optimized neuroscientific V1 models outperform CNNs in several V1 neural benchmarks.\n\n[Score updated 5 -> 6]", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors adequately addressed the limitations and potential negative societal impact of this study.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "7"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper shows that co-training on primate neural data improves the robustness of CNNs.  There were some questions about the relevance of the work with respect to other previous work, but this was handled well by the authors and the reviewers were satisfied by those clarifications (which should also be worked into the paper).  There were some ethical concerns raised, but these also were handled well.  In general, I believe the review process worked well here, the paper is stronger because of it, and should be accepted.
Decision: Accept (Poster)