Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: This paper proposes a benchmark for the comparison of neural network models of vision to rodent visual cortex. The benchmark compares models with various architectures (including various CNNs, transformers, and other architectures) and one model trained different tasks to a large database of neural recordings from mouse visual cortex. It relies on newly proposed metrics that are validated on a previous benchmark. These metrics evaluate both the fit to individual neurons and the entire population of neurons. Previous results are replicated and explained in the context of a large scale study. The paper provides analysis of the results as well as predictions on the function and structure of the rodent visual cortex areas.


main_review: The paper presents a benchmark, new evaluation metrics, a large scale evaluation of vision models trained on various tasks as models rodent visual cortex, many analyses of the results and interesting interpretations of these results. 

strengths:
- The benchmark is extensive, it explores a large number of models and tasks when compared to previous work in the field.
- They propose new metrics for neural regression which are faster and computationally inexpensive. Furthermore, the two metrics evaluate different aspects of the neural code (fitting individual neuron activity and fitting neural populations).
- Previous results on randomly initialized models are replicated and extended to other architectures. Which offers a new perspective on the inductive bias contributes to the scores.
- The layer-by-layer and area-by-area analyses offer many interesting hypotheses and predictions on rodent visual cortex.
- The paper is clear, thorough, well detailed and well explained. The code is provided for reproducing the results.
- The authors discuss the paper's limitations, provide detailed descriptions of the computational resources they used and acknowledge the ethical concerns behind their work. 

limitations_and_societal_impact: The authors addressed some of the papers limitations. However there are other issues to address, I have split them into major and minor weaknesses:

Major
- It's important for analysis papers to have a clear objective beyond fitting neural data. The paper lacks hypotheses and predictions on brain function that constrain the space of models and tasks, allow for more in-depth analysis which, in turn, gives solid conclusions.
- In brainscore's benchmark, data was recorded from monkeys tasked with object recognition, so evaluating only models trained on classification makes sense. To my understanding, the mice that are recorded in the dataset were only passively viewing images. Thus, it might be more reasonable to, for example, shift the analysis towards unsupervised/self-supervised objective functions and test models with different hypotheses about perception in that respect. 
- Inline with this remark, not all tasks included in the benchmark are behaviorally relevant for rodents. This might be why no clear conclusions can be deduced from the analysis of the task clusters. Another possible reason could be that some visual areas are responsible for detecting motion, which is why static stimuli do not allow for their differentiation from other areas. 
- In the absence of a guiding hypothesis and as mentioned by the authors in the discussion, the benchmark should include a wider range of architecture types and objective functions. Especially biologically plausible models of vision, such as recurrent models. Objective functions include, but are not restricted, self-supervised objectives and generative models of vision. 

Minor
- Calling a neural regression method state of the art is inaccurate and might cause confusion. A neural network architecture is state of the art if it performs better than previous architectures whereas the higher pearson's R on brainscore only highlights the new neural regression method's ability to extract more correlation from the data.
- The paper explores differences of neural regression on two axis; task and architecture. However, scores might change for different task-architecture combinations. It is definitely a formidable challenge to perform such an analysis at a large scale. But it is possible for a subset of architectures that do not share the same inductive bias (like convolutions), and using only behaviorally relevant tasks. 
- Results in 3.6 which claim the slight importance of architecture over task are far from conclusive because many biologically relevant tasks are not included and the space of architectures is not fully covered as mentioned previously.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: Extracts intermediate features (across all layers) for models pretrained on ImageNet classification (and other computer vision tasks) and tests its modeling performance of mice two-photon neural responses (from 6 cortical areas) using regression and RSA. Features at intermediate depths are better, there is a slight hierarchy between neural network depth and cortical area, and there seems to be no relation between the pretraining task and the cortical area the trained network models best.
main_review: Originality
Though techniques used are standard, the extent of the experiments and the animal model (mice) is novel and significant.

Quality
Experiments are sound and enough information is provided to allow replicability.

The main drawback of the paper in its current form is the figures, though they contain information to draw the conclusions drawn in the paper, it is not easy to see. Fig 1, for instance shows performance metrics of pretrained vs randomly initialized models but the models have been sampled (from the total of 110), the red bar goes below the blue bar if the value is lower and the error bars overlap so it is hard to see where they start or end. Perhaps a two-sided violin plot per model (instead of the bars) could show the distribution for blue and red better. Or maybe a scatter plot (x axis trained, y axis random) where each point is a model could show all models and more easily show which version is better (in comparison to the diagonal), error bars could also be included per point or perhaps omitted for clarity, and important points (i.e., models) can be labelled (with colors or marker styles or text above the point). All results from Sec 3.3 (other than the best network being Inception-ResnetV2) are close to impossible to see in Fig. 1. In Fig. 2, perhaps adding a table with the averages per task group will help show the trends and differences between the two metrics (as done for figure 3). Fig. 3 could have also been a table (or perhaps two), it will be easier to read the numbers and gauge the difference that way (it is also currently unclear what is your assumed cortical hierarchy and task hierarchy). Given the amount of work put into obtaining these good results, spending time finding a better way to convey them might be worthy and will surely make the paper more attractive/citable.

I do not think the method comparison with BrainScore regression (Sec 3.1) adds to this paper (and it may take away from it). I understand this was asked of you before but it does not actually show that results from this paper (in mice) will hold if using the BrainScore regression method, it only shows that both methods give similar regression results on the same data (which is expected given that the changes are minor). That the method transfers to primate data (with the very different visual system and recording methods) might be interesting but I presume so will many other methods (including BrainScore’s) and regression performance or transferability across animal models is not the focus of this paper so it feels disconnected. It may be worth considering bumping it to the end of the result section (as an extra transfer result) or even to the appendix.

Clarity
Well written, though sometimes wordy; for instance, the main results of the paper at the end of the introduction could be a lot bolder/more direct even if obviating some details (for instance, 1 → “Training matters: features from trained models predict/model neural responses better than features from randomly initialized models.” or phrases like “features learned in service of some task” → “learned features” or “there may be a slight upwards gradient in complexity” → “complexity increases”).  Of course, the work and results still hold but I think ML readers will appreciate a less hedged writing style.

For reproducibility, can you add a sentence about how images were modified to work as ImageNet images, how do you deal with the aspect ratio difference (if any), sizing and color. 

Perhaps it is worth providing the proportion of cells (out of the 6.6K) from each cortical area (and the name of the areas). A single sentence should do. “Dataset areas are divided as V1 (50%), AM (10%),  PM(5%), ….” (could be in supplementary).

For the Taskonomy models, you say “we choose to extract representations across all layers”(l.90), to confirm, this means through all encoder+decoder layers (i.e., different tasks may have different depths)?

I am confused about your ridge regression, do you cross-validate or not? because it says you use “a lambda penalty of 1.0” (l.132) but then in the same line you say you “monopolize generalized cross-validation” (?), I presume you do use LOOCV (if so, I would drop the sentence about using a lambda of 1 and perhaps provide the lambda grid you searched over).

Also, to confirm, once you cross-validate the penalty, you use all the data to fit the final linear regressor (with the chosen lambda), right? if so, the regression correlation may be optimistic (i.e., it reports cross-validation results). This is ok as long as what’s reported is clear.

The interpretation in Sec 3.4 seems premature given that it is only truly one task (Semantic segmentation) that moves up from SRP to RSA (and at the same time, scene classification moves down from SRP to RSA). It is “only” a hypothesis so it is up to the authors to decide whether they are ok with it.

Changes in performance in Sec. 3.6 are so small I do not think one can even say they point in any direction (l.251).

Results from Sec 3.8 are neat, I thought the lack of functional specialization across different cortical areas in mice was well known; maybe you can find some references for it and frame your results as further confirmation. I would also reference some of the discussions about cortical area hierarchies. 

In l. 296, pointing regression scores are lower than RSA scores seems unimportant. They are both correlation scores but in very different spaces.

Great discussion.

Other minor remarks

- the word “later” in l.33 is odd.
- you end up using 6619 cells (not 59K as claimed in the abstract)
- in l. 132, it will be better to name scikit-learn [52] rather than just the citation or “implemented with a popular machine learning toolkit [57]” if you don’t want to name it.
- in l. 152, drop “In this work,”
- i would reference the figures before the written results (i.e., closer to the top of each 3.x subsection rather than at the very end), for instance, the line in 195-196 could go up to l.184
- l.219 ‘writ large”?
- Reference to fig 4 (l.257) appears before reference to fig 3 (l.281), fig 3 should probably be fig 4 and viceversa.
- For 3.x sections, perhaps consider putting result directly in the titles, for instance, section 3.5: "how well do non-neural network baselines predict rodent visual cortex" -> "non-neural network baselines fail to predict rodent visual cortex”
- Showing the neural RDMs for different areas and perhaps some for a selected layer in a trained model could be interesting (perhaps in appendix)

Significance
It fills a research gap in visual neuroscience and will be a good reference for future studies.

Overall
This is a good paper that is on the verge of acceptance. With some improvements I would have no problem accepting it.


Update:
I have bumped my score from 5 to 6 after the authors addressed some of my concerns and those of other reviewers.
limitations_and_societal_impact: Biological datasets are always limited, though not directly pointed out, it is understood. Discussion about societal impact of a better understanding of the visual system is not directly provided, though I would think it may be unnecessary.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: Several other reviewers have expressed that they find the current scientific contributions of the paper significant enough, so I will increase my score to a 6, even though the authors did not quite address my major concern about the novelty and utility of the proposed neural regression (i.e. what is the empirical benefit of neural regression over just ridge regression? if that is not understood, the authors should disclose that and not put so much emphasis on the importance of the regression method (in the title, in the abstract, and in the main text)).  

I believe that the paper can benefit from reducing the amount of time spent discussing the neural regression method and the RSA vs encoding model comparisons, and more time focusing on the actually novel and significant part which is the comparison of the findings in mice to the previously established findings in monkeys and humans. And just to clarify, I think that only discussing what the findings are in mice is not enough on its own, because all the analyses that the authors carry out have been previously done in other organisms. So to actually have a significant contribution, I think it's important to emphasize the comparisons of the results in mice to the previously established results in other organisms.

=============== post-rebuttal =================

The manuscript presents an exploratory empirical study of the relationships between the mouse visual system and representations extracted from computer vision neural network models. The authors investigate representations from a large number of computer vision models, as well as representations from the same models but when the models are initialized randomly. They also explore two ways of relating the mouse recordings to the representations from the computer vision models: encoding models and representational similarity analysis. The authors additionally propose what they claim is a new encoding model methodology, though I remain unconvinced of its utility.
main_review: The manuscript was easy to read and provides a broad exploration of an interesting question. I do however have several concerns about this work. I've summarized these below.

1. Methodological concerns. The authors claim to propose a "novel, highly optimized neural regression method" for encoding models. However I'm unclear about what parts of the proposed method (sparse random projection + ridge regression via generalized cross validation) the authors are claiming to be novel, and I'm also unclear about which of those parts are actually leading to a boost in prediction performance. The authors claim that ridge regression is a less popular encoding model method, though it is in fact the default method in encoding models in the language literature (Wehbe et al 2014, Huth et al. 2016, Jain and Huth 2018, Toneva and Wehbe 2019,...). The authors use leave one out cross validation which is indeed an uncommon choice, but it's also the default setting on the RidgeCV method in sklearn, so it's unclear what is "highly optimized" about this method. Also, it would be helpful if the authors could provide some clarification (both in the rebuttal and in the paper) about exactly what samples are used to train the encoding models (both in the mouse and the monkey experiments). Were separate encoding models built for different individuals? If separate encoding models were built, it's not clear how the predictions or predictions performance is aggregated across individuals. How were the repetitions of different image stimuli handled in the leave one out CV setting? I imagine that leaving out the data corresponding to one image while keeping data that corresponds to other repetitions in the training set can lead to some unfair underestimate of the generalization error. It would be interesting to compare the zero-shot performance of the leave-one-out proposed method to the zero-shot performance of the default Brain Score encoding model (i.e. zero-shot meaning that all repetitions of a test image are removed from the training set).

2. Questions about some of the interpretations. 
- L214-220: The authors interpret the result that transformer-based architectures show the largest differences in prediction performance between the trained and randomly initialized computer vision models as evidence that some of the better performing randomly initialized models may be due to an underlying convolutional architecture. Another possibility is that the randomly initialized models predict the neural recordings well across architectures, but the trained transformer-based architectures are much better at predicting the neural recordings. This possibility is consistent with a large difference in prediction performance between the trained and randomly initialized transformer-based models. To disentangle these, it would be helpful if the authors compared the prediction performances across different randomly initialized architectures with a statistical test.
- The authors compare the results from RSA with those of encoding models and state that "with more probe images, we expect that the parametric linear mapping would be more competitive with the nonparameteric distances of representational similarity". It's not clear to me that the correlations from encoding models being lower than those from RSA means that the encoding model results are less "competitive". The results from an encoding model are from generalizing to a previously unseen datapoint, and the RSA results are not. It would be helpful if the authors provide an estimate of the chance performance in both the encoding model and the RSA setting to show that one is not inherently easier than the other.

3. Some lack of clarity in the methodology and experimental settings. The manuscript has a clear exposition for the most part but there are some methodologically details that need to be better explained. Some examples:
- The questions that I brought up in the previous point 
- L92-98: what is being clustered and how? Are these clusters from the original Taskonomy paper or did the authors actually obtain these "purely data-driven clusters". 
- L111-120: for a reader who is unfamiliar with RSA, the description in these lines may not be easy to parse. For example "We compute RDMs by correlating the activations of all neurons in a given neural site to the 118 images in the stimulus set" sounds like the authors are correlating neural activity to images, instead of computing pairwise correlations across the neural activations. Here it is again not clear how separate subjects are handled.
- L185-196: are these results using the mouse or the monkey data?
limitations_and_societal_impact: Yes
review_title: null

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: The authors present a large-scale empirical study comparing artificial neural networks against mouse brain recordings using correlation-based metrics. They utilize a variety of publicly available neural networks as well as a large brain data corpus provided by the Allen Brain Institute. Their metrics are sampled from the field and rely on linear correlation based comparisons. They do also provide a variant metric that is more performant than the recently proposed BrainScore. They demonstrate that an artificial network’s training task and architecture individually influence metric performance. They also replicate some analyses previously done with primate data to emphasize similarities and differences.
main_review: *Strengths:*
The study is timely and comprehensive. It provides a sufficient level of rigor to establish itself as a viable benchmark for relating mouse visual neuron recordings to artificial network activations. Overall it is well written with easy-to-parse plots. The appendix also provides a concise and convincing defense for the methods chosen in the main manuscript.

*Weaknesses:*
The claimed contribution of the work is to better understand what does/doesn't matter when comparing deep neural network activations against mouse brain activations. The abstract offers some argument for why we would want to do such a comparison in the first place, but in general the paper is pretty well fixated on the opening question of how well the networks fare as models of mouse cortex instead of what does comparing network responses to mouse cortex tell us. I realize that this work is building on an ongoing research thread from a couple of labs, who all provide their own motivations. Nonetheless, I still find myself wondering what new information the results gives us regarding biological & artificial networks. Maybe their emphasis on equivalences between primate and mouse is key? My recommendation is for them to try to make this point of *why the comparison matters in the first place* more salient in the intro or discussion.

The references need to be reviewed carefully. There are many [10, 26-28, 30-34, 36-42, 44-48] that do not include a publication venue. Others [24, 51, 74] are listed as arxiv preprints, but have since been published at peer-reviewed venues. Finally, [53] and [56] appear to be the same citation.

*Additional minor comments:*

1) Assuming you have this sort of resolution in your dataset, how well do the non-neural network baselines & neural networks perform on layer 4 primary cortex data? It has been suggested that this region has a dearth of “complex cells” as compared to primate data {1}. So I would imagine that the Gabor bank in particular will perform comparatively well in this area. I am asking because the Gabor filter bank model is still widely accepted as the standard among “interpretable” methods {2}, and given your discussion at the end about the meager overall performance it would be interesting to see if this focused area has a higher ceiling. Maybe this question is answered with the cre line split, but I’m not familiar with the different lines.

2) While I realize this is far from the authors’ problem to solve, I take issue with the phrasing of correlational scores as a “geometric” analysis. For example, one of a few quotes from the paper: “The resultant coefficient constitutes the score for how well a given model layer predicts the representational geometry of a given cortical area.” I found no mention of geometry in [56]. A quick internet search got me to {3}, which says “The dissimilarities can be interpreted as distances in the multivariate response space. The RDM thus describes the geometry of the arrangement of patterns in this space.” Maybe that’s where they got the term from? I find that it is a stretch to relate correlation values to distances and then further to a geometric description. My recommendation to the authors: use the term if you want, but I would suggest that you at least specify exactly what you mean by it. You should probably also differentiate your definition from others, e.g. that used by {4}, {5}, and {6}.

3) It appears to me that the overall differentiation among the models is quite small. This could lead to a possible alternative conclusion that none of these differentiations (task, architecture, depth, etc) really matter. Or, alternatively, that the metrics themselves are not useful for this type of differentiation. The noted winners/losers among architectures & tasks are certainly interesting observations, but I think it would help balance the paper if the authors commented on the general lack of variation in scores reported from figures 1 and 2. This would go well next to their already helpful discussion on the lack of total performance at the end of the paper.


{1} https://www.jneurosci.org/content/jneuro/28/30/7520.full.pdf

{2} https://www.annualreviews.org/doi/abs/10.1146/annurev-vision-091718-014731

{3} https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1003553

{4} https://www.nature.com/articles/s41586-019-1346-5

{5} https://elifesciences.org/articles/44526

{6} https://www.sciencedirect.com/science/article/pii/S0042698915003600
limitations_and_societal_impact: Yes, although see my comment #3 above.
review_title: null

Meta-review: This paper presents a large-scale / systematic study fitting many modern deep nets to mouse calcium data from the Allen institute. This paper received two accepts, one borderline accept and one borderline reject and was discussed on the forum. There was a bit of a discussion regarding the actual contributions of this paper. There was some agreement between the reviewers that the authors oversold the claim that they introduced a novel regression methods and the authors agreed to remove or significantly downplay this claim.There was also some discussion about the lack of clear hypotheses being tested or significant results contributing to our understanding of the visual cortex. The study is an impressive feat from the engineering perspective but at the end of the day from the dozens of models and results presented the fit are relatively low (correlation coefficients r<0.2 which means r^2<0.04). This is to be expected given how noisy calcium data are and given the small number of stimuli available (~100). However, given these scores it is possible that we are essentially fitting noise and indeed  the models are all performing very similarly so there is no really salient results. With that said, the positive reviewers also emphasized that the present study seem to be at least partially addressing a controversy in the field (because a previous study had found that random models did about as well as trained ones). Here at least the authors report the accuracy of trained models that significantly outperform untrained ones. The code is also made freely available and positive reviewers see that as a "launching point" for additional work on mouse (as opposed to primate cortex). All in all, this really is a borderline submission. Given the issues highlighted above, this paper could be accepted if space permits.
Decision: Accept (Poster)