Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: While deep networks are known to be biologically implausible, the strong similarities between the feature maps in such networks and the response characteristics (tuning curves) in different areas of the mammalian visual cortex (in particular the ventral stream) has prompted much work into trying to reconcile the known differences. The authors present work that contributes to such inquiries, demonstrating that at least some of the strong impediments in using such networks as plausible visual stream models may be more easily surmountable than they have thus far appeared to be.
main_review: The authors present thorough analyses of standard feedforward networks that quantitatively demonstrate that such networks can model the visual stream well while requiring far fewer coordinated parameter changes than occur with traditional training methods.  In particular, they show that very little supervision is required for strong matches to neural responses, that plausible distributions of initial random weights can match neural responses very well, and that weights can be updated very sparsely while still achieving quite high neural response matches. 

Such findings are highly relevant to allowing for such networks to be more plausibly accepted by the neuroscience community as valid models of cortical networks.  The striking finding that well-chosen random weights (Weight Compression) can predict neural responses with more than 50% accuracy is  especially interesting, as it suggests that indeed there may be less training required in mammalian brains compared to randomly initialized neuronal networks. This possibility opens up further avenues of study relating to the ability of real brains to learn with far fewer examples compared to ANNs.  

Figure 4b further demonstrates this possibility, showing that, under the ethologically relevant condition of little supervision, WC + Critical Training, where only a small subset of weights in so-called critical layers are trained, greatly outperforms traditional training.  We note that, in the ideal case, a biologically plausible method might be found that performs well both under little supervision, as WC+CT does here, and under a large amount of supervision, as traditional backprop does, the demonstration of principle for the former case already represents appreciable progress.

However, little in the way of tying model predictivity to accuracy on another task was explored. While tying brain predictivity to network performance is of course a very active area in its own right, we feel that addressing this directly within the context of the present work would facilitate readers' abilities to contextualize the present study with respect to ANN performance.

Overall, we find the authors' presented work to be of great interest to those interested in applying ML models to theoretical questions in neuroscience and to have a thorough set of analyses that both illustrates progress in this area while suggesting future avenues of work.

Minor:
- opening quotes are backwards throughout paper 
summary_of_the_review: The authors provide a compelling case that much more biologically plausible sets of initial conditions and sparse, selective supervision and synaptic updates might help substantially close the gap in viewing deep networks as plausible visuocortical models.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper is concerned with closing the gap between the amount of training in deep networks and in developing brains, as the current deep learning models use an unrealistically large number of synaptic updates. The authors address that with three strategies: less training, weight clustering and training in a subset of layers. All methods are tested individually and in combination with each other on primate ventral stream data.  
main_review: ## Strengths
The paper addresses an important problem in theoretical neuroscience: while trained deep networks appear to be good models of the visual stream, the amount of training they need to get there is incompatible with biology. 

Two of the proposed approaches to reduce the amount of training are somewhat straightforward: training for fewer epochs and freezing the amount of trained layers. However, I think the provided results are important as they show good fit to brain data can be achieved without closely following the training procedures of deep learning.

The last approach, weight clustering, is perhaps more interesting to the theoretical neuro community, as it works in the same way as genomically encoding the network (the “at birth” state). 

Importantly, the proposed ways to reduce the amount of training are tested not only using ImageNet performance, but also ventral stream data, which is arguably more important for neuroscience. 

Overall, the paper presents a novel set of results for an important problem, and supports its claims with a significant number of experiments.
 
## Weaknesses
As mentioned before, I think the weight compression part is the most valuable one, and therefore it should receive more attention in the paper.
1. Clustering should be better explained in the main text. Especially how the number of clusters was chosen.
2. In the beginning of Sec. 4, you discuss how many bits it would take to encode different networks. I think the same should be done for your experiments. E.g. what is the size of CORnet-S vs. the size of a compressed network?
3. I think similar weight compression techniques have been used before, see http://proceedings.mlr.press/v80/wu18h.html for example
4. In general, it’d be interesting to see how at-birth performance changes with the amount of “genetically encoded” information, i.e. the number of clusters passed through compression. E.g., would doubling the amount of clusters you use drastically improve performance? I’m not expecting this experiment to appear in the paper, but perhaps it’s worth discussing.

The majority of results is limited to one architecture, CORnet-S, which is not a very standard deep learning model. However, I think it is OK as repeating the experiments on something like ResNet50 would be very resource-consuming, and CORnet-S appears to be a better model of the visual stream anyway.

## Recommendation
As the paper addresses an important problem with a novel approach and extensive experiments, I recommend to accept the paper.

## Minor points
In Fig.1, were learning rates tuned individually for each length of training, or were short runs just snapshots of the longer ones? If it’s the latter, I’d imagine you can get better results by cranking up the learning rate and later decreasing it before you hit a plateau (e.g., by spacing out the same three LR decreases within a shorter run). I guess doing it for each point in Fig.1A is too much training, but repeating all points in Fig.1B-C should be possible?

> For instance, the fully trained model’s 55 million supervised updates translate to 5.5 updates every second, whereas the model with 1 million updates and 76% relative brain predictivity translates to one labeled image update every 10 seconds which appears more plausible given the upper limit of 2-3 saccades per second in humans

I would say that a more correct conclusion is that supervised learning is a bad concept for development, but I also think your results are applicable beyond supervised learning. Basically, you show that reducing the amount of updates in backprop retains good Brain-Score performance. But that probably tells more about SGD dynamics in deep networks rather than a specific loss function, and might easily translate to, say, more realistic self-supervised learning. (I guess you briefly mention this point in the discussion.)

> Abstract

> First, we find that only 2% of supervised updates (epochs and images) are needed to achieve ∼80% of the match to adult ventral stream.

80% of the best performing network -- currently it reads as 80% of absolute performance, which would presumably be 1.0 Brain-Score?

The caption of Fig.3C-D should better explain what exactly is trained in each setting

In Sec.6, you should specify that you used MobileNet V1 and ResNet-50.
summary_of_the_review: The paper provides novel results and important evaluations. There are some methodological issues, but they are not critical to the main point. I therefore favor acceptance.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: ## Updated the score

This paper proposes to address an important research question for connecting biological (BNN) and artificial neural networks (ANN). Although after training, ANN replicates various salient features of BNN, the way they are often trained is biologically implausible and thus, it is hard to argue that ANNs are suitable for modeling BNNs convincingly. In particular, this work focuses on the already existing CORnet who has shown a high Brain-score. The idea of the authors is to show that they can largely reduce the number of updates when using their methods while still retaining a high Brain-score, thus proposing a potential training mechanism for BNNs.
main_review: ### Strength:

First of all, the question addressed in the paper is essential for both the machine learning and neuroscience community and the author aim at addressing in a well-written piece of work that also clearly contextualizes and motivates the issue at hand. This work uses a state-of-the-art neural architecture, instead of a reduced version of models as is often the case in this type of work. The authors also propose a good analysis and clearly lay out their results, although their interpretation is one of the weaknesses of the paper. 

### Weakness:

Although it is true that this work has many strengths it is hard to overcome the following weaknesses, which question the motivation of the approach undertaken. First, there exists a set of more principled approaches that have been developed over the last 4-5 years for training ANNs in a biologically plausible way.  Second, the results obtained are very subjective to the authors as the result of the improvement of the score is minor. 

Although it is true that we don’t expect the authors to solve how the visual system develops and that this work corresponds to a proof of principle, it is still hard to navigate the plausibility of the assumptions made among all possibilities. On the other hand, there is a large set of work that has been proposed to build biologically plausible deep neural networks starting from clear objective function minimization and enforcing locality of learning rules. Compared to this set of works this one seems more adhoc than principled. This represents an important weakness when addressing the more neuroscience-friendly community. 

Indeed the approach proposed is more of a combination of existing approaches rather than a novel idea: the use of training of a subset of layers, different weights initialization based on the distribution of trained weights. It is not clear how this provides new insight to the statistical learning community. 


summary_of_the_review: This paper is well-written and addresses an important question possibly helping bridge neuroscience and machine learning. However, this work falls short on both the relevance of the experimental results and the ad-hoc nature of the proposed approach.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper addresses the question of how many weight updates are needed to train a deep network before it takes on biologically realistic representations. The paper uses CORnet-S (a network that has been proposed to resemble primate ventral stream), and BrainScore (a benchmark of how closely related deep network responses are to visual responses in primate ventral stream). Three ways of reducing the numbers of weight updates are explored, each of which is found to vastly reduce updates while moderately reducing BrainScore. First, the network is simply trained for fewer epochs. Second, weights are initialized with clusters of weights found after training. Third, only a subset of layers is updated. A combination of methods leads to 80% of full brain predictivity with 0.5% of the standard number of weight updates. 
main_review: Strengths: 
- The paper is clear and well written
- The analysis is thorough and supports the authors' arguments
- The method of initializing weights via clusters of trained kernels is interesting  
- The "critical training" method of training only selected layers is also interesting 

Weaknesses: 
- "Recently, such models have been criticized due to how their learning departs from brain development (Marcus, 2004; Grossberg, 2020; Zador, 2019)." This seems to be the motivation for the work, but those authors don't raise the issues that are addressed in this paper. The Marcus book predates deep learning. Grossberg focuses on catastrophic forgetting. Zador focuses on the sparsity of labelled data in life. I am not sure whether the number of weight updates needed to train deep networks presents a substantial independent concern about their biological plausibility, particularly given the possibility of constant unsupervised learning. If the issue has been raised in the literature, more specific references would help here. If not, I think the authors should argue more specifically that it is a problem, before going on to solve it. 
- The cluster-based initialization method seems to be presented as a way of bringing some learning results through a genetic bottleneck, but the biological plausibility isn't obvious. Further discussion of how this relates to and departs from biology would be helpful. 
- It's interesting that critical training worked so well, but in the context of the paper's goals, the biological relevance of this method should also be laid out more clearly. 

Minor: 
- I don't think it was shown how classification performance is affected by the cluster-based initialization method, but it would be nice to see that. 
- "37 bits per synapse (Zador, 2019)" This refers to the binary connectivity pattern (an issue of architecture) rather than the weights (which are addressed in this paper). 

summary_of_the_review: I think the approach and the results are new and interesting, but I have reservations about the motivation for the work and its connection to biology.
review_title: null

Meta-review: This paper experiments with what is required for a deep neural network to be similar to the visual activity in the ventral stream (as judged by the brainscore benchmark). The authors have several interesting contributions, such as showing that a small number of supervised updates are required to predict most of the variance in the brain activity, or that models with randomized synaptic weights can also predict a significant portion of the variance. These different points serve to better connect deep learning to important questions in neuroscience and the presence of the paper at ICLR would create good questions. The discussion between authors and reviewers resulted in a unanimous vote for acceptance, and the authors already made clarifications to the paper. I recommend acceptance.
Decision: Accept (Spotlight)