Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: This paper attempts to model the development of the human visual system in infants, by training deep neural network architectures inspired by the human visual system on images from ImageNet, and learning a linear decoder on the outputs of each layer (following Zhang et al 2017) to measure how much information useful for distinguishing between classes is contained within each layer in the architecture. The paper measures the amount of class information in each layer over the progress of training.

I agree that deep networks could serve as good models for various parts of the brain, including the visual system especially given that convolutional networks have been inspired from studies of the visual system. However, the paper doesn't seem to provide any evidence for how the training process used for deep neural networks should correspond to the development of the visual system in infants. In particular, backpropagation is considered biologically implausible [1], whereas backpropagation serves as the main method for learning in the neural networks. Furthermore, neural networks have randomly initialized parameters, whereas it seems unlikely that human infants' brains would lack existing organization to such a drastic extent. In order for the results in this paper to hold greater weight, I would expect to see more evidence about how the neural network training process (also including aspects such as batch normalization, and the self-supervised clustering method in DeepCluster) are expected to correlate with learning in human brains.

For the above reasons, I vote to reject the paper. My conclusions above are based on my surface-level knowledge of neuroscience, so I welcome any clarifications or corrections from the authors about the above points.

[1] https://arxiv.org/abs/1502.04156
review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: This paper aims to examine whether DNNs are a good model of infant
behavior and development.  The paper is very well written and easy to
read.

The goal was to compare the development of object representations
across layers with the development in children and to compare the
order of learning of different object classes.

The work did show that unsupervised training results in a different
pattern of layer learning than supervised learning, but neither form
of learning was able to model the development in children.  Perhaps a
self-supervised multimodal learning system should be tried?


The decision to train the DeepCluster type net in a supervised way for
a control on training method vs architecture type is nice, but it would also
have been good to try other kinds of networks.


It is not clear that age of acquisition of the verbal word should be
related to age of acquisition of the visual concept.  The author's
state "A number of linguistic factors are known to affect when words
are first used, including the frequency of the word in language and
its number of phonemes, but the second strongest factor is the
"concreteness" of the word (Braginsky et al., 2015). This suggests
that the strength of the visual representation of a class has an
effect on when its label is acquired."   It is not clear to me how the concreteness
of a word relates to the strength of visual representation.

I don't think there is enough new insight gained from this paper for ICLR publication
at this stage.

Minor comments:

What are dashed lines in Figure 2 top left box?

Funding acknowledgement (especially with grant number) should not be
in an anonymous submission.

magenetic


review_title: Official Blind Review #3

final_score: rating: 3: Weak Reject
review_content: Summary

The paper aims to understand how object vision develops in infancy and childhood by using deep learning models. In particular, it chooses two deep nets, CORnet and DeepCluster to measure learning. CORnet is supervised and is designed to mimic the architecture and representational geometry of the visual system. It tries to quantify the development of explicit object representations at each level of this network through training by freezing the convolutional layers and training an additional linear decoding layer. The paper evaluates the decoding accuracy on the whole ImageNet validation set for individual visual classes. DeepCluster differ in both supervision and in the convolutional networks. To isolate the effect of supervision, it ran a control experiment in which the convolutional network from DeepCluster (an AlexNet variant) is trained in a supervised manner. The paper tries to draw conclusions on how learning should develop across brain regions in infants. In all the networks, it also tested for a relationship in the order in which infants and machines acquire visual classes, and found only evidence for a counter-intuitive relationship. 

Limitations

The topic is extremely interesting and worth intense study. However, the approach is not convincing. CORnet may have some relevances. It is not clear how well it models the representational geometry of the visual system. It is even less clear whether DeepCluster is relevant. Why would it be related to infant learning? 

The whole idea of using DNN to infer biological learning is built on shaky ground given how little we know the learning mechanism of the brain. In particular, back propagation is not widely considered possible in biology. Given the learning mechanism may be very different. What is the basis of using DNN to study the infant learning?

The findings are also not very surprising and offer much for the community.

Given the paper lacks rigor and findings, it does not meet the bar of ICLR.
review_title: Official Blind Review #2

Meta-review: This paper investigates the properties of deep neural networks as they learn, and how they may relate to human visual learning (e.g. how learning develops across regions of the infant brain). The paper received three reviews, all of which recommended Weak Reject. The reviewers generally felt the topic of the paper was very interesting, but overall felt that the insights that the paper revealed were relatively modest, and had concerns about the connections between DNN and human learning (e.g., the extent to which DNNs are biologically plausible -- including back propagation, batch normalization, random initialization, etc. -- and whether this matters for the conclusions of the present study). In response to comments, the authors undertook a significant revision to try to address these points of confusion. However, the reviewers were still skeptical and chose to keep their Weak Reject scores.

The AC agrees with reviewers that investigations of the similarity -- or not! -- between infant and deep neural networks is extremely interesting and, as the authors acknowledge, is a high risk but potentially very high reward research direction. However, in light of the reviews with unanimous Weak Reject decisions, the AC is not able to recommend acceptance at this time. I strongly encourage authors to continue this work and submit to another venue; this would seem to be a perfect match for CogSci conference, for example. We hope the reviews below help authors to improve their manuscript for this next submission.
Decision: Reject