From: NeurIPS.cc/2019/Workshop/Neuro_AI/Program_Chairs
title: Paper Decision
decision: Accept (Poster)

From: NeurIPS.cc/2019/Workshop/Neuro_AI/Paper52/AnonReviewer1
title: Quantification of neural noise is poorly described, and alternative explanations not explored
importance: 1: Irrelevant
importance_comment: The authors suggest that noise corresponds to a regularization step that the brain does, like dropout. Single neuron noise could result in information from that neuron not propagating further down the network, like in dropout (but not equivalent for various reasons), but that would require more thoughtful comparisons with the neural activity, instead of just a comparison of distribution of activations.
rigor_comment: Controlling for state variables was a useful step to perform. Otherwise the comparisons are not well-quantified and other explanations of the noise/models are not explored.
clarity_comment: Figure 3 needs much more description, are the subspaces defined using trial-averaged responses? Why define a noise subspace instead of looking at the direction of the noise on each trial?
clarity: 3: Average readability
evaluation: 2: Poor
intersection_comment: Comparisons are too preliminary.
intersection: 2: Low
technical_rigor: 2: Marginally convincing
category: AI->Neuro

From: NeurIPS.cc/2019/Workshop/Neuro_AI/Paper52/AnonReviewer3
evaluation: 1: Very poor
intersection: 2: Low
importance_comment: Tries to argue based on which distributions fit best the noise measured in cortex that the networks are similar to ANNs with dropout. However, the link is tenuous and not strongly argued.
clarity: 2: Can get the general idea
technical_rigor: 1: Not convincing
intersection_comment: In principle showing that dropout was like neuronal noise could be interesting, but it's pretty tenuous here.
rigor_comment: It's a weak link to show that two distributions are similar, especially when only a very few distributions were fit and there are a lot of missing details about what is meant by a "dropout-like" distribution (early on it says "see Methods" but then isn't defined in the Methods).
comment: There are a number of claims that don't appear to be very well supported by what is actually shown.
importance: 1: Irrelevant
title: Tries to link observations of noise in cortex to dropout, but relationship is unclear
category: AI->Neuro
clarity_comment: Quite difficult to follow the chain of reasoning here.

From: NeurIPS.cc/2019/Workshop/Neuro_AI/Paper52/AnonReviewer2
evaluation: 2: Poor
intersection: 3: Medium
importance_comment: CNNs with dropout are proposed as a null model which seems to explain the response noise in neural data, but CNNs have no eye movements, while the neural data apparently does.  
clarity: 2: Can get the general idea
technical_rigor: 2: Marginally convincing
intersection_comment: While the analysis of the noise properties of neural responses in these datasets is important for the field of Neuroscience, descriptions of the datasets used and control for eye movements seems to be lacking.  This work appears to offer little new insight for the AI field in its current preliminary form.
rigor_comment: deltaF/F - explain.
For neuroAI symposium, better to define these terms.

Details on convnet too sparse.  Maxpooling layers? What was achieved performance on CIFAR-10 test set?

I would liked to have seen a log-norm example class in Fig 1.
The mixture of Gaussians depicted in Fig 1 (a), is one to consider <0.1 signal, or noise?  Perhaps its no response plus noise? Do you have any baseline "no stimulus" epochs to quantify the level of baseline noise for each of the units considered?

For Allen Institute data, and the head fixed mice, apparently eye movements were not paralyzed.
This is an important source of variability which is not accounted for in the CNN model.
i.e. for a given presentation of the stimulus, one has no idea if receptive fields are receiving remotely similar illumination.
Perhaps a way to proceed to control for this would be to inject the Allen Institute recorded eye movements into the CNN model input stream.


comment: AI->Neuro:
"We believe that research into the structure and role of biological noise will be useful for developing new methods to train neural networks with better generalization capabilities."
The inspiration appears to have propagated in the opposite direction, i.e. CNNs with dropout are proposed as a null model which seems to explain the observations in the data.

"Future work will study how different forms of subspace-aligned noise may help deep neural networks generalize better from fewer examples."
This is an interesting prospect of the work proposed in the abstract, but essentially left for future investigation.

With this in mind, while the analysis of the noise properties of neural reponses in these datasets is important for the field of Neuroscience, descriptions of the datasets used and control for eye movements seems to be lacking.  This work appears to offer little new insight for the AI field.

importance: 2: Marginally important
title: Dropout inspires a hypothesis on the functional role for noise in generalization, but analysis is superficial.
category: AI->Neuro
clarity_comment: See rigor.