Venue: NeurIPS 2021 Workshop SVRHM

Reviews:

final_score: rating: 7: Good paper, accept
review_content: The authors tested a simple modification of popular CNN architectures (ResNet and DenseNet) that led to moderate improvements on image classification (CIFAR-10) and one measure of robustness (robustness to JPEG compression artefacts). The authors consider the key of their modification to be a minimum operation between pairs of feature channels within one layer (see below for alternative explanations) and claim that this minimum operation relates to end-stopping in biological neurons. 

I find the mild improvement on two measures (classification and robustness to JPEG compression) somewhat interesting, given the simplicity of the architectural change. It is laudable that the authors controlled for number of model parameters while comparing performance. The paper states that Min-nets may be a computationally efficient alternative to Log-nets and FP-nets, although the paper does not make direct comparisons with these alternatives. These contributions notwithstanding, I do not think the results are necessarily due to the minimum operation, as the authors would like to claim. I also find the liberal analogies to biology (and other work in computer vision) insufficiently explained and possibly inaccurate.

My foremost concern is that improved performance of Min-nets may be due to increased depth, both directly and indirectly. Based on the text, each Min-block comprises 3 convolutional layers (Secs. 2 and 2.1), whereas the corresponding block it replaces in the control ResNet or DenseNet comprises only 2 convolutional layers. Moreover, the minimum operation is itself a nonlinear operation, and thus increases the effective nonlinearity, or depth, of the network. It is not clear whether the increased performance can be attributed to the direct increase in depth---thus bearing no relation to the minimum operation---or to the indirect increase in depth, which does arise from the minimum operation, but not necessarily in a biologically relevant way.

This connects to the second concern, that the biological relevance (and other analogies) claimed by the paper may be inaccurate and misleading. The paper repeatedly suggests that end-stopping is analogous to the AND (and thus MIN) operation, and even to curvature/corner tuning (lines 41--44). End-stopping describes the property that responses of, say, an orientation-selective cell to a straight bar, increases only up to a certain bar length, then decreases if the bar is further lengthened. (Thus, the maximum response is elicited when the bar is 'end-stopped.') This property does not necessarily imply an AND operation, or curvature/corner tuning: A bar fading out at the edge or terminated with any border (other than a straight edge with two right-angle corners) should also strongly activate an end-stopped cell, so no particular feature need to be AND-combined with the bar's orientation. Although end-stopping *can* be modeled by an AND operation, and although the MIN operation represents *some* AND operation, it is far from certain that a MIN operation with learned weights should represent end-stopping at all. End-stopping can be measured, but the paper has not done so.

The paper also makes an appeal to hyperselectivity as defined in Paiton et al., 2020 (which follows Vilankar & Field, 2017, the latter itself a meeting abstract that contains few details). First of all, hyperselectivity in Paiton et al. is defined in pixel space, by which standard any unit in a CNN past the first layer can potentially be hyperselective. The minimum operation guarantees hyperselectivity, but only along one dimension---all dimensions orthogonal to the plane defined by **v**, **g** (the pair of filters being MIN-combined) are not affected. This is far from the case in Paiton et al., where hyperselectivity is induced by local competition among all neurons in a layer, and thus affects many more dimensions. As above, hyperselectivity can be measured, but the paper has not done so. Along the same line, the paper contains several more gratuitous analogies to Gaussian curvature, product of eigenvalues, etc. (lines 12--21), all described too vaguely to be justifiable or useful.

Despite these criticisms, given the relative simplicity of adding a min operation, I think the results, and the reason for these results, may merit discussion in the workshop.
review_title: Promising results with questionable interpretation and biological relevance

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: ### Summary : 
The authors propose to include a new non-linear operation in the state of the art deep learning architecture. This operation is implemented with the minimum operation between 2 filters (the effect is similar to an AND operation). This operation leads to ‘end-stopped’ neurons (neurons encoding for a corner …) that have been found in the early visual cortex. In terms of performance, this min operation allows a better robustness to the jpeg compression artifact and improve the testing classification accuracy significantly.


### General comments : 
This article is in general well written and pleasant to read. The authors' claims in terms of improved classification accuracy and better robustness are clear, unambiguous and  backed by relevant experiments. I would have appreciated the section on the min-block (i.e. section 2.1) to be clearer (and ideally illustrated with a comparative scheme with the ResNet block). Given the short format of this workshop paper, and the promising results I would tend to accept the article. Please find more comments below (mostly suggestion to improve the current version of the article). My rating : 8

### More specific comments
* It is a bit disappointing that the section that is crucial for the understanding of the paper is the less clear one. I would suggest to include a scheme that is comparing, side by side, a ResNet block with a Min-Block. I would also suggest to rewrite the section 2.1. such that it is highlighting the main difference with ResNet blocks.

* The authors have successfully demonstrated that the MinBlock is bringing robustness to jpeg compression artifact. But is it also showing an increased robustness to different types of perturbation ? It would be interesting to run a systematic analysis of the network robustness with the classical adversarial attacks.

### Suggestion of improvement for future work
* The authors say that the AND operation leads to filters similar to those encountered in the early visual cortex (e.g. end-stopped cells). It would be interesting to show it qualitatively. For example, one can train a Min-network and a standard one on natural images dataset and then visualize the learned receptive fields (i.e. the projection in the input space of the filters located at different depth). Then, it would interesting to run a qualitative analysis of the differences between the 2 set of filters.
 
* Another Interesting way of illustrating the end-stopped neurons, would be to present a stimulus composed of numerous corner, line-ends... (at test time, the training would be standard) and to see if more neurons are responding to this kind of stimuli in the Min-network compared to a classical one.

* The shown results (both improved classification accuracy and better robustness) should be generalized on more network architectures and more importantly on different datasets (e.g. higher resolution image dataset).

* Another interesting analysis would be to analysis the sample image misclassified by the classical network and well-classified by the Min-Network. Is there any more end-stops patterns in those samples ? 
review_title: Good article, more qualitative analysis would have been interesting

final_score: rating: 7: Good paper, accept
review_content: I enjoyed reading this paper and think that exploring the addition of new, neurally inspired DNN architectural choices is an interesting and worthy goal. This submission also certainly fits into the scope of the workshop, as insight from visual neuroscience was a clear motivator. For these reasons, I recommend the submission to be accepted as is. However, I think addressing point 3 below will make it a stronger, more impactful paper. 

Strengths: 
1. The rationale for adding end-stop neuron like units into DNNs used in computer vision was motivated;
2. The implementation of this new method was detailed (both in the main text and the appendices);
3. The results were interesting and encouraging. Additionally, it was clear that Min-blocks could be extended to a variety of architectures, beyond what was tested;
4. There is clear potential for future directions.

Weaknesses: 
1. The fact that Min-Nets exhibited similar scaling in POCP as ResNets and DenseNets (despite slightly better robustness) slightly dampens the results;
2. The fact that Log-nets and FP-nets have had similar benefits using similar mechanisms [lines 180-181], also slightly dampens the novelty of this work; 
3. There are two points that are brought up in the submission that are particularly interesting, but not fully addressed. I understand that the page limit constrains discussion, but maybe it could be included in the appendices? These two points are: 1) why do Min-Nets exhibit similar scaling in POCP as ResNets and DenseNets? and 2) why does substituting the first block with a Min-block make more of a difference than substituting a later block? I don’t think a full answer is necessary, but possible hypotheses (especially those that draw influence from visual neuroscience) would be interesting and insightful. 

One final minor point. I am not familiar with FP-nets, and I imagine others might not be as well. It would be helpful if you would define what the acronym is the first time you use it [line 22].
review_title: Neurally inspired modification to existing CNNs leads to improved results

Decision: Accept (Poster)