Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: I first want to thank the authors for their proposed approach in this paper. Authors made an attempt to bridge the gap between natural (primate) vision and NN models. The paper is easy to read and understand. The authors proposed a Conv-LSTM-inspired model called V1Net. The model shows some merits in detecting the correct labels for noised inputs. 

Unfortunately, the paper lacks some critical analysis, and V1Net usability is limited in real-life. Commonly, the community expects a certain number of experiments to back a claim. Specifically I have the following questions:

1. “V1Net can be flexibly incorporated as a module in existing implementations of DCNs”, can you elaborate how? Current architecture rarely consider using conv-lstm to solve tasks such as object detection. Not saying the current trend is correct or incorrect in doing so, but lack of experiments and details leave this claim unsupported. 
2. The leap between horizontal connectivity and V1Net is rather unmotivated. Specifically, authors should explain why V1Net is the only (or the most suitable) way of implementing horizontal connectivity. 
3. Why is the FFT texturization the correct way to evaluate the robustness? Specifically, could it be that V1Net being a simpler model (such as the case in Fig 4, where the performance is lower than conv-lstm for clean data) simply generalizes better to noisy input. Simpler models sometimes have the tendency to remove noise better. 
4. Necessary comparisons are not made with at least a few state of the art models in CIFAR. As it stands the impact is limited on community who uses conv-lstm for CIFAR classification.
5. Figure 4 lacks the required standards of a scientific article figure. Borders on only 3 sides, and DPI seems to be low as text is pixelized. 
review_title: Official Blind Review #1

final_score: rating: 1: Reject
review_content: The paper proposes, inspired by the Primates brain, to add horizontal inhibitory and excitatory connections. In practice, the work proposes a variant of convolutional LSTM cells, that incorporates additional convolutions. 

Overall, the paper is hard to read, the experimental setting does not fully convince.  It seems hard to reproduce the results using the information given in the paper. A lot of the method section focuses on intuition, using vague vocabulary which makes it hard to understand concretely what is done in practice. In particular, the contributions are not clear enough. The difference between the proposed approach and existing works needs to be made clearer. 
The idea is interesting, however, and the paper would benefit greatly from addressing those issues. 



Main points


The claims seem somewhat bigger than the actual contributions. In practice, the contribution is a modification to an LSTM cell.

The paper is not easy to read, and mixes various terms without introducing them (e.g presynaptic activity is used to introduce the method but never introduced, not even in related work). It would be good to use standard notation and math font (e.g. small bold letter for vectors, etc) and to define notations. In Figure 1, the kronecker symbold represents convolutions but in (1), the hadamard product (*) represents convolutions. These inconsistensies make the paper harder to follow.

The problem of clarity extends to the experimental section, where the experiments are not clearly explained, 

The method section could be more detailed, in particular, a mathematical comparison of LSTM vs the proposed approachh would be useful. Figure 1 is unclear: what do the red, dark blue and light blue lines correspond to mathematically?

The related attempts in ML and Deep learning should be reviewed ( e.g. Lateral Inhibition-Inspired Convolutional Neural Networkfor Visual Attention and Saliency Detection, AAAI 2018).

The experimental setting is not convincing. It seems that a simple state-of-the-art CNN architecture would do better than the proposed approaches. Comparing a ConvLSTM with a single convolution does not seem fair.

Very little implementation details are given. The authors mention a ConvLSTM2D layer is used. However, the inputs are static images: how exactly was the experiment done? Where does the time come from since the dataset considered is static?

About emergent neurally plausible horizontal connections: that section is interesting but would benefit from being more detailed and rigorous. There is no actual measure or study of the emergence of said connections.

3.2 is misleading as it corresponds to a setting that is never used in the experimental setting. 

Most of the results are qualitative, except for Table 1. It would be useful to have quantitative comparisons on established benchmarks.
review_title: Official Blind Review #3

final_score: rating: 1: Reject
review_content: The authors propose to modify a convolutional variant of LSTM (ConvLSTM) to include horizontal connections inspired by known interactions in visual cortex: excitation, subtractive inhibition (linear) and shunting multiplicative gating (nonlinear). They evaluate their V1Net model on a new task they call the texturized challenge (spectrally perturbed CIFAR-10 images) and on contour segmentation on BSDS500 and show that their approach outperforms some baselines.

Strengths:
+ The biological motivation is quite clear
+ Architecture is simpler than that of previous related work (hGRU)

Weaknesses:
- Not clear what the objectives/contributions are
- No advancement of state of the art in computer vision
- No novel insights about brain function
- Motivation of the "texturized challenge" is unclear
- Performance on BSDS500 is far from state of the art
- Value of the qualitative analysis on stylized ImageNet is unclear

Overall, I'm not sure what the goal of this paper is. It neither presents an advance of the state of the art in any computer vision problem nor does it lead to any novel insights about the brain. The lack of a clear statement about the contributions of the paper seems to confirm this impression – the authors don't seem to know either.
review_title: Official Blind Review #2

Meta-review: The paper proposes a neurally inspired model that is a variant of conv-LSTM called V1net. The reviewers had trouble gleaning the main contributions of the work. Given that it is hard to obtain state of art results in neurally inspired architectures, the bar is much higher to demonstrate that there is value in pursuing these architectures. There are not enough convincing results in the paper to show this. I recommend rejection.
Decision: Reject