Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The authors propose to add a biologically inspired lateral inhibition mechanism into deep convolutional networks for image recognition. When incorporated into AlexNets and ResNets, LI seems to improve performance on ImageNet classification without increasing trainable parameters. The authors examine the LI filter weights and find a biologically-resemblant center-surround pattern of inhibition.
strength_and_weaknesses: Strengths:
1) There are considerable gains on AlexNet and ResNet while using LI, however, I would like to note that these are preliminary results as also highlighted by the authors.

Weaknesses:
1) The proposed work lacks novelty, several methods in the past have tried to apply a very similar lateral or divisive inhibition mechanism to deep convolutional networks and have reported gains in image classification performance (particularly when added to AlexNet). See [1] [2] and [3] for example. The proposed work is almost exactly similar to [1].
2) The evaluation is very preliminary and lacks comparison to suitable baselines (other kinds of normalization such as BatchNorm, LayerNorm, etc.) or other normalization techniques such as [1, 2] which are very relevant.

References:
1. Hasani, H., Soleymani, M., & Aghajan, H. (2019). Surround modulation: A bio-inspired connectivity structure for convolutional neural networks. Advances in neural information processing systems, 32.
2. Miller, M., Chung, S., & Miller, K. D. (2021, September). Divisive Feature Normalization Improves Image Recognition Performance in AlexNet. In International Conference on Learning Representations.
3. Pan, X., Giraldo, L. G. S., Kartal, E., & Schwartz, O. (2021). Brain-inspired weighted normalization for CNN image classification. bioRxiv.

clarity,_quality,_novelty_and_reproducibility: The proposed work is not novel and evaluations are preliminary. The authors don't discuss sharing code or trained models, which is troublesome to reproduce the presented results.
summary_of_the_review: The proposed work is preliminary and lacks the novelty and quality of work expected at ICLR. I do not recommend accepting this paper at this stage. I suggest the authors to please consider a suitable workshop for this work and a significant extension of this work could be suitable for the ICLR audience with wider evaluation using suitable baselines and tasks.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper incorporates the lateral inhibition of neuron into the Deep network for image classification. Results show the effectiveness of the proposed mechanism. The lateral inhibition is not novel but the idea to explore the usability of the lateral inhibition into artificial neural network is interesting.  The key weakness of this paper is that there are no any explanations why their design improves the performance.  

strength_and_weaknesses: The lateral inhibition based idea to explore the usability of the lateral inhibition into artificial neural network is interesting and useful. Such an approach should be strongly supported. 

The key weakness of this paper is that there are no any explanations why their design improves the performance.  
clarity,_quality,_novelty_and_reproducibility: 1\ Why the low-pass filter with eliminating the central weight could model the LATERAL INHIBITION is not clear from this paper. The design is not a typical Difference of Gaussian function which is generally adopt in image processing [1].  Eq (5) seems an OFF-receptive field ?

2\ The inhibition computation by subtracting a computed value from the previous one is commonly used in many previous models [1][2][3]. The traditional models simulated visual mechanisms should be clearly mentioned and discussed [1][2][3]. 

3\ The description of related work is very limited as the reason shown in above. 

4\ The learned weight W plays the similar function of channel attention. Hence, what’s the difference between channel attention and the inhibition computation?  

5\ Eq (5) seems an OFF-receptive field ? Author should visualize the learned kernels as many examples. 

[1] IEEE Transactions on Image Processing. 2015;24(8):2565{2578.
[2] Brain-inspired weighted normalization for CNN image classification.
[3] IEEE transactions on pattern analysis and machine intelligence. 2015;37(10):1973{1985.

Minors:

Section 3.3 INHIBITION PLACE

we place it right after that convolution, as shown in Fig.

summary_of_the_review: It can be seen that the work is not complete with only five pages as the main text. Many important issues such as explaining why the design of lateral inhibition improves the performance. Computationally, what’s the difference between channel attention and the inhibition computation?  In summary, the technique contribution is simple.  

I encourage authors to enhance their work by addressing 1-2 key questions mentioned above, adding content and depth to the paper.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: 
Summary:
This article starts from the perspective of neuroscience, hoping to design a neural network that conforms to human cognition. The original intention of the article is good, but there is nothing special about the method of the article. Regarding the writing, methodology, and experiments, this article is far below the acceptance criteria for a good conference like ICLR. Reviewers put a lot of effort into reviewing each article, so hopefully, authors will write each article well before submitting it.


strength_and_weaknesses: 
(Positive) Although this article is very poor, its exploratory spirit, specifically the exploratory spirit of exploring neural networks that conform to human cognition, is worthy of recognition.


(Negative) The authors claim some gain in their approach. But as we all know, after introducing attention or dynamic mechanism, a general neural network can have some improvement. In particular, the worse the network, the more obvious the improvement.

(Negative) In fact, filtering is not a new thing in neural networks.


(Negative) A very working method should be validated on large-scale tasks. In addition to classification, there is detection, segmentation, and so on.


(Negative) The writing of this article is very poor. There are multiple copies of the text in the article. For example, there are multiple repetitions of text in the abstract and introduction. A good article should have a better way of expressing it.


(Negative) Gaussian filtering and bilateral filtering are not new in neural networks. The method proposed in the article is equivalent to a bilateral filtering non-local neural network.

(Negative) The article associates its method with depth-wise convolution. This is very imprecise and arrogant. If this is true, is MLP-Mixer also a special case of this article?

(Negative) As I said before, adding attention or dynamics to the neural network, such as senet, sknet, acnet, dynamic nets, will bring gains. There are too many such examples. So the results in Table 2 are not dazzling at all.

(Negative) The numbers in Equation 5 are so intuitive and natural.


clarity,_quality,_novelty_and_reproducibility: 
The quality, clarity, and the originality are poor.


summary_of_the_review: 
See "Summary Of The Paper." Regarding the writing, methodology, and experiments, this article is far below the acceptance criteria for a good conference like ICLR.
review_title: null

final_score: rating: 1: strong reject
review_content: summary_of_the_paper: This paper proposes a type of neural networks that uses lateral inhibition inspired layers. Lateral inhibition in this paper refers to the effect where the contrast of nearby neuron excitation in the lateral direction is increased for recognition. Inspired by this, the paper proposes to use a Gaussian low-pass filter with a zero center weight along with a learnable channel weight to mimic this effect. A flexible alternative would be a depthwise convolution. Then this output is used to subtract from the input tensor to form the final output. This lateral inhibition inspired design is tested with AlexNet and ResNet18 for image classification on ImageNet, and shows improvements at little additional cost.
strength_and_weaknesses: This paper is motivated by a neurobiological effect named lateral inhibition. It is always fascinating to see methods that are inspired in this way because it may reveal some important angles that the academic community has overlooked.

But this paper seems to be less prepared for this conference. The method is interesting but very simple: it adds a couple of convolutional layers to existing architectures. The baseline architectures are not from recent works, and the model scales in the test are also small (e.g. ResNet18, AlexNet). I also didn’t find the visualizations mentioned in the paper. I hope that the authors could add more experiments to the paper, show larger experiment settings, and polish the paper a bit more.
clarity,_quality,_novelty_and_reproducibility: The paper is clear to read. But its quality is below the conference standard. The method is simple and the novelty is limited. The reproducibility seems good.
summary_of_the_review: This is a paper inspired by the lateral inhibition effect. The motivation is great, but the method is too simple, and the paper does not show the strength of the method through large-scale and comprehensive experiments. Overall, I think the paper is below the conference standard.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper proposes to add a lateral inhibition mechanism into CNN and shows performance improvements when paired with AlexNets and ResNets. There is consensus among the reviewers on the novelty of the approach and with evaluation that is still preliminary and not as thorough as one would expect from an application work I am afraid I have to reject it.
I encourage the authors to continue with their study and to provide more compelling empirical to potentially strengthen their contribution. 
justification_for_why_not_higher_score: There is consensus in rejecting this work.
Decision: Reject