Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The paper presents the Recursive Gated Convolution (g^n Conv) that performs high-order spatial interactions with gated convolutions and recursive designs. The proposed module can serve as a plug-and-play module for both transformer and convolutional neural networks.
Based on the proposed module, HorNet is further introduced, showing good results on ImageNet classification, COCO object detection and ADE20K semantic segmentation.


strengths_and_weaknesses: 
Pros:
- Well organized and good writing.
- Extensive experiments on multiple datasets and tasks.
- The design of g^nConv is interesting and contributes much to the performance.
- Limitations are stated in this paper.


Cons:
- The motivation is not very clear to me. Why do we need high-order interactions?
- Since the operator is Mul rather than MatMul, then the high-order interaction seems like an element-wise one rather than the spatial-wise interaction as stated in the paper. Or it can be recognized as a channel-wise one (element-wise multiply -> channel projection -> element-wise multiply ...)?
- Could the author provide the complexity analysis for different n?
questions: See the Strengths And Weaknesses part
limitations: See the Strengths And Weaknesses part
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper proposes a new convolution-based deep neural architecture named $g^nConv$. Its essential operation is the gated convolution which multiplies the output of linear projection layers and depth-wise convolution layers. It also introduces high-order interactions with recursive gating. The authors conduct various experiments to show the effectiveness of the proposed results, including ImageNet classification, COCO object detection, and ADE20K semantic segmentation.
strengths_and_weaknesses: Strengths:
1. This is a well-written paper with carefully designed experiments.
2. This paper combines several deep learning techniques in an interesting way to get a novel architecture for image classification, detection, and segmentation.

Weaknesses:
1. The experimental result section is weak. The authors did not compare their proposed method with the state-of-the-art methods. The authors are suggested to add more recent works for comparison, e.g., Dynamic Group Transformer [Liu etal, IJCAI 2022], MViTv2 [Li etal, CVPR 2022], Pale Transformer [Wu etal, AAAI 2022]
2. The authors did not compare the throughput of each method. Since the proposed method involves a lot of small matrix multiplications that cannot be parallelized, and depth-wise convolutions, I guess this method should be slower than the other methods, even though their FLOPs are similar.


questions: 1. It would be better if the authors could theoretically analyze why High-order interactions can improve network performance.
2. The authors are suggested to compare with more recent works and evaluate the throughput of each method.
limitations: Yes
review_title: null

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposes a new operation (gnConv) and network (HorNet) to perform computer vision tasks. Motivated by the success of vision Transformers, the key idea is to build an architecture that has input-adaptive, long-range and high-order spatial interactions. The authors did so by proposing a recursive form of gated convolution, which has the capability of going even higher order explicit spatial interaction than 2. On well-established computer vision benchmarks such as image classification on ImageNet, semantic segmentation on ADE20K, object detection on COCO, the authors showed consistent improvement over Swin Transformer and ConvNeXt. 

POST-REBUTTAL UPDATE:

I have read the authors' rebuttal. While I am a bit disappointed by the answer to my Q2, I am satisfied with the rebuttal overall and the paper. In my opinion / experience, in this day and age, it is not an easy feat to achieve the consistent improvement as the authors have shown in this paper, and I have decided to maintain my original rating of 8.
strengths_and_weaknesses: Strengths:
- The proposed architecture is very simple and effective. Higher-order interactions have been desired but have never been prevalent, possibly due to gradient problems. But the authors seem to have got it to work quite well.
- Writing is very clear. 
- Experiments are thorough, and solid improvements are observed throughout.

Weaknesses:
- No major weaknesses. I just have a couple of questions.

Overall, I think the originality, quality, clarity, and significance are all high.
questions: L316 states that HorNet currently runs slower than ConvNeXt. It would be good to clarify how many times slower. 

HorNet seems to outperform Swin Transformer and ConvNeXt pretty consistently. But have the authors considered comparison against models with better performance on ImageNet, such as CoAtNet [8]? Related, what are ways that will allow HorNet to achieve state-of-the-art on ImageNet classification?

Finally, I think the higher-order interaction really originated from the gating mechanisms in LSTM. In vision, "SORT: Second-Order Response Transform for Visual Recognition" from ICCV 2017 is one of the earliest works. It would be great to add proper discussion on these.
limitations: I am reasonably happy with the Limitations section of the paper from L315 to L318. As I wrote earlier, it would be good to be more transparent and specific.
review_title: null

Meta-review: metareview: 
This paper introduces a new operation **gnConv** and a computer vision network architecture **HorNet**.  Motivated by the success philosophy of vision Transformers, the key idea of gnConv is to build a recursive form of gated convolution. It make the module input-adaptive and with long-range and high-order spatial interactions. Consistent improvement are shown over Swin and ConvNeXt on well-established CV benchmarks such as image classification on ImageNet, semantic segmentation on ADE20K and object detection on COCO.

The paper receives receives unanimous accept from all reviewers (Reviewer 3Snz champions the paper with rating score 8), leading to an ``Accept'' decision. 
confidence: Certain
Decision: Accept