Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: In this paper, the authors introduce a novel layer, called Gated Relational Message Passing (GRMP), for efficient multi-relational modelling over graphs. Importantly, GRMP scales better than existing multi-relational models when increasing the number of considered relations.

The authors use GRMP as building block for Efficient multi-range relational graph neural networks (EurNet) that they apply to multi-scale modelling problems. Across image classification, object detection, image segmentation and protein function prediction tasks, the authors show that EurNet performs comparably or better than state-of-the-art methods.
strength_and_weaknesses: Strengths:
* Contributions are clear and important
* Claims are well supported; experiments are extensive (compared with multiple methods, using different model capacities, and considered different datasets / tasks)
* The paper is well written. Figures are clarifying.

Weakness:
* This is a minor point: Maybe the authors could more explicitly showcase the empirical advantages of GRMP speedup in an applied setting when comparing with graph-based multi-relational models. (this is somehow already discussed in the “Throughput analysis.” paragraph, but the authors could make it more evident)
clarity,_quality,_novelty_and_reproducibility: * Clarity: clearly written
* Novelty: Novel. New layer for multi-relational modelling over graphs at scale.
* Reproducibility: Good. Hyperparameter choices are well justified. Criteria on the choice of baseline models are discussed. Authors provided detailed architectures of the proposed models. Source code in supplementary material.

summary_of_the_review: This is an important contribution to multi-relational modelling and graph neural networks. The paper is clearly written, experiments are extensive and detailed supplementary info is provided.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper introduces a graph convolution module that learns representations of a node by aggregating the representations from its neighbour nodes in a graph. Compared with the standard one (RGConv from Schlichtkrull et al., 2018), the proposed module introduces factorised version of the kernel matrix and other adaptations to reduce the computational complexity. The paper then applied the proposed module to several tasks of computer vision as well as protein structure modeling.
strength_and_weaknesses: Strength:

1. The paper provides comprehensive experiments on various tasks of computer vision and protein structure modeling. In each of the tasks, the paper compares with strong baselines on large-scale datasets. Comprehensive ablation study is also provided to show the efficiency of the proposed module. The empirical study of the paper is strong and supports the claims of the paper well.

2. The proposed module shows promising results when applied with different backbone architectures in various tasks, which demonstrates the usability of the method.

3. The paper is well written and structured. Experimental details are elaborated.

Weaknesses:

1. Despite the comprehensive empirical study, I feel that the novelty of the proposed method might be a bit weak. The proposed method is an adaptation from a well studied method (RGConv) from the angle of efficiency. The technical depth might not be very strong. But I would not complain much as similarity might be a factor that the method can be applied in various tasks.

2.  The construction of the graphs may require additional tuning such as the number of KNN. It is also a bit heuristic as it is constructed by some predefined rules (although it can be dynamic with the learning of the model). Moreover, although the paper claims "spatial", I feel that it does not apply specific configurations to deal with "spatial" and "spatial" is just one of the predefined relations.
clarity,_quality,_novelty_and_reproducibility: I'm happy with the clarity, quality and reproducibility of the paper. I have minor concerns on novelty.
summary_of_the_review: Very comprehensive experiments and novelty is a bit weak.

I have read the response of the authors. Additional experiments are appreciated.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper introduces the EurNet for Efficient multi-range relational modeling. It constructs the multi-relational graph to encode the short-medium and long-range relation embedding. It also introduces a gated relational message passing layer to achieve that. Extensive experiments on ImageNet classification, COCO object detection and ADE20K semantic segmentation show the effectiveness of proposed approaches. 
strength_and_weaknesses: Strength 
1, The motivation is clear, and the writing is easy to follow. 

2, The experiments results are extensive. The proposed approach outperforms previous work FocalNet in many settings. 

3, The experiment results for protein function prediction are interesting. 

Weakness.
1, The idea or motivation of using different range graph structure is not new before the vision transformer.  The authors do not cite or compare the closely related works: 1, Graph-Based Global Reasoning Networks CVPR-2019. 2, Dynamic Graph Message Passing Network CVPR-2020. 3, Dual Graph Convolutional Network for Semantic Segmentation, BMVC-2019.

2, The core contribution in Equ.3 is not new or novel. It contains the different window sizes of MHSA-like operator and along with a channel attention. 
	
3, The core contribution in Equ.3 should be simplified for easier understanding.  For example, why not give a figure to better illustrate the core operation?

4, What the advantages of combining different range for classification tasks?
What are the advantages over CNN+ transformer-like models?

5, The performance on ImageNet and COCO are not very competitive. In particular, compared with Convnext, the GFlops and Throughput increases while the performance improvements are within 0.2 over different baselines. 

Moreover, the proposed framework does not show any performance gains over con-current methods.
CMT: Convolutional Neural Networks Meet Vision Transformers. CVPR-2022
CMT: Convolutional Neural Networks Meet Vision Transformers. Arxiv-2022.02





clarity,_quality,_novelty_and_reproducibility: Clarity, Quality are good, while the novelty is not new.
summary_of_the_review: I appreciate the author present detailed rebuttal. However, the related works I present share the similar ideas with the author work. The authors also admit they ignore these related works. Please do cite these paper for new draft.

After discussion with AC and other reviewers, I keep rating unchanged.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: 1, Introduce a task called multi-range relational modeling.
2, Their EurNet constructs the multi-relational graph, where each type of edge corresponds to short-, medium- or long-range spatial interactions.
3, The proposed GRMP separately performs (1) relational message aggregation on each individual feature channel and (2) node-wise aggregation of different feature channels

strength_and_weaknesses: Strength:
1, Propose a novel modeling layer, called gated relational message passing (GRMP), to propagate multi-relational information across the data.
2, Demonstrate EurNets in image and protein structure modeling cases.
Weaknesses:
1, Why your model is more "efficient" than other models?
2, Why do we need multi-range relational modeling in Image classification?
3, Do you have an inference time comparison between previous works?
4, The method part is somehow hard to follow. Do you have any figures to demonstrate your method?
5, Why relation-channel entangled aggregation is better?
clarity,_quality,_novelty_and_reproducibility: Clarity, Quality, Novelty, and Reproducibility are OK.
summary_of_the_review: The paper introduces EurNet to do multi-range relational modeling. The proposed GRMP separately models multi-relational graph.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: Based on the collected information from all reviewers and my personal judgement, I can make the recommendation on this paper, **rejection**. Here are the comments that I summarized, which include my opinion and evidence.

**Research Problem**

This paper studies the spatial relationship in various learning tasks including image classification and protein structure. 

**Related Work**

Several reviewers pointed out that many key missing references. 

**Motivation**

The authors argue that there exist different kinds of spatial relations, including short-range, medium-range, and long-range relations. Only involving a single type of spatial relations is not sufficient for the learning task. However, as the introduction says, there exists a branch of literature in multi-relational modeling. This motivation is not strong. Moreover, I am also confused about “separately modeling each spatial relation is a better solution to capture different tasks’ focus.” Why? More explanations and support are needed. 

**Philosophy**

The authors aim to propose a new method, named EurNet, for multi-range relational modeling. From the above motivation, I am not informed or confused about what challenges the authors aim to tackle. Therefore, it is unclear the philosophy to handle these challenges. As the authors mentioned the great importance of short-range relations in instance segmentation, and the upgraded importance of long-range relations in semantic segmentation, relational learning is task-dependent. Separately modeling each spatial relation will not solve the problem. The authors fail to answer what type of spatial relation is suitable for the learning task, or how to fuse different types of spatial relations for the learning task. 

**Presentation**

The presentation is difficult to follow and should be improved heavily. I have read this manuscript at least five times. Many parts are prolix and lack information. The authors might have a misunderstanding on problem definition in Section 3.1. The problem definition is the research problem that the authors aim to address, rather than the proposed model to tackle the research problem.  A formal problem definition is strongly suggested with clear notations

**Technique**

The authors propose multi-range relational edge construction and dynamic edge construction with two instantiations in image and protein structure modeling. However, the parameter settings in Section 4 are ad-hoc. 

**Experiments**

(1) The experimental results are extensive. 

(2) I guess that the performance gain comes from multiple types of inputs, which can be verified by the ablation study in Table 5. 

(3) The authors should also feed the multiple types of inputs for different message-passing strategies. By this means, the authors can demonstrate the effectiveness of their proposed GRMP in a fair fashion. 

No objection was raised from the reviewers on the rejection recommendation.

justification_for_why_not_higher_score: My major concerns lie in presentation, motivation, and philosophy, which makes this paper not self-standing and lacks novelty. Two reviewers also have concerns on the novelty, but they would like to insist their scores.
justification_for_why_not_lower_score: Although this paper receives four positive scores, several reviewers have concerns on the novelty. Moreover, it is weird to see that one reviewer increased the score due to its limited novelty. I asked the reason but did not receive any reply from the reviewer.

Based on my above comments, this paper does not reach the bar of ICLR.
Decision: Reject