Graph Vision Networks for Link Prediction

Authors: Yanbin Wei, James Kwok, Yu Zhang
Primary area: learning on graphs and other geometries & topologies
Keywords: Graph Neural Networks；Link Prediction；Multimodality

Abstract:
The potential of the vision modality for enhancing graph structural awareness has long been overlooked in the mainstream graph neural network (GNN) community. In this paper, we propose a simple yet effective framework called Graph Vision Networks (GVN), which first incorporates vision awareness into Message Passing Neural Network (MPNN) and achieves effective performance for link prediction, highlighting this unexplored but promising direction. Specifically, GVNs transform graph structures into images and extract Visual Structural Features (VSFs) from those images, where VSFs are considered a novel type of structural feature. Similar to previous structural features, VSFs also mitigate the limitations of traditional MPNNs in expressive power and substructure awareness. Additionally, unlike most previous heuristic-based structural features (e.g., common-neighbor-based and path-based ones), which typically depend on fixed structural priors, VSFs are adaptive and capable of capturing varying structural insights to better suit different scenarios. Extensive experiments across seven commonly used benchmark datasets demonstrate that GVNs and their variants can significantly enhance MPNNs in link prediction tasks. Additionally, the straightforward design of the framework makes it highly compatible with current methods, providing additional performance gains to achieve new state-of-the-art performance.

Decision: reject
Reviewer scores: 5, 5, 5, 6

Reviews and discussion:
<thread>
creation date:
1730112343541

thread_type:
official_review

<thread_object>
creator: reviewer_e7xg

<content>
summary:
The author noticed that the MPNN model based on message passing can only obtain coarse-grained structural feature information, which makes the model unable to adaptively model fine-grained structural features, which limits the expressive power of the model. Therefore, inspired by Das et al., the author proposed to use the visual modality of the graph to provide fine-grained structural features for the model, and proposed the GVN model. The model first generates edge-oriented k-order subgraphs and node-oriented k-order subgraphs based on the query node pairs, and visualizes these subgraphs as 2D images. Then the cross-attention model is used to fuse the image features with the node features of the graph, and finally the readout module is used to implement the downstream prediction task.

strengths:
1. Well-written, clearly organized, and easy to read and understand
2. The motivation of the article is clear. It is timely and necessary to use image modality for link prediction task on graphs.
3. Evaluating the model's effectiveness on multiple datasets with different scale and types.

weaknesses:
1. The innovation of the method proposed in the article is relatively neutral. The main innovation of this article is to classify and discuss link images and node images. The remaining designs are just stacking existing modules (GNN, cross-attention, ResNet). First, inspired by the prior work, the authors use images to express the local structure of the graph. Secondly, the article uses the existing pre-trained visual encoding model to extract image features. Finally, the fusion of visual features and graph structure features utilizes the existing cross-attention. 

I think the authors can further explore this idea in the following aspects. First, how to construct image representations of graph structures, such as the color and shape of nodes, etc. can be further explored. Secondly, how to construct effective image encodings of graphs instead of using existing pre-trained ResNet models can be explored. For example, can the features extracted by the visual encoder be used to reconstruct the local structure of the graph (regression graph adjacency matrix)?

2. The article claims to construct an adaptive fine-grained structure feature. However, the article only uses the image modality of the graph structure (which may be considered a fine-grained graph structure representation, but I have doubts about this), and cannot express adaptability.

3. There are some typos in the article, such as the period in line 025, the singular form of produce, and the misspelling of Empirical in line 032.

questions:
1. What does $lO(\cdot)$ mean in Section 4.3?
2. There is already a projector in the formula on line 215, and there is another projector in the formula on line 220. The expressiveness of two linear projectors put together is the same as that of one projector because the product of two matrices is equivalent to one matrix.
3. Why are there no mapping matrices $W_Q, W_K, W_V$ in cross-attention?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730462316177

thread_type:
official_review

<thread_object>
creator: reviewer_l8vi

<content>
summary:
This paper proposes a Graph Vision Networks (GVN) framework aimed at improving the expressive power of Message Passing Neural Networks (MPNNs) in link prediction tasks by integrating structural visual modalities. Extensive experiments on multiple datasets show that the framework exhibits potential in improving prediction accuracy and model robustness.

strengths:
Originality: The idea of embedding local structural information of graphs using visual models is innovative. 

Quality: The author has demonstrated good performance across multiple datasets, particularly in challenging large-scale graph datasets.

weaknesses:
1. While the idea of using visual modalities to model local neighborhoods is novel, the techniques utilized in this manuscript are very simple. For instance, the use of simple attention mechanisms to integrate node and neighborhood features. 
﻿
2. The author employs the graph visualizer (GV) to generate image modalities of subgraph structures. So what distinguishes the image modalities generated by GV for two isomorphic subgraphs? Are they random or identical? It appears that the improvement in representation learning performance is highly dependent on the effectiveness of GV.
﻿
3. The title and abstract of this manuscript are overly concise. It's hard to understand the work's significance and contributions.
﻿
4. The manuscript lacks a framework diagram to illustrate the proposed methodology, which could enhance clarity and comprehension.
﻿
5. The presentation of equations in the manuscript is uneven. For example, formulas in the PRELIMINARIES section are numbered, while those in the GRAPH VISION NETWORKS section are not, leading to potential confusion.
﻿
6. The writing quality of the manuscript requires improvement, with multiple errors. For instance, in the third paragraph of the INTRODUCTION, the quotation marks around “identity” are incorrectly used.

The rebuttal addresses my concerns, so I increase the score.

questions:
Please see the Weaknesses section.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730691035696

thread_type:
official_review

<thread_object>
creator: reviewer_zsnm

<content>
summary:
This paper focuses on the link prediction task. The authors introduce Graph Vision Network (GVN). By integrating vision awareness into MPNNs for link prediction, GVN models can adaptively extract learnable visual structural features (VSFs). Moreover, the authors propose two models under GVN framework: GVN-Link and GVN-Node. Experimental results demonstrate that the proposed method establishes new SOTA for link prediction.

strengths:
1 The writing and the organization of this paper are generally OK.

2 Integrating vision modality into MPNNs for link prediction sounds interesting.

weaknesses:
1 The motivation for incorporating vision modality into MPNNs for link prediction should be better clarified and discussed. Why is this design effective? Any theoretical evidence? Maybe a dedicated section for this discussion could be valuable.

2 The counterpart methods used for experimental comparison seem not SOTA enough. The authors should compare some 2024 SOTAs.

3 Minor Issues:  Ln 32 on Page 1, ‘Empiically’ should be ‘Empirically’

questions:
See above weaknesses.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730722973649

thread_type:
official_review

<thread_object>
creator: reviewer_g7xf

<content>
summary:
This paper proposes the Graph Vision Networks (GVN) for link prediction, which is designed to deal with the potential limitation of HSFs that it can only be derived based on the pre-defined structural prior and thus lacks sufficient adaptability to diverse real-world scenarios.

The core idea of GVN is very simple, it interprets either a link-centered subgraph or a node-centered subgraph as visual images and then extracts visual features from them as the structure features.

strengths:
1. The idea is simple and the experiments show the effectiveness.

weaknesses:
1. The technical novelty is limited. 
2. The method is not convincing. It first transforms the graph data to visual modality and extracts the visual features as the structural features, which is expected to be more effective than the HSFs learned based on structural priors in graph modality. Intuitively,  the visual features are more capable of learning appearance features or semantic features. In contrast, the graph data should have more expressive power than the visual modality in terms of the structural features.

questions:
It is suggested to given more theoretical analysis and qualitative comparison to show why the method can learn more effective structure features from the visual modality than the graph modality.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732507244216

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Thank You for Your Review and Invitation for Further Discussion

comment:
Dear Reviewers,

We hope this message finds you well. We would like to express our sincere gratitude for the thoughtful and detailed feedback you have provided on our manuscript. Your insights have been invaluable in guiding our revisions.

In response to your concerns, we have provided detailed explanations and conducted additional experiments to address each point raised. These responses and the corresponding results have been included in the revised manuscript. We have attached the updated version for your review.

We understand that your time is valuable, and we greatly appreciate your efforts in reviewing our rebuttal. As most reviewers have not yet had the opportunity to respond, we kindly ask for your feedback on whether the revisions meet your expectations. Your insights are crucial for us to ensure that all concerns have been adequately addressed.

Thank you once again for your dedication and support. We look forward to your valuable feedback and hope that our efforts have effectively resolved the issues you highlighted.

Best regards,

The Authors
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper proposes a message passing neural network that incorporates vision awareness by transforming the graph into images and extracting visual features from them. The objective is to provide fine-grained structural features to the model.

Strengths:
- The idea is simple.
- Integrating vision modality for link prediction is interesting.
- The experiments show effectiveness.

Weaknesses:
- Limited technical novelty. While the idea of using visual features is interesting, the techniques used are common.
- The motivation behind the use of visual features for graphs is unclear and unconvincing.
- The baselines should be reviewed and updated to include more recent methods.

There are major concerns regarding the limited novelty of the proposal and its insufficient motivation. The original concerns highlight the lack of baselines and problems with the experiments. Given these issues, I recommend rejecting the paper.