Jointly Training Task-Specific Encoders and Downstream Models on Heterogeneous Multiplex Graphs

Authors: Logan Labossiere, Khalil Al Handawi
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords: Multiplex, Heterogeneous, GNN, GraphSAGE

Abstract:
Learning representations on Heterogeneous Multiplex Graphs (HMGs) is an active field of study, driven by the need for generating expressive, low-dimensional embeddings to support downstream machine learning tasks. A key component of this process is the design of the graph processing pipeline, which directly impacts the quality of learned representations. Information fusion techniques, which aggregate information across layers of a multiplex graph, have been shown to improve the performance of Graph Neural Network (GNN)-based architectures on various tasks including node classification, edge prediction, and graph-level classification. 
Recent research has explored fusion strategies at different stages of the processing pipeline, leading to graph-, GNN-, embedding-, and prediction-level approaches. In this work, we propose a model extending the $\texttt{GraphSAGE}$ architecture, which simultaneously refines layer-wise embeddings produced by the encoder while training downstream models. We evaluate the model's effectiveness on an HMG on real-world and benchmark datasets, comparing it to models utilizing either graph-level or prediction-level fusion without jointly optimizing their vector embeddings. We demonstrate that our approach enhances the model's performance on downstream tasks, particularly node classification.

Decision: reject
Reviewer scores: 3, 3, 5

Reviews and discussion:
<thread>
creation date:
1730306233547

thread_type:
official_review

<thread_object>
creator: reviewer_ycoc

<content>
summary:
This paper proposes an** improved** GraphSAGE model that optimizes layer-wise embeddings during encoding while training downstream models. Experiments on a heterogeneous multiplex graph from a travel dataset show that this method outperforms models using only graph-level or prediction-level fusion, particularly for node classification tasks.

strengths:
1. This paper introduces a new method with three graph learning models and a dataset. The models improve performance by maintaining the graph structure early and targeting specific tasks. This leads to better results in link prediction and node classification on complex graphs.

2. The paper has a clear structure. Each section is well-organized, and the logic flows well. Readers can easily follow the research process.

weaknesses:
1. The proposed method lacks innovation as it combines existing models without introducing significant new ideas.

2. The encoder employs parallel GraphSAGE layers to sample and aggregate neighbors of nodes, but the paper does not assess the efficiency of this approach, necessitating further experiments.

3. The experiments only compare variations of the proposed model, lacking convincing evidence from comparisons with a broader range of baseline models.

questions:
Can the authors provide more detailed information about the computational efficiency of the HMGSAGE encoder? Specifically, how does the use of parallel GraphSAGE layers impact the model’s overall runtime and memory usage?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730608706541

thread_type:
official_review

<thread_object>
creator: reviewer_ae1x

<content>
summary:
This paper introduces SECSAGE, an extension of GraphSAGE for heterogeneous multiplex graphs, which refines layer-wise embeddings concurrently with downstream model training. Specifically, the model is optimized through joint encoder-classifier training, showing improvements over selected baselines on the Travel Dubai dataset. While technically robust and effective, it would be beneficial to include experiments on other datasets, code for reproducibility, expanded baseline comparisons, and improved writing clarity.

strengths:
1. The topic is interesting and relevant, addressing challenges in representation learning for heterogeneous multiplex graphs.
2. SECO combines prediction-level fusion with simultaneous optimization, resulting in improved task alignment in embeddings.
3. The proposed model demonstrates improvement over selected baselines.
4. The model’s performance is validated with multiple metrics, including accuracy, F1, and ROC-AUC.

weaknesses:
1. The reliance on GraphSAGE extensions and standard fusion techniques limits the novelty of the proposed architecture.
2. The absence of code hinders reproducibility, reducing accessibility for researchers who may want to apply the proposed approach.
3. Evaluating on only a single dataset may not capture sufficient diversity, thus limiting generalizability.
4. While the model outperforms the chosen baselines, authors should expand the comparison to include a wider range of recent and diverse approaches.
5. The manuscript would benefit from further refinement to improve clarity and readability.

questions:
1. Are there specific models you believe should be included in the baseline comparisons, especially in the context of heterogeneous/heterogeneous multiplex/knowledge graphs?
2. Do you have plans to make the code available publicly?
3. Since the model is evaluated on a single dataset, can you discuss the potential challenges in applying your model to other datasets?
4. Could you further clarify the novelty of the added components and explain how these modifications specifically address the unique challenges of heterogeneous multiplex graphs?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730972096872

thread_type:
official_review

<thread_object>
creator: reviewer_uuqc

<content>
summary:
This paper proposes a novel method for jointly training task-specific encoders and downstream models on heterogeneous multiple graphs (HMG). The author first defines key concepts such as heterogeneous multiple graphs, inductive learning, and simultaneous encoder-classifier optimization SECO. Subsequently, they designed an encoder called HMGSAGE, which can handle graph data with multiple node and edge types. Building on this foundation, the author further expanded HMGSAGE and constructed a graph autoencoder GAE capable of inductive link prediction and graph structure data node classification. The core contribution of this paper is the introduction of the SECO paradigm, which learns the optimal node embeddings for the downstream task through supervised learning by iteratively and simultaneously updating the encoder and downstream classifier, thereby improving the performance of the downstream task.
advantage:

strengths:
1.This paper proposes a new joint training framework that closely integrates the training process of the encoder and downstream models, providing an effective solution for tasks on heterogeneous multiple graphs.

2.The author clearly defines and explains concepts such as heterogeneous multiple graphs, inductive learning, and SECO, providing a solid theoretical foundation for readers.

weaknesses:
1.The writing quality of the entire paper is relatively lacking, with problems such as unclear expression and confusing logic, especially in the introduction section, where there are significant writing issues. The discussion of the recommendation system in the introduction is too long and not effectively connected to the core research content and motivation of the paper, making it difficult for readers to understand the actual significance and motivation of the research.

2. The overall organizational structure of the paper is rather confusing, leading to confusion for readers in understanding. There is a lack of clear logical connection between the various parts, making it difficult to trace.

3. The method proposed in this article is essentially a stack of existing technologies. Specifically, the author uses GraphSAGE as an encoder, stacks it with a joint training method based on SECO, and introduces the GAE model structure for learning. Although these technologies are effective in themselves, they do not demonstrate sufficient innovation and lack new theoretical or practical value.

4. Only one dataset was used for verification in the experimental part, and there was no sufficient comparative experiment. In particular, other methods listed in Table 1 mentioned in the paper were not effectively compared as a baseline for comparison.

questions:
Please refer to the weaknesses.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732743651263

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Major revisions to the paper

comment:
We thank the reviewers for their thorough comments which have helped us improve the manuscript. We respond to the reviewers’ comments below. We have revised the manuscript by correcting a few minor errors and making adjustments where necessary to improve its flow and readability. We also added new experiments to demonstrate the framework's generalizability. All changes to the manuscript are highlighted through the use of $\color{blue}\textbf{blue}$ fonts to help facilitate the review.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper extends GraphSAGE for representation learning on Heterogeneous Multiplex Graphs by refining layer-wise representations. Although it is an interesting topic, the novelty and contributions are very limited by leveraging existing GraphSAGE and fusing techniques. Besides, the evaluations are performed only on one dataset, and the baselines are old. Thus, it is not fully justified. Although authors provide the improvements on the expression, it also much more efforts to enhance the quality and clarity.