GMAgent: A Graph-oriented Multi-agent Collaboration Framework for Text-attributed Graph Analysis
Authors: Hang Lv
Pengxiang Zhan
Yanchao Tan
Zixuan Guo
Shiping Wang
Carl Yang
Venue: Accepted by TMLR ()

Abstract: Text-Attributed Graphs (TAGs) are crucial for modeling interconnected data in numerous real-world applications. Graph Neural Networks (GNNs) excel at efficiently capturing global structural information across TAGs, while Large Language Models (LLMs) offer strong capabilities in local semantic understanding. Despite the recent development of integrating GNNs and LLMs for TAG analysis, these approaches often fail to fully exploit their complementary strengths by relying primarily on a single architecture. Furthermore, LLM-based multi-agent collaboration systems have shown promising potential across diverse fields. However, their integration with GNNs for graph analytical tasks remains underexplored. To this end, we introduce GMAgent, a novel graph-oriented multi-agent collaboration framework that effectively and flexibly interacts between diverse GNN-based and LLM-based graph agents, facilitating comprehensive TAG analysis. First, we deploy multiple GNNs as graph agents to perform conflict evaluation, identifying conflict scenarios for further multi-agent collaboration. Then, we repurpose LLMs as graph agents via graph-driven instruction tuning and adopt a role-play expert recruiting strategy, thereby generating LLM graph experts' initial analyses for conflict scenarios. Finally, we conduct a graph-oriented multi-agent collaboration to effectively and efficiently guide collaborative self-reflection on graph experts and the final answer selection. Extensive experimental results on five datasets demonstrate significant improvements, showcasing the potential of our GMAgent in improving the effectiveness, interoperability, and flexibility of comprehensive TAG analysis.

Published in Transactions on Machine Learning Research (10/2025)
GMAgent: A Graph-oriented Multi-agent Collaboration Frame- work for Text-attributed Graph Analysis
Hang Lv
luhangkern&@gaail.cmm com
Fuzhou University, Fuzhou, China
Pengxiang Zhan
wnyshanpenngrionggl66...
Fuzhou University, Fuzhou, China
Yanchao Tan
ycton@ffa.edu.cr
Fuzhou University, Fushou, China
Zixuan Guo
8323048210fau.edu.co
Fushou University, Fushou, China
Shiping Wang
shipingwangphd@118...om
Fushou University, Fuzhou, China
Carl Yang
j. Barlyyngg@emory.ce edu
Emory University, Atlanta, USA
Reviewed on OpenReview: / wwtp:/vee...eeeee.emeee net/ forum? 4Bm79ePPea
Abstract
Text-Attributed Graphs (TAGs) crucial for modeling interconnected data in numerous real-world applications. Graph Neural Networks (GNNs) excel at efficiently capturing global structural information across TAGs, while Large Language Models (LLMss) offer strong capabilities in local semantic understanding, Despite the recent development of integrating GNNs and LLMs for TAG analysis, these approaches often fail to fully exploit their complementary strengths by relying primarily on single architecture. Furthermore, LLM- based multi-agent collaboration systems have shown promising potential across diverse fields. However, their integration with GNNs for graph analytical tasks remains underexplored. To this end, introduce GMAgent, novel graph-oriented multi-agent collaboration framework that effectively and flexibly interacts between diverse GNN-based and LLM--bsed graph agents, facilitating comprehensive TAG analysis. First, we deploy multiple GNNs as graph agents to perform conflict evaluation, identifying conflict scenarios for further multi-agent collaboration. Then, we repurpose LLMs as graph agents via graph-driven instruction tuning and adopt a role-play export recruiting strategy, thereby generating LLM graph experts' initial analyses for conflict scenarios. Finally, conduct graph-oriented multi- agent collaboration to effectively and efficiently guide collaborative self-reflection experts and the final answer selection. Extensive experimental results on five datasets demonstrate significant improvements, showcasing the potential of our GMAgent in improving the effectiveness, interoperability, and flexibility of comprehensive TAG analysis.
graph
1 Introduction
Text-Attributed Graphs (TAGs), where cach node can be associated with textual attributes, are common across various real-world applications. Due their rich semantics and complex structures, TAGs have been widely used in diverse domains, such social networks, academie networks, e-commerce networks, and web

---

Published in Transactions on Machine Learning Research (10/2025)
page analytics (Pan et al. 2024 Ren et al. 2024 supporting core tasks such node classification and link
prediction (Jin et al. 2024 Tan et al. 2025 j. To accurately handle TAGs, Graph Neural Networks (GNNs)
have been commonly adopted for capturing the global structural information of graphs (Wang et al. 2024c
Zhu ct al. 2021) However, GNNs often struggle to fully integrate the rich semantics embedded textual altribules. Inspired by the successes of Large Language Models (LLMs), some researchers have ex explored LLMs for accurately capturing contextual semantics of graph attributes (Tang et al. 2024a Fang al. 2024
Huang al.) 2024a), while their limited input tokens hinder the processing of large-scale graphs. Although these efforts have been made to combine GNNs and LLMs, they generally rely either GNNs LLMs the primary backbone, limiting their abilities to comprehensively exploit the strengths of both.
Recently, LLMs' advanced task understanding and self-planning capabilities have led to the development of LLM based multi-agent collaboration systems across diverse fields (Chen et al., 2025 Fan et al. 2025 Tang al., 2024b). Nevertheless, the integration of LLMSs and GNNs within multi-agent collaboration frameworks for graph analytical tasks remains largely underexplored. Figure I illustrates the potential of a multi-agent. collaboration framework that combines the global structural patterns captured by GNNs with the local semantic understandings provided by LLMs. In this multi-agent framework, GNNs and LLMs can work together as complementary backbones, learning from each other and collaborating to improve graph analytical tasks (e.g.. node classification, link prediction, and graph classification). To achieve these goals, we further address the following three key technical challenges.
Challenge I: How to effectively utilize GNNs as graph agents for interacting with other graph agents? GNNs excel in capturing global structural informa- tion across the whole TAG via message- passing and aggregation mechanisms, especially providing a clear computa- tional advanlage in large-scale graph analysis. While deploying GNNs graph agents significantly enhance the understanding of TAGs, the inher- ent architectures of GNNs limit their interpretability when interacting with other graph agents.
Text-attributed Graphs
Social Network
Academic Network
E-commerce Network
GNNS
LLMs
How to repurpose L.LMs as graph agents to accurately under- stand complex groph data and execute graph analytical tasks? To fully exploit the potential of LLMs for graph analy- sis, an essential obstacle is the accurate understanding and reasoning capabil- ities of LLMs given graph data with abundant attributes and flexible struc- tures. Despite L.Ms'' strengths in understanding semantics, they still struggle to precisely understand graph structures and the information needs of different graph analytical tasks.
Challenge II:
Global Structural Information
Local Semantic Information
Figure 1: An illustrative toy example of the potential for a multi- agent collaboration framework that can simultaneously harness the global structural learning power of GNNs and the local semantic richness of LLMs for TAG analysis.
Challenge III: How to properly perform multi-agent collaboration with GNN-based and LLM-based graph agents for comprehensive TAG analysis? The effectiveness of multi-agent, collaboration largely depends on how to integrate the strengths of multiple agents to facilitate comprehensive TAG analysis. Balancing the global structural learning power of GNN-based graph agents with the local semantic comprehension of LLM- graph agents presents significant challenge in robustly improving the accuracy and efficiency of graph-oriented multi-agent collaboration.
To lackle these challenges, propose Graph-oriented Multi-Agent collaboration framework for text- attributed graph analysis (GMAgent), which consists of three key steps: (i) Deploying GNNs as Graph Agents,
2

---

Published in Transactions on Machine Learning Research (10/2025)
where perform conflict evaluation based multiple trained GNN models to identify confficting scenarios for further multi-agent collaboration; (ii) Repurposing I.LMs as Graph Agents, where we obtain an I...-powered graph agent via graph-driven instruction tuning and adopt a role-play expert recruiting strategy, collecting LLM graph experts' initial analyses for conflicting scenarios; (iii) Graph-oriented Multi-agent Collaboration, where we enable advanced LLMs (e.g.; GPT-40) to assign a confidence score for each LLM graph expert and generate summary report, along with all GNN and LLM experts' analyses, to guide collaborative self-reflection of LLM experts and final answer selection.
Our overall contributions summarized as follows:
Formulation of the Graph-oriented Mulli-agent Framework. We establish a first multi-agent framework that effectively and flexibly engages interactions between diverse GNN-basod and LLM-basod graph agents, Iacilitating comprehensive TAG analysis.
Effective Model Designs. We design and implement a set of models and mechanisms, including conflict evaluation, graph-driven instruction tuming, role-play expert recruiting, summary report generation, and collaborative reff-reflection, constituting a robust graph-oriented multi-agent collaboration framework for comprchensive TAG analysis.
Extensive Experiments across Gruph Analytical Tasks and Datasets. We conduct thorough experiments to validate our proposed approach with five real-world datasets, demonstrating its superiority over existing state-of-the-art methods and highlighting its effective, interpretable, and flexible abilities in enhancing TAG analysis.
2 Related Work
2.1 Text-attributed Graph Analysis
Graph analysis methods, especially Graph Neural Networks (GNNs). have shown effectiveness in capturing structural information for graph data (Zeng et al. 2025 Wang et al.
2024c Zhu et al., 2021 Traditional
GNNs, such as Graph Convolutional Networks (GCNs) (Kipf & Welling, 2017 and Graph Attention Networks
(GATs) VVličković et al.) 2018)) are widely used for learning node and edge representations in graphs, excelling in extracting structural relationships. However, when it comes to processing rich textual attributes associated
with nodes or edges, these models face limitations (Tan et al., 2023 Wang et al. 2024c Fang et al.
2024 He
et al.
2024 Zhao et al.] 2023).
Recently, the emergence of Large Language Models (LLMs), with their robust semantic understanding, offers a way to incorporate textual information into Attt-butr Graph (TAG) learning (Fang et al. 2024
Tan et al. 2023). For instance, GraphGPT (Tang el al. 2024a) demonstrated how LLMs can process and understand complex graph structures by combining text-based knowledge with graph structural information.
Pan et al. (2024) distilled knowledge from LLMs for learning TAGs. While LLMs excel at interpreting and representing text, they are insufficient for capturing the global structural patterns that GNNs handle well, highlighting the need for a more integrated approach (Li et al. 2024).
Building on the complementary strengths of GNNs and LLMs, recent research has explored two main categories of integrated approaches, based on whether the backbone is a GNN or an LLM. In the first category, GNNs serve as the backbone, with LLMs providing additional semantic context. Methods such as ConGra'T (Brannon al., 2024) and GRENADE (Li et al. 2023) align node embeddings generated by GNNs with representations from LLMs, effectively combining the structural information from TAGs with the textual understanding of LLMs. In the second category, LLMs are used as the backbone, and GNNs are incorporated to infuse structural graph information into LLMs. Approaches like DGTL (Qin et al. 2023) and GraphAdapter (Huang et al. 2024b) co-train transformer layers in LLMs alongside graph neural layers, enabling LLMs to better capture the structural dependencies in TAGs. Despite the progress made by both categorics, a key limitation remains: these models typically rely on cither GNNs or LLMs as the primary backbone, which restricts their ability to fully leverage the strengths of both. As a result, they fail to simultaneously harness the structural learning power of GNNs and the semantic richness of LLMss.
3

---

Published in Transactions on Machine Learning Research (10/2025)
2.2 Multi-agent Systems
Multi-agent systems have been explored across wide range of applications and domains (Chen et al. 2025
Liu et al.) 2024b; Chen et al. 2023), demonstrating their capability to efficiently handle complex tasks through agent coordination. For example, AutoGen (Wu et al. 2024a) has focused on automating multi-agent collaboration in diverse tasks, such as math problem-solving, group chat, and coding. BadAgents (Yang et al. 2024b) formulated a framework for agent backdoor attacks, which can introduce malicious behavior in the intermediate reasoning process while keeping the final output correct. These approaches demonstrate the potential of multi-agent systems to scale and efficiently solve problems through agent coordination.
In recent years, LLM-based multi-agent systems have emerged as powerful tool, primarily divided into two categories: collaboration and competition (Wang et al. 2024b). The first category involves collaborative multi-agent systems, where multiple LLM-based agents work together toward a common goal. These systems have proven effective in areas like cooperative prciisonnmmkinng (Piatti et al. 2024), majority voting (Chen et al. 2024). medical domains (Fan et al. 2025 Tang et al. 2024b), and code generation (Islam et al. 2024). For example, GOVSIM (Piatti al.) 2024h represented the LLM's strategic interaction and cooperative decision-making capability. The second category includes adversarial multi-agent. systems, where agents have conflicting objectives. These methods have been applied in competitive environments, such as games {Feng 2024), debate (Liu et al. 2024c), and evaluation (Chan et al. 2024). For example, GroupDebate (Liu
ct al. let al. 2024c) involved dividing all agents into multiple debate groups, enhancing the performance and efficiency in the multi-agent debate system.
Despite the wide-ranging success of multi-agent LLM systems these fields, their application in graph analytical tasks remains relatively unexplored. In the graph domain, existing multi-agent systems often employ GNNs to support agent coordination structures to model agent states (Zhang et al. 2024 Duan
et a.l. 2024). For example, VillagerAgent (Dong et al., 2024) introduced directed acyclic graph framework to manage task assigmments and track agent states, while CAG-ODE (Huang et al.) 2024c employed a GNN as the ordinary differential equations function to model continuous agent interactions. Graph-Agent-Reasoonn
(Hu et al. 2024) further extended LLM-bases multi-agent collaboration for graph reasoning, though its application remains limited to this specific task. Wu et al.] (2024b) made progress by integrating GNNs and LLMs within a multi-agent framework to enhance graph learning for task planning. As closest to us, MARK (Fu et al., 2025) utilized GNNs to identify uncertain nodes via graph clustering, and then generated ranking-based guidance through LLM-based multi-agent collaboration. However, MARK's reliance clustering restricts its applicability to broader graph lasks like multi-label node classification or link prediction. Although both LLMs and GNNs have shown individual success in multi-agent systems, their integration for graph analytical tasks is still limited, leaving significant potential for deeper investigation and development in this area.
3
The GMAgent Framework
3.1 Problem Formulation and GMAgent Overview
Given Attt-Attr Graphs (TAG) G and multiple graph analytical т, (e.g., node classification TT), the goal of GMAgent is to develop a multi-agent collaboration framework that flexibly integrates the strengths of various Graph Neural Network (GNN) graph agents (GNN,) 2i and Large Language Model (LLM) graph agents [ Mr. improving graph analytical task performances on т.
To achieve this goal, we first deploy multiple GNNs trained on the specific TAG dataset graph agents, performing conflict evaluation to identify conflict scenarios for multi-agent collaboration. Then, generate an LLM-pooered graph agent via graph-driven instruction tuning, integrating CoT-based instructions from advanced LLMs (e.g., GPT-4o) with lask-specific instructions. Adopting a role-play expert recruiting strategy, diverse LLM graph experts provide initial analyses for conflict scenarios. Finally, we introduce a graph-oriented multi-agent collaboration between GNN and LLM experts, where advanced LLMs (o.g., GPT-4o) assign a confidence score each LLM expert and generate a summary report guide collaborative self-reflcction on LLM experts and final answer selection. The overall framework of our GMAgent is shown in Figure 2
4

---

Published in Transactions on Machine Learning Research (10/2025)
1
Deploying GNNS as Graph Agents
Training
Inference
Final Answer
>>>>
>>>>
Graph Analytical
GNNi
GNN:
Tasks
Conflict Scenario
Text-attributed Graph
GNN-based Agents
Conflict Evaluation
Repurposing LLMs as Graph Agents
Instruction Tuning >>>>
Experts Recruiting >>>>
S CoT-based Instruction
+
+
>>
Effective Graph
Graph Analytical Tasks
LLM
Description
Graph Agent
Multiple Experts
Graph-oriented Multi-ogent Collaboration
GNN Role:
Answer:
Confidence Analyses:
>>
Final Answer Selection
LLM Role:
Reasom:
Reason:
Reason:
Collaborative Self-reflection
Answer:
Key Insights: (l)... (2) ... (3) ...
Analysis: ...
GPT-4o
Global Summary:
Summary Report
Experts Analyses
Figure 2: The overall of our GMAgent framework, consisting of three key steps, which are ! Deploying GNNs (e.g., GCN and GAT) as Graph Agents, Repurposing LLMs (e.g., Qwen2-7B) as Graph Agents, and Graph-oriented Multi-agent Collaboration.
3.2 Deploying GNNs Graph Agents
2025 They GNNs demonstrate strong performance multiple training tasks and datasets [Luo et al. 2024
effectively enhance node representations with structural information via message- passing and aggregation tion
patterns, considering the global information across the whole TAG (Pan et al. 2024 Wang el al. 2024c Zhu 2021h. As shown in Figure 3 aa, GNN models, like GCN (Kipf & Welling 2017 j and GAT (Veličković et al.
et al.
2018 can achieve the average accuracy rates of 73.08% and 73.79% respectively. Notably, when both models provide consistent predictions (i.e., "GCN + GAT"), the accuracy improves further, demonstrating that GNNs capable of making reliable and accurate inferences. In addition their accuracy, GNNs offer clear computational advantage when dealing with large-scale graphs. Compared with LLMs that accurately capture contextual semantic understanding within limited input tokens, GNNs can efficiently procoss large-scale graph structures utilizing their architecture's inherent parallel message- passsegganiig mechanism (Yang et al. 2020a ) As shown Figure b(b), we compare the inference efficiency (seconds per response) of GCN, LLaMA3-SB (Dubey et al. 2024), and Qwen2-7B (Yang et al.] 2024a) across three datasets, where LLMs like LLaMA3-SB and Qwen2- 7B exhibit significantly higher inference times than GNNs due to producing interpretable answers.
To this end, propose to deploy GNNs graph agents within multi-agent framework for TAG analysis, which balances effectiveness and computational efficiency. Specifically, we first train multiple GNNs (e.g., GCN and GAT) tailored for specific datasets and graph analytical tasks (e.g., node classification). Then, we introduce a conflict evaluation to identify conflict scenarios based on each GNN-based graph agent's inference results. These results for conflict scenarios are used for further multi-agent collaboration.
3.2.1 GNN-based Graph Agents Training
To obtain the GNN-based graph agent, we train each GNN on the TAG dataset G = (V,,, for a graph analytical task Each node E V has an initial node embedding, fv S based on its shallow feature Se,- Then, the GNN adopts message-passing and aggregation patterns to learn structuro-aware embedding for node yn via exploring the global context across all nodes as follows:
h UPD (h AGG ((MSG (hr,, h k- (n)(())))
(1)
5

---

Published in Transactions on Machine Learning Research (10/2025)
ZZ
GCN
GAT
GCN GAT
GCN
LLaMA3-8B
XX
Qwen2-7B
100
Inference Time (log()) |+10
$2.61 82.50
30
Accuracy (%)
70.65
12
11 13.39
11.381140 11.40
11.30
64.38
10
60
40
2
Cora
Arxiv
IMDB
Cora
Arxiv
IMDB
(a) Accuracy Results
(b) Inference Time Results
Figure 3: (a) Accuracy of GCN and GAT on validations of three datasets. "GCN + GAT" represents the accuracy on samples where both models make consistent predictions. (b) Inference efficiency comparison among GCN, LLAMA3-8B, and Qwen2-7B.
where h denotes the embedding of node in the k-th layer of the GNN and ((w,,) denotes the neighbor nodes set of th- The MSG function receives from each neighbor node's message, the AGG function used for aggregating the neighbor node embeddings, and the UPD function updates the embedding of Vn, based on the aggregated neighbor node embeddings.
After obtaining the last layer's node embedding he..k via Eq. we apply the predictor and the objective function tailored to the graph analytical task T; for training the GNN, where take node classification example:
Dr.., = Softmax (MLP,, Ta (kv,.))),
(2)
C E,, EVE (Ør (Vr..))
(3)
where CE(.,) is the cross-entropy loss between the prediction Dra... and the ground-truth label yra. for node Un- K is the total number of GNN layers.
3.2.2 Conflict Evaluation via GNN-based Graph Agents Inference
Different GNN-based graph agents may generate inconsistent predictions for the same scenario on the TAG dataset, due to architectural differences. To fully integrate the opinions of various GNN experts, we propose conflict evaluation mechanism. Specifically, for each input instance (e.g., node in classification), if all GNN experts produce the same inference result, the result is directly accepted. Otherwise, the instance is trcated as conflict scenario and passed to LLMs for collaborative reasoning with the corresponding GNN outputs (ef., Section 3.4/. This output-level detection model-agnostic, enabling straightforward extension to settings with more experts, heterogeneous architectures, or different domains, while avoiding reliance on any single modcl's bias. We verify the impact of different GNN expert combinations and quantify the corresponding proportion of conflict scenarios in Section 4.3 Particularly, we automatically textualize each GNN expert 's characteristic and inference result on the conflict scenario via simplified template follows:
A simplified opinion template for GNN expert GNN,
GNN Role: GNN, is graph analysis expert, depending to form the node representations. Answer: {For node classification Tı, "Category Name".)
This mechanism balances effectiveness with computational efficiency based on multiple GNN-based graph agents, reducing the number of scenarios requiring further multi-agent collaboration. Mcanwhile, it also ensures that each GNN expert's opinion can interact with other agents in the multi-agent framework. Section 4.6 provides the detailed efficiency analysis of GMAgent.

---

Published in Transactions on Machine Learning Research (10/2025)
3.3 Repurposing LLMs as Graph Agents
TAGs with abundant attributes and llexible structures pose signilicant challenges for LLMs (Tang el al. 2024a Ye et al. 2024), where LLMs as graph agents may struggle to precisely understand these unfamiliar graph structures and process new graph analytical tasks. This leads to potential inaccuracies and limited reasoning capabilities on various datasets. Instruction tuning enables LLMs to understand and perform well a wide range of graph analytical tasks on multiple datasets and contexts (Wang al. 2023b c). The effectiveness of such tuning for LLMs largely depends on how the instructions are structured. However, manually constructing these task-specific instructions is often time-consuming and requires excessive resources.
To address these challenges, we repurpose LLMs as graph agents. We first generate an L.M.- graph agent via graph-driven instruction tuning, where we integrate CoT-based instructions from advanced LLMs (e.g,, GPT-4o (Achiam el al. 2023)) with lask-specific instructions (e.g., node classification) as the training corpus. Additionally, we employ the role-play expert recruiting strategy to gather diverse LLM graph experts into the multi-agent framework, and then obtain their initial analyses of conflict scenarios (c.f., Section 3.2.2
3.3.1 Generating LLM-pooered Graph Agent
To help LLMs understand complex TAGs and execute graph analytical tasks, it is crucial to construct an effective training corpus specific to TAG data. Inspired by traditional graph analysis techniques, such as neighbors [Kipf & Welling 2017 Hamilton et al.) 2017 Yang et al. 2020b) and random walks (Li et al. 2021b Ivanov & Burnaev] 2018: Jan et al. 2023), we propose an eflective graph description textualization mechanism describe graphs via these concepts. These techniques play complementary roles understanding TAGs, providing LLMs a comprehensive and enriched perspective on graph structures (Tan al. 2025 Fang el al., 2025). Particularly, neighbors facilitate the understanding of local connectivity and feature distributions, offering fine-grained insights into node attributes. In contrast, random walks serve as a dynamic strategy to capture global structural patterns and high-order relationships, highlighting the diversity of connectivity and paths. Specifically, we extract the key information from TAGs and convert it into a textual graph description for each node, consisting multiple one-hop neighbors and three random walks. A simplified example of an effective graph description is given below:
A simplified effective graph description example
The effective graph description of is listed follows:
Ego Graph Node: (w1: 01 's text attribute 31, vy's...)
One-hop Neighbor: {1z, Us, U4, 25}
Random Walk: {(1) U1 + 02 + 06 + 07 U8i...
Based on this graph description, we then construct the lask-specific instructions (e.g., node classification). To further improve LLM reasoning capabilities on unfamiliar graph structures or new tasks, we leverage the Chain-of-Thouge based on the graph descriptions for different graph analytical lasks, generating answers accordingly. We then integrate these outputs from GPT-4o into CoT-based instructions for fine-tuning LLMss. The detailed prompt templates are provided in Appendix A.7
(CoT) mothodology (Wei et al., 2022). This allows GPT-4o to reason step-by-step
For fine-tunin,, we adopt a general LLM Qwen2-7B 2-B with LoRA (Hu et al. 2022 as the starting point, which can be flexibly replaced with other powerful LLMss. Then, we utilize the negative liklliild loss as the fine-tuning objective as follows:
Po (Y,, Yj,ck) LLMo (I,, Yj.ck,,
(4)
1Y,
LFT = log (Yr,144i, Mj,ck),
(5)
where 0 is the learnable parameters of the LLM, the instruction I, E I is the input of LLM, and Y, is the output of LLM. After obtaining the fine-tuned LLM for TAG analysis, repurpose as the graph agent
7

---

Published in Transactions on Machine Learning Research (10/2025)
for accurately analyzing and executing graph analytical tasks. Appendix A.5 further evaluates the impact of various graph description strategies for LLM-based graph agents.
3.3.2
Graph Expert Recruiting
LLMs often produce similar answers when given the same prompts due to their homogeneity (Lu et al. Padmakumar & He 2024). This hinders discussions and decision-making in the multi-agent framework. To mitigate this issue, we introduce the role-play expert recruiting strategy. Here, we assign distinct roles to the graph agent from Scction 3.3.1 as LLM graph experts. These roles focus on different perspectives, such as analyzing random walks or one-hop neighbors. In this way, we gather diverse analyses of conflict. scenarios identified in Section 3.2.2] Specifically, given a conflict scenario Pe, and role prompt RLLM, we generate the analysis from cach LLM graph expert:
2024
ALLM, LLMo (PeLs RLLM)).
We can obtain initial analyses from all LLM graph experts A= (ALLM, ALLM, ALLMMr h. where Mr is the total number of LLM graph experts. Then, we combine each analysis with the correspouding LLM expert's 's characteristics via the following template:
A simplified opinion template for LLM expert LLM:
LLM Role: LLM, is graph analysis expert, specializing in ..... Its task is to analyze the graph based .....
Answer: {For node classification 7i "Category Namc".)
Analysis: {ALLMr}
3.4
Graph-oriented Multi-agent Collaboration
With LLMs demonstrating strong abilities in task understanding and self-planning an emerging research direction to build LLM-based multi-agent systems for graph analytical tasks (Ren et al. 2024 Hu et al.,
2024 Wang et al. 2023a): ) Recently, GraphAgent-Reasonee (Hu al. 2024) applied LLM- based col collabrraiin for graph reasoning. handling graphs with over 1,000 nodes. However, how to integrate both GNN-basod graph agents and LLM-based graph agents into the multi-agent collaboration for effectively and efficiently handling graph analytical tasks (e.g., node classification) remains unknown.
To address the issuc, we propose novel graph-oriented multi-agent collaboration method that fully leverages the strengths of GNN and LLM experts. This allows flexible interactions between diverse GNN-based and LLM-based graph agents, enhancing the accuracy of graph analytical tasks. Specifically, we propose to utilize advanced LLMs (c.g., GPT-4o) to assign a confidence score for each LLM expert and generate a summary report. This report, along with all experts' analyses from GNN experts and LLM experts, guides collaborative self-reflection among LLM experts and facilitates the final answer selection.
3.4.1 Summary Report Generation
Different LLM experts may offer varying answers based on their analytical perspectives (e.g., from random walks one-hop neighbors). LLM-based graph agents often struggle to detect and correct their own mistakes, leading to potential misguidance during multi-agent collaboration (Chih- Yao Chen et al. 2025 Wang et al. 2024a). To this end, we utilize GPT-4o to assign confidence score from to 5 for cach LLM expert based on their current. analyses and generale summary report by extracting key insights and a global summary (shown in Figure 2). With billions of parameters, GPT-4o, efficiently summarizes LLM expert insights (Tang et al., 2024b Yeh al., 2024). This summary guides LLM graph agents prioritizing their considerations based on the reliability of each expert's analysis. Since GNN experts focus on structure but lack interpretability, GPT-4o only processes I.I.M analyses for scoring and summarization.
8

---

Published in Transactions on Machine Learning Research (10/2025)
3.4.2 Collaborative Self-reflection Optimization
A After generating the summary report, LLM experts iteratively refine their analyses through self-reflection, combining insights from other experts and the report. To measure the degree of agreement among the experts, we define the agreement across expert predictions based on the concept of entropy (Shannon 1948) as follows:
N Σ
CONS, -
FRET.aa log (FEE,..,)
(7)
where N is the total number of candidate answers provided by experts and FREE,,, the frequency of each candidate answer aj appearing across experts' analyses at the current iteration T. In each iteration, LLM experts reflect on their previous analyses, where they also consider the summary report and insights from other experts to decide whether to revise their answers and analyses. We then leverage GPT-4o to generate new summary report based on experts' revised analyses for the next iteration. Note that, this collaboration cycle is repeated until either CONS,, > CONST,,, or the maximum iteration count Tmaz is reached.
Finally, we compute the final score for each predicted answer based on its frequency (c.f., Eq. 7 and each expert's confidence score CONET,,, selecting the highest-scoring answer as final answer:
f argmax
CONFT. FRET.
(8)
where M is the total number of all experts, consisting of Mc GNN-based graph agents and Mr. LLM-based graph agents. Te is the iteration at which the collaboration cycle concludes. Each LLM expert's confidence score is derived from the summary report at the iteration T.- Notably, since GNN-basod graph agents capture the global and structural information across the whole TAG, they are always assigned the highest confidence score of 5 by default. This graph-oriented multi-agent collaboration flexibly integrates the global and local insights from both GNN-based and LLM-based experts, effectively resolving conflicting scenarios. Furthermore, employing CONS,, as critcrion for determining when collaboration should stop, improves the efficiency of multi-agent collaboration. Appendix A.7 provides the prompt templates used in graph-oriented multi-agent collaboration.
3.5 Complexity Analysis
We analyze the time complexity of the proposed CMAgent framework by three major components: Deploying GNNs as Graph Agents, Repurposing LLMs as Graph Agents, and Graph-oriented Multi-agent Collaboration.
Deploying GNNs as Graph Agents. For each GNN agent, the time complexity per layer is O(Dvdd + [&|d), where [ and |&| denote the number of nodes and edges the graph, and is the dimension of node feature. With K layers and Mc GNN agents, the total cost is (MM K ((V] (El)d), which scales linearly with graph size and is efficient due the parallelizable message passing.
Repurposing LLMs as Gruph Agents. Given Mf. LLM agents, each inference time on a textualized graph description is OTT L'), where TŁ is the LLM's generation steps and L' is the input prompt length. Instruction tuning is a onc-time offline step with standard LLM finc-tuning complexity. Since LLMs process each conflict scenario independently, the per-scenario cost is O(Mr. Tr. L').
Graph-oriented Multi-agent Collaboration. In each self-reftcction itcration, all Mr LLM experts revise their outputs based on the summary report derived from GPT-4o. GPT-4o performs scoring and summarization with cost O(Ts T."), where Ts is the generation steps and T," is the combined input length. Since the Mr LLM experts generate responses independently, this step can be executed in parallel. Suppose the number of iterations is bounded by Tmaz, the total collaboration complexity per conflict scenario is (TTaz (Mz Tr. T/ + Ts. I.")).
In general, our proposed GMAgent balances accuracy and computational efficiency for graph analytical tasks, where the GNN-based agent deployment ensures low cost large-scale graphs, while L.M--based agent.
9

---

Published in Transactions on Machine Learning Research (10/2025)
Table Statistics of the used datasets.
Task
Node Classification
Link Prediction
Dalaset
Arxiv
Cora
IMDB
Products
PubMed
DBLP
# Nodes
169,343
2,708
21,420
2,449,029
63,109
18, 405
# Edges
1,166,243
5,429
86,642
61,859,140
244,986
67,946
Node Type
1
1
4
4
3
Link Type
1
1
6
10
Features
128
1,433
3,489
100
200
334
collaboration is selectively applied only to conflict scenarios, reducing the frequency of LLM invocations. Additionally, we provide a detailed efficiency comparison of our GMAgent in Scction 4.6
4 Experiment
In this section, we evaluate our GMAgent framework, focusing on the following five key research questions:
How does our proposed GMAgent framework perform in comparison to the representative graph- oriented methods, and how well does it scale when applied to large-scale graphs?
RQI:
RQ2: How does GMAgent perform when integrating different. GNN-based graph agents?
RQ3: How is the performance of GMAgent affected by the choice of LLM-based graph agents?
RQ4: How do different multi-agent collaboration strategies (e.g., number of agents, choices of summary agents, maximum iteration count, and self-reflection strategies) affect the performance of GMAgent?
RQQ:
How does GMAgent perform in terms of efficiency compared with the representalive baselines?
4.1
Experimental Setup
4.1.1 Datasets
comprehensively evaluate the effectiveness and efficiency our GMAgent, we utilize four real-world datasets for node classification (i.e., ogbu-arxiv (abbr. Arxiv Cora LMDE and ogbn-products (abbr. Products")
two for link prediction (i.e., PubMed and DBLF The detailed statistics shown in Table 1 We provide
further details for the datasets in Appendix A.1
4.1.2 Evaluation Protocols
For node classification, we adopt different ratios 11%%%/1%/444 12% /28 for Arxiv, 60% 10%/2099 220 220% for Cora, and for IMDB, which is consistent with Hu al. (2020a); He et. al. 2024]; ; Lv al. (202)) Notably, follow the official sales-ranking- rankiig-bbee split from Hu al. (2020a) for the large-scale Products dataset, where 9%/2%/902 of products are used for training, validation, and testing, respectively. For link prediction, we train all methods using the randomly selected 80% of links and evaluate them on the remaining 20% held-out links for PubMed, and employ the ratio 10%/10%%/14 /10% for DBLP, following Yang et al. (2020a); Nguyen et al. 2002) We use two commonly adopted evaluation metrics for node classification (Yang et al,, 2020a Lv et al. 2021 Tan et al. 2023): Macro-F1 (across all labels) and Micro-F1 (across all nodes). The Fl score is a metric of the model's 's accuracy in binary and multi-class classification tasks, which considers both precision and recall. For link prediction, compute the AUC and Accuracy (abbr. ACC) metrics
hhttps:/ /tt.d//ooaaooo..dddd/
Phttp:/ www.corra.jnstresaarde com/lander /lander
com karrriabaaaaaaaeeaaaa /ww.ooiinintttaattss
'httpss/wwwwnde" .ith.gov/ /ov/n hatpgggo//ppallllll pubmedd
5 httpps//ddblp.mii-trrrr.de
10

---

Published in Transactions on Machine Learning Research (10/2025)
suggested in Yang al. (2020a); Tan et al. [2023 Liu al. (2024a). AUC indicates the model's ability to distinguish between positive and negative classes across thresholds, while ACC represents the proportion of correctly classified instances overall.
4.1.3
Methods for Comparison
The following 14 characteristic baseline methods be classified into two categories:
GNN-based method: GCN (Kipf & Welling 2017). GAT
Veličković et al. 2018), RevGNN (Li et al.
2021a),
GraphSAGE (Hamilton et al. 2017), HGT (Hu al.
2020b), HINormer (Mao et al. 2023). and
TAPE
(He et al.
2024
LLM- based method: Baichuan2-7BB Yang et al.
2023 Qwen2-7B (Yang et al.
2024a LLaMA3-
8B (Dubey et al. 2024), GPT-3.5 Ouyang et al. (2022). GPT-4 AAchaam et al. 2023)
GraphGPT Tang
et al. 2024a), and MARK (Fu et al. 2025
For more details of the compared baselines, please refer to Appendix
A.2
4.1.4 Implementation Details
For our GMAgent, we utilize AutoGer and FastChal to enable collaboration among multiple agents. By default, select fine-tuned Qwen2-7B 7B the foundation model for our LLM-based graph agent and employ GCN and GAT for our GNN-based graph agents. The number of LLM-based graph agents Mr is set to 4, and the maximum iteration count Tmaz is set to 2. Qwen2-7B is fine-tuned using LLaMA-Factory (Zheng ett al. 2024) on a mixture of instructions from a wide range of tasks and datasets. We perform parameter- efficient fine-tuning via LoRA (Hu et al.) 2022) with rank r = 32 and scaling factor a = 64. We set the 10-5, the maximum input length of the LLM to 1200 tokens, and train for two epochs. GNN-based methods are trained and evaluated using CogDL (Cen et al. 2023) or HGB (Lv et al., 2021]. For LLM-based methods, we load the checkpoint of LIM from HuggingFaace" or call the official APT from OpenA P for evaluation. All LLM-based methods and GMAgent are evaluated using the same test instructions. All experiments are conducted using eight NVIDIA GTX 3090 Ti GPUs. Notably, due to the reliance on mamually customized prompts and GPT-3.5 APIs to generate high-quality explanations for each dataset, we only use publicly available TAPE features on the Cora and Arxiv datascts Therefore, the results for TAPE (He et al. 2024) are excluded from Table 2 However, we did incorporate TAPE into the GNN graph agents variation comparison in Section 4.3 on the Arxiv and Cora datasets, using feature files sourced from the TAPE repository. The full code for this work is available
learning rate
4.2 Overall Performance (RQ1)
In this subscetion, we provide comprchensive performance analysis of our proposed GMAgent framework across various graph analytical tasks and datasets, comparing it with the state-of-the-arl baselines. This evaluation focuses both node classification and link prediction tasks, assessing the model's ability to understand and predict using Text-Attributed Graphs (TAGs).
Overall, our proposed GMAgent demonstrates superior performance, attributed to its unique integration of GNNs for capturing global information and LLMs for interpreting textual attributes. As shown in Table 2 Table 3 and Figure 4 our GMAgent consistently surpasses all baseline models in every evaluation metric. By efficiently assigning simpler tasks GNNs and utilizing LLMs to handle complex, text-heavy scenarios, GMAgent achieves optimal resource allocation and high accuracy across six datasets. Tn addition, the comprchensive performance analyses of GMAgent link prediction are presented in Appendix A.3
httttp///gthuub.cmm/mmcoooo///// aautogg
Thttps://hthubbemm/llnsss Chat
Shttps://lnggiingfaccc.o
9hutps://plattormnoponnaa
hhtpp///ghhabcccmm//haaaggnnnn/n Agent.
11

---

Published in Transactions on Machine Learning Research (10/2025)
Table 2: Experimental results (%) on three datasets for node classification, where * denotes a significant improvement according the Wilcoxon signed-rank test (Woolson, 2007)
The best performances are
highlighted in boldface and the second runners are underlined.
Dataset
Arxiv
Cora
IMDB
Method
Micro-F1
Macro-F1
Micro-F1
Macro-F1
Micro-Fi
Macro-F1
GCN
71.73+0.21
51.1240.65
+0.57
81.29 20.76
64.35 20.73
58.17 #1.45
GAT
72.21
52.30 40.54
81.50 10.68
81.42 10.53
64.31 40.94
58.94 1:330
RevGNN
72.76 40.26
51.38 40.62
84.19 40.42
82.21 10.90
65.89 #0.81
59.90 #1.21
GraphSAGE
71.45 40.07
50.75 10.86
84.53
83.68
62.34 20.79
53.57 11.96
HGT
71.25
+0.52
51.39 20.75
82.61 40.31
81.05 10.64
67.12 #0.65
63.38 1.57
HINormer
71.08
+0.49
51.77
#0.81
82.84
+0.57
81.28
+0.87
67.47 #0.58
64.09 #1.09
Baichuan2-77 7B
2.43
+1.23
1.98
+1.09
8.90
#2.12
5.04 #2.55
40.55 #2.11
39 14 #:2.66
Qwen2-7B
40.25
22.84
18.56
#2.41
58.54
1:.35
48.35 1.883
64.48 #2669
61.27 2:225
LLaMA3-BB
23.16 12.57
11.26
22.15
14.76
4:206
8.49 11.9
37.65 #1.90
34.86
2:283
GPT-3.5
43.23 22.30
32.28
12.59
65.30
11.14
55.34 11.62
55.49
11.47
54.14
42.03
GPT-4
51.21 +1.5
43.40
#2.07
67.72
+1.89
56.14 +1.35
59.57 #1.19
58.18
#2.55
GraphGPT
31.48
#1.52
17.62
+2.30
24.47
+2.06
15.16 +2.11
44.25
#1.58
43.51
$2.69
MARK
55.42
+2.60
44.70
#2.50
72.35
42.13
69.32
42.67
GMAgent
78.72*
10.91
59.30*
10.85
85.97*
10.89
85.61*
11.02
74.36*
11.03
66.82*
11.2
Table 3: Experimental results (%) the Products dataset for node classification, where denotes significant. improvement according to the Wilcoxon signed-rank test (Woolson. 2007). Total runtime is measured on 50,000 randomly sampled test instances.
Dataset
Products
Method
GCN
GAT
Qwen2-7B
LLaMA3-8B
GMAgent
Micro-F1
76.38 10.26
73.18
20.38
58.02 12.30
44.23 42.53
11.12 44.27*
81.65*
Macro-F1
38.07 +0.69
35.06
=0.61
38.81 #2.15
32.57 +2.69
+1.08
Total Runtime
4min49s
23min25s
340h
367h
81h 16.7GB
Inference Memory Usage
15.3GB
23.1GB
15.8GB
16.6GB
4.2.1 Comparison with GNN-based Methods
Generally, our proposed GMAgent consistently out- performs GNN-based methods across all tasks and datasets, showcasing its precise understanding of graph data (shown in Table 2 Table 3 and Fig- ure GMAgent achieves significant performance gains in node classification and link prediction with an average of 7.63% and 3.58%, respectively. Notably, our framework achieves an average improvement of 11.84% over standard GCN and GAT metrics. While approaches like RevGNN excel in node classification and HGT in link prediction, both struggle to fully utilize the rich altributed lext in TAGs for challeng- ing scenarios. In contrast, GMAgent capitalizes on the LLMs' ability to understand semantic content, boosting performance in challenging scenarios where traditional GNN-based models fall short..
Qwen2-7B
GPT-4
GraphiGPT
HINormer GAT
70
70
60
ACC
20
10.
GNNs
LLMs
GMAgenr
GNNs
L.Ms
GMAgent
(a) PubMed
(b)
Figure 4: ACC results (%) on the PubMed and DBLP datascts for link prediction.
12

---

Published in Transactions on Machine Learning Research (10/2025)
Conflict Ratio (%)
15
Micro-Fi (%)
78
10
76
74
TAPE
TAPE
72
Grap
GraphSAGE
GCN
GCN
GAT
GAT
GAT
GAT
GraphSAGE
GraphSAGE
TAPE GCN
TAPE GCN
(a) Conflict Ratio
(b) Micro-F1
Figure 5: Influence of developing different GNNs as graph agents in our GMAgent on the Arxiv dataset.
4.2.2 Comparison with LLM-based Methods
As illustrated in Table Table and Figure GMAgent also surpasses all LLM-based methods with significant improvements various tasks and datasets, highlighting GMAgent 's superior capabilities in TAG analysis. Compared to Qwen2-7B, GMAgent achieves an overall average improvement of 179.20 20% in node classification and link prediction. Despite GPT-4o's robust generalization abilitics based on extensive parameter sets, it faces difficulties when dealing with complex graph structures and struggles with fine-tuning on unfamiliar datasets. By considering both efficiency and cost, GMAgent utilizes tne-tuned Qwen and LLaMA series models, yet surpassos GPT-4o with both GNNs' global structural insights and the LLMs' semantic understanding. Notably, MARK relies on graph clustering identify uncertain nodes for LLM-basod multi-agent collaboration, which limits its applicability to tasks such multi-label node classification (e.g., IMDB) link prediction (e.g., PubMed). Therefore, mark " in Table 2 for IMDB. In contrast, GMAgent offers a flexible multi-agent collaboration framework that effectively integrates the global structural learning abilities of GNNs and the local semantic richness of LLMs, enabling broad adaptability across diverse graph analytical tasks.
4.2.3 Evaluation on Large-scale Graphs
To assess the scalability and resource demands of GMAgent on large-scale graphs, we conduct experiments Products, which is an undirected and unweighted Amazon co-purchasing network with 2.45M nodes and 61.86M edges. As shown in Table g GMAgent achioves the best Micro-F1 and Macro-F1, while maintaining reasonable runtime (81h) and inference memory usage (16. (IB.7GB). Compared to LLM-based baselines such as Qwen2-7B (340h, 15.8GB) and LLaMA3-8B (367h, 16.6GB), GMAgent efficiently reduces total runtime by over 75% and significantly improves overall performance (achieving up an 84.60% gain in Micro-Fl over LLaMA3-8B). Moreover, its memory usage remains comparable to GNN-based methods (c.g., 16.7GB for GMAgent vs. 23.1GB for GAT). These results demonstrate that GMAgent scales well even on million-node graphs, achieving a practical balance between predictive accuracy and computational efficiency. They also highlight its potential for practical deployment in real-world industrial applications.
4.3 Varying GNN-based Graph Agents (RQ2)
Figure illustrates how different combinations of GNN-based graph agents (GCN, GAT, GraphSAGE, and TAPE (GCN)) affect the conflict. ratio and Miero-F1 on the Arxiv dataset. within GMAgent.
In general, we observe that integrating any GNN agent combinations leads to a certain proportion of conflict. scenarios, with an average conflict ratio of 13. 13.29% across all combinations and range of 11.61%-15.96%. This indicates that the conflict evaluation mechanism (Section 3.2.2 filters most consistent predictions final
13

---

Published in Transactions on Machine Learning Research (10/2025)
Table 4: Influence of different LLM-based graph agents.
Dataset
Arxiv
Cora
Metric
Micro-F1
Macro-F1
Micro-F1
Macro-Fi
LLaMA3-8B
73.84
54.68
80.51
79.64
Qwen2-7B
75.53
55.09
81.29
80.92
LLaMA3-8BPT
77.19
58.45
83.47
82.85
Qwen2-7BPT
78.72
59.30
85.97
85.61
86
GMAgent
GMAgent
78
GCN
85
GCN
Micro-F1 (%)
GAT
Micro-FI (%)
76
84
83
82
72
81
2
2
# LLM-basaa Graph Agents
# LLM-bused Graph Agents
(a) Arxiv
(b) Cora
Figure 6: Influence the number LLM-based graph agents.
oulputs, alleviating the frequent call to costly LLMs. Resolving the remaining conflicts via LLM collaboration yields potential improvements in predictive accuracy, with an average Micro-F1 improvement 6.70% over the corresponding single-GNN GNN basclincs.
Furthermore, we observe that performance varies across different combinations of GNN agents. For example, GCN+GAT achieves the highest Micro-Fl with the lowest conflict ratio, which aligns with their complementary structural focuses: GCN models global topology and GAT emphasizes node-level attention. In contrast, combinations that involve TAPE (e.g., GCN+TAPE) exhibit higher conflict ratios but only marginal gains on Arxiv. A likely reason the misalignment between TAPE's I.I.M-derieed semantic embeddings and the structure-oriented outputs of other GNNs, which increases disagreement without consistently improving accuracy. These observations suggest that selecting structurally complementary yet inference-aligned agents (e.g., GCN+GAT) is more effective, while semantically divergent agents should be introduced selectively and evaluated per dataset. Additional results on Cora are provided in Appendix A.4
4.4 Varying LLM-based Graph Agents (RQ3)
Table shows the influence of different LLM-based graph agents on two datasets, where fine-tuning significantly improves performance across all models. Qwen2-7Brr, in particular, consistently outperforms other models both datasets, achieving the highest values for both Micro-Fl and Macro-F1 with average 2.44% improvement over the second-best foundation model LLaMA3-8BFr.
ingly, the fine-tuned L.IaMA-8BB also shows considerable improvement compared to its base version, though it still lags behind Qwen2-7Brr These results suggest that fine-tuning plays a critical role in enhancing the performance of LLM-based agents in graph analytical tasks.
4.5 Impact of Varying Graph- Graph-oriented Multi-agent Collaboration Strategies (RQ4)
4.5.1 Varying Number of Agents
As shown in Figure d increasing the number of LLM-based graph agents consistently improves performance across both Arxiv and Cora datasets. However, adding agents requires additional prompt design and
14

---

Published in Transactions on Machine Learning Research (10/2025)
Table 5: Influence of different summary agents.
Dataset
Arxiv
Cora
Metric
Micro-F1
Macro-F1
Micro-F1
Macro-F1
LLaMA3-8B
72.54
49.36
81.14
79.43
Qwen2-7B
74.80
52.89
83.86
81.52
GPT-4o
78.72
59.30
85.97
85.61
79
Avg. Inference Time (s)
86
Avg. Inference Time (s)
15
Micro-F1 (%)
78
10
Micro-FI
84
77
83
76
82
2
3
4
5
2
3
4
5
(a) Arxiv
(b) Cora
Figure 7: Inftuence of the maximum iteration count.
increases computational costs, as more interactions and coordination between agents are needed. To balance between accuracy and computational efficiency, an optimal choice of agent number 4.
4.5.2 Varying Summary Agents
Table B illustrates the influence of different LLM-based summary agents on performance. GPT-4o consistently outperforms LIaMA3-SB and Qwen2-7B, particularly on the text-heavy Arxiv dataset. This can be attributed GPT-44''s superior capability in handling complex language and generating detailed, coherent summaries. The role of the summary agent is crucial in our proposed GMAgent framework as it consolidates insights from multiple graph agents, guiding the collaborative process toward consensus. By providing global overview and prioritizing key insights, GPT-4o effectively enhances the decision-making process within the multi-agent collaboration, contributing to more accurate final predictions.
4.5.3 Varying Maximum Iteration Count
As shown in Figure we observe that moderately increasing the maximum iteration count Tmaz from 1 to 2 can improve accuracy, as additional self-reftection rounds enable LLM-based graph agents to better refine and consolidate their predictions. However, beyond Tmaz = 2, the performance improvement declines, while the inference time continues increase. This indicates that excessive iterations may lead redundant reasoning and reduce the effectiveness of prediction refinement. Therefore, we adopt Tmaz = 2 a practical trade-off, balancing predictive accuracy and computational efficiency while ensuring sufficient iterative refinement.
4.5.4 Varying Self-reflection Strategies
To study the offectiveness of different strategics during the multi-agent collaboration process, we compare three variants of our proposed method in the self-reflection process. We study GMAgent as follows:
GMAgent w/o. R represents GMAgent without using the summary report in the self-reflection process.
GMAgent w/o. Rot represents GMAgent without using other agents' analyses in the self-reflcction process.
GMAgent w/o. Rey represents GMAgent without using prior self-analysis in the self-reflection process.
15

---

Published in Transactions on Machine Learning Research (10/2025)
Table 6: Influence of different self-reflectinn strategies.
Dataset
Arxiv
Cora
Metric
Micro-Fi Fl
Macro-FT
Micro-F1
Macro-FT
GMAgent
78.72
59.30
85.97
85.61
w/o. Ri
70.03
49.47
79.14
78.65
w Rox
74.54
54.91
83.05
81.89
w/o. Prr
76.81
57.25
84.62
83.24
GCN GAT
Qwen2-7B LLaMA3-8B
GMAgent
90
90
10GB
17GB
0.5GB
17GB
Micro-FI (%)
70-
Micro-F1 (%)
70
0.3GB
3GB
16GB
50-
50-
16GB
30
30
17GB
17GB
10
10
1min 5min
100h
350h
2s
10s
30min 5h
Total Runtime
Total Runtime
(a) Arxiv
(b) Cora
Figure 8: Comparison of different methods on Arxiv and Cora terms of Micro-F1 (y-axis), total runtime (x-axis), and memory usage (dotted circles). Shapes denote method types (O GNNs, A LLMs, t GMAgent).
As shown in Table 6 removing the summary report (GMAgent w/o. Rre) causes the most significant drop in performance, emphasizing the crucial role the report plays in guiding the self-reflection process. The summary report prompts agents to reconsider their earlier analyses based on consolidated feedback, facilitating deeper reflection and improvement in subsequent iterations. Without the summary report, the graph agents lack a clear direction for refinement, resulting in less effective self-reflectinn. Moreover, removing the analyses from other agents (GMAgent w/o. Rox) or ignoring prior self-analysis (GMAgent w/o. Rp,) also leads performance degradation, further underscoring the importance of collaborative feedback and the iterative self-reflection mcchanism within the multi-agent framework. We provide simplified scenario to illustrate the effectiveness of our collaborative self-reflection mechanism in Appendix A.6
4.6 Efficiency Analysis (RQ5)
As shown in Figure g GMAgent consistently achieves favorable trade-off between predictive accuracy and computational efficiency across both datasets. Compared with LLM-based baselines (e.g., Qwen2-7B), GMAgent reduces total runtime by 74.85% Arxiv and 90.27% Cora, while achieving the best predictive accuracy and maintaining comparablc memory usage. This substantial efficiency gain stems from our conflict scenario evaluation mechanism (cf., Section 3.2.2,, where the majority of instances can be accurately handled by strong GNN experts without invoking expensive LLM-based inferences. Additionally, despite a moderate increase runtime and memory usage, GMAgent significantly outperforms GNN-based baselines (c.g., GCN and GAT) in predictive performance and provides superior interpretability and transparent reasoning via its multi-agent collaboration mechanism.
16

---

Published in Transactions on Machine Learning Research (10/2025)
5 Conclusion and Future Work
In this paper, we introduce GMAgent, an effective and flexible graph-oriented multi-agent collaboration framework for text-attributed graph analysis. By enabling seamless interactions between diverse GNN-based and LLM-based graph agents, GMAgent integrates the global structural learning power GNNs and the local semantic richness of LLMs to enhance graph analytical tasks. Specifically, GMAgent includes innovative deploying GNNs as graph agents, repurposing LLMs as graph agents, and enabling graph-oriented multi-agent. collaboration among these graph agents. Extensive experiments on five datasets demonstrate GMAgent's superior performance over the state-of-the-art baselines, improving not only the comprehension of graph data but also the accuracy and interpretability of graph analytical tasks.
Looking ahead, future work could improve GMAgent's capabilities to tackle complex and conflicting scenarios in graph analysis. This includes creating better selection mechanisms (e.g,, learned selectors or threshold-bed based confidence aggregation) to identify conflicting outputs, designing more effective graph-to-text transformation strategies to balance redundancy and information preservation, and adding a wider range of agents for decision-making. Moreover, it is also interesting to explore real-time agent collaboration strategies, where agents adjust roles based on problem complexity, thereby enhancing efficiency and adaptability.
Acknowledgments
This work was supported in part by the Fujian Provineial Artificial Intelligence Industry Development Technology Project under Grant (2025H0042), Fujian Provincial Natural Science Foundation of China under Grants (2025J01540), National Natural Science Foundation of China under Grants (No.62302098.. Carl Yang was not supported by any fund from China.
References
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, llge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. In arXiv preprint ar NKW::230300877, 2023.
William Brannon, Wonjune Kang, Suyash Fulay, Hang Jiang, Brandon Roy, Deb Roy, and Jad Kabbara. Congrat: Self-supervised contrastive pretraining for joint graph and text embeddings. In Proceedings of TextGmphs-11 17: Graph-bused Methods for Natural Language Processing, pp. 19 39, 2024.
Yukuo Cen, Zhenyu Hou, Yan Wang, Qibin Chen, Yizhen Luo, Zhongming Yu, Hengrui Zhang, Xingcheng Yao, Aohan Zeng, Shiguang Guo, et al. Cogdl: A comprehensive library for graph deep learning. In Proceedings of the ACM Web Conference 2029, pp. 747-758, 2023.
Chi-Min Chan, Weize Chen, Yusheng Su, Jianxuan Yu, Wei Xue, Shanghang Zhang, Jie Fu, and Zhiyuan Liu. Chateval: Towards better blseb evaluators through multi-agent debate. In The Twelfth International Conference on Learning Representations, 2024.
Feng Chen, Xinwei Chen, Rong-Jun Qin, Cong Guan, Lei Yuan, Zongzhang Zhang, and Yang Yu. Efficient multi-agent cooperation learning through teammate lookahead. Transactions on Machine Learning Research, 2025. ISSN 2835-8856.
Lingjiao Chen, Jared Quincy Davis, Boris Hanin, Peter Bailis, Ion Stoica, Matei A Zaharia, and James Y Zou. Are more 1lm calls all you need? towards the scaling properties of compound ai systems. Advances in Neural Information Processing Systems, 37:45767 45790, 2024.
Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang, Chenfei Yuan, Chi-Min Chan, Heyang Yu, Yaxi Lu, Yi-Hssi Hung, Chen Qian, et al. Agentverse: Facilitating multi-agent collaboration and exploring emergent. behaviors. In The Twelfth International Conference on Learning Representations, 2023.
Justin Chih-Yao Chen, Archiki Prasad, Swarnadoep Saha, Elias Stengel-Eskin, and Mohit Bansal. Magicore: Multi-agent, iterative, coarse-to-fine refinement for reasoning. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 2025.
17

---

Published in Transactions on Machine Learning Research (10/2025)
Yubo Dong, Xukun Zhu, Zhengzhe Pan, Linchao Zhu, and Yi Yang. Villageragent: A Sraph-basedl multi-agent framework for coordinating complex task dependencies in minecraft. In Findings of the Associalion for Computational Linguistics ACL 2024, pp. 16290 16314, 2024.
Wei Duan, Jie Lu, and Junyu Xuan. Group-aware coordination graph for multi-agent reinforcement learning.
In Proceedings of the 33th International Joint Conference on Artificial Intelligence, 2024.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd models. In Proceedings of the 41st International Conference on Machine Leurning, 2024.
Zhihao Fan, Lai Wei, Jialong Tang, Wei Chen, Wang Siyuan, Zhongyu Wei, and Fei Huang, Ai hospital: Benchmarking large language models in a multi-agent medical interaction simulator. In Proceedings of the 31st International Conference on Compulational Lünguistics, pp. 10183 10213, 2025.
Yi Fang, Dongzhe Fan, Daochen Zha, and Qiaoyu Tan. Gaugllm: Improving graph contrastive learning for text-attributed graphs with large language models. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 747-758, 2024.
Yi Fang, Dongzhe Fan; Sirui Ding, Ninghao Liu, and Qiaoyu Tan. Uniglm: Training one unified language model for text-attributed graphs embedding. In Proceedings of the Eighteenth ACM International Conference on Web Search and Dala Mining, pp. 973 981, 2025.
Xidong Feng, Yicheng Luo, Ziyan Wang, Hongrui Tang, Mengyue Yang, Kun Shao, David Mguni, Yali Du, and Jun Wang. Chessgpt: Bridging policy learning and language modeling. Advances in Nearal Information Processing Systems, 36, 2024.
Yiwei Fu, Yuxing Zhang, Chunchun Chen, JianwenMa JianwenMa, Quan Yuan, Rong- Rong-Cheng Tu, Xinli Huang, Wei Ye, Xiao Luo, and Minghua Deng. Mark: Multi-agent collaboration with ranking guidance for trib-atd graph clustering. In Findings of the Associalion for Compulational Linguistios: ACL 2025, pp. 6057-6072, 2025.
Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in Neural Information Processing Systems, 30, 2017.
Xiaoxin He, Xavier Bresson, Thomas Laurent, Adam Perold, Yann LeCun, and Bryan Hooi. Harnessing explanations: Llm-to-lm interpreter for enhanced text-attributed graph representation lcarning. In Proceedings of the 12th International Conference on Learning Representalions, 2024.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In Proceedings of the 10th International Conference on Learning Representations, 2022.
Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovee. Open graph benchmark: Datascts for machine learning on graphs. Advances in Newral Information Processing Systems, 33, 2020a.
Yuwei Hu, Runlin Lei, Xinyi Huang, Zhewei Wei, and Yongchao Liu. Scalable and accurate graph reasoning with Ilm-based multi-agents. Xi preprint arXiv:2, 10. 0..05130 2024.
Ziniu Hu, Yuxiao Dong, Kuansan Wang, and Yizhou Sun. Heterogeneous graph transformer. In Proceedings of the ACM Web Conference 2020, pp. 2704 2710, 2020b.
Jin Huang, Xingjian Zhang, Qiaozhu Mei, and Jiaqi Ma. Can LLMs effectively leverage graph structural information through prompts, and why? Transactions on Machine Learning Research, 2024a. ISSN 2835-8856.
Xuanwen Huang, Kaiqiao Han, Yang Yang, Dezheng Bao, Quanjin Tao, Ziwei Chai, and Qi Zhu. Can gun be good adapter for lmss? In Proceedings of the ACM on Web Conference 2024, pp. 893 904, 2024b.
18

---

Published in Transactions on Machine Learning Research (10/2025)
Zijie Huang, Jechyun Hwang, Junkai Zhang, Jinwoo Baik, Weitong Zhang, Dominik Wodarz, Yizhou Sun, Quanquan Gu, and Wei Wang. Causal graph ode: Continuous treatment effect modeling in multi-agent, dynamical systems. In Proceedings of the ACM on Web Conference 2024, pp.
2024c.
Md Ashraful Islam, Mohammed Eunus Ali, and Md Rizwan Parvez. Mapcoder: Multi-agent code generation for competitive problem solving. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 4912-499 112--944, 2024.
Sergey Ivanov and Evgeny Burnaev. Anonymous walk embeddings. In Proceedings of the 35th International Conference on Machine Learning, pp. 2186-2195, 2018.
Bowen Jin, Gang Tiu, Chi Han, Meng Jiang, Heng Ji, and Jiawei Han. Large language models on graphs: A comprehensive survey. IEEE Transactions on Knowledge and Data Engineering, 2024.
Thomas N Kipf and Max Welling, Semi-supervised classification with graph convolutional networks. In Proceedings of the 5th International Conference on Learning Representations, 2017.
Guohao Li, Matthias Müller, Bernard Ghanem, and Vladlen Koltun. Training graph neural networks with 1000 layers. In Proceedings of the 38th International Conference on Machine Learning, pp. 6437-6449. PMLR, 2021a.
Jianxin Li, Hao Peng, Yuwei Cao, Yingtong Dou, Hekai Zhang, S Yu Philip, and Lifang He. Higher-order attribute-enhancing heterogeneous graph neural networks. IEEE Transactions on Knowledge and Data Engineering, 35(1):560 574, 2021b.
Yichuan Li, Kaize Ding, and Kyumin Lee. Grenade: Graph-centric language model for self-supervised representation learning on text-attributed graphs. In 2023 Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 2745-222222 2757. Association for Computational Linguistics (ACT.), 2023.
Yuhan Li, Zhixun Li, Peisong Wang, Jia Li, Xiangguo Sun, Hong Cheng, and Jeffrey Xu Yu. A survey of graph moets large language model: Progress and future directions. In Proceedings of the 33th International Joint Conference on Artificial Intelligence, 2024.
Hao Liu, Jiarui Feng, Lecheng Kong, Ningyue Liang, Dacheng Tao, Yixin Chen, and Muhan Zhang, One for all: Towards training one graph model for all classification tasks. In Proceedings of the 12th International Conference on Learning Representations, 2024a.
Jie Liu, Yinmin Zhang, Chuming Li, Zhiyuan You, Zhanhui Zhou, Chao Yang, Yaodong Yang, Yu Liu, and Wanli Ouyang. MaskMA: Towards zero-shot multi-agent decision making with mask-based collaborative learning. Transactions on Machine Learning Research, 2024b. ISSN 2835-8856.
Tongxuan Liu, Xingyu Wang, Weizhe Huang, Wenjiang Xu, Yuting Zeng, Lei Jiang, Hailong Yang, and Jing Li. Groupdebate: Enhancing the efficiency of multi-agent debate using group discussion. arXie preprinl Nxii:2409.14655 2024c.
Li- Chun Lu, Shou-Jen Chen, Tsung-Min Pai, Chan-Hung Yu, Hung-yi Lee, and Shao-Hua Sun. Lim discussion: Enhancing the creativity of large language models via discussion framework and role-play. In Conference on Language Modeling, 2024.
Yuankai Luo, Lei Shi, and Xiao-Ming Wu. Classic gnns are strong basclines: Reassessing gnns for node classification. Advances in Neural Information Processing Systems, 37:97650- 97600999769999 2024.
Yuankai Luo, Lei Shi, and Xiao-Ming Wu. Can classic gnns be strong baselines for graph-level tasks? simple architectures meet excellence. In Forty-second International Conference on Machine Learning, 2025.
Qingsong Lv, Ming Ding, Qiang Liu, Yuxiang Chen, Wenzheng Feng, Siming He, Chang Zhou, Jianguo Jiang, Yuxiao Dong, and Jie Tang. Are we really making much progress? revisiting, benchmarking and refining heterogeneous graph neural networks. In Proceedings of the 27th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1150-1160, 2021.
19

---

Published in Transactions on Machine Learning Research (10/2025)
Qtheng Mao, Zemin Liu, Chenghao Liu, and Jianling Sun. Hinormer: Representation learning heterogeneous information networks with graph transformer. In Proceedings of the ACM Web Conference 2023, pp. 599-69997 610, 2023.
Trung-Kiee Nguyen, Zemin Liu, and Yuan Fang. Link prediction on latent heterogeneous graphs. In Proceedings of the ACM Web Conference 2028, pp. 263-273, 2023.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances Neural Information Processing Systems, 35, 2022.
Vishakh Padmakumar and He He. Does writing with language models reduce content diversity? In Proceedings of the 12th International Conference on Learning Representations, 2024.
Bo Pan, Zheng Zhang, Yifei Zhang, Yuntong Hu, and Liang Zhao. Distilling large language models for text-attributed graph learning. Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, Boise, ID, USA, October 21-25, 2024, 2024.
Giorgio Piatti, Zhijing Jin, Max Kleiman- Weiner, Bernhard Schölkopf, Mrinmaya Sachan, and Rada Mihalcea. Cooperate or collapse: Emergence of sustainable cooperation in a society of 1lm agents. Advonces Neural Information Processing Systems, 37:111715 111759, 2024.
Yijian Qin, Xin Wang, Ziwei Zhang, and Wenwu Zhu. Disentangled representation learning with large language models for text-attributed graphs. arXic preprint arXiv:2310.15 18152, 2023.
Xubin Ren, Jiabin Tang, Dawei Yin, Nitesh Chawla, and Chao Huang. A survey of large language models for graphs. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 6616-6626, 2024.
Claude Elwood Shannon. A mathematical theory of communication. The Bell system technicol journal, 27 (3):379-423) 1948.
Yanchao Tan, Zihao Zhou, Hang Lv, Weiming Liu, and Carl Yang. Walklm: A uniform language model fine-tuning framework for attributed graph embedding. Advances in Neural Information Processing Systems, 36, 2023.
Yanchao Tan, Hang Lv, Pengxiang Zhan, Shiping Wang, and Carl Yang. Graph-oriented instruction luning of large language models for generic graph mining. IEEE Transactions Pattern Analysis and Machine Intelligence, 2025.
Jiabin Tang, Yuhao Yang, Wei Wei, Lei Shi, Lixin Su, Suqi Cheng, Dawei Yin, and Chao Huang. Graphgpt: Graph instruction tuning for large language models. In Proceedings of the 4 7th ACM SIGIR International Conference on Research and Development in Information Retrieval, pp. 491-500, 2024a.
Xiangru Tang, Anni Zou, Zhuosheng Zhang, Ziming Li, Yilun Zhao, Xingyao Zhang, Arman Cohan, and Mark Gerstein. Medagents: Large language models as collaborators for zero-shot medical reasoning. In Findings of the Association for Computational Linguistics ACL 2024, pp. 599-621, 2024b.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pictro Liò, and Yoshua Bengio. Graph attention networks. In Proceedings of the 6th International Conference on Learning Representations, 2018.
Kuansan Wang, Zhihong Shen, Chiyuan Huang, Chieh-Han Wu, Yuxiao Dong, and Anshul Kanakia. Microsoft. academic graph: When experts are not enough. Quantitative Science Studies, 1(1):396 -413, 2020.
Qineng Wang, Zihao Wang, Ying Su, Hanghang Tong, and Yangqiu Song. Rethinking the bounds of Ilm
reasoning: Are multi-agent discussions the key? In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1106-6131 Association for Computational Linguistics, 2024a.
20

---

Published in Transactions on Machine Learning Research (10/2025)
Qinyong Wang, Zhenxiang Gao, and Rong Xu. Graph agent: Explicit reasoning agent for graphs. arXiv preprint Xiv:22330. 16421, 2023a.
Wei Wang, Dan Zhang, Tao Feng, Boyan Wang, and Jie Tang. baldleagentbenc: A benchmark for evaluating cooperation and competition capabilities of language models in multi-agent systems. arXiv preprint arXiv:2408.15071 2024b.
Yaoke Wang, Yun Zhu, Wenqiao Zhang, Yueting Zhuang, Liyunfei Liyunfei, and Siliang Tang, Bridging local details and global context in text-attributed graphs. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 14830-14841, 2024c.
Yizhong Wang, Hamish Tvison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Chandu, David Wadden, Kelsey MacMillan, Noah A Smith, Iz Beltagy, et al. How far can camels go? exploring the state of instruction tuning on open resources. Advances in Neural Information Processing Systems, 2023b.
Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language models with self- self-rener instructions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguisties, 2023c.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Fdl Chi, Quoc Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35, 2022.
Robert F Woolson. Wilcoxon signed-rank test. Wiley encyclopedia of clinical trials, pp. 1 3, 2007.
Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, et al. Autogen: Enabling next-gen Ilm applications via multi-agent conversations. In First Conference on Language Modeling, 2024a.
Xixi Wu, Yifei Shen, Caihua Shan, Kaitao Song, Siwei Wang, Bohang Zhang, Jiarui Feng, Hong Cheng, Wei Chen, Yun Xiong, and Dongsheng Li. Can graph learning improve planning in Ilm-based agents? In Proceedings of Neural Information Processing Systems, 2024b.
Aiyuan Yang, Bin Xiao, Bingning Wang, Borong Zhang, Ce Bian, Chao Yin, Chenxu Lv, Da Pan, Dian Wang, Dong Yan, et al. Baichuan 2: Open large-scale language models. arXiv preprint arXiv:2309. 10305, 2023.
An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. Qwen2 technical report. arXiv preprint Xrxi::2477 10671, 2024a.
Carl Yang, Yuxin Xiao, Yu Zhang, Yizhou Sun, and Jiawei Han. Heterogencous network representation learning: A unified framework with survey and benchmark. IEEE Transactions on Knowledge and Dala Engineering, 34(10):4854 4873, 2020a.
Carl Yang, Jieyu Zhang, Haonan Wang, Sha Li, Myungwan Kim, Matt Walker, Yiou Xiao, and Jiawei Han. Relation learning on social networks with multi-moda graph edge variational autoencoders. In Proceedings of the 13th International Conference on Web Search and Data Mining, pp. 699-707, 2020b.
Wenkai Yang, Xiaohan Bi, Yankai Lin, Sishuo Chen, Jie Zhou, and Xu Sun. Watch out for your agents! investigating backdoor threats to Ilmebased agents. Advances in Neural Information Processing Systems, 37:100938-100090 2024b.
Ruosong Ye, Caiqi Zhang, Runhui Wang, Shuyuan Xu, and Yongleng Zhang. Language is all a graph needs. In Findings of the Association for Computational Linguistics: EACL 2024, pp. 1955-1973, 2024.
Chun-Hsiao Yeh, Jiayun Wang, Andrew D Graham, Andrea J Liu, Bo Tan, Yubei Chen, Yi Ma, and Meng C Lin. Insight: A multi-modal diagnostic pipeline using 1ms for ocular surface discase diagnosis. In International Conference on Medical Image Computing and Computer-Assiss Intervention, pp. 711-721. Springer, 2024.
21

---

Published in Transactions on Machine Learning Research (10/2025)
Xianlin Zeng, Yufeng Wang, Yuqi Sun, Guodong Guo, Wenrui Ding, and Baochang Zhang. Graph structure refinement with energy-based contrastive learning. In Proceedings of the AAAT Conference on Artificial Intelligence, volume 39, pp. 22326 22335, 2025.
Yulun Zhang, He Jiang, Varun Bhatt, Stefanos Nikolaidis, and Jiaoyang Li. Guidance graph optimization for lifelong multi-agent path finding. In Proceedings of the Thirty Third International Joint Conference Arlificial Intelligence, pp. 311 320, 2024.
Jianan Zhao, Meng Qu, Chaozhuo Li, Hao Yan, Qian Liu, Rui Li, Xing Xie, and Jian Tang. Learning on large-scale text-attributed graphs via variational inference. In The Eleventh Indernational Conference Learning Representations, 2023.
Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, and Zheyan Luo. Llamafactory. Unified efficient finc-tuning of 100+ language models. In Proceedings of the 62nd Annual Meeting of the Association for Compulational Linguislics, 2024.
Jason Zhu, Yanling Cui, Yuming Liu, Hao Sun, Xue Li, Markus Pelger, Tianqi Yang, Liangjie Zhang, Ruofei Zhang, and Huasha Zhao. Textgnn: Improving text encoder via graph neural network in sponsored search. In Proceedings of the Web Conference 2021, pp. 2848 2857, 2021.
22

---

Published in Transactions on Machine Learning Research (10/2025)
A Appendix
A.1 Detailed Descriptions of Datasets
This section provides detailed descriptions of each graph dataset used in our experiment.
(1) Node classification: Node classification assigns the target node to predefined categories by utilizing the diverse relationships and attributes present in the graph.
ogbu-arxiv (abbr. Arxiv) represents directed graph that captures the citation network among computer science arXiv papers indexed by MAG (Wang et al. 2020). Each paper in the dataset is associated with a research category, manually labeled by the authors and arXiv moderators. These research categories are selected from a set of 40 subject arcas.
Cora comprises 2,708 scientific publications classified into one of seven classes case-based, genetic algorithms, neural networks, probabilistic methods, reinforcement learning, rule learning, and theory, with citation network consisting of 5,429 links.
TMDB is a website about movies and related information, including a subset from the Action, Comedy, Drama, Romance, and Thriller genres. Each labeled movie has one or multiple labels.
ogbn-products (abbr. Products) is undirected and unweighted Amazon co-purchasing network with 2.45M nodes and 61.86M edges, where each product is assigned to one of 47 categories for multi-class node classification task.
(2) Link prediction: Link prediction predicts the likelihood of a future or missing connection between nodes in graph.
PubMed contains a graph of genes, discases, chemicals, and species. It performs word2vec computations on all PubMed papers and aggregates the word embeddings generate 200-dimensional features for each type of node.
DBLP includes substantial collection of papers the web, authors, conferences, and terms, providing comprehensive dataset. The target nodes, representing authors, are categorized into four rescarch areas: database, data mining, machine learning, and information retrieval.
A.2 Detailed Descriptions of Baselines
The following characteristie baseline methods can be classified into two categories: (1) GNN-based methods and (2) LLM-based methods.
(1) GNN-based methods:
GCN Kiipf & Welling, 2017) scales linearly in the number of graph edges and learns hidden layer representations that encode both local graph structure and features of nodes.
GAT (Veličković et al. 2018 utilizes masked self-attention mechanisms to enhance the processing of graph data by addressing limitations in traditional graph convolution methods.
RevGNN (Li et al. 2021a) captures long-range interactions in graph data and reduces memory complexity with grouped reversible connections, enabling more effective training of deep and wide GNNs.
GraphSAGI AGE (Hamilton et al. 2017) generates node embeddings by sampling and aggregating features from a node's local neighborhood, enabling scalable learning on large graphs.
HGT (Hu al. 2020b) extends the transformer architecture to handle heterogencous graphs to capture diverse node and edge interactions.
23

---

Published in Transactions on Machine Learning Research (10/2025)
2023 uses graph transformers to learn node representations on heterogeneous information networks by capturing both local structure and heterogeneity.
HINormer (Mao et al.
TAPE (He et al. 2024 leverages LLMs' explanations to generate informative node features for Lext- attributed graphs, boosting the performance of various GNNs.
(2) LLM-based methods:
Baichuan2-7B-BB (abbr. Baichuan2-7B) (Yang et al., 2023 is an open-source, bilingual language model developed by Baichuan Inc., trained on 2.6 trillion tokens with billion parameters.
InnstucB--nstruct (abbr. Qwen2-7B) Yang et al. model, designed to excel in tasks like language understanding, generation, and more, with support for processing up to 131,072 tokens in context.
2024a is an instruction-tuned billion parameter
LLaMA3-8B (Dubey et al. 2024 succeeds LLaMA2, offering improved performance with 8 billion parameters through advancements in archilecture, training data, and optimization.
GPT-3.5 (Ouyang et al. parameters, capable of generating iuman-like text and understanding complex contexts.
2022 is large-scale language model developed by OpenAl with 175 billion
2023 builds upon GPT-3.5, providing advanced language generation and under- standing capabilities with greater scale and improved performance.
GPT-4 (Achiam
GraphGPT (Tang el al. 2024a uses LLM as backbone and integrates LLMs with graph knowledge using a graph structural instruction tuning paradigm, enhancing understanding through gext-graph grounding and step-by-step reasoning.
MARK Fu et al. 2025 multi-agent framework that enhances text-attributed graph clustering by leveraging three LLM-bbsed agents provide ranking- supervision signals for refining uncertain nodes near cluster boundaries.
A.3 Overall Performance on Link Prediction
In this subsection, we comprehensively analyze the performance of proposed GMAgent framework link prediction tasks across PubMed and DBLP datasets, comparing it with state-of-the-ard base- lines. Overall, our proposed GMAgent consistently demonstrates superior performance, which can be attributed to its effective integration of GNNs for capturing global structural information and LLMs for interpreting complex textual attributes. As shown in Figure and Figure g. our GMAgent outperforms all baseline models in both ACC and AUC met rics. By efficiently assigning simpler tasks to GNNs and utilizing LLMs to handle more complex, text-heavy scenarios, GMAgent achieves optimal resource alloca- tion and high accuracy across both datasets.
HINormer
GAT
Qwea2-7B
GPT-4
GraphGPP
HINormer
80
50
1 LLMs
6
10
30
GNNS
(a) PubMed
(b) DBLP
Figure 9: AUC results (%) on the PubMed and DBLP datasets for link prediction.
A.3.1 Comparison with GNN-based Methods
Generally, our proposed GMAgent consistently surpasses GNN-based methods on link prediction tasks across PubMed and DBLP datasets, demonstrating its precise understanding of graph structures (shown in Figure g and Figure Specifically, GMAgent (HINormer GAT) yields a significant performance improvement, with an average gain of 5.22%. Moreover, GMAgent (GCN + GAT) and GMAgent (HINormer + GAT) achieve average improvement of 6.03% and 6. 6.18 over their corresponding standard models, respectively.
24

---

Published in Transactions on Machine Learning Research (10/2025)
Conflict Ratio (%)
Micro-FI (%))
15
86
10
5
TAPE
TAPE
82
Prr
GraphSAGE
GCN
GCN
GAT
GAT
GAT
GAT
GraphSAGE
GraphSAGE
TAPE GCN
TAPE GCN
(a) Conflict Ratio
(b) Micro-F1
Figure 10: Influence of developing different GNNs as graph agents in our GMAgent on the Cora dataset.
This showcases the accuracy of our graph-oriented multi-agent collaboration framework for TAG analysis. Traditional approaches, like HGT, struggle to fully utilize the rich text attributes of TAGs, particularly in more complex scenarios. In contrast, GMAgent fully harnesscs the LLMs' ability to comprehend semantic content, boosting performance in challenging scenarios where the GNN-based models fall short.
A.3.2 Comparison with LLM-based Methods
As illustrated in Figure and Figure g GMAgent also outperforms all LLM-based methods with significant improvements on link prediction tasks across both datasets, highlighting GMAgent 's superior capabilities in TAG analysis. Compared to Qwen2-7B, CMAgent (HINormer + GAT) achieves overall average improvement of 446.43% in link prediction, indicating the inherent limitation for general LLMs of understanding complex graph structurcs. Despite GPT-4o's robust generalization abilitics based on extensive paramcter sets, it faces difficulties when dealing with complex graph structures and struggles with fine-Luning on unfamiliar datasets. By considering both efficiency and cost, GMAgent utilizes fine-tuned Qwen and LLaMA models, yet surpasses GPT-4o with both GNNs global structural insights and the LLMs' semantic understanding.
A.4 Impact of Varying baseb Graph Agents on the Cora Dataset
Figure 10 illustrates how different combinations of GNN-based graph agents (GCN, GAT, GraphSAGE, and TAPE (GCN)) affect the proportion of conflict scenarios (i.e., conflict. ratio) and Micro-F1 on the Arxiv dataset within GMAgent.
In general, observe that integrating any GNN agent combinations leads to certain proportion of conflict scenarios, with an average conflict ratio of 11.62% across all combinations and range of 4.42%-19.00 This indicates that the conflict evaluation mechanism (Section 3.2.2p filters most consistent predictions as final outputs, alleviating the frequent call to costly LLMs. Resolving the remaining conflicts via LLM collaboration yields potential improvements in predictive accuracy.
On Cora, performance also varies notably across GNN agent combinations. Single TAPE, which enriches GCN backbone with semantic node attributes, achieves the highest standalone accuracy. This suggests that in datasets with relatively simple and homogeneous graph structures, single strong GNN agent may already capture most of the task-relevant information, thereby performing accurate predictions. In such cases, incorporating additional agents can lead to redundant conflicts and even degrade overall performance. This finding reflects the dataset-specific advantage of TAPE rather than a limitation GMAgent. On more complex datasots, like Arxiv, intograting TPAE with GCN/GAT with the help of our GMAgent consistently yields better performance than using TAPE alone (cf., Section 4.3). Overall, the conflict evaluation mechanism benefits from structurally complementary and inference-aligned pairings (e.g., GCN+GAT) complex
25

---

Published in Transactions on Machine Learning Research (10/2025)
Table 7: Influence of different graph description strategies.
Dataset
Arxiv
Cora
Metric
Micro-F1
Macro-F1
Micro-F1
Macro-FT
Base
75.53
55.09
81.29
80.92
+ Ego Graph Node
76.21
56.85
82.34
82.01
+ One-hop Neighbour
77.46
58.02
84.15
83.74
GMAgent
78.72
59.30
85.97
85.61
GAT Answer:
Answer: cs.DS
GCN Agent
Round ()
Experts Analyses
Summary Report
Round /
Collaborative Self- Reflection
Hop Neighbors Expert:
Confidence Analyses:
One-Hop Neighbors Expert:
Answer: [es. DS, es. ...::
One-Hop Neighbors Expert: 5 (Strong confidence)
Answer: Des.D, cs. ...::
Reasoning: Its one-hop neighbors, mullt-resolttion hashing and memory-efficient sketches emphasize essential data structures, illustrate importance officient algorithms....
works
Centrality Expert: (Moderate confidence).
Reasoming: the paper's references locus on "datu structures". "algoritha design" dealing with "sketching algorithms" ... Given that the neighbors development of efficient systems related computational methods
representation and
Key Insights:
"Kemel Densiy Esstimation" foundational method in machine learning for "sketching alsorithm"" that compresses high-dimensional data making
Centrality Expert:
Centrality Expert:
Answer: [exSS, Ds.
"hiih-dimemisionll streaming data" focuses on handling......
Answer: [es_D8, cs.
Ressening: Given the locus Tbe most likely category cs.S.. This is further reinforeed by the "information propagation". and applications social networks throughout the paper.
Ressoning: es.D encompusses "duta processing". "analysis", "modeling". which closely aligns
frequent references
Global Summary:
The problem of how to perform efficient density estimation in "high streaming data"
paper focuses on "kemel density cstimation" and its connections efficient "high-dineensionll streaming data" make ... the paper's 's connections esscurial sketching hashing, ridge regression.
CONS, 1.27
Ground-truth es.
cONs; 1.10
Ground-truth: es.DS
Figure 11: A node classification conflict scenario Arxiv to illustrate the effectiveness of our collaborative self-reflection mechanism in GMAgent.
graphs, while semantically divergent agents (e.g., TAPE) require more selective integration based on graph structure and attribute richness.
A.5 Impact of Different Graph Description Variants
We evaluate the effect of various graph description strategies for LLM- agents the Arxiv and Cora datascts as follows: (1) "Basc" directly utilizes vanilla Qwen2-7B as graph agents without any graph-aware fine-tuning; (2) *+ Ego Graph Node" fine-tunes the LLM using only the central node attributes, providing basic node-level semantics; (3) "+ One-hop Neighbour" further integrates one-hop neighbors, enriching the local structural context; (4) GMAgent combines one-hop neighbors with random walks (cf., Scction 3.3.1] encode both local and global structural semantics, enabling a more comprehensive graph understanding. As shown in Table 7 model performance consistently improves with the integration of richer structural signals. Notably, our GMAgent setup achieves the best performance across all metrics on both datasets, demonstrating the effectiveness of combining local neighborhood information with global path-based context in the textualization process. The abundant textual graph descriptions provide diverse and complementary semantics, which enable LLMs to significantly comprehend and analyze graph structures.
A.6 Case Studies
To assess the effectiveness of GMAgent's collaborative self-reflection mechanism in enhancing LLM graph agents' understanding of graph data and executing graph analytical lasks, we provide a node classification conflict scenario on the Arxiv dataset. Figure [I shows the distinct responses the One-Hop Neighbors Expert and Centrality Expert at different rounds of collaborative self-reflection, and the summary report generated by GPT-4o. In the first iteration, the One-Hop Neighbors Expert tends to predict cs. influenced by neighboring nodes associated with
hashing for fast pnirwise summatioos". Tn contrast,
26

---

Published in Transactions on Machine Learning Research (10/2025)
the Centrality Expert, which focuses centrality metrics (e.g., degree and closeness), suggests that es.SI is more appropriate category. Nevertheless, accurately identifying the ground-truth label (i.e., cs.DS) remains challenging due to the existing conflict scenario within different agent analyses.
Additionally, GPT-4o assigns a confidence score from 1 (Poor Confidence) to 5 (Strong Confidence) for each LLM expert based on their analysis (for instance, assigning a score of to the One-Hop Neighbors Expert). Acting as summary agent, GPT-440 extracts key insights from the various LLM graph agents' analyses, providing a global overview tailored to this conflict scenario. By integrating this summary report with the GNN graph agents' answer candidates, our GMAgent stabilizes the answer distribution of all graph agents during the collaborative self-reflection phase, thereby improving the prioritization of the ground-truth label. Particularly, guided by the summary report and insights from other agents, the Centrality Expert is able to identify the correct answer. These results strongly support the effectiveness of GMAgent, demonstrating that our collaborative self-reflcction mechanism enables the LLM to focus on crucial information and generate accurate analyscs, espocially in complex graph structures with rich semantic content.
A.7 Full Prompt Template
we provide the prompt templates used in repurposing LLMs as graph agents (cf., Section 3.3 and graph-oriented multi-agent collaboration (cf., Section 3.4.
As shown in Table and Table g
Table 8: Prompt templates used for repurposing LLMs as graph agents on the Arxiv dataset.
Type
Prompt
Task- specific Instruction
Given the target PAPER with the effective graph description in the Arxiv dataset, which of the following categories does this PAPER belong to: { {Category List}. Directly provide the answer to this PAPER's categories. Ana The effective graph description of this PAPER is listed as follows: Title: {iille of PAPER) Abstract: {Abstract of PAPER] Ego Graph Node: {Ego Gruph Node Lisl} One-hop Neighbor: {One-hop Neighbor List}
Input:
Random Walk: {Random Walk Paths}.
Output: (Ground-truth Category).
Querying GPT-4o
(Taak-speciftt Instruction Input) and the answer is { Task- specific Instruction Output). Imagine that you have made the correct choice and proceed with step-by-step reasoning. Your reason needs to incorporate Ego Graph Node, One-hop Neighbor, and Random Walk in the given effective graph description.
I have a question as below:
CoT-based Instruction
Input: Given the target PAPER with the effective graph description in the Arxiv dataset, which of the following categories does this PAPER belong to: {Category List). Please think about the calegorization in step-by-step manner and avoid making false associations. Then provide your reason. Using the following format: Answer: [Answer]; Reason: [Reason). An A The effective graph description of this PAPER is listed as follows: Title: { Tillle of PA PAPRR Abstract: (Abstruct of PAPER) Ego Graph Node: {Ego Graph Node List) One-hop Neighbor: (One-hop Neighbor List} Random Walk: { (Random Walk Paths). Output: Answer: (Ground-trey truth Category); Reason: { (Generated by GPT-40).
27

---

Published in Transactions on Machine Learning Research (10/2025)
Table 9: Prompt templates used for graph-oricnted multi-agent collaboration on the Arxiv datasct.
Type
Prompt
LLM- powered Graph Agent
You are a graph analysis expert, specializing in Grraph Analysis Perspectives). Your task to analyze the graph based on { Gruph Analysis Perspeclines) within the Arxiv datasel. Consider the following question: (Task-specifie Instruction Inpuf). Utilizing your expertise in (Graph Analysis Perspectives); interpret the graph's conditions and emphasize key aspects related to (Graph Analysis Perspectives). Please provide three likely categories as comma-separated list, arranged from most likely to least likely. For each category, explain your reason. Ensure that your explanation aligns with the answer you provide. Using the following format: Answer: (Top-3 Likely Calegory Lisl} Reason: {Reason for the Selected Categories}.
Global Summary Agent
Analyze the expert reports related to the target node graph. Your main goal to evaluate each expert's overall answer based on a holistic confidence score and provide a reason, leading to a cohesive global summary. 1. Confidence Analysis: Evaluate each expert's entire analysis by assigning single confidence score on scale of 1 to 5: 5: Strong confidence; 4: High confidence; 3: Moderate confidence; 2: Low confidence; 1: Poor confidence. For each score, provide a clear and concise justification that reflects the expert's overall depth of knowledge, accuracy, and reliability in their analysis. Ensure that the score represents the expert's report as a whole and not specific sections or categorics. 2. Extract Key Insights: Summarize significant. insights relevant to the target node from each expert's report. 3. Global Summary: Create unified summary that synthesizes the insights and highlights any critical agroements controversics. Using the following format: Confidence Analyses: One-Hop Neighbors Expert: (Confidence Score and Reason] Random Walks Expert: { [Confidence Score and Reason] Centrality Expert: CConfdence Score and Reason) Key Insights: {Key Insights] Global Summary: Global Summary).
Collaborative Self- Reflection
Consider the following question: specific Instruction Input). Based on the question, you have prepared a preliminary analysis: {mitiial Analysis). You will receive preliminary analysis from other experts and a synthesized report. Critically review and analyze these insights. 2. Tf you find aspects of other experts' analysis that are more rational than yours, incorporate these into your analysis for improvement. 3. If you believe your analysis is more scientifically sound compared to others, maintain your stance. Please provide three likely categories as comma-separated list, arranged from most likely to least likely. For cach category, explain your reason. Using the following format: Answer: { (Top-3 Likely Calegory List} Reason: (Reason for the Selected Categories).
28


Reviews:
[{"review_id": "iNuhI8JSPf", "reviewer": "Reviewer_cjkt", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "# Strengths\n- The approach is understandable and sensible. The methods for wrapping of individual GNN or LLM approaches and their combination become clear from the manuscript.\n- The evaluation uses a diverse set of individual GNN and LLM approaches\n\n# Weaknesses\n- The evaluation lacks a multiagent baseline of any sort. There is relevant related work which could be implemented at this point (or simple combination strategies).\n- Presentation could/should be improved: The introduction would benefit from giving explicit examples of relevant tasks, not only mentioning text-attributed graphs in general.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a multiagent system solution for tasks related to text-attributed graphs. The authors therefore propose to combine different agents, either being a graph neural network (GNN) or an LLM. Individual prompts wrap the predictions of used GNNs and LLMs, and an agent collaboration mechanism based on so-called collaborative self-reflection and a performance report is proposed.\n\nThe authors evaluate their method on node classification and link prediction tasks for datasets arxiv, cora, imdb, pubmed, dblp. Various baselines from competing GNN and LLM approaches are chosen.\n\nThe results show that the proposed approach surpasses all baselines for both tasks. Ablations also show that some agent combinations work better than other for the proposed system.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Related work on Multiagent systems: Recent work [1] should be mentioned. In addition, where do you situate your multiagent collaboration approach in? Is it completely new in the multiagent / expert advice literature? If so, why can't we reuse other combination strategies? Quite some are cited, but only a general statement is given.\n- Multiagent baselines: Please add recent multiagent baselines for text-attributed graph tasks. This could be either only based on LLMs [1] or mentioned close competitors [2] or something similar but representative. Otherwise a strong argumentation is necessary why this is not needed.\n- Discussion of ablation results: Please add deeper discussion on the agent ablation in 4.3 wrt if your system is able to best combine the available agents. Is it that the systems makes more mistakes when combined some agents, which could hypothetically achieve better results?\n\n[1] MARK: Multi-agent Collaboration with Ranking Guidance for Text-attributed Graph Clustering, https://aclanthology.org/2025.findings-acl.314.pdf\n[2] Can Graph Learning Improve Planning in LLM-based Agents?, https://arxiv.org/pdf/2405.19119", "broader_impact_concerns": "-", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "wdwERrtOvn", "reviewer": "Reviewer_Mq2L", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The paper is well-organized and presents a clear and intuitive framework that is easy to follow.\n2. The integration of GNNs and LLMs in a role-based, multi-agent collaboration framework is novel and addresses key limitations in previous methods.\n3. The framework is validated through experiments, showing substantial improvements in both node classification and link prediction tasks across diverse datasets.\n\nWeaknesses:\n1. The paper assumes LLMs can understand graph structures through textual descriptions (e.g., one-hop neighbors, and random walks), but does not evaluate or justify the effectiveness of this graph-to-text transformation. This modality shift may introduce ambiguity or information loss, and warrants further investigation. It is unclear how LLM can understand the graph description based on text. The mechanism of how much LLM can effectively comprehend the give graph information requires separate studies.\n2. The selection mechanism for triggering LLM-based collaboration in conflict scenarios is only briefly described. It remains unclear how this policy would generalize to more complex or ambiguous cases in real-world applications, and whether it introduces bias or inefficiency.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces GMAgent, a framework that integrates GNNs and LLMs into a multi-agent collaboration system for analyzing text-attributed graphs (TAGs). It leverages GNNs for structural understanding and LLMs for semantic reasoning, selectively applying LLM-based collaboration in conflict cases. Extensive experiments across five datasets demonstrate significant performance gains over state-of-the-art methods.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Clarify and evaluate the graph textualization mechanism. Provide ablation studies or user studies that compare different graph-to-text strategies (e.g., with or without random walks or structural context) to demonstrate how effectively LLMs can comprehend graph information through text alone.\n2. Detail the conflict scenario selection policy. Elaborate on how the selection of conflict scenarios is performed and provide a quantitative analysis of its impact (e.g., how often conflicts are detected, how conflict filtering affects downstream performance). Consider alternative strategies such as learned selectors or threshold-based confidence aggregation.", "broader_impact_concerns": "There is no concerns on ethical implication in this work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Ea4WKjEauA", "reviewer": "Reviewer_JHwe", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n1. Synergistic Integration. By combining GNNs’ global structural learning with LLMs’ local semantic reasoning within a unified multi‐agent loop, the framework leverages complementary strengths in a way not seen in prior work  ￼.\n2. Empirical Gains. GMAgent yields an average improvement of 6.67 % on node classification and 3.58 % on link prediction over individual GNN backbones, showing clear quantitative benefits  ￼.\n3. Architectural Flexibility. Ablations on varying combinations of GNN agents (e.g., GCN+GAT vs. single TAPE) and LLM agents demonstrate that the framework can be tailored to dataset characteristics without redesigning core components  ￼.\n\nWeaknesses\n1. Computational Overhead. Reliance on LLM inference (e.g., GPT‐4o) for conflict scenarios introduces nontrivial latency and energy costs, which may hinder real‐time or large‐scale deployments.\n2. Hyperparameter Sensitivity. Key settings—number of self‐reflection iterations (Tmax), number of LLM experts, conflict‐detection thresholds—are presented without sensitivity analysis, leaving questions about stability under different configurations.\n3. Reproducibility Gaps. While high‐level descriptions of prompts and LoRA fine‐tuning are provided, exact templates, hyperparameters, and training details are deferred, making precise replication difficult.\n4. Scalability Uncertainty. Experiments are limited to mid‐sized academic and citation graphs; the paper does not evaluate performance or resource usage on industrial‐scale TAGs with millions of nodes.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "1. Formulation of a Graph‐oriented Multi‐Agent Framework. The paper introduces GMAgent, the first multi‐agent collaboration framework that unifies diverse GNN‐based and LLM‐based “graph agents” to jointly analyze Text‐Attributed Graphs (TAGs)  ￼.\n2. Effective Model Designs. It presents a suite of novel mechanisms—conflict evaluation among GNN agents, graph‐driven instruction tuning for LLM agents, a role‐play expert recruiting strategy, summary‐report generation, and collaborative self‐reflection—to enable agents to iteratively refine their outputs  ￼.\n3. Extensive Experiments. Comprehensive evaluation on five real‐world datasets (node classification and link prediction) demonstrates that GMAgent significantly outperforms state‐of‐the‐art baselines, validating its effectiveness, interpretability, and flexibility", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Detailed Efficiency Metrics: Report end‐to‐end wall‐clock runtime and approximate energy consumption for typical datasets, rather than only theoretical complexity.\n2. Hyperparameter Ablations: Include experiments varying Tmax, agent counts, and conflict‐threshold parameters to demonstrate robustness.\n3. Full Prompt & Tuning Specifications: Provide exact CoT‐based and graph‐description prompts, LoRA ranks, learning rates, and training corpora in an appendix or supplementary material.\n4. Larger‐Scale Evaluation: Add at least one large‐scale dataset (e.g., full PubMed citation network) to assess scalability and resource demands.", "broader_impact_concerns": "1. Environmental Footprint: Frequent LLM invocations, especially with GPT‐4o, carry substantial carbon and energy costs, raising sustainability issues.\n2. Bias Amplification: Text‐attributed features (e.g., author metadata) may encode societal biases; multi‐agent aggregation could inadvertently reinforce unfair patterns in downstream decisions.\n3. Potential Misuse: A powerful TAG‐analysis pipeline could be repurposed for large‐scale social surveillance or automated misinformation generation in networked media.\n4. Accessibility Barrier: Dependence on proprietary LLMs limits adoption in resource‐constrained research or industry settings, potentially widening the gap between well‐funded and under‐resourced groups.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)