MARK : Multi-agent Collaboration with Ranking Guidance for Text-attributed Graph Clustering (2025)

Abstract: This paper studies the problem of text-attributed graph clustering, which aims to cluster each node into different groups using both textual attributes and structural information. Although graph neural networks (GNNs) have been proposed to solve this problem, their performance is usually limited when uncertain nodes are near the cluster boundaries due to label scarcity. In this paper, we introduce a new perspective of leveraging large language models (LLMs) to enhance text-attributed graph clustering and develop a novel approach named Multi-agent Collaboration with Ranking Guidance (MARK). The core of our MARK is to generate reliable guidance using the collaboration of three LLM-based agents as ranking-based supervision signals. In particular, we first conduct the coarse graph clustering, and utilize a concept agent to induce the semantics of each cluster. Then, we infer the robustness under perturbations to identify uncertain nodes and use a generation agent to produce synthetic text that closely aligns with their topology. An inference agent is adopted to provide the ranking semantics for each uncertain node in comparison to its synthetic counterpart. The consistent feedback between uncertain and synthetic texts is identified as reliable guidance for fine-tuning the clustering model within a ranking-based supervision objective. Experimental results on various benchmark datasets validate the effectiveness of the proposed MARK compared with competing baselines.

## introduction graph neural networks (gnns) @xcite , a popular method to handle graph, can learn the overall structural information within graphs but are unable to directly process the informative text associated with * equal contribution, co-first authors. existing methods @xcite , after obtaining cluster assignments from the clustering model, utilize an llm-based agent to decompose the task of querying the clustering categories of nodes into several sub-tasks, which are then processed sequentially to derive the final feedback. after filtering out potential incorrect feedback based on the llm's prediction confidence scores, the clustering model is then fine-tuned using crossentropy loss with the feedback-based pseudo-labels. however, three challenges still remain: (1) can we use multi-agent cooperation to query clustering categories? queries based on a single agent lack breadth of knowledge, resulting in an overly narrow perspective @xcite . (2) can we develop filtering strategies that consider both text semantics and graph topology? the noisy feedback filtering strategy that relies solely on prediction confidence utilizes the prior knowledge of llms but fails to leverage the topological perspective from the graph. ## related work recently, research on the integration of large language models (llms) and graph neural networks (gnns) in text-attribute graphs has garnered increasing attention @xcite @xcite . researches can be divided into three categories: (1) llm-as-encoder. @xcite @xcite uses pre-trained language models (plms) @xcite to encode texts, achieving a hybrid architecture of plm and gnn. to obtain expressive representations, @xcite utilizes the hidden embeddings of llms to construct node embeddings. (2) llm-as-predictor. @xcite @xcite serializes the graph into various sequences, which are then fed into the llm for prediction. @xcite adjust traditional llm architecture to conduct joint text and graph encoding. (3) llm-as-reasoner. under the guidance of carefully designed prompts, @xcite @xcite enables llms to perform inference tasks for node class or link existence. the above works almost focus on node classification and link prediction, while text-attributed graph clustering remains under-explored. to the best of our knowledge, gclr @xcite is currently the only study dedicated to text-attributed graph clustering. gclr employs a contrastive @xcite and pooling-based @xcite @xcite attributed graph clustering model as its backbone. gclr adopts an llm-as-reasoner architecture described above, where the llm solely offers feedback for the fine-tuning of the gnn, thereby eliminating the costly pretraining and fine-tuning of llms. despite the success, the three challenges discussed in the previous section require further exploration in this paper. ## conclusion in this paper, we address the challenge of textattributed graph clustering by leveraging the complementary strengths of graph neural networks (gnns) and large language models (llms). our proposed mark is a novel framework that integrates gnns with multi-agent llm collaboration to generate robust, ranking-based guidance. to overcome the limitations of current approaches, such as narrow single-agent perspectives, insufficient noise filtering, and fragile supervision, we de-sign three collaborative agents: the concept agent extracts cluster semantics, the generation agent synthesizes topology-aligned text for uncertain nodes, and the inference agent provides ranking-based feedback by contrasting uncertain and synthetic texts. by using consistent llm feedback as supervisory signals and incorporating a ranking-aware loss, mark effectively reduces noise interference and enhances clustering robustness. more importantly, extensive experiments validate that mark offers a powerful pipeline for augmenting existing gnn-based clustering methods with llms.