Can LLM s Convert Graphs to Text-Attributed Graphs? (2025)

Abstract: Graphs are ubiquitous structures found in numerous real-world applications, such as drug discovery, recommender systems, and social network analysis. To model graph-structured data, graph neural networks (GNNs) have become a popular tool. However, existing GNN architectures encounter challenges in cross-graph learning where multiple graphs have different feature spaces. To address this, recent approaches introduce text-attributed graphs (TAGs), where each node is associated with a textual description, which can be projected into a unified feature space using textual encoders. While promising, this method relies heavily on the availability of text-attributed graph data, which is difficult to obtain in practice. To bridge this gap, we propose a novel method named Topology-Aware Node description Synthesis (TANS), leveraging large language models (LLMs) to convert existing graphs into text-attributed graphs. The key idea is to integrate topological information into LLMs to explain how graph topology influences node semantics. We evaluate our TANS on text-rich, text-limited, and text-free graphs, demonstrating its applicability. Notably, on text-free graphs, our method significantly outperforms existing approaches that manually design node features, showcasing the potential of LLMs for preprocessing graph-structured data in the absence of textual information. The code and data are available at https://github.com/Zehong-Wang/TANS.

## introduction graph-structured data are prevalent in many realworld domains, including chemistry @xcite , social networks @xcite , and recommendation systems @xcite . graph neural networks (gnns) @xcite @xcite @xcite have emerged as powerful tools for processing such data lms enable the feature alignment for tags. text text text text text text text text text text text (c) limitation: how to collect text-attributed graphs? we need to collect graphs from scratch, focusing on those with inherent node semantics, such as citation networks. can we convert existing graphs to textattributed graphs using llms? text text by learning node representations that capture both the structural and attributed properties of graphs. in each gnn layer, the model first applies a linear transformation to project the input node features or previous layer embeddings into a new space. then, through message passing @xcite , the model aggregates information from neighboring nodes to update node embeddings. although gnns have shown remarkable success across various applications, a significant limitation is their inability to handle multiple graphs jointly, especially when these graphs have different node feature dimensions @xcite , as illustrated in figure 1 @xcite . this is primarily because the input dimension of messagepassing gnns is fixed, making it challenging to accommodate graphs with varying feature spaces. in practice, many real-world graphs possess diverse feature dimensions, preventing a single gnn from being applied across multiple graphs. this limitation poses a substantial barrier to deploying gnns in scenarios requiring cross-graph learning, such as transfer learning @xcite , domain adaptation @xcite , out-of-distribution detection @xcite , and graph foundation models @xcite . overcoming this challenge is crucial for extending the applicability of gnns to more complex real-world tasks. to address the challenge of aligning node features across multiple graphs, two main approaches have been proposed. the first approach involves using singular value decomposition (svd) @xcite to decompose the original node features of different graphs, aligning the number of dimensions across graphs. however, this method has two limitations: (1) svd is only applicable to graphs with node features, making it ineffective for graphs that lack such features. (2) while svd aligns the feature dimensions, it does not ensure that the semantic meaning of the features is consistent across different graphs @xcite . a different approach introduces the concept of text-attributed graphs (tags) @xcite , where each node is associated with a textual description. researchers use textual encoders @xcite to transform these descriptions into textual embeddings, better aligning node features across graphs and enabling a single gnn to handle multiple graphs, as shown in figure 1 @xcite . compared to svd, this method provides more meaningful feature alignment and can generalize to unseen tags by simply processing the node descriptions @xcite . however, despite these advantages, tags face a significant limitation: the collection of high-quality textual descriptions for all nodes is often impractical, making it challenging to apply this method in real-world scenarios. in this paper, we address the challenge of collecting textual descriptions for feature alignment in tags. inspired by the success of llms in generating synthetic data @xcite , we propose a novel method named topology-aware node description synthesis (tans) that leverages llms to automat-ically generate node-level textual descriptions for existing graph datasets @xcite , acting as the first work towards this direction. tans focuses on the node classification task and uses the inherent topological information @xcite of each node to guide the llms in producing meaningful textual descriptions that capture the role of each node within the graph. our method is versatile, as it not only applies to graphs without textual descriptions but also enhances graphs with existing rich or limited textual data by generating more informative descriptions. we evaluate our approach on five diverse graph datasets, covering text-rich, text-limited, and text-free scenarios, each with distinct scales and topologies. the experimental results demonstrate the effectiveness of our method, showing superior performance in generating node properties. moreover, our approach is robust in transfer learning and domain adaptation, further showing the value of llm-generated descriptions in aligning node features across different graphs. ## backgrounds preliminary. we analyze why existing gnns struggle to handle multiple graphs with different feature spaces. to understand this limitation, we first review the message-passing process in gnns. given a graph @xmath0, e) with a node set v and edge set e, where each node @xmath1, a gnn encoder ϕ takes the graph as input and learns node embeddings @xmath2, e) through message passing. a typical gnn layer is defined as follows: this fixed dimensionality presents a key challenge: when a new graph with different feature dimensions is introduced, the same transformation matrix cannot be used, as it does not accommodate the differing input size. this inherent mismatch is the core reason why a single gnn model cannot effectively handle multiple graphs with different feature spaces. step 2: node description step 3: prompting llm step 1: node properties graph degree centrality: 3 clustering coefficient: 0.25 betweenness centrality: … node property node text neighbor text node type: … graph type: … (optional) node texts: … (optional) neighborhood texts: … node properties: … step 4: explanation llm output the potential class of the node among {classes of nodes} and provide reasons for your assessment. the potential class for the node is ……, the reason is …… final node descriptions related works. we present the most relevant works on feature alignment for graphs in this section, with additional references available in appendix a. a common approach is to apply svd to decompose node features across graphs into a shared feature space @xcite . however, svd struggles to ensure finegrained feature alignment, especially across different graphs. to address this, some methods design advanced gnn models that learn projectors for different graphs @xcite , but these approaches fail to generalize to unseen graphs @xcite . tag-based methods (liu et al., 2024a; wang et al., 2024c) align features by encoding node textual descriptions using textual encoders. yet, as noted by chen et al. (2024b), these embeddings can still reside in different subspaces, limiting their effectiveness. to mitigate this, some works leverage llms to generate additional node descriptions for better feature alignment. for instance, tape (he et al., 2024) uses llms to infer node classes and augment original texts with generated explanations. kea (chen et al., 2024a) enhances node embeddings by explaining key terminologies from the original descriptions. however, both methods rely on existing text and cannot handle graphs without textual data. ## method: tans the core idea behind tans is to leverage topological information as auxiliary knowledge to enhance llm-generated descriptions for each node. our method is versatile, applying not only to graphs without textual descriptions but also improving the quality of graphs with existing textual data. ## discussion expanding to more graph-related tasks. in our experiments, the proposed tans achieves desirable performance on node classification tasks for citation and airport networks, demonstrating the potential of llms in understanding node properties based on graph topology. this success motivates us to explore the potential of llms in understanding edge and graph properties, extending our method to edge-level and graph-level tasks. we plan to investigate these extensions in future work. converting basic attributed graphs. our proposed tans converts existing graphs into textattributed graphs, facilitating feature alignment in graph preprocessing. although our focus is on graphs classified by their associated textual descriptions, we believe that attributed graphs, where node features are generated through feature engineering, can also be converted into text-attributed graphs due to the inherent semantics of each feature dimension. for instance, cora uses a 1433dimensional one-hot encoding, with each dimension corresponding to a keyword, and pubmed uses a 500-dimensional tf-idf vector, where each dimension represents a keyword. by leveraging these inherent semantics, we can convert original node features into textual descriptions. whether these converted graphs are classified as text-rich or textlimited will depend on the specific case. we plan to explore this conversion process in future work. ## conclusion in this work, we explore the ability of llms to convert existing graphs to text-attributed graphs by generating node descriptions in graphs, regardless of whether the graphs contain textual information. our proposed tans enables llms to incorporate graph topological information when generating node descriptions, allowing for the alignment of node features across graphs. experimental results demonstrate the superiority of our method across text-rich, text-limited, and text-free graphs in training from the scratch, domain adaptation, and transfer learning settings.