‘Hello, World!’: Making GNN s Talk with LLM s (2025)

Abstract: While graph neural networks (GNNs) have shown remarkable performance across diverse graph-related tasks, their high-dimensional hidden representations render them black boxes. In this work, we propose Graph Lingual Network (GLN), a GNN built on large language models (LLMs), with hidden representations in the form of human-readable text. Through careful prompt design, GLN incorporates not only the message passing module of GNNs but also advanced GNN techniques, including graph attention and initial residual connection. The comprehensibility of GLN’s hidden representations enables an intuitive analysis of how node representations change (1) across layers and (2) under advanced GNN techniques, shedding light on the inner workings of GNNs. Furthermore, we demonstrate that GLN achieves strong zero-shot performance on node classification and link prediction, outperforming existing LLM-based baseline methods.

## introduction graph neural networks (gnns) are designed to process graph-structured data, and they have demonstrated strong performance in various downstream tasks such as node classification and link prediction @xcite . a key to their success lies in their message passing module, which updates the representation of a node by aggregating information from its neighbors @xcite . however, the high-dimensional embeddings (i.e., vectorized representations) obtained via existing gnns are generally not comprehensible. in this work, we propose gln (graph lingual network), where an llm is prompted to aggregate neighbor information to update a node's representation. therefore, all hidden node representations of gln are human-readable texts. moreover, we propose a tailored llm prompting framework incorporating advanced gnn techniques, specifically graph attention @xcite and initial residual connection @xcite . compared to existing gnns, our gln offers several advantages. first, its hidden representations are comprehensible and human-readable, since they are text descriptions of nodes generated by the llm. second, using an llm as the message passing module enables gln to solve graph-related tasks in a zero-shot manner, without any training or task labels. third, gln can be further prompted to explain its decisions on graph-related tasks, facilitating human understanding of its reasoning. thanks to the comprehensibility of gln's hidden representations, we provide an intuitive analysis regarding how the node representations change (1) across gln layers and (2) under advanced gnn techniques. drawing from this analysis, we offer several key insights into the mechanisms underlying gnn message passing and its advanced techniques. moreover, we demonstrate the zeroshot capability of gln on popular downstream tasks (node classification and link prediction), demonstrating its superiority over existing llmbased baseline methods. code, datasets, and example node representations generated by gln are in https://github.com/kswoo97/gln-code . ## related work and preliminaries in this section, we cover related work and preliminaries of our research. ## proposed method: gln in this section, we introduce gln (graph lingual network), a graph neural network where an llm serves as its message passing module. we first give an overview of gln (sec. 3.1) and describe our specialized prompt that incorporates gnns' advanced techniques (sec. 3.2). refer to figure 1 for an overview of gln. ## analysis and experiments in this section, we analyze representations obtained by gln and demonstrate its zero-shot capability in several graph-related tasks. ## conclusion in this work, we propose gln, a gnn that uses an llm as its message passing module (sec. 3). leveraging the comprehensibility of its hidden representations, we provide intuitive insights into message passing and advanced gnn techniques (sec. 4.1). moreover, gln outperforms baselines on zero-shot graph-related tasks (sec. 4.2).