Title: UniGTE: Unified Graph–Text Encoding for Zero-Shot Generalization across Graph Tasks and Domains
Venue: NeurIPS 2025
Accepted: yes
Keywords: Large Language Models

Graph Neural Networks

Zero Shot Learning
Abstract: Generalizing to unseen graph tasks without task-specific supervision is challenging: conventional graph neural networks are typically tied to a fixed label space, while large language models (LLMs) struggle to capture graph structure. We introduce UniGTE, an instruction-tuned encoder–decoder framework that unifies structural and semantic reasoning. The encoder augments a pretrained autoregressive LLM with learnable alignment tokens and a structure-aware graph–text attention mechanism, enabling it to attend jointly to a tokenized graph and a natural-language task prompt while remaining permutation-invariant to node order. This yields compact, task-aware graph representations. Conditioned solely on these representations, a frozen LLM decoder predicts and reconstructs: it outputs the task answer and simultaneously paraphrases the input graph in natural language. The reconstruction objective regularizes the encoder to preserve structural cues. UniGTE is instruction-tuned on five datasets spanning node-, edge-, and graph-level tasks across diverse domains, yet requires no fine-tuning at inference. It achieves new state-of-the-art zero-shot results on node classification, link prediction, graph classification and graph regression under cross-task and cross-domain settings, demonstrating that tight integration of graph structure with LLM semantics enables robust, transferable graph reasoning.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes an instruction-tuned encoder-decoder framework for zero-shot generalization across graphs from different domains without any task-specific fine-tuning. The  proposed method is equipped with structure-aware attention mechanism to preserve the permutation invariance and graph structural information.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The presentation of this paper is good and the paper is well-written.\n2. The experimental results show that the proposed method outperforms most baseline methods across 9 datasets in three types of tasks.\n3. The motivation of this paper is clear.\n\nWeaknesses:\n1. The intuition of including the additive biases in the attention mechanism need further justification.\n2. In the introduction (line 42), the authors mention that one drawback of GOFA is its high computational cost. The proposed method also include different types of embeddings as the input of the encoder. However, this paper does not include the time complexity of the proposed method.\n3. In the ablation study, the authors do not validate the necessity of three additive biases, which the authors claim to preserve the permutation invariance and structural awareness.\n4. There is a misuse of symbol in the paper. In line 106, $n$ is denoted as the number of hop, while in line 109, $n$ is denoted as number of nodes, which is inconsistent. n-hop neighbors of a node are the nodes located at a shortest distance of n from that node, rather than the number of nodes in the subgraph.\n\nQUESTIONS:\nQ1: I am confused about adding the additive biases in the cross-attention mechanism in equation 2. The output of equation 2 only computes the cross attention weight for different tokens, while the original positional encoding is added to the token embedding before the self-attention mechanism. How does it encode any node-permutation invariance information and structural awareness information as attention weight only adjusts the importance of different token embeddings?\n\nQ2: In the introduction (line 42), the authors mention that one drawback of GOFA is its high computational cost. The proposed method also includes different types of embeddings as the input of the encoder. Then, what's the time complexity of the proposed method?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes UniGTE, which is an encoder-decoder framework, simultaneously aware of graph and text modalities in a unified architecture, particular focus on transferable/ zero-shot graph reasoning scenarios.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n\n1. This paper is well-written and easy to follow.\n\n2. The statement of introduction and motivation makes sense. It is time to advance the graph-text unified architecture.\n\n3. Experiment results demonstrate that the proposed methods can promote the zero-shot and transferable graph reasoning ability.\n\nWeaknesses:\n\n1. One core limitation is that, as a fundamental architecture/framework work, this paper only focuses on zero-shot performance, which is not common for such types of work. A unified framework can do well on zero-shot cases, but it cannot only work for zero-shot. It is also necessary that also present more promising results in fundamental tasks under SFT settings.\n\n2. Though the work has presented many downstream tasks to show its effectiveness. It lacks intrinsic/fundamental graph understanding and reasoning abilities for analysis under basic graph tasks. e.g., zero-shot question-answering in some graph analysis/algorithm problem, like cycle and connectivity detection, shortest path, hamilton path, topological sort, like performed by other graph reasoning works like NLGraph, GITA, etc\n\n[1] NLGraph, Can language models solve graph problems in natural language?, NeurIPS 2023\n\n[2] GITA, GITA: Graph to Visual and Textual Integration for Vision-Language Graph Reasoning, NeurIPS 2024.\n\nQUESTIONS:\n1. Please provide more results on SFT settings.\n\n2. It is well to directly show the models' basic graph understanding performance (as well as the transferable performance).\n\nLIMITATIONS:\nNo. The architecture should also consider the fundamental adaptability to the SFT setting, not only for zero-shot, to ensure its better usability.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work fundamentally advances how graphs and language interact in foundation models. A Unified Encoder-Decoder framework UniGTTE is proposed in the paper. Unlike existing approaches that either (a) serialize graphs losing structure, or (b) separately train GNN+LLM components, UniGTE's end-to-end instruction tuning enables structure-preserving zero-shot adaptation. The auxiliary reconstruction objective further ensures semantic-structural consistency.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\nUniGTE innovatively achieves a tight coupling of graph structure and natural language cues through learnable alignment tokens and graph-text attention mechanisms, solving the multimodal split problem. The dynamic graph embedding generated by this framework can adapt to different task requirements and demonstrate excellent zero-shot generalization capabilities in cross-task and cross-domain scenarios, significantly outperforming existing methods. The frozen LLM decoder design reduces training parameters while providing self-supervision signals through graph cue reconstruction tasks, effectively improving representation quality. Experimental verification shows that this method achieves state-of-the-art performance on diverse tasks such as node classification, link prediction, and graph regression.\n\nWeaknesses:\nUniGTE has obvious bottlenecks in computational efficiency. The serialization of graph structures and the design of frozen decoders lead to latency issues when processing large-scale graph data. The method has limited data coverage and lacks verification of dynamic graphs and industrial-grade ultra-large-scale graphs, which affects its practical application potential. The model's interpretability is insufficient, the cross-modal mechanism of alignment tokens lacks theoretical support, and the quality of graph reconstruction lacks quantitative evaluation indicators. Although it innovatively combines graph-text alignment, some technical components (such as adapter tokens and reconstruction targets) have conceptual overlaps with existing methods, and their differentiated contributions need to be further clarified.\n\nQUESTIONS:\n1. The claimed \"efficiency\" lacks comparative metrics against baselines.\n2. The method has limited data coverage and lacks verification on dynamic graphs and industrial-grade ultra-large-scale graphs, which affects its practical application potential.\n3. The cross-modal fusion mechanism lacks interpretability.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper titled “UniGTE: Unified Graph-Text Encoding for Cross-Modal Retrieval and Captioning” proposes a unified framework that encodes both graph and text modalities into a shared embedding space. It introduces graph-aware attention and a contrastive training strategy to bridge the semantic gap between graphs and text. Experiments on benchmark datasets demonstrate that UniGTE achieves strong performance on tasks such as cross-modal retrieval and graph captioning, outperforming existing baselines and validating its generalization ability across modalities.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1.\tUnified Framework: UniGTE presents a novel approach to jointly encoding graphs and text into a shared representation space, which is a significant step toward bridging cross-modal understanding.\n2.\tGraph-Aware Attention Mechanism: The model effectively integrates graph structures into the attention computation, allowing better alignment of textual and structural information.\n3.\tStrong Experimental Results: The method achieves state-of-the-art or competitive performance on multiple tasks, including cross-modal retrieval and graph captioning, demonstrating its effectiveness and generalizability.\n4.\tTask Versatility: UniGTE supports both retrieval and generation tasks, showing the model’s adaptability across modalities and applications.\n\nWeaknesses:\n1.\tLimited Modal Diversity: While the model unifies graphs and text, it does not extend to other modalities (e.g., images, audio), which may limit its applicability in broader multimodal contexts.\n2.\tScalability Concerns: The paper lacks detailed analysis of computational efficiency and scalability when dealing with large-scale graphs or long text inputs.\n3.\tAblation Study Scope: Although some ablation is provided, further breakdown on the contribution of each graph component (e.g., edge types, node features) would strengthen the empirical analysis.\n4.\tInterpretability: Like many Transformer-based models, UniGTE may suffer from limited interpretability, especially in understanding how graph-text interactions are formed.\n\nQUESTIONS:\n1.\tAlignment Strategy Clarification:\nHow does UniGTE ensure that the learned representations of graphs and texts are semantically aligned beyond contrastive loss? Have you considered other alignment signals such as mutual information or co-attention mechanisms?\n2.\tGraph Variability Handling:\nHow does UniGTE perform when the input graphs vary significantly in size, sparsity, or topology? Does the model require any normalization or preprocessing to handle such variations?\n3.\tModality Imbalance:\nIn training or inference, does the model exhibit bias toward one modality (e.g., favoring text features over graph structure)? How is this addressed, if at all?\n4.\tGeneralization to Unseen Modalities or Tasks:\nCould the UniGTE framework be extended to incorporate additional modalities such as images or tables? If so, what would need to be modified?\n\nLIMITATIONS:\nNo.\nThe paper does not explicitly discuss the limitations or potential negative societal impacts of UniGTE. While the technical contributions are clear, a more responsible and comprehensive treatment of broader implications would strengthen the work. Here are a few constructive suggestions:\n1.\tBias and Misalignment Risk: The model could potentially learn and reinforce biases present in the training data, especially if the text or graph inputs reflect real-world inequalities. The authors should address how this risk is mitigated or detected.\n2.\tMisuse Potential: Cross-modal retrieval systems could be used for surveillance or misinformation if misapplied. A short section acknowledging such risks and encouraging ethical use would be valuable.\n3.\tLimitations in Generalization: The paper would benefit from more discussion of performance trade-offs when transferring to out-of-domain tasks or graphs with drastically different structure or semantics.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}