Bridging Local Details and Global Context in Text-Attributed Graphs (2024)

Abstract: Representation learning on text-attributed graphs (TAGs) is vital for real-world applications, as they combine semantic textual and contextual structural information. Research in this field generally consist of two main perspectives: local-level encoding and global-level aggregating, respectively refer to textual node information unification ( e.g. , using Language Models) and structure-augmented modeling ( e.g. , using Graph Neural Networks). Most existing works focus on combining different information levels but overlook the interconnections, i.e. , the contextual textual information among nodes, which provides semantic insights to bridge local and global levels. In this paper, we propose GraphBridge, a multi-granularity integration framework that bridges local and global perspectives by leveraging contextual textual information, enhancing fine-grained understanding of TAGs. Besides, to tackle scalability and efficiency challenges, we introduce a graph-aware token reduction module. Extensive experiments across various models and datasets show that our method achieves state-of-the-art performance, while our graph-aware token reduction module significantly enhances efficiency and solves scalability issues. Codes are available at https://github.com/wykk00/GraphBridge.

## method in this section, we will introduce the notations used in this paper. subsequently, we will present the proposed graph-aware token reduction method in section 3.2. finally, the multi-granularity integration framework will be discussed in section 3.3. ## experiments in this section, we first introduce the used datasets in section 4.1. we then detail the baseline methods and experimental setup in sections 4.2 and 4.3, respectively. experiments are presented to evaluate our proposed method in section 4.4. we further investigate the use of a causal large language model as the backbone in section 4.5. finally, we provide an analysis of hyper-parameters, assess scalability and efficiency, and conduct an ablation study. furthermore, additional experiments (e.g., link prediction) are in appendix. ## conclusion in this paper, we introduce graphbridge, an innovative multi-granularity integration framework for text-attributed graphs. our method emphasizes the importance of bridging local and global perspectives by incorporating contextual textual information, thereby enhancing the fine-grained understanding of tags. an efficient fine-tuning and inference framework for text-attributed graphs. it co-trains large language models and gnns using a ladder-side approach, optimizing both memory and time efficiency. for inference, en-gine utilizes an early exit strategy to further accelerate. we reproduce its results using the official source code ¶ .