Unifying Text Semantics and Graph Structures for Temporal Text-attributed Graphs with Large Language Models
Authors: Siwei Zhang
Yun Xiong
Yateng Tang
Jiarong Xu
Xi Chen
Zehao Gu
Xuehao Zheng
Zi'an Jia
Jiawei Zhang
Venue: NeurIPS 2025 poster ()
Keywords: Temporal Text-attributed Graph
Large Language Models
Data Mining
Primary area: deep_learning
TL;DR: We address the problem of temporal text-attributed graph (TTAG) modeling and propose CROSS. CROSS is the first framework designed to unify text semantics and graph structures with LLMs for TTAG modeling.

Abstract: Temporal graph neural networks (TGNNs) have shown remarkable performance in temporal graph modeling. However, real-world temporal graphs often possess rich textual information, giving rise to temporal text-attributed graphs (TTAGs). Such combination of dynamic text semantics and evolving graph structures introduces heightened complexity. Existing TGNNs embed texts statically and rely heavily on encoding mechanisms that biasedly prioritize structural information, overlooking the temporal evolution of text semantics and the essential interplay between semantics and structures for synergistic reinforcement.
To tackle these issues, we present $\textbf{CROSS}$, a flexible framework that seamlessly extends existing TGNNs for TTAG modeling. CROSS is designed by decomposing the TTAG modeling process into two phases: (i) temporal semantics extraction; and (ii) semantic-structural information unification. The key idea is to advance the large language models (LLMs) to $\textit{dynamically}$ extract the temporal semantics in text space and then generate $\textit{cohesive}$ representations unifying both semantics and structures.
Specifically, we propose a Temporal Semantics Extractor in the CROSS framework, which empowers LLMs to offer the temporal semantic understanding of node's evolving contexts of textual neighborhoods, facilitating semantic dynamics.
Subsequently, we introduce the Semantic-structural Co-encoder, which collaborates with the above Extractor for synthesizing illuminating representations by jointly considering both semantic and structural information while encouraging their mutual reinforcement. Extensive experiments show that CROSS achieves state-of-the-art results on four public datasets and one industrial dataset, with 24.7\% absolute MRR gain on average in temporal link prediction and 3.7\% AUC gain in node classification of industrial application.

Unifying Text Semanties and Graph Structures for Temporal Text-attributed Graphs with Large Language Models
Siwei Zhang', Yun Xiong', Yateng Tang', Jiarong Xu', Xi Chen'
Zehao Gul, Xuehao Zheng', Zi'an Jia', Jiawei Zhang*
'Shanghai Key Laboratory Data Science, School of Computer Science, Fudan University Schooo of Management, Fudan University Tencentt Weixin Group
FIFM Lab, University of California, Davis
(swzhang24, x_ che2211, guzh22, 241 24110240035) fudan.uuaee.. eatan.edducom jiawei@ifmlab. org fyunx, jiarongxu) edu. cn (fredyttang, xuehaozheng) com
Abstract
Temporal graph neural networks (TGNNs) have shown remarkable performance in temporal graph modeling. However, real-world temporal graphs often possess rich textual information, giving rise temporal text-attributed graphs (TTAGs). Such combination of dynamic text semantics and evolving graph structures introduces heightened complexity. Existing TGNNs embed texts statically and rely heavily on encoding mechanisms that biasedly prioritize structural information, overlooking the temporal evolution of text semantics and the essential interplay between seman- tics and structures for synergistic reinforcement. To tackle these issues, we present CROSs, a flexible framework that seamlessly extends existing TGNNs for TTAG modeling. CROSS is designed by decomposing the TTAG modeling process into two phases: (i) temporal semantics extraction; and (ii) semantic-struce strucurra informa- tion unification. The key idea is to advance the large language models (L..Ms) to dynamically extract the temporal semantics in text space and then generate cohesive representations unifying both semantics and structures. Specifically, propose a Temporal Semantics Extractor in the CROSS framework, which empowers LLMs to offer the dynamic semantic understanding of node's evolving contexts of textual neighborhoods, facilitating semantic dynamics. Subsequently, we introduce the -ttrutti-sttuctural Co-encoder, which collaborates with the above Extractor for synthesizing illuminating representations by jointly considering both semantic and structural information while encouraging their mutual reinforcement. Extensive periments show that CROSS achieves state-of-the-arr results on four public datasets and one industrial dataset, with 24.7% absolute MRR gain average in temporal link prediction and 3.7% AUC gain node classification of industrial application.
1 Introduction
Temporal graphs are crucial for modeling dynamic interaction data, where objects are represented as nodes and timestamped interactions are depicted edges [ 2] Unlike static graphs, temporal graphs continuously evolve over time 3 To capture the temporal dependencies and realize representation learning for such graphs, extensive research has developed temporal graph neural networks (TGNNs)
4 7 g. These works typically adopt structural encoding mechanisms 9 10 to encapsulate the
dynamics of graph structures [II], thus enabling representations for downstream tasks.
Meanwhile, besides the dynamically evolving graph structures, real-world temporal graphs are also often accompanied by rich text attributes, giving rise to temporal text-attributed graphs (TTAGs) [2]. As shown in Fig. [[), e-commerce networks, nodes may encompass elaborate text descriptions such profile product introduction, while timestamped edges can attach transaction details
Corresponding Author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Archbond peReEAAe haperal asidiclel gragh (TTNG)
Structural Ensoder
Paisting TGNNs
Minadianal layer
Leggig
Jual Ahnamic iesi acEaRks
unffer snnamn--taain ovetesting
Unale reprodtaiion
Teusporul Semmantics Extructor
Sammatic-itucttrr Co-enoder
Ours (This poper]
Semantic leee
Dysanic of kor for "Apple" "knoahhorherd
Laye
Al tine
Aa Illustrative example of semantic dynamks in ITAGs
(b) Compartion between existing IGNNS and our frumework.
Figure 1: (a) An illustrative example of semantic dynamics in TTAGs. Temporal text-atrrbbuted graphs (TTAGs) inherently exhibit semantic dynamies, where text semanties around nodes dynamically evolve across temporal dimensions. (b) Comparison between existing TGNNs and framework. Going beyond existing TGNNs that biasedly focus on structural dynamics, our framework seamlessly unifies text semantics and graph structures of TTAGS, promoting both dynamic semantic understanding and semantic--truutuue reinforcement.
including price, transaction type, review, etc. Representation learning in TTAGs presents unique challenges due to the intricate combination of dynamic text semantics and evolving graph structures, which remains largely under-explored in the community. Despite the success of existing TGNNs, they still encounter two fundamental limitations that hinder their accommodation to TTAG modeling.
Limitation (i): Neglect of semantic dynamics. Text semantics arise from the textual attributes of node and its surrounding neighborhoods. These semantics exhibit changes across timestamps due to the temporal evolution neighborhood contexts within TTAGs. For example, as depicted in Fig. [a). the term "apple" may take different meanings according to user preferences (driven by its neighborhoods), such as "smartphone" when focused on technology while "fruit" when turning to daily life. Such characteristics necessitate temporal-awer -wwre design for extracting text semantics over time. However, existing TGNNs always use pre-trained language models, e.g., MiniLM [13], to statically embed texts as pre-processed features, failing to adapt to such dynamic semantic shifts and leading to suboptimal representations for effectively capturing the semantic dynamics.
I Limitation (ii): Ineffective encoding for semantic-structural reinforcement. Another limitation of existing TGNNs is the rigid reliance their structural encoding mechanisms, which predominantly focus on topological information without adequately incorporating semantic considerations. We argue that the semantics and structures of TTAGs can mutually reinforce each other, making biased encoding mechanism that solely emphasizes structures untenable. This hypothesis is reasonable due to the inherent complementarity between textual content and structural connectivity [14 15| (also
empirically validated in Sec. A). For instance, product recommendations are shaped not only by goods' descriptions but also by users' historical behaviors [16]. Existing TGNN encoding mechanisms do not account for such nuanced semantic-strucctural reinforcement in TTAGs, resulting in insufficient representations that overly reliant on simplistic (or sometimes noisy) structural information.
On the other hand, recent surveys [7 [8] reveal that large language models (LLMs), e.g., DeepSeek- v2/3 [19], exhibit notable capabilities in semantic understanding and generation, which have been suc- cessfully leveraged in graph modeling through text augmentation (00 21) However, LLMs employed in these methods are typically limited by static corpora input for reasoning, making them ill-suited to capture the fine-graine dynamics of TTAGS. This limitation inspires to ask: Can advance LLMs for TTAG modeling as a promising solution to the aforementioned limitations of existing TGNNs?
Key innovation. Beyond existing TGNNs that solely prioritize structural dynamics, in this paper, we tackle above issues by decoupling the TTAG modeling process into two phases: (i) temporal semantics extraction; and (ii) semantic-structurr information unification. By extracting both dynamic semantics and evolving structures of TTAGs, we cohesively unify them to harness their dual-strengths for more informative representations. Consequently, as shown in Fig. [bb). the key idea of this work is to elevate LLMs with dynamic reasoning capability to extract semantic dynamics, which subsequently allows for a unified, integrated, and non-biased encoding mechanism obeying semantic-structural reinforcement.
Fig.
Present work. We extend existing TGNNs for TTAG modeling and introduce CROSS (Cohesive Representations Of Semantics and Structures), novel framework that seamlessly unifies text

---

TGNN
Ang Methods
Ours [This paper]
TONNS
L.M Methods
Ours [This paper]
Pralen cafler cocodias
Frase
Stalie Peesening
Dynamii
encoding
encoding
ETMI Unilivation
Modality Fusson
Figure 2: Technical difference comparison from the perspective of LLM utilization and modality fusion.
semantics and graph structures coupled with LLMs. CROSS can significantly boost existing TGNNs, and comprises two main components: (i) Temporal Semantics Extractor, and (ii) Semantic- structural Co-encoder. To capture the semantic dynamics for nodes within TTAGs, we propose a Temporal Semantics Extractor, which empowers the LLMs with the dynamic semantic summarization reasoning capability for TTAG modeling. It automatically detects semantic dynamics by constructing a temporal reasoning chain to dynamically prompt the LLMS to summarize the textualized, evolving neighborhoods of nodes, thus revealing the linguistic nuances across temporal dimensions. Besides the Extractor, to effectively achieve semantic-structurt reinforcement, we further propose the Semantic- structural Co-encoder that simultancously exploits both semantic and structural information through an iterative, multi-layer design. Each layer in the proposed Co-encoder bidirectionally transfers the cross-modal information between semantics and structures, allowing both modalities to blend deeply and fully illuminate each other. By performing such a unification, we can generate modal-cohesive representations that are both semantic- -emiicic-enriched and crructural-infommer
The main contributions of this paper are summarized as follows:
We address an under-explored problem of temporal text-attributed graph (TTAG) modeling and propose CROSS. To the best of our knowledge, CROSS is the first framework designed to unify text semantics and graph structures with LLMs for TTAG modeling.
We design a temporul-aware LLM prompting paradigm for TTAG modeling and develop Temporal Semantics Extractor. It enhances LLMs with dynamic reasoning capability to offer the evolving semantics of nodes' neighborhoods, effectively detecting semantic dynamics.
We introduce a modal-cohesive co-encoding architecture for TTAG modeling and pro- pose Semantic-structure Co-encoder, which jointly propagates semantic and structural information, facilitating mutual reinforcement between both modalities.
We conduct extensive experiments on four public datasets and one practical e-commerce in- dustrial dataset. CROSS outperforms baselines with 24.7% absolute MRR gain on average in temporal link prediction and 3.7% AUC gain in node classification of industrial application.
2 Preliminaries and Related Work
2.1 Preliminaries
Definition I. Temporal Text-attributed Graph. Given a set of nodes V. node text attributes D, and edge text attributes R, a temporal text-attributed graph (TTAG) can be represented as sequence of interactions = {u,, v, f)), where u, V and 2 Each node u E V is associated with node text attribute d, 'D and each interaction (u, 1) E G attaches an edge text attribute Ta,u,t E R.
We use (t) {(u, v, 22,7 T < 1) U {(1, u, T) |r < t) to denote the set of historical interactions involving node before time 1.
Definition 2. Temporal Text-attributed Graph Modeling, Given node u, time t, and all available historical interactions before t, {uu', v', T)(T < <), temporal text-attributed graph modeling aims to learn a mapping function f : (u, f) + z. (f), where (t) E Rdd denotes the representation of node u time 6, and d the vector dimension.
Definition Language Model vs. Large Language Model. We clearly distinguish between pre- trained language models (LMs) and large language models (LLMss. LMs, e.g., MiniL.,, are relatively smaller models designed to embed texts from the text space into the feature space; and LLMS refer to significantly larger models far surpassing the linguistic capabilities of LMs, like DeepSeek-v2/3.
2.2 Related Works
We provide two groups of related works: TGNNs and LLM-for-Grand Graph-Text-A Tuxt-Augmentatioon (LLM4GTAug) methods. As illustrated in Fig. 2 we summarize these methods from the perspective of LLM utilization and modality fusion. A discussion of related works is detailed in Sec. E
TGNNs. L.M utilization within TGNNs remains limited. Existing TGNNs typically pre-embed the raw texts input features and rely on their structural encoders 9 to capture the dynamics of
3

---

Co-encoder
Large Longuag
Toil
Pronph
Row Tants TTAGS
modill
Miser
Co Layer
Temporal Semanties Extractor
Co-enesder
Senarrik Lager
Siructural Layer
Co Co-eneoder Layer
Crossasedal
Temporal Resooning Chain
Sonusetle
Sinurioral
Trat MuTRARtII
Figure 3: Architecture of the proposed framework, CROSS. Our Temporal Semantics Extractor constructs the temporal reasoning chain to empower the LLMs with dynamic reasoning capability to dynamically provide the semantic understanding of nodes' neighborhoods, thereby extracting semantic dynamics in text space, Subsequently. the Semantic-strucct Co-encoder iteratively mixes and consolidates both semantic and structural information, ensuring their mutual reinforcement for effectively generating modal-cohesive representations.
graph structures. These methods overlook the temporal evolution of text semantics within TTAGs. Besides, they integrate semantic and structural information only at the input stage, resulting in shallow modality fusion that occurs before encoding.
LLM4GTAug methods. Existing LLM4GTAug methods 20 21 are designed for static graphs and employ LLMs through static, one-off reasoning for each node. This makes them ill-suited for capturing the temporal dynamics of TTAGs. In addition, they mix structural and semantic representations only the output stage, leading to shallow fusion after encoding.
As summarized in Tab. I in this pa- per, our CROSS aims to (i) enhance LLMs with dynamic reasoning capa- bilities to capture semantic dynamics; and (ii) facilitate hierarchical modality fusion throughout the entire encoding process. Our proposed method funda- mentally opens new avenues for future research in TTAG modeling.
Table I: Summary of related works.
LLM4GTAug
TGNNs
CROSS
Structural Dynamies
X
Semantic Dynamics
X static
X no L.M
M.I Utilization
dynamic hierarchical
shallow at outpor (after encoding)
shallow at input (before cacoding)
Modality Fusion
(during encoding)
3 Methodology
As mentioned before, unifying text semantics and graph structures for TTAG modeling requires considering both semantic dynamics and semaatic-structural reinforcement. To this end, as depicted in Fig. the proposed CROSS framework comprises two stages, i.e., the Temporal Semantics Extractor first detects the dynamic text semantics for nodes' evolving neighborhoods; then the Semantic- structural Co-encoder jointly unifies both semantic and structural information to ensure mooal reinforcement. We will introduce these two stages in the following subsections.
3.1 Temporal Semantics Extractor
In this section, we we propose the temporal reasoning chain to advance the LLMs with dynamic reasoning capability to extract semantic dynamics for TTAG modeling. It will be described below.
Temporal Reasoning Chain. Existing LLM utilization paradigm within LLM4GTAug methods are ill-suited for semantic extraction in TTAGs, as it fails to effectively capture the dynamics of node's semantic contexts. To address this challenge, we design novel temporal reasoning chain that empowers the LLMs with the dynamic semantic summarization reasoning capability across temporal dimensions, discerning the evolving linguistic nuances along timestamps. However, performing LLM reasoning at every timestamp remains computationally impractical. To ensure scalability and stability, for each node among TTAGs, we strategically sample reasoning timestamps at equal interaction intervals. This strategy enables CROSS to constrain the number of LLM calls and guarantee that each L.M reasoning step accesses a balanced set of interactions, striking a delicate balance between temporal granularity and cost efficiency.

---

Formally, for node u, we first derive the set of its interaction timestamps To = {t1, tg...., ) where t2... S in and the node degree. Then we sample the reasoning timestamps at intervals of among Tu with predefined maximum reasoning count m as follows:
41 tn
== 1, m - 1, E To T
Ts
Es
&
=
(1)
i ==m
Here, Tu S Tr depicts the LLM reasoning timestamps to summarize text semantics around node and we will also analyze the dataset-specific
m in Sec. Oof the Appendix.
Summarization, To align with the dynamics of semantic contexts within TTAGs, we facilitate the LLMs to dynamically summarize nodes' neighborhoods across different timestamps. Specifically, we perform multi-turn calling with the LLMs along the previously sampled reasoning timestamps.
For node u at reasoning time E Tu, we first obtain its neighborhood by retrieving u's historical interactions H., (t) = {u, r) |T < i U {(v, u, T) |T < i}. and then we prompt the LLMs to generate neighborhood summaries and semantic details for that time. This can be represented by:
(f) LLM (dust, (ra).eH.. PROMPT)
LLM(:: PROMPT) denotes the LLM calling with the prompt, which comprises a fixed template along- side variable pieces, including node u's raw text du, time i, and the neighborhood ( (e)..-)) di) the LLM-generated text, denoting the semantic summary for u's contexts at time f. We present simplified example of PROMPT for clarity, and the complete version is given in Sec. Flof the Appendix.
simplified example of PROMPT
Goal: [Request summarize the current neighborhoods and emphasize the response format.] Descriptions: [Provide the text attribute node d...]
Current time: [Specify the reasoning timestamp i.]
Historical interactions:
[List textualized neighborhoods using recent interactions (r.).EN.,(E))
After performing all reasoning, we can obtain a set of LLM-gerrrated textual summaries for each node u. We represent them as D. = (f)(f) f E F.). Notably, to incorporate the raw texts in TTAGs, we set d,, (0) = du. These summaries encapsulate the evolving semantics around nodes derived from the dynamically-enhanced LLMs, promoting semantic dynamics for TTAG modeling.
3.2 Semantic-structuraal Co-encoder
As mentioned in the Introduction, existing TGNNs solely account for structural dynamics, overlooking the necessary consideration of semantic dynamics and the reinforcement between these two modalities. To address this issue, our Semantic-structural Co-encoder performs iterative integration of semantics and structures at the layer level. Each layer of the proposed Co-encoder comprises three types of encoding components: (i) the semantic layer that encodes the text semantics from LLM- LLM- generated summaries to produce semantic representations, (ii) the structural layer that encodes the graph structures from neighborhood information to capture structural representations; and (iii) the cross- modal mixer that facilitates transformation between these two unimodal representations to cheer their mutual reinforcement. We will discuss them detail below.
Semantic Layer. To better incorporate the semantic modality, we implement our semantic layer using a Transformer encoder [22) due its widespread usage textual modeling. From the ablation results in Tab. 3, such a design effectively harnesses the full potential of text semantics for TTAG modeling, demonstrating clear advantages over purely structural encoding mechanisms.
We first prepare the inputs for the semantic layer. For node u at time f, the inputs of the semantic layer are the corresponding LLM-generated summaries before from our Extractor. We represent
them as D_, (t) (dx ( t4 < t). where t S ... S tg. We then use a pre-trained LM, such as MiniLM [13], to embed the texts into the d-dimensional semantic features, which is depicted as
(tx)x) = LM (d (tx)) E Rd To capture the temporal differences, we subsequently concatenate time information into these semantic features follows:
(tx (t+) + (t tx) 2d
(3)
5

---

Here, ((-) is the time encoding function introduced by [3], which is widely used in recent TGNNs [9] These enriched features then feed into the L-layer Transformer encoder blocks to derive the semantic representations for node at time L. We have:
(4) 6 (4x) = TRM (4)- (
(1+))
where e!l E 2d corresponds to the pre-mixed semantic representation at the I-th layer and As we will mention below, the latest semantic repre- (tz). has integrated information from its structural representation in the
Cu (0) (fi),...,c (0) (tx)
sentation of node u, e previous layer. This allows the 1-th semantic layer to receive the information from graph structures, achieving deep fusion and promoting their synergistic reinforcement.
Structural Layer. Meanwhile,
conduct our structural layer to encode the graph structures around node u at time t, G.. (t). using the structural encoding block in TGNNs. Such a design renders our framework inherently aagN-igo and flexibly integrable with any TGNN backbone.
Formally, the I-th structural layer aggregates the neighborhood information of node from current layer H ( (1) and its structural representation from the previous layer h (1-1) (t) with 2-layer Multi- Layer Perceptron (MLP). This can be expressed as:
( (t) MLP(() (h_ (t) || AGG ( (H))))
(5)
Here, (000) (1) is the pre-mixed structural representation at the I-th layer and || denotes concatenation. AGG(.) is a pooling function to aggregate the neighborhood information matrix into d-dimensional vector, with specific implementations (e.g., mean, sum) varying across various TGNNs 5
The neighborhood information H (t) is derived using a temporal attention mechanism 4 via message passing from the L-th layer neighborhood. This process assigns attention weights to scale the contribution and importance of neighbors (t, for node u as follows:
H ()) (1) Sof (1) (L),
(6)
where a (t) [a (t) represents the attention weight vector, and the matrix V i (t)
[ve (t))ENN((t) condenses the messages from u's 1-th layer neighborhood. Each element Oue (t) in
au (t) is the attention weight for node u to its neighbor E (t), and each row v! (t) from (i)
(1)
is the message carried from u's neighboring node v. These can be computed by: T
Sa ( 1) (0) fk (h (n!-""())
(7)
(8)
V k)
In Eqs. 7
f((+)(+ E {q. k, v)) denote the encoding functions for queries, keys, and values, respectively 22 These functions may be implemented differently across various TGNNs 4 5 and we do not discuss their details as they are beyond the scope of our work.
As we explain in the next paragraph, the structural representations from the previous layer, h (t). have integrated with the semantic representations. Consequently, the pesssage-passingg aggregation in Eq. 5 ennables the propagation between both types of information, effectively facilitating semantic- structural reinforcement during encoding.
Cross-modal Mixer. Finally, to enable the deep unification between the semantic and structural modalities for TTAG modeling, we introduce a novel and interesting cross-moda mixer that automat- ically transfers and integrates the bimodal information at a layer-wise wise granularity.
For node the 1-th layer, the inputs of cross-modal mixer are the most recent pre-mixed semantic representation e(!) (fx) and the corresponding pre-mixed structural representation 6.." (t).
This means that the remaining historical semantic representations, i.e., (t1), ... e!!) (tx-1): will be not involved the mixture process. Such a design is driven by: (i) efficiency consideration; (ii) cemporal-awarene consideration that the latest semantic representation carries the most contextually relevant information; and (iii) empirical consideration that mixing all semantic representations does not necessarily lead to better performance (See Sec. 4.5) Therefore, concatenate e!!' (tx) with

---

1 (t), (1), the post-mixee semantic and structural representations. These processes can be formulated as follows:
then pass through our cross-modal mixer, and finally split the fused representation to derive
(tx); h (t) Mixer (el) (t))
(9)
We implement Mixer() (.) using 2-layer MIP,, and it can be alternatively conducted in other fusion components. By iteratively performing such a mixture operation, we can deeply unify both text semantics and graph structures, enabling cohesive representations for TTAG modeling.
3.3 Training CROSS
Cohesive Representations. For node u at time t, its representation is derived from: (i) semantic (t) = Mean Cu (tz), c. (tx))
outputs from the L-th semantic layer with mean pooling, z, =,tm(t
(ii) structural outputs from the L-th structural layer, z (t) = h (t): and (iii) the unified outputs from the cross moosl mixer, mix (t) = Cu (tx) h (L)
(t). This can be denoted as:
Zu (t) = MLP Pou (z (t) || (1) || mix (t)
(10)
where zu (1) E Rd and MLP (-) is a 2-layer MLP that maps the dimension of the input vector to d. Loss Function. We adopt the temporal link prediction task (7 as training signals for TTAG modeling. For link (u, U, t), we compute its occurrence probability Pur (t) by feeding the concatenated represen-
tations of nodes and u, zu (t) (t), into 2-layer MLP. Cross-entropy loss is then applied:
L - Σ
Pwo (t) log (1 Pow' (f))
(11)
(u.o,t)EG
The e' denotes the randomly sampled negative destination node. Additionally, we will also present the theoretical analysis to prove the effectiveness of CROSS in Sec.
Experiments
4.1 Experimental Settings
Datasets. We implement experiments with five datasets, including four public datasets and one industrial dataset from real-world e-commerce systems. The four public datasets Enron, GDELT, ICEWS1819, and Googlemap_ CT are recently collected and released by [12]. Besides, the industrial dataset constructed with three months of transaction data sampled from private e-commerce trading network in WeChaf Mobile Payment Details of these datasets are summarized in Sec. D. due to page limitations. AlT datasets are chronologically split by 60%, 20%, and 20% for training, validation, and testing, respectively.
Baselines. We select eleven existing methods as our baselines, including JODIE 24 DyRep
225. TGAT [4) TGN (5), CAWN (26), PINT (27]. TCL (28). GraphMixer 29), Dy Gormer [7].
LKD4DyTAG [30], and FreeDyG [1O]. We also select powerful DeepSeek-v2 [19] and evaluate its zero-shot and one-shot performance as LLM baselines, denoted as LLM Detailed descriptions of all baselines are provided in Sec. [.2] Moreover, we choose three representative TGNN models, i.e., TGAT, TGN, and DyGFormer, the backbones of CROSS due to their superiority. For simplicity, we employ the well-established MiniLM [3] to embed texts into feature space. Additionally, we adopt DeepSeek-v2
the default LLM. We also report the results with other LLM backbones in We evaluate the learned representations using two downstream tasks, ie., temporal link prediction and node classification in an industrial application of financial risk management.
Sec. 4.5
4.2
Temporal Link Prediction
We begin our experimental evaluation by comparing the temporal link prediction performance of our model with baselines. We conduct this under two settings: (i) transductive setting, which predicts links between nodes that have appeared during training: and (ii) inductive setting, where predictions are performed with unseen nodes. Implementation details can be found in Sec. D.3 of the Appendix.
The results are presented in Tab. 2 performance of all three TGNN backbones across all datasets in both transductive and inductive
Clearly, our CROSS framework significantly improves the
https://pay. weixin. www.ia.....oo qq. com
Thhe dataset is sampled solely for experimental purposes and does not imply any commercial affiliation. All personally identifiable information (PII) has been removed.
7

---

Table 2: MRR results (%) for temporal link prediction in transductive and inductive settings. The results for LeM represent the gero-/one-sho performance the LLM DeepSeek-v2 v2 [9]] Results highlighted with a blue background indicate the performance and corresponding improvements our CROSS framework using various TGNN backbones. The best results are highlighted bold.
Transductive setting ICEWS1819
Inductive setting ICEWS1S19
Enron
GUELT
Gooplemap_co CT
Industrial
Enron
GDELT
Gonglempp_ C
Indastrial
JODIE
66.59 2.0
48.81
1.1
71,47 4
56.72 a
49.75
53.41
2.5
37,90 2.4
57.75 4
55.21 1.0
30.38 1 0.3
DyRep
38.85
45.61
2.8
63.13 3,6
69.01 2 0.4
36.49 0.0
42.95
8.3
4 3.1
50.42
2.8
4
23.47 4 L.8
TCL.
59.49 0.7
0.5
0.3
68.98
55 + 1.5 50.87 0.5
47.13 1.0
77.06 0.3
6..36:00.2
33:42 + 0.5
CAWN
0.
0..
82.
63.58
61.58
2.
53.12
PINT
2.
32.71
2.7
53.31 1.06
31,82
63.16
70.02 0.6
39.72 0.6
GraghMiice
62.68 1.3
53.33
0.4
80.69
50,50
43.75 1,5
41.18
67.09
51.36:0.22
34.06 0.7
PreeDyG
8.
68.27
0.7
75.91 0.7
70.38 0.1
5 0.3
2..1
66.01 2
56.18 0.5
LKDADYTAG
0.3
80.62
77.73
45.75
0.3
1.0
$7.91
24.18
7.99
33.68
30.50
11:27
17:73
10.08
32.26
38.21
2.62
LLMore
46.27
28.91
50.82
48.79
30.29
48.14
28.91
44.69
43.83
20.28
TGAT
6.1
85.81
03
46.74
42.01
0.5
04
60
30.04
TGAT
0.7
81.63 x
93.05 1.6
99:01 +00
96.07 2.8
81:52:-2200
64.56
1.8
82:25 2.0
91.59 0.0
62.22 2.1
Arg. 1 26.50
n 20.02
24.90
7.24
36.78
40.23
1 33.772
22.35
8,15
1.36633
1 12.18
TGN
73.05 a 1.7
54.28 + 1.6
#4.79
71.38 0.5
$4.46 3.0
54.98 2.3
37.48 a 2.8
69.09 0.8
67.88 0.2
38.28 + 4.1
TGN+
0.
77,95
2..88
99.92
94.26 0.8
82338
36.05
84.01 =92
92.08
2.6
Avg. 1 25.45
1. 222.79
23.67
9.95
28 67
39. 80
1. 17240
19.17
1. 14.32
1.24.
44.02
DyGiFormer
79.93
54.82 a
14.45
50.61
78.14
2.5
54.20
95.31 2.8
$1.28 4
95.77 a 0.3
99.82
94.77 =
4.9
66.37
4.4
87.90 = 0.6
91.74::001
82.30 1.2
Arg. 1 22.03
1 15.38
1 19.93
1 8 26
44.99
1 20.33
1 19.15
15. 76
1 19.66
1.38. 76
1 26. 10
settings. Meanwhile, CROSS achieves SOTA performance with a substantial margin over the best baseline. This observation proves the effectiveness of unifying text semantics and graph structures in TTAG modeling. Although LLM's zero/one-shot performance is suboptimal, CROSS still performs well. This suggests that LLMs may struggle to directly comprehend the dynamics of graph structures in TTAG modeling, but our proposed CROSS framework helps them to resolve this issue effectively. Moreover, our framework tends to result in closer performance across different TGNN backbones. We infer that this is due to the robustness of text semantics, which successfully reduces model reliance on simplistic structural information. Inspired by this, we detail robustness study in Sec. 4.4
4.3 Industrial Application
We conduct another downstream task using node classification in a real-world industrial appli- cation of financial risk management Indus- trial, where we predict whether nodes are in- volved in fraudulent activities. DyGFormer [7] is used as the backbone, and details can be seen in Sec. D.3 of the Appendix. CROSS achieves the best performance as shown in Fig. 4 indi- cating that the learned representations of CROSS are also effective for node-level tasks. CROSS'S success on Industrial demonstrates its scalability for large industrial-scale TTAGs. We will also provide scalability clarification in Sec. G
80
Figure 4: AUC results (% ) for node classification of a real-world industrial application in financial risk management, which predicts whether nodes are involved in fraudulent activitics on Industrial.
4.4 Robustness Study
Robustness for noise. To empirically validate the assumption of noisy structural information
mentioned in Sec. [ we strategically introduce noise into graph structures with perturbation rates of p E (10%, 20%, 30%, 40%, 50%). Implementation details are put in Sec. d to page limitations. To further investigate the impact of noise during encoding, we randomly select a subset of nodes and visualize the attention weights of their perturbed neighbors using box plots under p 50%.
200IE
TGAT
DyRey
TGN
DyCPurmer
TCL
GraphMiner
FreeDsG
TGN:
TCAT:
DyCFormer
400
Trunstedhe 110 56
60
80
60
311
20
40
50
20
31
10
20
30
10
40
20
Pettrlatioo Rate
ICENSUS0S
Pestuclation
Porusiation
Figure 5: Robustness study for noise on Enron and ICEWS1819 with different perturbation rates.
8

---

Table 3: Ablation study for the raw texts and L.M--generate texts. Semantic/Structural Encoding refer the encoding mechanisms that independently perform semantic/structural layers in Sec. 3.2 impry. indicates the performance improvements of LLM- --geeerateed texts TEXtLeM over the raw texts Textro- Our proposed co-encoding mechanism unlocks the full potential of LLM generated texts and achieves the best performance.
Dutaeets
Methods
Tos
Semantic Encoding
TexiLLM
impors
Textaa
Structural Encuding
TesiLLM
imprs.
Textow
Semantie-stracture Covencuding TesiLUMI (ouns)
TGAT TGN
49.73 1733+000
63.07 0.5
13.34
66.06
6365 1.7
2.41
70.27 0.2
05.58 0.7
26.31
49.73
63.07
13.34
73.0
42
95.84 =
DyCiFrr
49.73 0.8
63.07
13.34
79.93
8046 0.5
0.53
80.91
95.31
15 0.
1.59
$5.81 0.2
ICEWS1819
TGN
85.04 0. 1.5
7.59
84.79 0.6
85.69 0.4
0:990
85.96 0.8
94.74 5.7
8.78
DyGFormer
1.5
7.59
87.51 2 0.3
88.11
0.000
36,72 2 0.4
95.77 = 0.3
9:05
TGAT
4 0.734+077 1.1
18. 30
47.80 0.8
45.01 1.3
2.70
53.26
1.0
#1.52
28. 26
Incuctive
Ennon
TGN
0.794
45.24 1
13. 30
54.98 2.3
53.93 4.0
1.05
58.92 1/4
82,38 1.2
23. 46
DyCiFormer
45.24 0.7
67.64
N6.01
17:74
71.45+004 60
10. 82
0.
0.
75.19 0.2
82.25 =
7.06
ICEWS1819
TGN
71.45
1
69.69
70.39
0.70
70:01 0.6
84.01
DyGFormer
60.63 2 0.8
71.45 a 0.5
10.82
78.14 0.3
77.30 0.8
0.44
79.79 0.4
87 $7,50 =: 0.6
8.03
We report the results in Figs. 5 and 6 CROSS framework consistently performs best and exhibits remarkable robustness even under high perturbation rates. This may be due to CROSS's 's ability to effectively harness the valu- able temporal semantic information, which aids in mitigating the adverse impact of structural noise attacks. Furthermore, the results of atten- tion weights reveal that the CROSS framework can effectively down- wwinh the noisy neighbors during encoding. This may be the key reason why CROSS could achieve superior performance and exceptional robustness. Building on this observation, we conduct a case study to visualize the attention weights and the learned representations during encoding in Sec.
Our
Attestbu Weigle
volar
Byltheneer
Figure 6: Attention weights from randomly selected nodes to their perturbed neighbors on GDELT with the perturbation rate of 50%.
Robustness for encoding layers. Next, we study the model robustness to the number of encoding layers. Specifically, we conduct a series of experiments with varying numbers of encoding layers L = ( 2, 3, 4, 5). A larger number of encoding layers facilitates a deeper integration of the cross-modal information. Other details are provided in Sec. Of the Appendix. The results are depicted in Fig. Our CROSS framework also reveals outstanding robustness the encoding layers, where the performance improvements over their respective backbones become more pro- nounced as the number of layers increases. Such robustness likely stems from the in- valuable semantic information and the suf- ficient fusion between semantics and struc-
Laww
Law
(a) TGAT as the backbone. (b) TGN as the backbone.
Figure 7: Robustness study for encoding layers Enron. tures, whereas graph structures alone may carry high-order irrelevant or spurious information. We also find that the CROSS framework always achieves peak performance with a larger L. This can be attributed to the deeper information exchange of our cross-moda. mixer, which fully amplifies the mutual reinforcement between semantics and structures.
4.5
Ablation Study
We conduct three groups of ablation experiments. including model components, textual inputs and their encoding strategies, as well the choice of LLMs.
Ablation for model components. We start the ablation study by evaluating the contributions of the key components of our model. Detailed in-
Cile Minee
formation for each variant provided in Sec. K
of Appendix. The results are detailed in Fig. g We can see that incorporating all components re-
sults in the best performance, while the removal of any single component leads to a performance drop. This highlights the efffectiveness of cach
Figure 8: Ablation study for model components using TGN model as the backbone.

---

component in CROSS. Notably, mixing all semantic representations does not improve model perfor- mance. This may be attributed to the unexpected inclusion of overly outdated semantic information when passing through our cross-modal mixer, thus hindering the quality of final representations.
Ablation for raw texts and L.LM-generated texts. We then conduct ablation study compare the raw texts (i.e., the original text attributes in TTAGs) with the LLM LLM-ggneraaed texts (i.e., neighborhood summaries produced by the LLMs) using various encoding mechanisms. We present the results in Tab. 3 and other details can be seen in Sec. K Firstly, our framework outperforms all other variants, reaffirming its effectiveness. Besides, the results under semantic encoding prove the validity and expressiveness of the pure LLM- -LMe-genrrtted texts. Interestingly. the performance of LLM-generated texts yields only marginal improvements or even slight degradation when performing structural encoding. We infer that this occurs because the structural encoding introduces excessively irrelevant information from high-order relations, whereas the other two encodings directly capitalize on the node's ss own LLM- ----eerrrteed texts, thus integrating more focused and relevant information. This strongly demonstrates the necessity and superiority of the design of our co-encoding mechanism.
Ablation for different LLMs. We extend our ablation study with various LLMs, in- cluding DeepSeek-v2 [9] (default), GPT-
Eoeplick
Breplind Lanaatek
4o [3I. Llama3-a 8b
32 Vicuna-7b 333,
and Mistral- 7b [34.. Details and results are 9 respectively. We
put in Sec. K and Fig.
find that DeepSeek-v2 performs best and all variants outperform their corresponding TGNN backbones, reaffirming the effec- tiveness of the proposed CROSS framework.
TCM
T0AT
Figure 9: Ablation study for different LLMS on GDELT.
Conclusion and Future Work
In this paper, we focus on the under-explored problem of TTAG modeling and propose CROSS, which extends existing TGNNs to effectively unify text semantics and graph structures with LLMs. By introducing the Temporal Semantics Extractor, we can enhance the LLMs to dynamically extract the text semantics within nodes' neighborhoods. The Semantic-structuuall Co-encoder then integrates semantic and structural information, enabling bidirectional reinforcement between both modalities. As for future work, we will consider more complex designs of our cross- modal mixer for achieving better representation fusion, such using the time decay mechanism.
Acknowledgments
This work is funded in part by the NSFC under grant No. 62372013 and No. 62206056, the NSF through grant IIS-2106972, the CIPSC-SMP- Large Model Cross-Disciplinary Fund, and also supported by Tencent Wechat Group. The authors would like to express their gratitude to the reviewers for their feedback, which has improved the clarity and contribution of the paper. The first author, Dr. Zhang, also wants to thank Yifeng Wang for his efforts and support in this work.
References
[1] Seyed Mehran Kazemi, Rishab Goel, Kshitij Jain, Ivan Kobyzev, Akshay Sethi, Peter Forsyth, and Pascal Poupart Representation learning for dynamic graphs: A survey, Journal of Machine Learning Research, 21(70):1-73, 2020.
[2] Guotong Xue, Ming Zhong, Jianxin Li, Jia Chen, Chengshuai Zhai and Ruochen Kong. Dynamic network embedding survey. Neurocomputing, 472:212-223, 2022.
[3] Tao Zou, Yuhao Mao, Junchen Ye, and Bowen Du. Repeat-aware neighbor sampling for dynamic graph learning. In Proceedings of the 30th ACM SIGKDD Conference Knowledge Discovery and Data Mining, pages 4722-4733 2024.
[4] Da Xu, Chuanwei Ruan, Evren Korpeoglu, Sushant Kumar, and Kannan Achan, Inductive representation learning temporal graphs. arXiv preprint a7Xiv:2000.07962, 2020.
10

---

[5] Emanuele Rossi, Ben Chamberlain, Fabrizio Frasca, Davide Eynard, Federico Monti, and Michael Bronstein. Temporal graph networks for deep learning on dynamic graphs. arXiv preprini arXiv:2006. 10637, 2020.
[6] Yao Zhang. Yun Xiong, Yongxiang Liao, Yiheng Sun, Yucheng Jin, Xuehao Zheng, and Yangyong Zhu. Tiger: Temporal interaction graph embedding with restarts. In ACM Web Conference, 2023.
[7] Le Yu, Leilei Sun, Bowen Du, and Weifeng Lv. Towards better dynamic graph learning: New architecture and unified library. arXiv preprini arXiv:2303. 13047, 2023.
[8] Shenyang Huang, Farimah Poursafaei, Jacob Danovitch, Matthias Fey, Weihua Hu, Emanuele Rossi, Jure Leskovec, Michael Bronstein, Guillaume Rabusseau, and Reihaneh Rabbany. Tempo- ral graph benchmark for machine learning on temporal graphs. Advances in Neural Information Processing Systems, 36, 2024.
[9] Siwei Zhang, Xi Chen, Yun Xiong, Xixi Wu, Yao Zhang, Yongrui Fu, Yinglong Zhao, and Jiawei Zhang. Towards adaptive neighborhood for advancing temporal interaction graph modeling. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2024.
pages
[10] Yuxing Tian, Yiyan Qi, and Fan Guo. Freedyg: Frequency enhanced continuous-ting dy- namic graph model for link prediction. In The Twelfth International Conference on Learning Representations, 2024.
[11] Xiaodong Lu, Leilei Sun, Tongyu Zhu, and Weifeng Lv. Improving temporal link prediction via temporal walk matrix projection. arXiv preprint arXiv:2410.04013, 2024.
[12] Jiasheng Zhang, Jialin Chen, Menglin Yang, Aosong Feng, Shuang Liang, Jie Shao, and Rex Ying. Digb: A comprehensive benchmark for dynamic text-attributed graphs. Advances in Neural Information Processing Systems, 2024.
[13] Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang. and Ming Zhou. Minilm: Deep self-attention distillation for task-agnostic compression of pre-trained transformers. Advances in Neural Information Processing Systems, 33:5776-5777 2020.
[14] Wenting Zou, Xiao Hu, Zilong Pan, Chenglu Li, Ying Cai, and Min Liu. Exploring the relationship between social presence and learners' prestige in mooc discussion forums using automated content analysis and social network analysis. Computers Human Behavior, 1 115:106582, 2021.
[15] Jun Hao, Lili Pei, Yongxi He, Zhenzhen Xing, and Yuhan Weng. Tekgen: Graph convo- lutional network for aspect-based sentiment analysis with three-channel knowledge fusion. Neurocomputing, 600: 100:128163, 2024.
[16] Xubin Ren, Wei Wei, Lianghao Xia, Lixin Su, Suqi Cheng, Junfeng Wang, Dawei Yin, and Chao Huang. Representation learning with large language models for recommendation. In Proceedings of the ACM on Web Conference 2024, pages 3464 3464-3475, 2024.
[17] Zhikai Chen, Haitao Mao, Hang Li, Wei Jin, Hongzhi Wen, Xiaochi Wei, Shuaiqiang Wang, Dawei Yin, Wenqi Fan, Hui Liu, et al. Exploring the potential of large language models (Ims) in learning on graphs. ACM SIGKDD Explorations Newsletter, 25(2):42-61 2024.
[18] Haoyu Wang, Shikun Liu, Rongzhe Wei, and Pan Li. Model generalization on text attribute graphs: Principles with large language models. arXiv preprint arXiv:2502///836 2025.
[19] Aixin Liu, Bei Feng, Bin Wang, Bingxuan Wang, Bo Liu, Chenggang Zhao, Chengqi Dengr, Chong Ruan, Damai Dai, Daya Guo, al. Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model. arXiv preprint Xiw:224055004444 2024.
[20] Xiaoxin He, Xavier Bresson, Thomas Laurent, Adam Perold, Yann LeCun, and Bryan Hooi. Harnessing explanations: Llm-to-Im interpreter for enhanced text-attributed graph representation learning, arXiv preprint arXiv:2305.19523, 2023.

---

[21] Yi Fang, Dongzhe Fan, Daochen Zha, and Qiaoyu Tan. Gaugllm: Improving graph contrastive learning for rext-attributed graphs with large language models. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 747-758, 2024.
[22] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Fukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[23] Seyed Mehran Kazemi, Rishab Goel, Sepehr Eghbali, Janahan Ramanan, Jaspreet Sahota, Sanjay Thakur, Stella Wu, Cathal Smyth, Pascal Poupart, and Marcus Brubaker. Time2vec: Learning vector representation of time. arXiv preprint wrXiv:197.0053211 2019.
[24] Srijan Kumar, Xikun Zhang, and Jure Leskovee. Predicting dynamic embedding trajectory in temporal interaction networks. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 1269-1278, 2019.
[25] Rakshit Trivedi, Mehrdad Farajtabar, Prasenject Biswal, and Hongyuan Zha. Dyrep: Learning representations over dynamic graphs. In International conference on learning representations, 2019.
[26] Yanbang Wang, Yen-Yu Chang, Yunyu Liu, Jure Leskovee, and Pan Li. Inductive representation learning in temporal networks via causal anonymous walks. arXiv preprint arXiv:2101.05974, 2021.
[27] Amauri Souza, Diego Mesquita, Samuel Kaski, and Vikas Garg. Provably expressive temporal graph networks. Advances in Neural Information Processing Systems, 11:32257-32263 2022.
[28] Lu Wang, Xiaofu Chang, Shuang Li, Yunfei Chu, Hui Li, Wei Zhang, Xiaofeng He, Le Song, Jingren Zhou, and Hongxia Yang. Tel: Transformer-basee dynamic graph modelling via contrastive learning. arXiv preprint arXiv:2105.07944 2021.
[29] Weilin Cong, Si Zhang, Jian Kang, Baichuan Yuan, Hao Wu, Xin Zhou, Hanghang Tong, and Mehrdad Mahdavi. Do really need complicated model architectures for temporal networks? arXiv preprint arXiv:2302.1 1636, 2023.
[30] Amit Roy, Ning Yan, and Masood Mortazavi. Llm-driven knowledge distillation for dynamic text-attributed graphs. arXiv preprint arXiv:2502. 10914, 2025.
[31] OpenAl. Gpt-4o system card, May 2024. Available online.
[32] AI@Meta. Llama 3 model card, 2024. Available online.
[33] Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. Vicuna: An open-source chatbot impressing gpt-4 with 90%* * chatgpt quality. See https://vicuna. Imsys. org (accessed 14 April 2023), 2(3) 2023.
[34] Albert Q Jiang. Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. arXiv preprint 210.02231. 2023.
[35] Xi Chen, Yongxiang Liao, Yun Xiong, Yao Zhang, Siwei Zhang, Jiawei Zhang, and Yiheng Sun. Speed: Streaming partition and parallel acceleration for temporal interaction graph embedding. arXiv preprini arXiv: /4/29, 2023.
[36] Zhihao Wen and Yuan Fang. Trend: Temporal event and node dynamics for graph representation learning. In Proceedings of the ACM Web Conference 2022, pages 11 1159-1169 2022.
[37] Sheng Xiang, Mingzhi Zhu, Dawei Cheng, Enxia Li, Ruihui Zhao, Yi Ouyang, Ling Chen, and Yefeng Zheng. Semi-supervised credit card fraud detection via attribute-driven graph representation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 14557-14565, 2023.
12

---

[38] Kuo Yang, Zhengyang Zhou, Qihe Huang, Limin Li, Yuxuan Liang, and Yang Wang. Improv- ing generalization of dynamic graph learning via environment prompt. Advances in Neural Information Processing Systems, 37:70048-70077 2024.
[39] Yannis Karmim, Mare Lafon, Raphaël Fournier-S' Niehhota, and Nicolas Thome. Supra-laplacian encoding for transformer on dynamic graphs. Advances in Neural Information Processing Systems, 37:17215-17244 2024.
[40] Jintang Li, Ruofan Wu, Xinzhou Jin, Boqun Ma, Liang Chen, and Zibin Zheng. State space models on temporal graphs: A first-principles study, Advances Neural Information Processing Systems, 17:127000-227058 2024.
[41] Katherine Tieu, Dongqi Fu, Yada Zhu, Hendrik Hamann, and Jingrui He. Temporal graph neural tangent kernel with graphon- guaranteed. Advances in Neural Information Processing Systems, 37:94568-94606 2024.
[42] Xingtong Yu, Zhenghao Liu, Xinming Zhang, and Yuan Fang. Node-time conditional prompt learning in dynamic graphs, 2025.
[43] Ke Cheng, Linzhi Peng, Pengyang Wang, Heng Chang, Junchen Ye, and Bowen Du. On the scalability of temporal relative positional encoding for dynamic link prediction. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2, pages 298-309, 2025.
[44] Ke Cheng, Peng Linzhi, Junchen Ye, Leilei Sun, and Bowen Du. Co-neighbor encoding schema: A light-cost structure encoding method for dynamic link prediction. In Proceedings of the 30th ACM SIGKDD Conference Knowledge Discovery and Data Mining, page 1222-432,220 2024.
[45] Yuanyuan Xu, Wenjie Zhang, Xuemin Lin, and Ying Zhang. Unidyg: A unified and effective representation learning approach for large dynamic graphs. IEEE Transactions on Knowledge 2025.
and Data Engineering, pages
[46] Yuanyuan Xu, Wenjie Zhang, Ying Zhang, Xuemin Lin, and Xiwei Xu. Unlocking multi-modal potentials for link prediction on dynamic text-attributed graphs, 2025.
[47] Xi Chen, Yun Xiong, Siwei Zhang, Jiawei Zhang, Yao Zhang, Shiyang Zhou, Xixi Wu, Mingyang Zhang, Tengfei Liu, and Weiqiang Wang. Dtformer: A transformer-based method for discrete-time dynamic graph representation learning, 2024.
[48] Palash Goyal, Sujit Rokka Chhetri, and Arquimedes Canedo. dyngraph2vec: Capturing network dynamics using dynamic graph representation learning. Knowledge- Based Systems, 187:104816. 2020.
[49] Ryan A. Rossi, Rong Zhou, and Nesreen K. Ahmed. Deep inductive graph representation learning. IEEE Transactions on Knowledge and Data Engineering, 32(3):438-452, 2020.
[50] Xiaofu Chang, Xuqin Liu, Jianfeng Wen, Shuang Li, Yanming Fang, Le Song, and Yuan Qi. Continuous dynamic graph learning via neural interaction processes. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pages 145-154, 2020.
[51] Aldo Pareja. Giacomo Domeniconi, Jie Chen, Tengfei Ma, Toyotaro Suzumura, Hiroki Kaneza- shi, Tim Kaler, Tao Schardl, and Charles Leiserson. Evolvegen: Evolving graph convolutional networks for dynamic graphs. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 1363-5370. 2020.
[52] Yiwei Wang, Yujun Cai, Yuxuan Liang, Henghui Ding, Changhu Wang, and Bryan Hooi. Time-aware neighbor sampling for temporal graph networks. arXiv preprint arXiv:2/12.0988 2021.
[53] Ziwei Fan, Zhiwei Liu, Jiawei Zhang, Yun Xiong, Lei Zheng, and Philip S Yu. Continuous-time sequential recommendation with temporal graph collaborative transformer. In Proceedings of the 30th ACM international conference on information knowledge management, pages 433 442, 2021.
13

---

[54] Jianfei Gao and Bruno Ribeiro. On the equivalence between temporal and static equivariant graph representations. In International Conference on Machine Learning, pages 7052-7076. PMLR, 2022.
[55] Yizhou Chen, Anxiang Zeng, Qingtao Yu, Kerui Zhang, Cao Yuanpeng, Kangle Wu, Guangda Huzhang, Han Yu, and Zhiming Zhou. Recurrent temporal revision graph networks. Advances in Neural Information Processing Systems, 36, 2024.
[56] Zeyang Zhang, Xin Wang, Ziwei Zhang, Haoyang Li, Yijian Qin, and Wenwu Zhu. Llm4dyg: Can large language models solve spatial-temporal problems on dynamic graphs? In Conference on Knowledge Discovery and Data Mining (ACM SIGKDD), 2024.
[57] Siwei Zhang, Yun Xiong, Yao Zhang, Xixi Wu, Yiheng Sun, and Jiawei Zhang. ilore: Dy- namic graph representation with instant long-term modeling and re-oceurrence preservation. In Proceedings of the 32nd ACM International Conference Information and Knowledge Management, pages 3216-3225, 2023.
[58] Yiwei Wang, Yujun Cai, Yuxuan Liang, Henghui Ding. Changhu Wang, Siddharth Bhatia, and Bryan Hooi. Adaptive data augmentation temporal graphs. Advances in Neural Information Processing Systems, 34:1440-1452, 2021.
[59] Xuhong Wang, Ding Lyu, Mengjian Li, Yang Xia, Qi Yang, Xinwen Wang, Xinguang Wang, Ping Cui, Yupu Yang, Bowen Sun, et al. Apan: Asynchronous propagation attention network for real-time temporal graph embedding. In Proceedings of the 2021 international conference on management of data, pages 2628 2638, 2021.
[60] Unai Alvarez-Rodriguee Federico Battiston, Guilherme Ferraz de Arruda, Yamir Moreno, Matjaž Perc, and Vito Latora. Evolutionary dynamics of higher-order interactions in social networks. Nature Human Behaviour, 5(5):586-555, 2021.
[61] Hongkuan Zhou, Da Zheng, Israt Nisa, Vasileios loannidis, Xiang Song, and George Karypis. Tgl: A general framework for temporal gnn training on billion-scale graphs. arXiv preprint arXiv:2203. 14883, 2022.
[62] Yuhong Luo and Pan Li. Neighborhood--waare scalable temporal network representation learning. In Learning on Graphs Conference, pages 1-1. PMLR, 2022.
[63] Siwei Zhang, Yun Xiong, Yao Zhang, Yiheng Sun, Xi Chen, Yizhu Jiao, and Yangyong Zhu. Rdgsl: Dynamic graph representation learning with structure learning. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, pages 3174-3183, 2023.
[64] Xi Chen, Yateng Tang, Jiarong Xu, Jiawei Zhang, Siwei Zhang, Sijia Peng, Xuchao Zheng, and Yun Xiong. Rethinking time encoding via learnable transformation functions, 2025.
[65] Yanchao Tan, Zihao Zhou, Hang Lv. Weiming Liu, and Carl Yang. Walklm: A uniform language model muning framework for attributed graph embedding. Advances Neural Information Processing Systems, 36, 2024.
[66] Jianan Zhao, Meng Qu, Chaozhuo Li, Hao Yan, Qian Liu, Rui Li, Xing Xie, and Jian Tang. Learning on large-scale text-attributed graphs via variational inference. arXiv preprint 22X1::2200.14709, 2022.
[67] Michihiro Yasunaga, Jure Leskovec, and Percy Liang. Linkbert: Pretraining language models with document links. arXiv preprint arXiv:2203. /5827, 2022.
[68] Junhan Yang, Zheng Liu, Shitao Xiao, Chaozhuo Li, Defu Lian, Sanjay Agrawal, Amit Singh, Guangzhong Sun, and Xing Xie. Graphformers: Gnn-nested transformers for representation learning on textual graph. Advances in Neural Information Processing Systems, 34:28798- 28810, 2021.
[69] Xikun Zhang, Antoine Bosselut, Michihiro Yasunaga, Hongyu Ren, Percy Liang, Christopher D Manning, and Jure Leskovec. Greaselm: Graph reasoning enhanced language models for question answering. arXiv preprint 200.0020,..8866, 2022.
14

---

[70] Bowen Jin, Yu Zhang, Qi Zhu, and Jiawei Han. Heterformer: Transformer-baze deep node representation learning on heterogeneous text-rich networks. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 1020-1031, 2023.
[71] Bowen Jin, Yu Zhang. Yu Meng, and Jiawei Han. Edgeformers: Graph-empooered tranffommess for representation learning on textual-edge networks. arXiv preprint arXiv:2302.110500 2023.
[72] Bowen Jin, Wentao Zhang, Yu Zhang, Yu Meng, Xinyang Zhang, Qi Zhu, and Jiawei Han. Patton: Language model pretraining text-rich networks. arXiv preprint arXiv:2305.12266: 2023.
[73] Yun Zhu, Yaoke Wang, Haizhou Shi, and Siliang Tang. Efficient tuning and inference for large language models on textual graphs. arXiv preprint arXiv:2401. 2024.
[74] Xixi Wu, Yifei Shen, Caihua Shan, Kaitao Song, Siwei Wang, Bohang Zhang, Jiarui Feng, Hong Cheng, Wei Chen, Yun Xiong, and Dongsheng Li. Can graph learning improve planning in llm-based agents? Proceedings of Neural Information Processing Systems, 2024.
[75] Xixi Wu, Yifei Shen, Fangzhou Ge, Caihua Shan, Yizhu Jiao, Xiangguo Sun, and Hong Cheng. When do I1ms help with node classification? a comprehensive analysis, 2025.
[76] Yijun Tian, Huan Song, Zichen Wang, Haozhu Wang. Ziqing Hu, Fang Wang, Nitesh Chawla, and Panpan Xu. Graph neural prompting with large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 19080-19088, 2024.
[77] Jiabin Tang. Yuhao Yang, Wei Wei, Lei Shi, Lixin Su, Suqi Cheng. Dawei Yin, and Chao Huang. Graphgpt: Graph instruction tuning for large language models. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 491-500, 2024.
[78] Jiapu Wang, Sun Kai, Linhao Luo, Wei Wei, Yongli Hu, Alan Wee-Chung Liew, Shirui Pan, and Baocai Yin. Large language models--uided dynamic adaptation for temporal knowledge graph reasoning. Advances in Neural Information Processing Systems, 37:8384-844 2024.
[79] Yutai Duan, Jie Liu, Shaowei Chen, Liyi Chen, and Jianhua Wu. G-prompt: Graphon- prompt tuning for graph classification. Information Processing & Management, 6133) 2024.
[80] Xiaoxin Hc, Yijun Tian, Yifei Sun, Nitesh Chawla, Thomas Laurent, Yann LeCun, Xavier Bresson, and Bryan Hooi. G-retriever: Retrieval-augmentee generation for textual graph understanding and question answering. arXiv preprint arXiv:2402.07630 2024.
[81] Xin Li, Dongze I Lann, Zhihe Lu, Jiawang Bai, Zhibo Chen, and Xinchao Wang. Graphadapter: Tuning vision lasion-language models with dual knowledge graph. Advances in Neural Information Processing Systems, 36, 2024.
[82] Zhuofeng Li, Zixing Gou, Xiangnan Zhang, Zhongyuan Liu, Sirui Li, Yuntong Hu, Chen Ling, Zheng Zhang, and Liang Zhao. Teg-db: A comprehensive dataset and benchmark of textual-edge graphs, 2024.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We clearly mentioned our contributions and scope in both the Abstract and Introduction sections.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We have outlined the potential limitations of the proposed CROSS in Sec. L.5
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
16

---

Justification: We have provided theoretical analysis in Sec. to prove the effectiveness of the proposed CROSS framework with full set of assumptions and a complete proof.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: have provided the descriptions datasets Sec. D.I model architec- ture in Sec. 3 training procedures in Alg. T and the implementation details in Sec. D.3
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
17

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We have provided the codes and data in the supplemental material, as well as at htttps:/annonymous. 4open. Ccckence/r/CRSSArevviiie
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to rum to reproduce the results. See the NeurIPS code and data submission guidelines (https
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: have provided the detailed information of data splits in Sec. 4. and both ayper-parameters and type of optimizer in Sec. D.3
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: As mentioned in Sec. D.3 we repeat the experiments for runs with seeds ranging from 0 to 2 to ensure evaluation reliability and report the averaged performance with the corresponding standard deviations.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
18

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: As mentioned in Sec. D.3 all training is performed on a single server with 72 cores, 128GB memory, and four Nvidha Tesla V100 GPUs.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that didn't 't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: Our work conforms to the NeurlPS Code Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. if there special consid- eration due to laws or regulations in their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification:
We focus on the problem of TTAG modeling, which has no socictal impact.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
19

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our paper does not pose risk of misuse.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best. faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have cited the original paper that produced the code package or dataset used in our paper.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
20

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
batasets.papersstt
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out the asset's s creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The assets such the code introduced in the paper well documented, and will be provided after the publication.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
21

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification: Our work enhances LLMs with dynamic reasoning capability to extract semantic dynamics for TTAG modeling. We have described the usage of LLMs in several parts, including the introduced prompting paradigm Sec. prompt template in Sec. F implementation details of LLM calls in Sec. D.3 scalability clarifications of LLM usage in Sec. as well as the experimental results with different LLMs in Sec. 4.5
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (bttps://neur /wt.e/ucipp..co/mmmeeeee/e////// for what should or should be described.
22

---

f
Row tocka
Rost
Raw Inal
LLM setterstiiii locts
geserssed boxts
M wodeggatdddddddde teats Rosi
Acteation weights bedkbone model Allantiun backlane Prediction probabiliy: 0.17
Mitention weights bodshone stodel Prodliction pexbubility 0.47
Attention weights the bedkbone model Prediction probubilliy: 0.36
Prediktion probubility 0.62
Artcotion weights model Prediction probahilit: 0.25
Attention weights model Prediction probubiltty: 0.79
Attemtion weights Prediction probubility: 0.85
Atteation weights Prediction probabileyr 0-94
Figure 10:
Case study for text semantics and graph structures. Thicker edges denote higher attention weights during encoding. CROSS exhibits exceptional prediction performance and robustness by successfully unifying text semantics and graph structures within TTAGs, which enables the model to adaptively adjust the attention weights to concentrate on more relevant neighbors, thereby achieving improved prediction performance.
A Case Study
In this section, we conduct a case study to qualitatively investigate the effectiveness of CROSS.
Case study for text semantics and graph structures. As illustrated in Fig. TO we select representative node from Googlemap_co CT CT and visualize its raw texts, the L.M--gnneratdd texts from Temporal Semantics Extractor, the attention weights assigned among neighborhoods by both the TGAT backbone and CROSS during encoding, as well as the prediction probability. The value of prediction probability refers to the predicted probability for the corresponding positive edges as defined in [q. [T] where higher values indicate better performance. We find that CROSS demonstrates remarkable capability to unify semantics and structures, which allows the model to adaptively adjust attention weights to concentrate on more relevant neighbors during encoding, thereby achieving improved prediction performance. For instance, CROSS effectively detects the semantic shift in preferences for the target node "Ryan Blanck" across temporal dimensions, which is from "restaurant" to "shopping" (t-2 + fz). Subsequently, automatically reduces the attention weights of neighbors assigned to restaurant nodes (e.g.. "Frank Pepe Pizzeria Napoletana") while increasing the weight for store-related neighbors (e.g., "Stafford Thrift Shop").. This highlights the complementarity of semantics and structures, enabling CROSS to prioritize preferred neighbors with semantic contexts.
Case study for the learned representations. We further conduct a case study for the learned representations directly. Specifically, we ex- tract semantic and structural representations of a selected node among GDELT, visualize their distributions using Kernel Density Esti- mation (KDE), and compare the differences between CROSS and the backbone. As shown in Tab. I| CROSS redistributes the feature space and produces more cohesive representa- tions between semanties and structures. This can be attributed to our co-encoding architecture, which facilitates synergistic reinforcement between
YIN roald
struttuaal imprrsetation
iepiinataiion
20
Figure 11: Case study for the learned representations. CROSS effectively synthesizes more cohesive, unified rep- resentations between semanties and structures.
23

---

modalities. Additionally, we find that semantic and structural representations from backbones have different distributions. This directly proves the complementarity of the two modalities, each cap- tures distinct information that contributes to the final representations. As result, CROSS effectively integrates them and leads to unified representations that incorporate the most salient features both.
B Theoretical Analysis
CROSS is designed from the extraction and unification of semantics and structures within TTAGs. In this section, present theoretical analysis for this idea to prove the effectiveness of CROSS, showing that unifying text semantics and graph structures yields more expressive information than existing TGNNs that rely solely on graph structures. This holds under two key conditions:
Integrity: Text semantics can accurately reflect the graph structures among the neighborhoods of the target node, as they are derived from the LLM -.M--geeratedd texts enriched by the dynamically- informed LLMs' parameterized knowledge.
Complementarity: Text semantics complement the graph structures among the neighborhoods of the target node, providing additional contextual cues that help distinguish subtle relationships.
Theorem 1. Consider the conditions as follows:
1) Integrity: Zr serves as a reliable proxy for Za, thus we have
H (Zc) =, <>
(12)
2) Complementarity: Zr encapsulates orthogonal information that is not captured by Ze, and that is
H 7G, Zr) H (ul - -é', 6'>c.
(13)
Under these conditions, it follows that:
H (y) Zr) < H (y Za).
(14)
where Za denotes the information derived from graph structures, ZT represents the information captured from text semantics, is the target for prediction, and H (-1-) depicts the condition entropy.
We also provide an alternative theoretical analysis prove the model effectiveness, showing that unifying text semantics and graph structures instead of solely considering graph structures is bounded.
Theorem 2. Given the target y, there exists constant B E (0, such that:
I(Za, ZT: 2 I(Za;v) v) + 3 min (y) Za), (())
(15)
where Zg denotes the information derived from graph structures, ZT represents the information captured from text semantics, 1(-I-) denotes mutual information, and (-I-)) depicts the condition entropy.
Proof of Theorem 1. We aim prove that the conditional entropy of unifying both graph structures Za and text semantics Zr, Le., H Za, Zr), is strictly less than the conditional entropy of y solely based on Zo, i.e., H ZG).
We begin with:
Hly Zr).
(16)
Next, we decompose this using the properties of entropy into two phases:
H Hy Za. Zr) = H Zo. aas Zr) + [y: ZM Zr),
(17)
where Z denotes the information arising from the mixed representations of text semantics and graph structures.
We then apply the following upper bound on conditional mutual information as follows:
I(y; Zm Za. Zr) H HZZ Zo, Zr) H( 1y. ZG. Zr)
(18)
< HHHHN Zo, Zr).
Here, the first equality follows from the definition of mutual information, and the inequality holds due to the nonnegativity of conditional entropy.
24

---

By substituting Eq. 18 into Eq. 17 we obtain:
H Z,, Zr) s H (y ZM, ZT) + H (ZM Zc. Zr).
(19)
Since conditional entropy increases when conditioning on fewer variables, it follows that: Zo, ZM, Zr) + (Z ZT) Hy ZT) (Za ZT).
(20)
By applying the "Integrity" and "Complementarity" conditions, we arrive at:
Hly Za: 7m) HM Zr) & H (y) - +c.
(21)
Finally, since G, we conclude:
Hyyl é é' +e Hy)))))
(22)
Consequently, have proven that:
Hy 7r) < Hly Za)-
(23)
This completes the proof.
Proof of Theorem We aim prove that the mutual information of unifying both graph structures Za and text semantics Zrr, i.e. I(Za, v), is lower bounded by the mutual information of solely based on Zg. i.e., I ((Zgy)).
We begin with:
I(Zc, Z):y).
(24)
Using the chain rule of mutual information, I(2G, Zr; can be decomposed as:
T(Za. Zrz3) T(Zo; y) T(Zr: y Ze),
(25)
where I(Z:; y) quantifies the contribution of graph structures ZG about y. and I(Zr:y/ y ZG) reflects the additional information provided by text semantics Z7 conditioned on Ze-
Then, we compute lower bound for the term [(T;yy Za). Based on fundamental properties of entropy, we can obtain:
1(Zriyl Zo) Hy Za) H ZG) = ([r ZG) - H(Zr y, ZG).
(26)
Here, the conditional mutual information /(Tr: LZ) could be as high as the minimum of the conditional entropies Hy Za) and HZrr Zc). Considering any unavoidable loss in information during the unification process, there exists constant (0, 1), such that:
I(Zr:yl Za) 2 3 min H Za). ((Zr Zc)
(27)
Substituting the lower bound from Eq. 27 into Eq. 25
obtain the final bound:
T T(Z: v) (Za: T(ZT: y Zg)
(28)
T (Z + min (H()) (a)
This completes the proof.
L
C Notations and Algorithms
We provide the important notations used in this paper and their corresponding descriptions as shown in Tab, Additionally, for clarity, we present the pseudo-codes CROSS in Algorithm |
D Details of Experimental Setting
D.1 Datasets
In this paper, we select four public datasets 12 from different domains and one real-world industrial dataset. We present their detailed descriptions below, and their statistics summarized in Tab. 5
25

---

Algorithm 1: Training CROSS (one epoch).
node set V; A TTAG 9 = ((u, v, 1)} with node text attributes D and edge text attributes R: The maximum reasoning count m; The number of encoding layer L. Initialize all model parameters and prepare the LLMs;
Input:
//
Temporal Semantics Extractor
2 foreach V do
Derive reasoning timestamps Tu with m by Eq.
Summarize u's textualized neighborhood at i E Tu, with LLM by Eq.
end
// Semantic-struucuraal Co-encoder foreach batch (u, v,t) C G do
foreach 1.2,..., do
Retrieve text semantics from generated summaries and graph structures from neighborhoods for nodes w/v;
Compute pre-mixed semantic representations e(0) (ta) with semantic layer by Eqs.
3
Compute pre-mixed structural representations ( u// (t) with structural layer by Eqs. g
10
Mix and propagate cross-modal representations by Eq.
12
end
13
Derive the final representations (t)v(t) (t) by Eq. 10
Compute loss C by Eq. 11 and backward;
end
Table 4: Important notations and descriptions.
Notations
Descriptions
du
Raw text attribute of node Raw text attribute of edge (u, v,
Tu,0.t
du (f)
LLM L-geneeerd text summary for u's neighborhood at reasoning time
al (t)
Pre-mixed semantic representation for node u at time in the [-th layer Post-mixed semantic representation for node u at time t the 1-th layer Pre-mixed structural representation node u time in the 1-th layer Post-mixed structural representation for node at time the I-th layer Final representation for node at time
e (t)
6 (4)
h (t)
z.(1)
Enron" Enror" originates from a collection of email exchanges among employees of the ENRON energy corporation spanning three years (1999-200 2002). In this dataset, nodes represent employees, and edges correspond to emails exchanged between them. Each node has text attributes that are derived from the employee's 's department and role if such information is available. Each edge attaches text attributes consisting of the raw content of the emails. Edges are sequentially ordered based on the e-mail sending timestamps.
GDELT originates oriiginaassss from the Global Database of Events, Language, and Tone, project aimed at cataloging political behaviors across nations worldwide. In this dataset, nodes represent political entities, such "Egypt" "Kim Jong Un". The textual attributes of nodes are directly taken from the names of these entities. Edges capture the relationships between entities (e.g.. "Make Empathetic Comment" or "Provide Aid"), with the textual attributes of edges being derived from the descriptions of these relationships. Edges are sequentially ordered based on the occurroccu timestamps.
ICEWS18110 sourced from Integrated Crisis Early Warning System project, which serves larger temporal knowledge graph for tracking political events compared to Enron.
https://www.t cmu edu adu/"enron
httpa:// wew.tprojectt....ooo
htipe:://vv..deellpojjeett
https://dataverse harvard. adth/dataaverse1cees
26

---

This dataset is built using events occurring between January 1, 2018, and December 31, 2019. The textual attributes of nodes include the name, sector, and nationality of each political entity, while the textual attributes of edges represent descriptions of the political relationships. All edges are sequentially ordered based on the event-occurring timestamps.
Googlemap_ _ is sourced from the Google Local Data project, which compiles review data from Google Maps along with user and business information in the United States up to September 2021. This dataset specifically focuses on business entities located in Connecticut. Nodes represent users and businesses, while edges correspond to user reviews of businesses. Textual attributes are assigned exclusively to business nodes, encompassing the business name, address, category, and self-introduction. All edges are sequentially ordered based on the review timestamps.
Industrial is sourced from real-world e-commerce transaction records sampled from a mobile payment company, spanning March to June 2024. Nodes in this dataset represent users or merchants while edges denote their transaction records. Each node is enriched with text attributes, such as the user/mmerhaan name and affiliation. Text attributes of each edge include textual details such as price, transaction type, and user review. Besides, all edges are sequentially ordered based on the transaction timestamps, and each node is assigned label indicating whether it fraudulent.
Table 5: Detailed statistics of datasets.
Datasets
Nodes
Links
Times
Duration
Domains
Time Granularity
Enron
42,711
797.907
1,006
years
E-mail
one day
GDELT
6,786
1,339,245
2,591
2 years
knowledge graph
15 minutes
ICEWS1819
31,769
1,100,071
730
2 years
knowledge graph
24 hours
Googlemap_ CT
111,168
1,380,623
55,521
Recommendation
Unix Time
Industrial
1,112,094
3,196,008
months
E-commerce
one day
D.2 Baselines
We evaluate the performance and discuss the capabilities of eleven existing TGNN methods. The details of these methods are as follows:
JODIE (24) is designed to manage temporal graphs in bipartite user-item settings. It employs two Recurrent Neural Networks (RNNs), one for updating the user states and another for the item states. To prevent the issue of outdated node representations, projection layer is added to track the evolution of the embeddings over time.
DyRep (55 incorporates neighborhood information by utilizing a temporal attention-based aggregation mechanism. This approach helps capture the evolving structural features of nodes' local environments in the temporal graph, allowing for more accurate dynamic representations.
TGAT 4 leverages a temporal attention model to aggregate data from temporal-topological neighbors, facilitating the creation of temporal node embeddings. It also introduces a trainable time encoding function that ensures each temporal step is distinctly represented, concept widely adopted in later TGN architectures.
TGN [5] builds upon earlier methodologies by introducing memory system that stores a state vector for each node. This memory is refreshed whenever a node participates in an interaction. The model also features modules for processing messages, updating memory states, and embedding temporal features, which collectively enable the generation of dynamic node representations.
CAWN (6] creates node embeddings using temporal walks. It generates multiple anony- mous random walks starting from target node and encodes them using a Recurrent Neural Network. These encoded walks then combined to form the final temporal representation, which is particularly effective for predicting temporal links.
https://datarepo. eng. eng. ucsd. wwtp://naawlll.oooooeeeeeeeeeeee hut.g//www///nuuuoggroooo//////d
27

---

TCL [28] uses breadth- first search to form temporal dependency sub-graphs. extract- ing sequences of interactions for analysis. Transformer encoder is applied to integrate temporal and structural information, enabling central node representation learning. Addi- tionally, TCL, incorporates a cross-attention mechanism within the Transformer to capture interdependencies between interacting node pairs.
PINT (27] applies injective message passing with temporal focus and incorporates relative positional encoding to improve the model's capability in capturing dynamic patterns within neighborhoods.
GraphMixer [29 employs a link encoder inspired by the MLP-Mixer framework to create temporal embeddings for nodes. Its design includes a fixed time encoding scheme, which demonstrates superior performance compared traditional learnable approaches. The model also utilizes node encoder with mean-pooling aggregate link-based information. DyGFormer ? relies information from I-hop neighbors to learn temporal graph representations. A Transformer encoder with a patching method is used to capture long- range dependencies across nodes. To preserve correlations between source and target nodes, DyGFormer integrates a Neighbor Co-occurrence Feature.
[3 conducts preliminary exploration of dynamic text-attributed graphs. It leverages LLMs as text embedders and introduces an auxiliary knowledge distillation loss to enhance model performance.
LKD4DyTAG
FreeDyG [10] delves the temporal graph g modeling into the frequency domain and proposes a node interaction frequency encoding module that both effectively models the proportion of the re-occurred neighbors and the frequency of corresponding interactions of the node pair.
In addition to the above existing deep learning- TGNNs, we also explore the performance of the LLMs for TTAG modeling. We employ the widely adopted and well-performing LLM, DeepSeck-v2 an open-source project released by DeepSeek, Inc ? DeepSeek-ve v2 is a strong, economical, and efficient mixture-of cocuure-of-eppeeee of-experts language model. For comparison, we test its seoo and one-shot performance for temporal link prediction, which denoted as LLM and LLM respectively. Similar to our temporal reasoning chain in Sec. 3.1 we design task-specific prompt to call with LLMs. Specifically, given the historical interactions of two nodes, we prompt DeepSeek-v2 to directly predict whether these two nodes will interact at a specific future timestamp.
D.3 Implementation Details
Tasks and Metrics. We follow 66] and conduct temporal link prediction under two settings: (i) transductive setting, which predicts links between nodes that have appeared during training; and (ii) inductive setting, where predictions are performed with unseen nodes. During training, we sample an equal number of negative destination nodes as described in Eq. [T Inspired by [8], we employ Mcan Reciprocal Rank (MRR) as the evaluation metric with 100 negative links per positive link during evaluation. We also report the AP and AUC results in Tabs. & g Additionally, we further conduct the node classification task in a practical industrial application for financial risk management using the Industrial dataset from the e-commerce domain. The objective of this task is to predict whether a node is involved in fraudulent activity. Specifically, we follow 66) and pass the learned representations through a two-layer MIP to get the probabilities of fraudulent activity for each node. We adopt the Area Under the Receiver Operating Characteristic Curve (AUC) as the evaluation metric for this task.
Model Configurations. For the training and evaluation, we follow [12 and train all models for 50 epochs and adopt the early stopping strategy under the patience of 5 with an evaluation interval of 5. The learning rate and the batch size across all models and datasets are set to 0,0001 and 256, respectively. We repeat the experiments for runs with seeds ranging from evaluation reliability and report the averaged performance with the corresponding standard deviations. All training performed on a single server with 72 cores, 128GB memory, and four Nvidia Tesla V100 GPUs As for the ayper-parameters, the representation dimensions across all models and datasets are consistently set to 384, and the introduced syper-paramettr, maximum reasoning count m, is set to 8 for all datasets by default. Other hyper-parameters among baselines follow the critical
ensure
V2
https://deepseek.comm
28

---

TCAT
DyRap
TGN
DyCFor TCL
Graphilisor
FreebyG
TGNI
DyGemerrr
MRB (50
10
201
30
Googeenap (TT
30
4u
10
30
30
CDGLT
Peturletion Ghoglesntp
50
30
40
50
Figure 12: Robustness study for noise on GDELT and Googlemap_coo CT with different perturbation rates. (Supplementary results for Fig. 5
Table 6: Ablation study for the raw texts and L.L.-generated texts. Semantic/Structural Encoding refer to the encoding mechanisms that independently perform semantic/stractuuaal layers in Sec. 3.2 impry. indicates the performance impravements of L.M M-generattd texts TeXtLIM over the raw texts Textraw (Supplementary results for Tab. B)
Semantic Encading
Struetural Eacoding
Stractural- Co-cncoding
Damasets
Methods
Textuo
TesiLLM
imprr:
Texbaw
TesiLLM
Testeu
TestLLs((our))
ingir:
TGAT
49.64 0.9
52.02 88
2.38
56.73 04
57.11 +03
0.288
58.20 + 0.2
$1.6 + 1.7
23.34
GDELT
TGN
09
2.38
1.6
56.27
77.95
DyLifermeer
49.64
=2.02
1 2.38
63
6254
1.19
62.73 0.5
$1.28
1
Thandletiv
IGAT TGN DyGFormer
47.03
03
5.56
65.46
Googlemap_com CT
47.03 0.2
98.38 0.1
51:35
71.35
11.601100
10:25
72.64 a
99.92 :: 0.0
1 27 28
47.03
9..
51:35
54.82 2.7
61.76.1...1
6.94
57.02 a 0.4
99.82 à 0.0
42 80
TGAT
78.15=188
48.68
46.74 3.9
$3.33
6.50
47.62 + 2.0
86.97 + 2.8
1 30.36
Industrial
TGN
0.
75.15 =
48.68
54.46 3.0
$3.20 1.0
1.26
+5.45
94.26 + 0.8
34 81
Dytiformer
24.47
48
74.45
74.05
0..
94.78 1.4
IGAI
0.7
19.74
42.01
0.5
15.79
3.78
0.3
64.56 : 1.8
GDELT
TGN
37.72-995 0.
9.74
37.48
34
2.87
38,81
3.8
7
DyGieocmer
0.7
37.72 9.3
1 9.74
0.2 50.61
52.33 + 04
1.72
52.19 0.3
66.37 = 4.4
7 14.18
Indhetive
TEAT TGN DyGFormer
89.53 44
45.10
60.96
66.98
6.02
62.17+002
91.59 0.0
29.40
Googlemap_Co CT
89.53 24
0.7
45.10
67.88
0.2
78.08 1: 1.0
10.20
1.2
92.68 : 0.1
24.51
0.2
0.7
45.15
2.5 $2.98
59.67 0.1
6.69
53.81 0.6
91.74 a 0.1
37.95
IGAT TGN
20.71
2..
25.63
30.04 3.0
33.86 1.2
3.32
34.19 1.2
62.22 2.1
28 08
Indistrial
2.
25.63
38.28
34.11 1.4
40.17
83.23 2.6
43300
DyGivermer
20.71 0.5
23.
25.63
54.20
0..
0.02
$2.30
7 which has performed an exhaustive grid search to identify the optimal syper-parameters across different models. For the details of LLM calls, in addition to the ablation study on different LLMs of Sec. 4.5 we adopt DeepSeck-v2 as the default LLM. All LLM calls on DeepSeek-v2 are performed Language Model as a Service cCoMaaS--coppaible manner via its official Application Programming Interface (API 11
hyper-pane parameters in the widely-used library DyGLib 10
E Related Work
Temporal Graph Neural Networks (TGNNs). Temporal graph neural networks (TGNNs) 35 36
37 38
are designed to generate node representations in temporal graphs where they typically develop various structural encoding mechanisms to summarize the dynamic
graph structures among neighborhoods of the target node 47 48 49
50
[1
[2] [3]
54 [55] [6].
Based on how these mechanisms operate, existing TGNNs can be broadly categorized into two types: message-encoding TGNNs (ME-TGNNs) and walk-encoding TGNNs (WE-TGNNss. ME-
64 capture changing graph structures via message passing mechanisms, where node representation is refined by aggregating messages from neighbors through various aggregation functions [9]. In contrast, WE-TGNNs (26] 27 incorporate temporal structural information into node representations in a different way. They typically sample multiple temporal walks originating from the target node and encode these walks based node occurrence information. Despite their success, above existing TGNNs focus solely on biased encoding mechanisms that prioritize topological dynamics, neglecting the rich text semantics present in temporal text-attributed graphs (TTAGs) [12].
TGNNs [57 58 59
60
61
62,
63
For completeness, note the existence of very recent work [30 that conducts preliminary exploration of TTAG modeling, where LLMS are used to embed texts into features, and a distillation
10 http:://aap--docsddepeeeee
http://pppcccsc..oee
29

---

Table 7: AP results (%) for temporal link prediction in transductive and inductive settings. Results highlighted with blue background indicate the performance and corresponding improvements of our CROSS framework using various TGNN backbones. The best results are highlighted in bold.
Transdactive setting ICEWS1S19
Inductive setting
Encon
GDELT
Gooplemap_CT
Industral
Enron
GDELT
ICEWSIS19
Gonglmap_CT
Induscial
JODIE
97.26
94.87
0.1
0.
93.91 0.4
= 4
52.33
DyRep
a
93.76 0.5
06.04:10.
77:21 1.2
87.64 0.2
90.99 2.7
9
91.76
74.44 2: 1.6
77.23 0.4
TCL.
96.16 9743+
0.00
99.23
89.80
94.33 0.2
93.49
+
9..
-6.17=00
58,02 +00
82.98 + 0.3
CAWN
95.50 47.34
96.37 0.11
91.48
91.19
=0
86229 1.00
88.14 1.1
PINT
0..
96.26 0.2
0.)
00
00
02
=0.11=011
90.91 0..1
Graphidiser
96.27
94.96 0.1
98.00
00
94.18 0.1
0.
90.
96.00=011
7..
82.66 0.2
FreeDyG
97.
95.02 1:0.1
29.08.200.
84.20 0.1
96.22 0.3
92.46:
9.
97.29:003
79.22
$4.11
LKD4DYTAG
9..1
96.26 0.3
99.17
84,88
97.20 0.1
93.02 0.2
91
98.21
80.3
85,88 0.5
TGAT
95.70 0.1
0.1
99.10 0.0
0.
93/60 0.6
91 0.2
0.1
07.81
$1.25 0.2
TGAT+
10.
98.11
29.61
000
02
93,41
Arg.
5.55
1 2:73
2.41
1 0.54
12.86
1 13.11
5.46
1.2.81
1 0.82
1 11.83
12.16
TGN
97.08
95.35
99.05
91.52+011
95.00 0066
94.58
90.23
97.48
90000
85.48 1.5
TGN+
99.71 0.1
98007 0.3
0.3
99.98:00
97.50 0.3
97.60=003
93.36+
98.74
97.45
94.69 #.1
Arg. 4.06
1 1.73
1 2:72
+ 0.68
8.40
2.90
7 3,02
1.133
? 1.26
7.45
1 9.221
DyGFermer
97.90
96.43 0.0
99.17 0.0
97. 21
94.99 0.3
93:45 à 0.1
98.13 0.0
2.
89,81 0.1
DyGiFomerr
98.50 00
99.78
98.25 0..
95.83
99009
97:25
95.90
Arg.1 5.07 1 1.73
2.07
1 0.61
18.81
1 1.22
3.226
2.33
7. 0.96
1 18.51
1: 6.09
Table 8: AUC results ( (%) ) for temporal link prediction in transductive and inductive settings. Results highlighted with blue background indicate the performance and corresponding improvements of our CROSS
framework using various TGNN backbones. The best results are highlighted in bold.
Transdactive setting
Inductive setting
Enno
GDELT
ICEWS1819
Googlemap_Co CT
Induscraa
Earun
GDELT
ICEWS1819
Googlemap CCT
Industrial
JODIE DyRop
0
95.55
0.2
97. 37.56x00 06
85:25
94.34
93.93 0,3
91.41
93.68 2.0
83.83 s 0.4
80,55 0.7
0.9
94.63
95.83
7802
0.3
91.60
90.71
90.60
74.60
0.7
TCL
0.0
99.18 1
89.75 0.2
91.97 0.2
93 0.4
92.69 0.1
98.04 0.02
87.95 1.0
80.58 0.5
CAWN
0.0
95.62 0.0
98.86 06 001
68.51
96.57 0.1
90.92 0,
0..
97.15 0.04
66.39:00..
87.17 0.1
PINT
0..00004
96.27 0.2
99.01 0.1
89.36
98 11 0.3
0.5 98.27 0..
58.24
87.25 0.3
GraphMixer
96.26 0.2
0.1
8008
10.
89.99
95.27
76.57
FreDDyG
0.
96.37 0.2
99.08 0.01
84.37 0.2
95.79 0.1
95.27
0.1
97.92 0.1
$0.58 0.1
84.29 0.2
LKDADyTACI
97.01 0.1
91.22 0.1
98:00 0.1
83.29 0.1
96.44 6:6
92.82 0.4
93.75
96.22+00
78.76 0.2
82.98 0.1
TGAT
95.93 0.
0.1
99.05 0.0
0.4
94.43 0.4
9..
97.67 0..
0.1
05.33*
78.88 0.6
TGAT+
00:0000004
98.17
99
99:97
97.04 0:2
97.53+
14:28:00:
98,54+0.3
97.20
92.15 0.4
5.47
2.52
2.24
0.57
1.444
2.54
0.887
11.87
13.27
TGN
02
95.67 6
99.02
0.11
95. 95.49 0.4
94.86
90.53 a 0.5
97.47 0.0
90:50
83.51 1.6
IGN+
99.70:011
9..14.0.3
003
2000
47.09 E07
97.48
93,89 0.7
9.772
93.17 9004
Arg. 1 3.86
1 1.55
1 2.47
1 0.71
1 8.00
+: 2,20
1 2.62
3.16
1 1.25
6.92
1 9.966
DyGFere
97.78 a
96.52 a 0.0
99.08 a 0.02
80.96 2.2
97.61 0.1
94.24 0.6
93.12 a 0.1
97.93 :: 0.0
77.99 2.7
87,89 0.2
DyGFormere
98.53 93
0.3
99.75 a 0000
99.97
98.24 0.1
98,03 0.7
95.57 : 0.5
98,99 0.1
91.2410.1
94.70 0.1
5.75
1.82
2.01
0.67
19.01
0.63
3.79
2.45
1.06
19.25
6.81
loss is applied to transfer knowledge from LLMs to a TGNN model. However, this method still suffers from the limitations of neglecting both semantic dynamics and cemantic-structrr reinforcement, as continues to rely on the TGNN encoding mechanism during inference (empirically validated in Tab. ( Instead, our proposed CROSS tackles these issues by unifying text semantics and graph structures, which effectively generates cohesive representations that are both context- and structure-aware.
Text-attributed Graphs (TAGs). Text-attributed graphs (TAGs) have been widely adopted in numerous real-world applications [16] (65] To enable representation learning in such graphs, existing methods often combine graph learning approaches with language modeling techniques. Early works focus on integrating pre-trained language models (LMs) with graph neural networks (GNNs). Some of them [66] 67 conduct cascaded architecture. They first use LMs to independently embed texts as node features, which are then fed into GNNs for representations. Unlike these pipelines, other methods adopt a hybrid GNN-LM architecture to jointly detect both semantic and structural information. Unfortunately, all these methods overlook the potential temporal information inherent in graphs, limiting their applicability to TTAG modeling. Moreover, they cannot be directly applied to TTAGs, as TTAGs and TAGs differ technically. For instance, TTAGs involve a sequence of timestamped interactions with both node and edge attributes, while TAGs typically rely on adjacency matrices with only node attributes.
[68] 69
70
7 72 [
30

---

Large Language Models (LLMs) for Graph Text Augmentation (LLM44TAug). In recent years, the rising prominence of large language models (LLMs) [8] 74 75] such as DeepScek-v2/3 [9]] has
underscored their exceptional potential to revolutionize graph modeling 76 77 78 They typically harness the prompt response of LLMS as the external knowledge to fortify overall model performance,
either for text augmentation [20] 79 80) or for structure refinement [21 [1]. However, all the above methods focus on static text-attributed graphs. LLMs used in these methods are typically limited by static corpora input for reasoning, making them ill-suited to capture the temporal dynamics of TTAGs. this paper, we propose to enhance LLMs with dynamic semantic summarization reasoning capability, effectively detecting semantic dynamics for TTAG modeling.
F Prompt Template
we provide simplified example of the prompts used to construct the temporal reasoning chain within our Temporal Semantics Extractor. Here, we present complete example of the prompts applied to the Googlemap_Co dataset as depicted in Fig. 13 Only minor keyword adjustments are required for the prompts when applied to other datasets. Specifically, for the Enron dataset, the terms "item" and "review" are replaced with "user" and "email". Similarly, for the GDELT and ICEWS1819 datasets, these terms are substituted with "entity" and "relation". For the industrial datasets, "item" and "review" are replaced with "user" and "transaction".
In Sec. 3.1
Gual 4: Summarize the historical reviews Maureem Sobel' at the current timestamp and provide your semantic understanding for them.
Descriptions xSx
Current timestamp #: 27639
Recent reviews of 'Maureen Sobel #:
timestamp: 27630
permed by a very experienced itzm:: 'Name: XXS. Description: xox renew: 'Always clean and monitared, Have good selection of stachines ...'
'Name: x5x. Descriptio: xax
timestamp: 27246
2. timestamp: 26330
ite:: 'Name: XXx. Descriptin: xax review: 'Hasic auto parts store with lackadaisical stereorype staff who ...'
.....
Provide the summary STRICTLY this form: Summary: XXx.
Figure 13: An example of the prompt used to query the LLMs Googlemap_ CT.
G Scalability Clarification of LLM Usage
In this section, we provide a scalability clarification of LLM usage within the CROSS framework, including presenting cost statistics and outlining the strategies employed to improve efficiency.
Cost statistics of LLM usage. DeepSeek-v2 [19 is renowned for its exceptional performance and cost-efficiency, which provides excellent balance between quality and affordability. Therefore, we select DeepSeek-v2 as the default LLM in our CROSS framework. Here, we present the cost statistics for calling DeepSeek-v2 API across the five datasets in Tab. for reference. Additionally, we want to emphasize that the DeepSeek-v2 v2 API imposes no theoretical rate limits. During practical implementation, we leverage multithreading techniques conduct multiple network requests simul- tancously under 80 concurrent processes, enhancing program parallelism and optimizing response time consumption. As for the money cost, DeepSeek-v2 prices at $0.00027 per 1,000 input tokens and $0.0011 per 1,000 output tokens at the time of our experiments.
Table 9: Cost statisties of LLM usage.
Datasets
# Input Tokens
Output Tokens Time Consumption (s)
Money Cost ($)
Enron GDELT
20,640,423
4,471,765
4,528
10.49
4.194,490
1,049,842
4,145
2.29
(CEWS1819
21,844,799
5,410,145
6,517
11.85
Googlemap CT Industrial
144,712,376
21,214,481
21,581
62.41
296,273,271
30,294,172
41,729
113.3
Strategies to improve the efficiency of LLM usage. It is important to highlight that we have implemented several strategies to enhance the efficiency of LLM usage as follows: (i) The frequency of LLM calls for each node is carefully constrained. We introduce maximum reasoning count to
31

---

constrain the number of reasoning steps for each node as described in Eq. D This design reduces the computational complexity of LLM calls from O(E)) to O(V)). where El and [V] represent the total number of edges and nodes respectively. (ii) Smaller L.LMs can serve cost-effective alternatives. Smaller L.Ms within the CROSS framework still demonstrate notable performance as shown in Sec. S.3 providing more cost-effective alternative to larger LLMs like DeepSeek-v2 or GPT-4o. (iii The LLM- LLM-generated texts are consistently reused to avoid repeated querying. Our method requires only single query to the LLMs, with the summaries being stored for subsequent use. These LLM -LM-generated texts can be reused for other tasks or integrated into other methods. We will make the LLM -gMeeraed texts publicly available if this paper can be accepted.
Based on the clarifications above, we argue that the proposed CROSS framework is applicable and effective for large industrial-scale TTAGs. This is because CROSS's success on our Industrial e-commerce systems has demonstrated its practical scalability, and the scale We will also provide the complexity
dataset from
of this dataset surpasses that of many common domains [82]
analysis for CROSS's components in Sec.
H Efficiency Study
In this section, we conduct an efficiency study by comparing model performance and training time per epoch. The results are presented in Fig, [4] where the top-left indicates better performance with higher efficiency. From the results, we observe that CROSS strikes better trade-off between effec- tiveness and efficiency, achieving the best performance while maintaining moderate, acceptable computational cost. These results further highlight the scalability and practicality of our proposed framework, making it well-suited for large-scale and industrial applications.
JNDEE
Datay
2
Selia
tyllyy
HOX
Hi
Tuh
Cann
Mispittion
Chack
200
(a) Enron.
(b) GDELT.
(c) ICEWS1819.
(d) Googlemap_
Figure 14: Comparison between model performance and training time per epoch. CROSS exhibits better trade-off between effectiveness and efficiency, achieving the best performance while maintaining moderate, acceptable computational cost.
I Parameter Study
To balance granularity and efficiency, we set maximum reasoning count (m in Eq. [ to constrain the number of LLM reasoning steps. Now we study how this hyper-parameter impacts performance and plot the results with varying in Fig. [5 Reasoning too infrequently (small m) may make the L.Mss cannot effectively com- prehend the semantic dynamics around nodes and thus result in degraded per- formance, while reasoning too frequently (large m) may cause the LLMs to fail to capture long-term semantic shifts. also worth noting that different models exhibit varying sensitivities, and m 8 seems to be generally sweet choice.
1000000
TeAT
(a) TGAT the backbone. Figure 15: Parameter study with differrnt maximum rea- soning count yn on GDELT. Results of w/o TSE are also pre- sented for reference.
(b) TGN as backbone.
J Details for Robustness Study
Robustness study for noise. As stated in Sec. the learned representations from existing TGNNs may rely solely on noisy structural information. To empirically validate this assumption, we strate- gically introduce noise into graph structures surrounding a target node by replacing its neighbors
32

---

with randomly sampled nodes at perturbation rates of p {10%, 20%, 30%, 40%, 50%). To further investigate the impact of noise and assess the model ability handle such conditions. we randomly select a subset of nodes and visualize the attention weights of their perturbed neighbors during encoding using box plots under p 50%.
Robustness study for encoding layers. Next, we study the model robustness to the number of encoding layers. Specifically, we conduct a series of experiments with varying numbers of encoding layers L = 2, 4, 5J. A larger number of encoding layers facilitates a deeper integration of the cross- cross-modal information. We employ TGAT and TGN as the backbones. Unlike previous experiments, we set the neighbor sampling size in each layer to 5 in this group of experiments due to computational constraints. Other
remain set to their default values detailed in Sec. D.3
T0ST
VIAA OI KUSTE 23 KW BEE n/O PI TAXTS
DrenT
Figure 18: Ablation study for model compo- nents using TGAT model the hackbone. Results of the backbone are also included reference. (Supplementary results for Fig. (8)
Figure 16: Attention weights from randomly se- lected nodes to their perturbed neighbors on Enron with the perturbation rate of 50%. (Supplementary results for Fig.
(a) TGAT backbone. (b) TGN as backbone. Figure 19: Parameter study with different max- imum reasoning count m on Enron. Results of w/o TSE are shown for reference. (Supplcmentary results for Fig. 15
(a) TGAT as backbone. (b) TGN as backhone. Figure 17: Robustness study for encoding lay-
ICEWS1819. (Supplementary results for Fig.
K Details for Ablation Study
Ablation study for model components. We start the ablation study by evaluating the contributions of the key components of our model, including the Temporal Semantics Extractor (TSE) See. 3.1 the Semantic-structural Co-encoder (SC) in Sec. 3.2 and the Cross-modal Mixer (CM) in Eq. 9 We remove each component individually, resulting in three variants: w/o TSE. w/o SC, and w/o CM. Moreover, disrupt the Temporal Reasoning Chain constructed by the TSE component via scrambling the chronological order in T. among Eq. which results in variant named w/o TRC. Additionally, to validate the rationale behind the design of our Cross-modal Mixer, we present results indiscriminately mixed. This variant is referred to w/ CM,.
where all semantic representations
A Ablation study for raw texts and I.M--generated texts. We then conduct an ablation study to com- pare the raw texts in TTAGs (i.e,, the original node or edge text attributes) with the I.L.-generated texts (i.e., neighborhood summaries produced by LLM-based Temporal Semantics Extractor). Ex- periments are performed using three encoding mechanisms, including the semantic encoding that performs semantic layers for semantic representations described Sec. 3.2 the structural encoding that conducts semantic layers for structural representations described in Sec. 3.2, and the semantic- structural co-encoding that is used to generate final representations in CROSS. These combinations lead to six variants.
blation study for different LLMss. We further extend our ablation study with different LLMs. In addition to the default LLM DeepSeek-v2 within the CROSS framework, we conduct experiments using various L.LMs with the GDELT dataset, including GPT-4o (3I, Llama3-8h (32), Vicuna-7h
33 and Mistral-7b [34]. Specifically, GPT-40 is proprictary large language model developed
33

---

by OpenA built upon the GPT-4 architecture and optimized for multimodal comprehension and generation. We also invoke it via its official AP! Llama3-8b an open-source auto-regressive language model with an improved transformer structure. Its tuned versions use supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to align with human preferences for helpfulness and safety. Vicuna-7b is built on Llama 2 and fine-tuned using supervised instruction. Its training data comes from user-shared conversations found online. Mistral-7b uses grouped-query attention (GQA) for faster processing and sliding window attention (SWA) to handle long sequences more efficiently, reducing the cost inference.
From the results in Fig. 9 we can find that DeepSeek-v2 performs best, demonstrating its effectiveness. Additionally, is worth noting that all variants significantly outperform their corresponding backbones and the performance differences across variants are relatively minimal. This demonstrates the robustness of the CROSS framework.
L Discussions
L.1 Empirical Analysis for Semantic Dynamics
In this subsection, we provide empirical analysis for the semantic dynamics in TTAGs. To achieve this, we conduct an empirical experiments with three additional variants, including:
w/ FTI (Fixed Time Interval): Instead of the timestamp sampling using fixed interaction intervals in Eq. [ for each node, we compute its interaction time span and uniformly sample timestamps to enable the LLM to summarize at fixed time intervals.
w/ RN (Recent Neighbors): For each node at each timestamp, we directly instruct the LLM to summarize the most recent 20 neighbors.
w/o TE (Time Encoding): We directly remove the time encoding mentioned in Eq.
The results presented in Tab. 10 From the results, find that the w/o TE exhibits only marginal performance drop. This indicates that time encoding alone captures limited temporal dynamics. Instead, our prompting paradigm guides LLMs capture the semantic evolution over time, effectively facilitating text-driven dynamics for TTAG modeling. Furthermore, either summarizing fixed time intervals recent interactions yields suboptimal results. This strongly validates the importance of semantic dynamics.
Table 10: MRR results (%) for temporal link prediction Enron with TGAT and TGN backbones.
Transductive TGAT
Inductive TGAT
TGN
TGN
w/ FTI
82.47
85.20
70.21
71.77
RN
78.46
83.38
65.60
68.87
w/o TE
92.87
93.62
81.40
80.33
CROSS
95.58
95.84
81.52
82.38
L.2 Extension to Edge Classification
CROSS can be easily extended to edge classification. For completeness, we evaluate two edge classification settings: (i) Supervised setting where models are trained directly using edge labels, and (ii) zero-shot setting where models are first trained on temporal link prediction, and 2-layer MLP subsequently fine-tuned on top of the frozen encoder for edge classification. Both implementation and model configurations follow DTGB [12), with DyGFormer adopted as the backbone. From the results, we find that CROSS still achieves the best performance in both supervised and zero-shot edge classification, proving its strong generalization and transferability across tasks. Additionally, all models exhibit performance drop in the zero-shot setting. This highlights the presence of transfer learning challenge in TTAG modeling, which warrants future research.
12 "httpp://openai: com https://openai.
34

---

Table 11: Precision, Recall, and F1 (%) results for supervised and zero-shot edge classification Enron. We omit some baselines, as they have shown inferior performance [12].
Supervised Edge Classification
Zero-shot Edge Classification
JODIE DyRep GraphMixer DyGFormer CROSS
JODIE DyRep GraphMixer DyGFormer CROSS
Precision
65.68
66.25
63.13
66.01
78.62
47.52
50.28
44.28
52.82
65.28
Recall
64.72
63,90
57.35
58.06
72.74
40.25
46.91
42.82
47.36
60.22
F1
64.78
64.32
55.07
56.04
70.91
42.88
44.58
43.66
45.20
58.29
L.3
When Handling Conflicting Modalities
CROSS is developed from the unification of text semantics and graph structures with the assumption of complementarity between both information, which is discussed in Sec. [ and validated in Sec. A While building upon this, we argue that CROSS remains effective even in the presence of potential cross-modal conflicts. This claim is supported by:
Quantitative results. Noise can reduce alignment and amplify conflicts between semantics and structures. Our robustness study for noise in Sec. S.c.5.44 simulates these inconsistencies, and the results show that CROSS consistently performs best and exhibits strong robustness. This validates CROSS's ability to effectively handle such cross- moddll conflicts.
Qualitative results. Similar to our case study in Sec. A we also visualize the learned represen- tations representative node on GDELT as shown in 20g The completely disjoint repre- sentations from backbones confirm the presence of cross-moda conflicts, and CROSS effectively narrows the feature space to produce relatively more unified representations. Such capabilities to handle conflicts can be attributed to our co- encoding mechanism, which adaptively facili- tates synergistic alignment between modalities.
TDW wood
arrecied regrnsezaiion
Figure 20: Case study for the learned representa- tions when handling conflicting modalities.
Vake
0.04
L.4 Complexity Analysis
Now, we provide a theoretical complexity analysis of the two main components of CROSS. Let | represent the number of nodes, D denote the average degree of the node, L indicate the number of encoding layers, and depict the maximum reasoning count. To simplify the calculations, we assume that the input, hidden, and output dimensions are uniformly set to d. In the Temporal Semantics Extractor, construct temporal reasoning chain with maximum reasoning count of m for each node, resulting in a computational complexity of (m Vl). For the Semantic-structure Co-encoder, utilize various existing TGNN encoding blocks the structural layers. Therefore, complexity analysis focuses on the additional cost introduced by the semantic layers. As mentioned in Sec. [3.2 these layers employ series of standard Transformer layers for each node, thus leading to total complexity of O +
L.5 Limitation
One potential limitation of CROSS is that we only focus on 1-hop historical interactions of a specific node as input to the LLMs within our Temporal Semantics Extractor. While effective in many cases, this approach may be suboptimal for scenarios where high-order temporal semantics are critical. However, directly incorporating multi-hop textual neighborhoods into LLM could substantially escalate computational costs and dilute the model ability to capture relevant semantic information. Future work could explore innovative methods to efficiently and effectively capture nodes' high-order semantics, unlocking further potential for improved TTAG modeling.
Another issue could be that A.L.-generated texts do not always perform optimally under different encoding mechanisms. Although LLMs are renowned for their powerful text generation and under- standing capabilities, it is crucial to explore effective encoding mechanisms that can fully maximize the potential of LLM-generated texts, such the proposed Semantic-structust Co-encoder.
35


Reviews:
[{"review_id": "MpSpYTwl9J", "reviewer": "Reviewer_Fqmi", "summary": "The authors tackle the under-explored setting of temporal text-attributed graphs (TTAGs), where both the graph topology and the associated node/edge texts evolve over time. They decompose the problem into (i) extracting dynamic text semantics and (ii) unifying those semantics with structural signals. Specifically, they propose CROSS, which first builds a Temporal Semantics Extractor that prompts an LLM at carefully chosen timestamps to summarize a node’s evolving textual neighbourhood, then feeds those summaries—together with structural features—into a Semantic-Structural Co-encoder that mixes the two modalities layer-by-layer via a lightweight cross-modal MLP mixer. Plugged on top of several standard TGNN backbones, CROSS delivers large gains on four public TTAG benchmarks and an industrial e-commerce graph, and it exhibits robustness to structural noise and depth.", "questions": "- The co-encoder currently mixes only the latest semantic embedding each layer. Have you explored attention-based fusion over the full history rather than simple exclusion of older summaries?\n- Why not compare with strong text-aware TGNNs such as LLM4DyG [1] with dynamic prompting?\n- The theoretical section would be more convincing if there is proof that the error introduced by summarising neighbourhood texts instead of feeding raw sequences is bounded. Such analysis or empirical evidence correlating summary quality with link-prediction gains would be beneficial. \n- Why are the performances of different models with Semantic Encoding in Table 2 exactly the same? \n\n[1] Llm4dyg: Can large language models solve spatial-temporal problems on dynamic graphs?", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "## Strengths\n- CROSS is conceptually novel and insightful: the framework remains TGNN-agnostic and easy to retrofit to existing models. \n- The experimental results are solid: consistently strong empirical gains across backbones, datasets, and transductive/inductive regimes.\n- The paper backs up design choices with extensive ablations (including removing components, different LLMs, varying depth, and synthetic noise) and even demonstrates an industrial fraud-detection case, underscoring practical relevance. \n\n## Weaknesses\n- The novelty of the proposed CROSS is limited. Temporal Semantics Extractor reduces to periodic LLM-based text summarisation, and Semantic-Structural Mixer is a shallow two-layer MLP. Both mechanisms resemble prior LLM-for-graphs adapters and do not introduce principled advances in temporal reasoning or cross-modal fusion.\n- Cross may suffer from complexity and scalability. For web-scale graphs, LLM'based text generation quickly exceeds practical latency and budget limits. The authors report neither empirical runtimes nor FLOPs, and there is no ablation on how accuracy degrades when $m$ or LLM capacity is reduced, so deployability remains unclear. \n- Several competing models (e.g., LKD4DyTAG[1]) also support dynamic textual features, but the experiments only feed them static embeddings; hyper-parameter tuning procedures for baselines are also missing. This makes it hard to attribute performance gains to CROSS rather than to sub-optimal baseline configurations. \n\n\n[1]  Llm-driven knowledge distillation for dynamic text-attributed graphs.", "was_revised": true, "final_justification": "All questions and issues have been addressed. I maintain my recommendation.", "rebuttal": "Many thanks to Reviewer Fqmi for providing thorough and insightful comments. \n\n>**W1:** Comparison with existing works.\n\nUnlike existing works in both LLM4Graph and TGNN domains, we propose **a concise yet effective framework for LLM4TTAG**, which fundamentally opens new avenues for future research in TTAG modeling.\n\nTo improve readability, we first summarize the key differences between existing works and CROSS, with detailed explanations in the subsequent discussion.\n\n||LLM4Graph Domain|TGNN Domain|CROSS\n|:-|:-|:-|:-\n|Structural Dynamics|✘|✔|✔\n|Semantic Dynamics|✘|✘|✔\n|**LLM Dynamic Reasoning**|✘|No LLM|✔\n|**Modality Fusion Strategy**|shallow at output (after encoding)|shallow at input (before encoding)|**hierarchical (during encoding)**\n\n**Problem Novelty.**\n\nWe address a largely underexplored yet practically important problem of TTAG modeling, with limited methods in this area.\n\n**Methodological Novelty.**\n\nWe emphasize the novelty of CROSS by addressing the **fundamental limitations** inherent to both the LLM4Graph and TGNN domains:\n\n* **LLM4Graph domain: LLMs in existing LLM4Graph methods lack dynamic reasoning capabilities and typically adopt shallow output modality fusion after encoding.** \n\t* Most LLMs are pretrained on static corpora without temporal consideration, which inherently limits their capacity for dynamic reasoning. Existing LLM4Graph methods rely on static graphs and still integrate these LLMs through static, one-off reasoning for each node, rendering them ill-suited to capture the temporal dynamics within TTAGs (Line 61).\n\t* Furthermore, these methods integrate structural and semantic representations **only at the output stage**, resulting in shallow modality fusion **after encoding** (Line 782). This limits the model’s ability to capture the intricate interplay between structures and semantics, ultimately leading to suboptimal representations for downstream tasks.\n* **TGNN domain: Existing TGNNs cannot effectively capture the semantic dynamics of TTAGs and often employ shallow input modality fusion before encoding.** \n\t* Existing TGNNs rely heavily on their structural encoders to capture the dynamics of graph structures, overlooking the temporal evolution of text semantics within TTAGs (Line 39). As a result, their learned representations tend to be biased and overly reliant on structural information.\n\t* Additionally, they typically pre-embed textual information as input semantic features and incorporate them **only at the input stage**, leading to shallow modality fusion **before encoding** (Line 45). This early fusion fails to enable mutual reinforcement between structures and semantics, thereby neglecting their interplay for TTAG modeling.\n\nTo tackle the above limitations, in this paper, we propose CROSS, which (1) enhances LLMs with dynamic reasoning capabilities to capture semantic dynamics; and (2) facilitates hierarchical modality fusion throughout the entire encoding process. Specifically:\n\n* **We introduce a temporal-aware prompting paradigm to empower LLMs with dynamic reasoning capabilities for extracting semantic dynamics.** \nUnlike statically employing LLMs, as discussed in Line 125, we propose a novel temporal reasoning chain paradigm to advance LLMs with dynamic semantic reasoning capabilities for TTAG modeling. It adopts a recurrent prompting paradigm to dynamically summarize the evolving neighborhoods of nodes, effectively extracting semantic dynamics within TTAGs.\n* **We propose a cohesive architecture to facilitate hierarchical modality fusion throughout the entire encoding process.** Unlike shallow fusion solely at input or output, as stated in Line 158, we design a novel co-encoding architecture that encourages information propagation **at the layer-encoding stage**, enabling hierarchical modality fusion **during encoding**. It facilitates deep cross-modal unification and allows both modalities to reinforce each other, leading to cohesive representations for TTAG modeling.\n\n>**W2:** Scalability and ablation.\n\n**Scalability.** We provide the scalability clarification in Sec. G (Line 806) and present an empirical efficiency study in Sec. H (Line 835).\n\n* **From the cost results of LLM calls, we find that CROSS remains applicable for large-scale TTAGs, validating its scalability.** We report the token cost and time consumption of LLM calls in Tab. 7 (Line 817). From these results, we find that CROSS maintains acceptable LLM costs even on our industrial dataset, proving its scalability for large-scale TTAGs.\n* **From the training time results, we find that CROSS exhibits no significant extra overhead compared to its backbone, further validating its scalability.** We also report the training time of CROSS in Fig. 13 (Line 841). From these results, we find that the training cost of CROSS does not increase significantly compared to its backbone, proving its scalability once again.\n\n**Ablation.** We perform an ablation study across different LLMs in Fig. 8 (Line 354) and a parameter study for $m$ in Fig. 14 (Line 852). **Lowering LLM capacity or reducing $m$ will lead to slight performance drops.** Nevertheless, we emphasize that all these variants still outperform baselines, reaffirming the effectiveness of CROSS.\n\nWe will revise our manuscript to highlight the above aspects more prominently.\n\n>**W3:** Baseline and hyper-parameter.\n\n**Directly using existing TGNNs to encode dynamic semantic features brings marginal gains or even slight degradation.** In the \"Structural Encoding\" column of Tab. 2 (Line 275), we report the performance of existing TGNNs when fed with raw static texts or dynamic LLM-generated texts. Under this setting, the performance of LLM-generated texts yields marginal gains or even slight degradation. We attribute this to the excessive irrelevant temporal semantics from high-order relations, which pollutes the representations.\n\n**CROSS and its backbone share the same set of well-tuned hyper-parameters, ensuring a fair comparison.** As noted in Line 750, we strictly follow the hyper-parameters from DyGLib for CROSS and baselines, which have been carefully selected via an exhaustive grid search.\n\n>**Q1:** Motivation of Mixer.\n\n**Motivation Analysis.** As discussed in Lines 216–219, the Mixer integrates only the last semantic representation based on three considerations: **efficiency, temporal awareness, and empirical evidence** where mixing all semantics leads to suboptimal results.\n\n**Empirical Evaluation.** Our ablation study (Line 322) provides empirical support for the Mixer design. From the results of the w/ CM$_{\\text{all}}$ variant (mix all semantics), we find that **mixing all semantic representations leads to suboptimal performance.** This is because aggregating all semantic representations into a single vector will lead to uniform attention scores in the next Transformer layer (since this vector closely resembles other semantic representations), making representations indistinguishable and ultimately degrading their quality.\n\n>**Q2:** Comparison with LLM4DyG.\n\n**Technical Comparison.** We first provide a technical comparison between CROSS and LLM4DyG:\n\n* **Similarity.** Both works explore the spatiotemporal reasoning capabilities of LLMs on dynamic graph-structured data.\n* **Key Differences.** CROSS is an LLM-based framework that enhances LLMs' dynamic reasoning capabilities for TTAG representation learning, while LLM4DyG is a benchmark that promotes LLMs' spatiotemporal reasoning capabilities via its designed prompting schemes. Moreover, LLMs in CROSS act as Enhancers for providing dynamic semantics, while in LLM4DyG, they serve directly as Predictors. Additionally, CROSS adopts the time-ordered text stream to describe the graph, whereas LLM4DyG employs triplets with node IDs.\n\nThe key differences are outlined below.\n\n||LLM4DyG|CROSS\n|:-|:-|:-\n|Scope|LLM benchmark with designed prompts|LLM-based framework for representation learning\n|Role of LLMs|Predictor|Enhancer\n|Graph Description in Prompt|Triplets|Time-ordered text stream\n\n**Empirical Comparison.** We also conduct an empirical comparison between CROSS and LLM4DyG. For LLM4DyG, we adopt its prompting strategy (illustrated in Tab. 7 of its publication) to instruct DeepSeek-v2 to perform temporal link prediction. The results are reported in Tab. 1.\n\nFrom the results, we observe that **CROSS significantly outperforms LLM4DyG**. This demonstrates that LLM-as-Enhancer can better exploit the spatiotemporal modeling capabilities of LLMs for TTAG modeling. \n\nTable 1: F1 result (%) for temporal link prediction. \n\n||Enron|GDELT|\n|:-:|:-:|:-:|\n|LLM4DyG (Trans)|41.27|58.74|\n|CROSS (Trans)|**86.72**|**75.28**|\n|LLM4DyG (ind)|33.19|49.01|\n|CROSS (ind)|**75.35**|**60.47**|\n\n>**Q3:** Alternative theoretical analysis.\n\nHere, we provide an alternative theoretical analysis to prove the effectiveness of CROSS, showing that unifying text semantics and graph structures instead of solely considering graph structures is bounded.\n\n> **Theorem 1 (Information Bound).** Given the target task $Y$, there exists a constant $\\beta\\in(0,1]$ such that:\n> \n> $$I(Z_G,Z_T;Y)\\geq I(Z_G;Y)+\\beta\\min\\\\{H(Y|Z_G),H(Z_T|Z_G)\\\\}, \\tag{1}$$\n> \n> where $Z_G$ denotes the information derived from graph structures, $Z_T$ represents the information captured from text semantics, $I(\\cdot|\\cdot)$ denotes mutual information, and $H(\\cdot|\\cdot)$ depicts the condition entropy.\n\nDue to character limitations, we could provide the corresponding proofs at the discussion phase (if permitted). Please feel free to request.\n\n>**Q4:** Clarification of Tab. 2.\n\nAs stated in Line 275, the \"Semantic Encoding\" column of Tab. 2 refers to the setting where **only** the semantic encoder is used, without any structural encodings from TGNNs. Consequently, **the results under this setting are independent of TGNNs**, and thus identical values are reported across them.\n\nThanks again for your invaluable suggestions! We hope we have addressed your concerns."}, {"review_id": "erYY5WTce9", "reviewer": "Reviewer_B85N", "summary": "CROSS (Cohesive Representations Of Semantics and Structures) is a framework designed for modeling Temporal Textual Attribute Graphs (TTAGs). TTAGs combine dynamic textual semantics with evolving graph structures, introducing significant complexity. The study highlights that existing Temporal Graph Neural Networks (TGNNs) do not adequately consider the textual features of dynamic networks.\nTo address this, CROSS decomposes the TTAG modeling process into two stages:\n- Temporal Semantic Extraction: Leveraging large language models (LLMs), language models (LMs), and time encoding to capture temporal semantics in text.\n- Semantic-Structural Information Integration: Using a Cross-modal Mixer to automatically transform and fuse bimodal information at hierarchical granularities.", "questions": "Questions:\n1. Eq. 1 uses n/m as the sampling step. How does the model perform when only sample the nearest m neighbors?\n2. In Line 174, do the k timestamps refer to the previously sampled m timestamps, or to k neighbors (i.e., not sampled timestamps)?\n\nOther questions please refer to Weaknesses.", "limitations": "yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n- The paper addresses a very novel and timely problem, with limited existing work in this area.\n- The writing is clear and easy to follow.\n- The experiments are comprehensive and well-executed.\n\nWeaknesses:\n1. In Eq. 2, m timestamps are sampled for each node u, and for each timestamp, all historical neighbors before that time are retrieved. This implies that the LLM-based summarization process needs to be performed n × m times. Is this computational cost too high?\n2. For the LLM(·; PROMPT) module, since all neighbor text information is retrieved, the input can become extremely long. How do the authors address this issue? \n3. The summarization ability of LLMs typically lies in extracting and rewriting important information from long text. However, in this work, the LLM is neither jointly trained nor specifically fine-tuned to identify task-relevant information. Thus, it is unclear whether the LLM can effectively summarize useful information from all retrieved neighbor texts. The current method simply feeds all neighbor texts into the LLM and applies summarization. Is this approach reasonable? How do the authors ensure that the summarized information is useful to the downstream task?\n4. For temporal text information extraction, the paper first uses an LLM to summarize neighbor texts, then applies a Transformer encoder to embed the summaries into d-dimensional semantic features, and finally concatenates time encodings to capture dynamic patterns.\nHowever, it is unclear whether LLMs can effectively extract temporal dynamics from text, especially without task-specific training. Given this, is most of the model's ability to capture temporal dynamics actually comes from the time encoding, rather than from the textual summaries themselves?\n5. Why is Mean Reciprocal Rank (MRR) used as the evaluation metric for the link prediction task? In most existing temporal link prediction works, AUC/AP is typically adopted. Using MRR makes it difficult to clearly compare the effectiveness of the proposed model with prior methods.\n6. What is the purpose of the robustness study? How is it related to the core design of the proposed model? It is unclear how robustness is a key aspect of the method, yet it occupies a significant portion of the experimental section. Could the authors clarify the motivation and relevance of this analysis?\n7. The paper claims to be the first work on TTAG modeling; however, reference [1] addresses a highly related problem with a similar approach. The authors should discuss the differences and similarities with [1], and if possible, provide experimental comparisons to clarify the contributions of this work.\n\n[1] Unlocking Multi-Modal Potentials for Dynamic Text-Attributed Graph Representation, KDD 2025", "was_revised": false, "final_justification": "", "rebuttal": "Many thanks to Reviewer B85N for the thorough and insightful comments, and we will revise our paper accordingly.\n\n>**W1:** LLM cost.\n\n**Clarification for LLM cost.** We would like to clarify the misunderstanding of the costs for LLMs:\n\n* **For each node, timestamps are sampled over its full interaction timestamps, resulting in only $m$ LLM calls per node instead of $n \\times m$.** As stated in Eq. 1 (Line 138), we sample $m$ timestamps from the complete $n$ interaction timestamps for each node. This reduces the number of LLM calls per node from $n$ to $m (m \\ll n)$.\n* **LLMs are not repeatedly invoked at every timestamp.** As illustrated in the pseudo-code (Line 635), CROSS pre-generates $m$ summaries for each node in advance. During co-encoder training, for node $u$ at time $t$, we only need to derive $u$'s pre-generated summaries that occur before $t$ to construct its representation, thus avoiding redundant LLM calls.\n* **Sampling ensures that the number of LLM calls remains controllable, effectively balancing temporal granularity and cost efficiency.** As shown in Line 133, by restricting the number of LLM calls to $m (m \\ll n)$, we avoid the impractical costs from invoking LLMs at every timestamp. We acknowledge that this sampling may unavoidably introduce information loss, but it indeed helps CROSS to achieve the trade-off between temporal granularity and cost efficiency. We also report the results across different sampling intervals in Fig. 14 (Line 851).\n\n**Evaluation for LLM cost.** We also present the empirical cost statistics of LLM calls in Tab. 7 (Line 817). From the results, we observe that **CROSS maintains acceptable LLM costs even on our industrial dataset**, demonstrating its scalability for large-scale TTAGs.\n\n> **W2:** Lengthy prompt.\n\nAs illustrated in Fig. 12 (Line 805), historical interactions within the prompt are organized in reverse chronological order. For each prompt, we impose a token budget of 4K tokens. Once the prompt exceeds this limit, **earlier interactions are truncated**.\n\n> **W3:** Rationality of model design.\n\nCROSS consists of two decoupled components with different responsibilities for TTAG modeling:\n\n* **From the data perspective**, the LLM is dynamically enhanced to generate evolving summaries that capture semantic dynamics, which is responsible for enriching the textual modality for the original TTAG data.\n* **From the model perspective**, the Co-encoder hierarchically unifies semantic and structural information to produce representations, which is responsible for specific spatiotemporal modeling and task adaptation.\n\nBased on the above clarification, we argue that the design of CROSS (including LLM usage) is reasonable. This is because:\n\n* **In CROSS, the LLM is designed as an Enhancer for data augmentation, rather than a Predictor that requires fine-tuning for task adaptation.** CROSS dynamically harnesses the LLM to capture the semantic dynamics over time, generating reusable dynamic summaries for semantic data augmentation. This LLM is not designed for label generation or task-specific prediction, making fine-tuning unnecessary.\n* **In CROSS, task adaptation is achieved through the Co-encoder, not the LLM.** Recent studies [1, 2] reveal that directly using LLMs for task prediction exhibits limited effectiveness in spatiotemporal problems. Empirical results from Q2 to Reviewer Fqmi and W2 to Reviewer C47o further confirm these limitations in the context of TTAG modeling. These observations highlight the important role of a Co-encoder in bridging the gap between LLMs and TTAG modeling, enabling the learning of expressive representations for downstream predictions.\n* **Off-the-shelf LLM already yields highly effective summaries that enable CROSS to achieve state-of-the-art performance.** As presented in Tab. 1 (Line 242), off-the-shelf LLMs can already generate sufficient summaries to support the best performance in TTAG modeling. This empirical evidence further validates the rationality of model design.\n\n[1] Are Language Models Actually Useful for Time Series Forecasting?, NeurIPS 2024.  \n[2] Can LLMs Understand Time Series Anomalies?, ICLR 2025.\n\n> **W4:** Text-driven dynamics.\n\n**Our prompting paradigm guides LLMs to capture the semantic evolution over time, effectively facilitating text-driven dynamics for TTAG modeling.** This is evident by:\n\n* **Ablation study for text-driven dynamics.** In our ablation study, we evaluate the w/o TRC variant, where reasoning timestamps and their order are shuffled. This prevents the LLM-generated summaries from reflecting the accurate temporal dynamics within TTAGs. As shown in Fig. 7 (Line 330), the w/o TRC leads to **a substantial performance drop**, confirming the presence and significance of text-driven dynamics.\n* **Ablation study for time encoding.** We further evaluate the contribution of time encoding. We introduce an additional variant, w/o TE, where time encoding is directly removed. From the results in Tab. 1, we find that the w/o TE exhibits **only a marginal performance drop**. This indicates that time encoding alone captures limited temporal dynamics for TTAG modeling.\n\nTable 1: MRR results (%) for temporal link prediction using TGN backbone.\n\n||Enron|GDELT|ICEWS1819| \n|:-:|:-:|:-:|:-:|\n|w/o TE (Trans)|93.62|77.05|91.72|\n|CROSS (Trans)|**95.84**|**77.95**|**94.74**|\n|w/o TE (ind)|80.33|55.31|82.65|\n|CROSS (ind)|**82.38**|**56.65**|**84.01**|\n\n> **W5:** MRR metric.\n\nWe report model performance using **AUC and Hits@10 in Tab. 6** (Line 797), and we choose MRR as the main metric because:\n\n* **Similar trends are observed across all metrics**, where CROSS performs best across all datasets and metrics.\n* **MRR has become an increasingly popular metric in temporal graph learning** [1, 2].\n* CROSS boosts many backbones to achieve high scores (> 98%) on metrics like AUC/AP, resulting in similar values across variants. This makes it hard to compare CROSS's performance across different backbones.\n\n[1] TMetaNet: Topological Meta-Learning Framework for Dynamic Link Prediction, ICML 2025.  \n[2] TGB 2.0: A Benchmark for Learning on Temporal Knowledge Graphs and Heterogeneous Graphs, NeurIPS 2024.\n\n> **W6:** Clarification of robustness study.\n\nOur robustness study is both well-motivated and reasonable:\n\n* **Motivation.** We clearly state the motivation of our robustness study in **Lines 274–276**: From the results in Tab. 1 (Line 242), we find that CROSS produces similar results across different structural encoding backbones. We hypothesize this stability is due to the robustness of text semantics, and thus design a robustness study to validate this assumption.\n* **Rationality.** We conduct a robustness study under different graph structural environments, including structural perturbations (e.g., injected noise) and structural expansions (e.g., increased encoding layers). This allows us to evaluate model performance in more degraded or complex structural scenarios. The observed robustness of CROSS emphasizes the significance of text semantics in maintaining effectiveness for TTAG modeling.\n\n> **W7:** Comparison with MoMent.\n\nWe would like to clarify that we do **not** claim to be the first to explore TTAG modeling. In fact, we explicitly mention a recent TTAG method, LKD4DyTAG (AAAI 2025), and include it in our empirical comparison (Tab. 1, Line 242). Our work presents the first attempt to unify text semantics and graph structures with LLMs for TTAG modeling.\n\n**Technical Comparison.** We first compare CROSS with MoMent [1]:\n\n* **Similarity.** Both works adopt a multi-modal perspective for TTAG modeling.\n* **Key Differences.** CROSS emphasizes modality fusion, facilitating deeper unification of semantic and structural signals. In contrast, MoMent focuses on modality alignment via a pre-existing alignment loss. Additionally, CROSS advances LLMs to explicitly capture temporal dynamics within the semantic modality, while MoMent still relies on static node texts to derive semantic features.\n\nThe key differences are summarized below.\n\n||MoMent|CROSS\n|:-:|:-:|:-:|\n|Modality Handling|Alignment|Fusion\n|LLM Usage|✘|✔\n|Semantic Dynamics|✘|✔\n|Structural Dynamics|✔|✔\n\n**Empirical Comparison.** We notice that MoMent was not accepted by KDD 2025, which remains **an arXiv preprint without official code**. Therefore, we re-implement it based on the details in its manuscript. We use TGN as the backbone.\n\nFrom the results in Tab. 2, we find that **CROSS consistently outperforms MoMent**, further validating the effectiveness of CROSS.\n\nTable 2: MRR results (%) for temporal link prediction.\n\n||Enron|GDELT\n|:-:|:-:|:-:|\n|MoMent (Trans)|83.19|88.40\n|CROSS (Trans)|**95.84**|**77.95**\n|MoMent (ind)|67.62|51.50\n|CROSS (ind)|**82.38**|**56.65**\n\n[1] Unlocking Multi-Modal Potentials for Dynamic Text-Attributed Graph Representation, Arxiv 2025.\n\n> **Q1:** Summarizing recent neighbors.\n\nWe conduct additional experiments by summarizing the $b$ most recent neighbors for each node at each timestamp, leading to the variant w/ RN. Notably, this significantly increases the number of LLM calls, resulting in substantial costs. As shown in Tab. 3, **simply summarizing recent neighbors results in inferior performance**. This is likely due to the narrow focus on short-term contexts, which fails to capture the long-range semantic dynamics for TTAG modeling. \n\nTable 3: MRR results (%) for temporal link prediction on Enron under $b=20$.\n\n||TGAT|TGN\n|:-:|:-:|:-:|\n|w/ RN (Trans)|78.46|83.38\n|CROSS (Trans)|**95.58**|**95.84**\n|w/ RN (ind)|65.60|68.87\n|CROSS (Ind)|**81.52**|**82.38**\n\n> **Q2:** Clarification of $k$.\n\nDuring training, for node $u$ at time $t$, we incorporate all $k$ summaries that occur before $t$ to derive its semantic representation. **This avoids information leakage from future texts.** \n\nThank you again for your valuable comments. Your feedback truly helps a lot!"}, {"review_id": "cUTlxGHn2m", "reviewer": "Reviewer_db5y", "summary": "This paper focuses on learning temporal text-attributed graphs (TTAGs) with LLMs. The proposed method first extracts temporal semantics using LLMs and introduces a modal-cohesive co-encoding architecture to integrate semantic information and structural information (generated by existing TGNNs). The experiments demonstrate the superiority of the proposed method.", "questions": "Q1. Does the prompt need to be adapted according to the datasets?\n\nQ2. Why does the cross-modal mixer use only the last semantic embedding as input, while the cohesive representations (Section 3.3) incorporate all $k$ historical semantic embeddings as input? And why is the cross-model mixer not applied only to the output of the last layer?", "limitations": "L1. The applications of the proposed method are limited to node classification and link prediction, even with the inclusion of LLMs.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "S1. The framework is simple and effective. The experiments are well-conducted and demonstrate the superiority of the proposed method.\n\nS2. The writing is good and easy to follow.\n\nW1. The method is relatively straightforward and lacks significant technical innovation. The graph temporal attention layer and transformer encoder are directly borrowed from existing methods without modification. The only novel aspect is the use of LLMs to generate semantic texts, thereby amplifying the semantic information.\n\nW2. However, despite incorporating LLMs, the method does not fully capitalize on their potential. It lacks the ability to generalize to different tasks through zero-shot learning and also cannot be applied to edge classification or text generation.\n\nW3. The overall computational cost is substantial, considering both the calls to LLMs and inference on a traditional DGNN. Furthermore, the temporal graph attention layer incurs a significant computational overhead when there are two layers, as demonstrated in previous studies [R1, R2] (see the costs of TGAT). This issue becomes even more pronounced when there are many candidate destinations, such as one positive destination and 100 negative destinations, as in the case of evaluating the MRR metric.\n\n[R1] Do We Really Need Complicated Model Architectures For Temporal Networks?\n\n[R2] TGB-Seq Benchmark: Challenging Temporal GNNs with Complex Sequential Dynamics", "was_revised": true, "final_justification": "This paper proposes a novel and effective approach to TTAG learning. While I noted its limitation in generalizing to broader tasks through the zero-shot capabilities of LLMs, I believe it provides a strong foundation for future work in this area. Therefore, I recommend accepting the paper.", "rebuttal": "We would like to sincerely thank Reviewer db5y for providing a detailed review with insightful questions.\n\n>**W1:** Comparison with existing works.\n\nUnlike existing works in both LLM4Graph and TGNN domains, we propose **a concise yet effective framework for LLM4TTAG**, which fundamentally opens new avenues for future research in TTAG modeling.\n\nTo improve readability, we first summarize the key differences between existing works and CROSS, with detailed explanations in the subsequent discussion.\n\n||LLM4Graph Domain|TGNN Domain|CROSS|\n|:-|:-|:-|:-|\n|Structural Dynamics|✘|✔|✔|\n|Semantic Dynamics|✘|✘|✔|\n|**LLM Dynamic Reasoning**|✘|No LLM|✔|\n|**Modality Fusion Strategy**|shallow at output (after encoding)|shallow at input (before encoding)|**hierarchical (during encoding)**|\n\n**Problem Novelty.**\n\nWe address a largely underexplored yet practically important problem of TTAG modeling, with limited methods in this area.\n\n**Methodological Novelty.**\n\nWe emphasize the novelty of CROSS by addressing the **fundamental limitations** inherent to both the LLM4Graph and TGNN domains:\n\n* **LLM4Graph domain: LLMs in existing LLM4Graph methods lack dynamic reasoning capabilities and typically adopt shallow output modality fusion after encoding.** \n\t* Most LLMs are pretrained on static corpora without temporal consideration, which inherently limits their capacity for dynamic reasoning. Existing LLM4Graph methods rely on static graphs and still integrate these LLMs through static, one-off reasoning for each node, rendering them ill-suited to capture the temporal dynamics within TTAGs (Line 61).\n\t* Furthermore, these methods integrate structural and semantic representations **only at the output stage**, resulting in shallow modality fusion **after encoding** (Line 782). This limits the model’s ability to capture the intricate interplay between structures and semantics, ultimately leading to suboptimal representations for downstream tasks.\n* **TGNN domain: Existing TGNNs cannot effectively capture the semantic dynamics of TTAGs and often employ shallow input modality fusion before encoding.** \n\t* Existing TGNNs rely heavily on their structural encoders to capture the dynamics of graph structures, overlooking the temporal evolution of text semantics within TTAGs (Line 39). As a result, their learned representations tend to be biased and overly reliant on structural information.\n\t* Additionally, they typically pre-embed textual information as input semantic features and incorporate them **only at the input stage**, leading to shallow modality fusion **before encoding** (Line 45). This early fusion fails to enable mutual reinforcement between structures and semantics, thereby neglecting their interplay for TTAG modeling.\n\nTo tackle the above limitations, in this paper, we propose CROSS, which (1) enhances LLMs with dynamic reasoning capabilities to capture semantic dynamics; and (2) facilitates hierarchical modality fusion throughout the entire encoding process. Specifically:\n\n* **We introduce a temporal-aware prompting paradigm to empower LLMs with dynamic reasoning capabilities for extracting semantic dynamics.** \nUnlike statically employing LLMs, as discussed in Line 125, we propose a novel temporal reasoning chain paradigm to advance LLMs with dynamic semantic reasoning capabilities for TTAG modeling. It adopts a recurrent prompting paradigm to dynamically summarize the evolving neighborhoods of nodes, effectively extracting semantic dynamics within TTAGs.\n* **We propose a cohesive architecture to facilitate hierarchical modality fusion throughout the entire encoding process.** Unlike shallow fusion solely at input or output, as stated in Line 158, we design a novel co-encoding architecture that encourages information propagation **at the layer-encoding stage**, enabling hierarchical modality fusion **during encoding**. It facilitates deep cross-modal unification and allows both modalities to mutually reinforce each other, leading to cohesive representations for TTAG modeling.\n\n>**W2, L1:** Zero-shot and extension to edge classification.\n\nOur focus is on TTAG representation learning. Therefore, we argue that text generation falls outside the scope of this work. Nevertheless, CROSS can be easily extended to edge classification. For completeness, we evaluate two edge classification settings:\n\n* **Supervised setting**: Models are trained directly using edge labels.\n* **Zero-shot setting**: Models are first trained on temporal link prediction, and a 2-layer MLP is subsequently fine-tuned on top of the frozen encoder for edge classification.\n\nBoth implementation and model configurations follow DTGB, with DyGFormer adopted as the backbone. The results on Enron are reported in Tabs. 1 & 2. From these results, we observe that:\n\n* **CROSS still achieves the best performance in both supervised and zero-shot edge classification**, proving its strong generalization and transferability across tasks.\n* All models exhibit a performance drop in the zero-shot setting. This highlights the presence of a transfer learning challenge in TTAG modeling, which warrants future research.\n\nTable 1: Precision, Recall, F1 (%) results for supervised edge classification.\n\n| |JODIE|DyRep|GraphMixer|DyGFormer|CROSS| \n|:-:|:-:|:-:|:-:|:-:|:-:|\n|Precision|65.68|66.25|63.13|66.01|**78.62**|\n|Recall|64.72|63.90|57.35|58.06|**72.74**|\n|F1|64.78|64.32|55.07|56.04|**70.91**|\n\nTable 2: Precision, Recall, F1 (%) results for zero-shot edge classification.\n\n| |JODIE|DyRep|GraphMixer|DyGFormer|CROSS| \n|:-:|:-:|:-:|:-:|:-:|:-:|\n|Precision|47.52|50.28|44.28|52.82|**65.28**|\n|Recall|40.25|46.91|42.82|47.36|**60.22**|\n|F1|42.88|44.58|43.66|45.20|**58.29**|\n\n>**W3:** Costs of LLM calls and model training.\n\nWe analyze the cost of LLM calls in **Sec. G (Line 806)** and provide an empirical efficiency study of model training in **Sec. H (Line 835)**.\n\n* **Despite involving LLM calls, CROSS remains applicable for large-scale TTAGs.** We report the token cost and time consumption of LLM calls in Tab. 7 (Line 817). From these results, we find that CROSS maintains acceptable LLM costs even on our industrial dataset, demonstrating its scalability for large-scale TTAGs.\n* **During training, CROSS exhibits no significant extra overhead compared to its backbone, as they share the same number of encoding layers.** We report the training times of models in Fig. 13 (Line 841). The cost of CROSS does not increase significantly compared to its backbone. This is because we strictly follow the hyper-parameters of DyGLib during training (Line 750), ensuring the same number of layers between CROSS and its backbone.\n* **CROSS can adjust the number of encoding layers to fit different computational resources while consistently achieving the best results.** As shown in Fig. 6 (Line 314), CROSS demonstrates remarkable robustness to the number of layers. This indicates that CROSS can flexibly adapt its encoding layers as needed while achieving the best performance.\n\nWe acknowledge that computing MRR is relatively more time-consuming than AUC. However, this overhead is not introduced by CROSS itself. We adopt MRR because it is a discriminative and increasingly popular metric in temporal graph learning, as detailed in W5 to Reviewer B85N. We also report other metrics with AUC and Hits@10 in Tab. 6 (Line 797).\n\n>**Q1:** Prompt adaptation for different datasets.\n\nAs detailed in **Lines 801–805**, we have carefully considered prompt adaptation across datasets. **Only minor keyword adjustments are required** for the prompts when applied to different datasets. For instance, in the Enron dataset, the terms “item” and “review” are replaced with “user” and “email”, respectively. Further details of prompt design can be found in Sec. F (Line 798).\n\n>**Q2:** Motivation of Mixer.\n\nWe discuss the motivation of our Mixer in **Lines 213-219**, and all design choices are empirically validated in **ablation study (Line 322)**.\n\n**Motivation Analysis.** As mentioned in the Introduction, the key motivation of our Mixer is the assumption that **semantics and structures can reinforce each other.** Consequently, **unification should occur throughout the entire encoding process rather than solely after encoding** (Lines 181–185, 206–209, and 223-224). Moreover, as discussed in Lines 216–219, the Mixer integrates only the last semantic representation based on three considerations: **efficiency, temporal awareness, and empirical evidence** where mixing all semantic representations leads to suboptimal performance.\n\n**Empirical Evaluation.** Our ablation study (Line 322) provides empirical support for the Mixer's design:\n\n* **Mixing at output leads to suboptimal performance.** The w/o CM variant (mix at output) exhibits performance degradation. This suggests that fusing bi-modal information only after encoding fails to exploit their mutual reinforcement, resulting in suboptimal performance.\n* **Mixing all semantic representations also leads to suboptimal performance.** The w/ CM$_{\\text{all}}$ variant (mix all semantics) also exhibits performance degradation. This is because aggregating all semantic representations into a single vector will lead to uniform attention scores in the next Transformer layers (since this vector closely resembles other semantic representations), making representations indistinguishable and ultimately degrading their quality.\n\nOnce again, thank you for your invaluable suggestions. Your questions were truly helpful, and we hope our responses have addressed your concerns satisfactorily."}, {"review_id": "4DT1JKrOPl", "reviewer": "Reviewer_C47o", "summary": "This paper focuses on representation learning for temporal text-attributed graphs. To address the limitations of existing models in overlooking the temporal evolution of text semantics and the interactions between semantic structure and text, the authors propose a CROSS framework that can seamlessly extend existing TGNNs for TTAG modeling. The proposed framework consists of a temporal semantics extraction component that uses LLM to summarize the historical semantics of each node, and the Cross-modal Mixer layer to perform information interaction between the node's evolving semantics and structures. Extensive experimental results on real-world datasets verify its superiority.", "questions": "1. For equation 1, the interaction frequency between entities can vary significantly across different time periods, being dense in some intervals and sparse in others. Does the use of a fixed sampling interval risk underrepresenting historical information during sparse periods? How does this strategy compare to an alternative that samples the most recent m interactions instead?\n\n2. While the semantic layer effectively captures the historical semantic information of nodes, why does the structural layer not incorporate the historical structural context of the nodes? I suggest the authors clarify this design choice and discuss whether integrating historical structural information into the structural layer could further improve the model’s performance.", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength:\n\n1. The paper is clearly written and easy to follow.\n\n2. The technical design of the proposed model is well-justified and thoughtfully developed.\n\n3. Extensive experiments have been conducted to demonstrate the effectiveness of the proposed framework.\n\nWeakness:\n\n1. In Section 2, Definition 2 lacks rigor. It would be more precise to refer to it as Temporal Text-attributed Graph Representation Learning, as the term “modeling” is ambiguous in this context. Additionally, Definition 3 seems to have limited relevance to the core contributions of the paper.\n\n2. The Related Work section does not adequately discuss recent models that employ LLMs for dynamic graph modeling [1,2,3]. Furthermore, the baseline comparisons should also include models that do not incorporate textual information, as they address similar tasks such as temporal graph node classification and link prediction.\n\n[1] Large Language Models-guided Dynamic Adaptation for Temporal Knowledge Graph Reasoning\n\n[2] Back to the Future: Towards Explainable Temporal Reasoning with Large Language Models\n\n[3] TimeR4: Time-aware Retrieval-Augmented Large Language Models for Temporal Knowledge Graph Question Answering", "was_revised": true, "final_justification": "Accept", "rebuttal": "We would like to sincerely thank Reviewer C47o for providing a detailed review with insightful questions! We will revise our paper accordingly.\n\n>**W1:** Explanation of terminology definition.\n\nThank you for your advice on the rigor of Definition 2. Although some works [1] use \"modeling\" as a substitute for \"representation learning\", this indeed lacks precision. To ensure clarity and rigor, we will revise the term \"Temporal Text-attributed Graph Modeling\" to \"Temporal Text-attributed Graph Representation Learning\" in our revised manuscripts.\n\nAs for Definition 3, we explicitly distinguish between LLMs and LMs to prevent potential misunderstandings, as some literature [2, 3] considers LLMs to be a specific subclass of LMs. However, in our work, we enhance the **LLM** with dynamic reasoning capabilities to generate evolving textual summaries, which are subsequently encoded into d-dimensional vectors by an **LM**. This separation clarifies their respective roles and helps prevent conceptual confusion.\n\n[1] Towards Adaptive Neighborhood for Advancing Temporal Interaction Graph Modeling, KDD 2024.  \n[2] A Comprehensive Overview of Large Language Models, ACM TIST, 2023.  \n[3] GLBench: A Comprehensive Benchmark for Graph with Large Language Models, NeurIPS 2024.\n\n>**W2-1:** Comparison with temporal knowledge graph reasoning (TKGR).\n\n**Technical Comparison.** In the domain of NLP, several LLM-based TKGR methods [1, 2, 3, 4, 5] have been proposed to enhance the graph reasoning capabilities of LLMs within temporal knowledge graphs (TKGs). We compare these methods with CROSS as follows.\n\n* **Similarity.** Essentially, both works can predict future knowledge based on historical knowledge in dynamic graph-structured data.\n* **Key Differences.** CROSS enhances LLMs for representation learning on TTAGs while LLM-based TKGR methods aim to improve LLMs for graph reasoning over TKGs. Moreover, CROSS extracts structural information via GNN-based aggregation, while LLM-based TKGR methods typically rely on explainable rule-based quadruples [1]. \n\nLLM-based TKGR methods cannot be fully applied to TTAGs, as **TTAGs are not necessarily TKGs**. For example, the Googlemap_CT dataset records user–restaurant review interactions rather than structured knowledge triples. Additionally, although LLM-based TKGR methods primarily focus on graph reasoning tasks, CROSS is designed to generate node representations to support various downstream applications. Therefore, these methods, while valuable, **may fall outside the scope of our work**. We will discuss, analyze, and appropriately cite these works in our revised paper. \n\nKey differences are summarized as follows.\n\n||LLM-based TKGR methods|CROSS|\n|:-|:-|:-|\n|Domain|NLP|ML|\n|Task|Knowledge graph reasoning|Representation learning|\n|Data|TKGs|TTAGs|\n|Role of LLMs|Predictor/Enhancer|Enhancer|\n|Structure Extraction|Rule-based quadruples|GNN-based aggregation|\n\n**Empirical Comparison.** We conduct empirical comparison with a recent LLM-based TKGR method, **LLM-DA**, using TiRGN as its graph-based reasoning module [1]. Other details are available in its original publication. Experiments are conducted on the ICEWS1819 dataset, as it can be structured as a temporal knowledge graph. The results are presented in Tab. 1.\n\nFrom the results, we observe that **CROSS consistently outperforms LLM-DA**, highlighting its effectiveness once again.\n\nTable 1: MRR result (%) for temporal link prediction on ICEWS1819. \n\n| |Transductive|Inductive|\n|:-:|:-:|:-:|\n|LLM-DA|76.42|57.43|\n|CROSS|**95.77**|**87.80**|\n\n[1] Large Language Models-guided Dynamic Adaptation for Temporal Knowledge Graph Reasoning, NeurIPS 2024.  \n[2] Back to the Future: Towards Explainable Temporal Reasoning with Large Language Models, WWW 2024.  \n[3] TimeR4: Time-aware Retrieval-Augmented Large Language Models for Temporal Knowledge Graph Question Answering, EMNLP 2024.  \n[4] Chain of History: Learning and Forecasting with LLMs for Temporal Knowledge Graph Completion, ACL 2023.  \n[5] ChatRule: Mining Logical Rules with Large Language Models for Knowledge Graph Reasoning, PAKDD 2025.\n\n\n>**W2-2:** Baseline clarification.\n\nWe would like to clarify that **our baselines already cover both types of models**: (1) those that explicitly incorporate textual information (e.g., LKD4DyTAG), and (2) those that do not (e.g., FreeDyG, DyGFormer). Under this comprehensive comparison, CROSS consistently achieves the best performance, demonstrating its effectiveness.\n\nWe will highlight this more prominently in our revised version.\n\n>**Q1:** Sampling of LLM reasoning timestamp.\n\nWe would like to clarify that **CROSS samples LLM reasoning timestamps based on fixed interaction intervals, rather than fixed time intervals.**\n\n**Motivation Analysis.** As noted in Line 138, for each node, we first derive its interaction timestamps and then sample timestamps for LLM reasoning **at fixed interaction intervals**. As discussed in Line 135, this strategy ensures that each LLM reasoning step accesses a balanced subset of interactions, thereby mitigating the issue of uneven interaction density across different time periods.\n\n**Empirical Evaluation.** To empirically validate the above analysis, we conduct an ablation study with two additional variants:\n\n* **w/ FTI (Fixed Time Interval)**: For each node, we compute its interaction time span and uniformly sample $m$ timestamps to enable the LLM to summarize at fixed time intervals. \n* **w/ RN (Recent Neighbors)**: For each node at each timestamp, we directly instruct the LLM to summarize the $b$ most recent neighbors.\n\nThe corresponding results are reported in Tab. 2. From the results, we find that:\n\n* **Summarizing at fixed time intervals leads to inferior performance.** This confirms the presence of uneven interaction density across different time periods. CROSS addresses this issue by sampling at fixed interaction intervals, ensuring balanced input interactions at each LLM reasoning step.\n* **Summarizing recent interactions also yields suboptimal results.** This is likely due to its narrow focus on short-term contexts, failing to capture the long-range semantic dynamics crucial for TTAG modeling.\n\nTable 2: MRR results (%) for temporal link prediction on Enron under $b=20$.\n\n||TGAT|TGN|\n|:-:|:-:|:-:|\n|w/ FTI (Trans)|82.47|85.20|\n|w/ RN (Trans)|78.46|83.38|\n|CROSS (Trans)|**95.58**|**95.84**| \n|w/ FTI (ind)|70.21|71.77|\n|w/ RN (ind)|65.60|68.87\n|CROSS (Ind)|**81.52**|**82.38**|\n\n>**Q2:** Motivation of Co-encoder.\n\nAs stated in Line 189 and illustrated in Fig. 2 (Line 132), we clarify that **the structural layer already incorporates historical structural information via TGNN-based aggregation.**\n\n**Motivation Analysis.** In CROSS, the semantic layer effectively captures semantic information, while the structural layer fully models structural contexts. As mentioned in the Introduction, the fundamental motivation of our Co-encoder is the assumption that **semantics and structures can reinforce each other.** Therefore, **unification should occur throughout the entire encoding process**, rather than shallow fusion solely at input or output. To achieve this, we propose to **iteratively and hierarchically** unify these two modalities during encoding, enabling their mutual reinforcement and producing cohesive representations for effective TTAG modeling (Lines 181–185, 206–209, and 223–224).\n\n**Empirical Evaluation.** Our ablation study (Tab. 2, Line 275) provides empirical support for the encoder design. Specifically, we conduct experiments to evaluate different design choices of the encoder, including semantic-only encoding, structural-only encoding, and the proposed co-encoding architecture. Under such extensive comparison, **CROSS consistently outperforms all variants**, demonstrating the effectiveness and necessity of the Co-encoder design.\n\nOnce again, thank you for your valuable review. You truly have helped us to improve our work!"}]

Author rebuttal:

Meta-review: In this paper, the authors address the problem of temporal TAG (TTAG) modeling, which is an underexplored topic. They propose the framework of CROSS, leveraging LLM for TTAG modeling. Unlike existing TGNNs that focuses on structural dynamics only, the proposed approach considers temporal semantics, and aims for semantic-structural unification. Experiments on four public datasets and one industrial dataset demonstrate the SOTA performance of CROSS.

Strengths:
- The paper is well motivated, addressing an underexplored yet important problem. The proposed concepts are intuitive and well justified. 
- The proposed framework can flexibly work with existing TGNN models, making it widely applicable. 
- The experimental results are strong across backbones, datasets, and learning settings.

Weaknesses:
- Complexity and scalability maybe a concern for web-scale graphs.
- Experiments protocol, in particularly, treatment of baselines, can be improved.
- Several pieces of related work should be discussed to better position this paper, and to further clarify and strengthen the technical novelty.

Overall, the paper studies a timely and underexplored topic of TTAG + LLM integration. Key questions have been addressed satisfactory in the rebuttal, and they should be incorporated into the camera ready, if accepted.
Decision: Accept (poster)