Node-Time Conditional Prompt Learning in Dynamic Graphs
Authors: Xingtong Yu
Zhenghao Liu
Xinming Zhang
Yuan Fang
Venue: ICLR 2025 Poster (2025)
Keywords: Graph mining
prompt learning
pre-training
dynamic graph
Primary area: learning on graphs and other geometries & topologies

Abstract: Dynamic graphs capture evolving interactions between entities, such as in social networks, online learning platforms, and crowdsourcing projects. For dynamic graph modeling, dynamic graph neural networks (DGNNs) have emerged as a mainstream technique. However, they are generally pre-trained on the link prediction task, leaving a significant gap from the objectives of downstream tasks such as node classification. To bridge the gap, prompt-based learning has gained traction on graphs, but most existing efforts focus on static graphs, neglecting the evolution of dynamic graphs. In this paper, we propose DyGPrompt, a novel pre-training and prompt learning framework for dynamic graph modeling. First, we design dual prompts to address the gap in both task objectives and temporal variations across pre-training and downstream tasks. Second, we recognize that node and time patterns often characterize each other, and propose dual condition-nets to model the evolving node-time patterns in downstream tasks. Finally, we thoroughly evaluate and analyze DyGPrompt through extensive experiments on four public datasets.

Published as a conference paper at ICLR 2025
NODE-TIME CONDITIONAL PROMPT LEARNING IN DYNAMIC GRAPHS
Xingtong Yu Zhenghao " Xiimming Zhang Yuan Fang Singapore Management University', University of Science and Technology of China? (xingtongyu, yfang) @smu.edu.sg, salzh @ edu..edu.nnn xinming ustc.edu.cn
ABSTRACT
Dynamic graphs capture evolving interactions between entities, such as in social networks, online learning platforms, and crowdsourcing projects. For dynamic graph modeling, dynamic graph neural networks (DGNNs) have emerged as a mainstream technique. However, they are generally pre-trained on the link pre- diction task, leaving significant gap from the objectives of downstream tasks such as node classification. To bridge the gap, prompt-based learning has gained traction on graphs, but most existing efforts focus on static graphs and neglect the evolution dynamic graphs. this paper, we propose DYGPROMPT, a novel pre- rre-training and prompt learning framework for dynamic graph modeling. First, we design dual prompts to address the discrepancy in both task objectives and temporal variations across pre-training and downstream tasks. Second, we recog- nize that node and time patterns often characterize each other, and propose dual condition-nets model the evolving node-time patterns in downstream tasks. Fi- nally, we thoroughly evaluate and analyze DYGPROMPT through extensive exper- iments on four public datasets.
INTRODUCTION
Graph data are pervasive due to their ability to model complex relationships between entities in various applications, including social network analysis (Fan et al. 2019: Yu et al.) 2023c Web mining (Xu et al. 2022 Wang et al., 2023). and content recommendation systems (Qu et al. 2023] Zhang et al. 2024). In many of these applications, the graph structures evolve over time, such as users commenting on posts in social network (Kumar et al. 2018 Iba et al., 2010), editing pages in crowdsoureing project like Wikipedia (Kumar et 2015). or interacting with courses in an online learning platform (Liyanagunawardena et al. 2013)) 3.. Such graphs are termed dynamic graphs Barros et al., 2021; [kkrdinng al.) 2021), in contrast conventional static graphs that maintain an unchanging structure.
On dynamic graphs, dynamic graph neural networks (DGNNs) (Pareja et al. 2020 Xu et al. 2020) have been widely applied. In typical design for DGNNs, each node updates its representation by iteratively receiving and aggregating messages from its neighbors While DGNNs are often pre-trained on the link prediction task, the downstream application could involve a different task, such as node classification, leading to a significant gap between pre-training and task objectives. More advanced pre- training and fine-tuning strategies on dynamic graphs (Bei ett al. 2024, Chen et al. [002]] Tian et al., 2021) also suffer from the same limitation. They first pre-train model for dynamic graphs based various task-agnostic self-supervised signals the graph, and then update the pre-trained model weights in a fine-tuning phase based on task- specific labels pertinent to the downstream application. Likewise, the pre-training and downstream objectives can differ significantly, resulting in a notable gap between them.
time-dependent manner.
To bridge the gap between pre-training and downstream objectives (Liu 2023a), prompt leamn ing has first emerged in language models (Brown et al.) (2020). Fundamentally, a prompt serves to reformulate the input for the downstream task to align with the pre-trained model, while freezing
"Co-first authors. 'Correpooning authors.

---

Published as a conference paper at ICLR 2025
Morning
Noon User speciftc timeline
Night
night
mide silii pner behavior
Professor
Post
500G
Divergent ubjective
RODE
2003
1005
Link prediction morning?
Node classification
night
End
sclive
at noom?
Student
Dynamic variation
Ruspended
User
(a) Users' comments over time
(b) Envolving node-time patterns
Figure/ I: Motivation of DYGPROMPT. (a) Users comment on different topics over time. (b) Evoly- ing node-time patterss in Wikipedia, node and time mutually characterize each other.
the pre-trained weights. Given far fewer parameters in a prompt than the pre-trained model, prompt learning is parameter-efficient and effective especially in data-scarce scenarios (Yu al.. 2024a) Inspired by the success of prompts in the language domain, recent studies have explored prompt learning on graphs (Liu et al., 2023b; Sun al., 20226 2023). However, these studies only focus on static graphs, neglecting the unique challenges brought by dynamic graphs. In this work, we explore Prompt learning for Dynamic Graphs, and propose framework called DYGPROMPT. The problem is non-trivial due to two key challenges.
First, how do we design prompts to bridge temporal variations across time, in addition to diver- gent task objectives? Generally, both task objectives and the timing of pre-training and downstream tasks can be inconsistent. For example, consider a user discussion network such as Reddit.com as illustrated in Fig. a where users and posts are represented as nodes, and an edge is formed when a user comments on a post. On this dynamic graph, a typical pre-training task involves link whether
(i.e., links to) a video game post at time t1- In contrast, the downstream task could be the status classification of the user's 's account, e.g., deter- mining whether it has been suspended at time 12. While previous work on static graphs (Liu al.] 20236 [Sun et al., 20226] 2023) employs prompts to narrow the task objective gap, the temporal gap remains unsolved. In particular, even when the downstream task also involves link prediction, the evolving graph structure introduces a temporal gap between the pre-training and downstream phases. In DYGPROMPT, we introduce dual prompts, consisting of node prompt and a time prompt. On the one hand, a node prompr alters the node features to reformulate task input and bridge the task gap, similar to previous work (Liu et al., 2223b; Fang et al. 2024)) On the other hand, time prompt adjusts the time features, capturing the temporal evolution of the dynamic graph, thereby narrowing inconsistencies that arise from varying priorities different times.
rrediction-predicee
comments
Second, how do we capture evolving patterns across disferent nodes and time points, driven by the dynamic interplay between them? Different nodes exhibit unique timelines, evolving distinctively over time nodes and time mutually influence each other. For example, in the user discussion network, a student may comment on open source code in the morning, while commenting on video games in the evening. However, shown in Fig. [a). a professor may comment academic policies at noon, while discussing family topics at night. Therefore, node and time patterns mutu- ally characterize each other: the behavior of the same node may be influenced by different times, while the behavior of node at a specific time may be influenced by the characteristics of the node. Hence, the evolving node-time patterns, as shown Fig. [Tb). may differ from the patterns observed in the pre-training data. However, previous studies neglect the fine- ane-grained interplay between the node and time patterns downstream. Note that some works (Tan et al. 2023] Wen Fang. 2023) have designed individual prompts or tokens tailored to each node, and contemporary approach (Chen et al. 2024) has developed time-aware prompts. However, they do not account for the mutual characterization between nodes and time, failing to capture the fine-grained, evolving node-time patterns. In DYGPROMPT, inspired by conditional prompt learning (Zhou et al. 2022). we propose series of node and time prompts generated by dual condition-nets. Specifically, time condition- net generates a sequence of time- conditiond node prompts for each node. These conditiond node prompts reflect temporal subtleties at different times for the same node, thereby better aligning
2

---

Published as a conference paper at ICLR 2025
the node's pattern at different times with those observed during pre- pre-training compared using fixed node prompt across all time points. Likewise, a node condition-net generates a sequence of node-conditioned time prompts at each timestamp. The node-conditioned time prompts account for node characteristics, thereby better aligning the time patterns of different nodes with those captured by the pre-trained model compared using a fixed time prompt for all nodes. It is worth noting that the condition-nets generate prompts conditioned on the input features rather than directly parame- terizing the prompts, significantly reducing the number of learnable parameters in the downstream prompt-tunee tonntttutngggggggg phase.
In summary, the contributions of this work are threefold. (1) We introduce DyGPROMPT for dy- namic graph modeling, proposing dual prompts to narrow the gaps arising from both task and dy- namic differences between pre-training and downstream phases. (2) We recognize that node and time patterns mutually characterize each other and further propose dual condition-nets to generate series of node and time prompts, thereby adapting to downstream node-time patterns. (3) We con- duct extensive experiments on four benchmark datasets, demonstrating the superior performance of DYGPROMPT in comparison to state-of-the-ar approaches.
2 RELATED WORK
Dynamic graph learning. Real-world graph structures often evolve over time. To model such dy- namic evolution, various continuous-time dynamic graph learning techniques have been proposed. Typically, these methods update node representations by iteratively receiving and aggregating mes- sages from their neighbors in time-dependent manner (Skarding et al. 2021, Duan et al.] 2024). Some approaches employ dynamic random walks to depict structural changes (Nguyen et al. 2018 Wang et al. 2021). while others incorporate time encoder to integrate temporal context with struc- tural modeling (Xu et al. 2020 Cong et al. 2022: Rossi et al. 2020; Yu al. 2023a). Furthermore, some researchers utilize temporal point processes to model structural evolution node communi- cations (Kumar et al., 2019; Trivedi et al.] 2019 Wen & Fang. 2022). However, these methods are typically trained on link prediction tasks, whereas downstream applications may involve different tasks, such as node classification, creating a significant gap between training and task objectives. This gap impedes effective knowledge transfer, adversely impacting downstream task performance.
Graph pre-training. More advanced dynamic graph pre-training methods also face similar prob- lems. Extending from graph pre-training methods (Hu et al.] 2020a b) Jiang et al.] 2023), dynamic graph pre-trainnnn pre-training captures inherent properties of dynamic graphs using various task aagnostnos self- supervised signals through strategies such as structural and temporal contrastive learning (Bei al.) 2024 Tian et al., (02T) Li et al. 2022). dynamic graph generation (Chen et al. (022), and curvature- varying Riemannian graph neural networks (Sun al., 2022a). They then attempt to transfer the prior knowledge to downstream tasks through fine-tuning with task-specific supervision. Nonethe- less, a gap persists between the objectives of pre-training and fine-tuning (Liu et al. [202a]] Yu al.) 2023d). While pre-training seeks to extract fundamental insights from graphs without supervision, fine-tuning is tailored to specific supervision for a given downstream task.
Graph prompt learning. First emerging in the language domain, prompt learning effectively bridges the gap between pre-training and downstream objectives (Brown et al. 2020). With uni- fied template, prompts are specifically tailored for each downstream task, aligning it more closely with the pre- pre-trained model while freezing the pre-trained weights. Prompt learning has been widely adopted on static graphs (Liu et al., 2023b; (Sun et al. 2023 Fang et al. 2024 Tan et al. 2023 Yu et al., 2023d 2025b 2024b) 2025a c 2023b), yet its application in dynamic graphs remains an open question. A contemporary study, TIGPrompt (Chen et al.) 2024), proposes prompt generator to learn time-aware node prompts. However, it only considers the temporal factor in node features, overlooking that time prompts for each node can also be influenced by node features.
3 PRELIMINARIES
In this section, we present related preliminaries and introduce our scope.
3

---

Published as a conference paper at ICLR 2025
Dynamic graph. Dynamic graphs are categorized into discrete-time dynamic graphs and contimuous-timg dynamic graphs (Skarding et al. 2021). The former consists of a series of static graph snapshots, whereas the latter is depicted In this work, we adopt the more general continuous-time definition, where a dynamic graph is rep- resented by G (V, T). Here, V denotes the set of nodes, F. denotes the set of edges, and T represents the time domain. In particular, each edge (1,, 2z.t) t) E E indicates a specific interaction (or event) between nodes 2 and vj at time Each node is associated with a temporal feature vector X1,7 E Rd evolving over time, represented as a row in the temporal feature matrix X, E R
sequence of events a continuous timeline.
Dynamic graph neural network. A popular backbone for static graphs is wressage-passing graph neural networks (GNNs) (Wu al. 2020). Formally, in the I-th GNN layer, the embedding of node v, represented as h'g. is calculated based on the embeddings from the preceding layer:
h Aggr(h'. :bh² : N..))
where N₂ denotes the set of neighboring nodes of v, and Aggr() (+) is neighhorhood aggregation function. Extending to dynamic graph neural networks (DGNNs), researchers integrate temporal contexts into neighborhood aggregation (Pareja et al. 2020: Xu et al., 2020), often utilizing a time encoder (TE) to map timestamps or intervals (Cong et al. 2002 Rossi et al. (020) within a dynamic graph encoder (DGE). Consequently, in the 1-th layer of a dynamic graph encoder, the embedding of node at time f, denoted by h . is derived as
h = Aggr(Fuuse(in) 1-1 TE(t)), (Fuse(h, TE(t')) (u, (), E No)),
(2)
where N2, contains the historical neighbors of u, with (u, E N₂ indicating that u interacts with e at time t' < t. Fuse(.) is a fusion operation that integrates structural and temporal contexts, such as concatenation (Rossi et al., 2020) or addition (Trivedi al. 2019). For simplicity, the output node embedding from the final layer is denoted as he.e, which is then utilized in loss function for optimization. Note that the time encoder maps the time domain to vector space, TE : T + Rd. common implementation is TE(()= = va [cos(w)l), sin(()) (),................... (os(wa/2()) sin(wa/2()), where fwi..... Way2) } is a set of learnable parameters (Xu et al. 2020 Cong et al. 2022
Scope of work. In this work, we pre-train a DGNN with a time encoder on dynamic graphs, em- ploying the task of temporal link prediction (Wang et al.] (02T). For downstream tasks, we target two popular dynamic graph-based tasks, namely, temporal node classification and temporal link predictior Specifically, we focus on data-scarce scenarios with only limited data for downstream adaptation, as labeled data are often difficult or costly to obtain for node classification (Zhou et al.) 2019 Yao et | (2020), while nodes with sparse interactions are common in real-world link predic- tion tasks (Lee et al. 2019 Pan et al. 2019).
4 PROPOSED APPROACH
In this section, we present our proposed model DYGPROMPT, starting with its overall framework, followed by its key components.
4.1 OVERALL FRAMEWORK
We illustrate the overall framework of DYGPROMPT in Fig. 2 which consists of two phases: pre- training and downstream adaptation. First, given a dynamic graph in F [[a., we pre-train a DGNN b In particular, we employ a universal task template based on similarity calculation (Liu et al. 2023b), which unifies different graph- based tasks, such as link prediction and node classification, across the pre-training and downstream phases. Second, given pre-trained DGNN, to enhance its adaptation to downstream tasks, we pro- pose dual prompts, as shown in Fig. c). The dual prompts comprise a node prompt and a time prompt, aiming to bridge the gaps caused by different task objectives and dynamic variations, spectively. Moreover, extending the dual prompts, we propose dual condition-nets to generate a
based on temporal link prediction, as shown in Fig.
As dynamic graphs focus on the evolving structures within graph, graph classification is rarely evaluated task (Skarding al. 202T Pareja al. 2020 Xu et al. 2020 Chen al. 2024
4

---

Published as a conference paper at ICLR 2025
N Saetsmon Closo
Node class
Time encoder
Dynamic graph encoder
Pre-trained time encoder
Pre-trained dynamic graph encoder
Downstream Loss (Eq. 11)
Link Link prodiction
Time feature
Node
Tuned da Frozen
4
Sim
Time prompt
Time condition-net
time prompt
conditioned
to Sim
Time- conditioncd
T Node prompi
Node condition-ne
node prompt
Pre-training loss (Eq. 3)
Node feature
(a) Toy dynamic graph (b) Pre-training
(c) Prompt tuning for downstream tasks
Figure 2: Overall framework of DYGPROMPT.
series of condicond node prompts and node-conditioned time prompts. These conditional prompts are designed to capture the evolving patterns across nodes and time points, along with their mutual characterization, in fine-grained and parameter--fficient manner.
4.2 PRE-TRAINING
Pre- Pre-training techniques Le- Khac et al. 2020) have been widely explored on dynamic graphs (Bei al., 2024, Chen al. 2022 Tian et al. (021). particular, the most common pre-training task is temporal link prediction (Xu et al., 2020 Rossi et al. 2020), where links are readily available without the need for manual annotations. Following these prior works, we adopt link prediction for Fig. pre-training, illustrated in Fig. [b). Its simplicity underscores the effectiveness and robustness of our prompt-based learning in downstream tasks.
Specifically, consider event (v, a, t). i.c., interaction between nodes and at time t. Corre- spondingly, we construct a tuple (v, a, b, t) such that nodes u and are not linked at f, serving as a contrastive signal. For each tuple, the pre-training objective is maximize the similarity between nodes v and a while minimizing that between v and b. Similar to previous work (You et al. 2020 Yu et 2023d), we optimize the following pre-training loss over pre-training data, Dpre.
Cpee DD pre: θ) = = - In
exp (bs exp (h. b..)))
In Eq. (3 e, h1,a: h4. represent the node embeddings of v, a, b time t, respectively. r a temperature hyperparameter. & denotes the set of learnable parameters of the DGNN, including those of the dynamic graph encoder and the time encoder. The pre-trained parameters, Ope arg mine fre (D p O), will be used to initialize the downstream models.
4.3 DUAL PROMPTS
To reconcile task and temporal variations across pre-training and downstream tasks, we propose dual prompts, comprising a node prompt and time prompt, as shown in Fig. Bec.
Node prompt. In language models, a prompt is a purposely designed textual description that re- formulates the input for downstream task, aligning the task with the pre-trained model (Brown) et al. 2020 Jia et al. (021). Likewise, a prompt for static graphs adjusts downstream node features (Liu et al. 2023b Fang al. 2024). In line with previous graph prompt learning methods graphs (Liu et al, 2023b Yu et al. 2023dle; 20255), for each downstream task, we define a learn- able vector p midee the node prompt modify the node features at a given time via element-wise multiplication. Concretely, the input features of node at time t, denoted by X1,v, are modified as:
static
node X
node
Xt,07
where @ is element- wise multiplication. The node prompt mode has the same dimension as node features, altering the importance of various node feature dimensions.
5

---

Published as a conference paper at ICLR 2025
Time prompt. We adopt time encoder TE to map the continuously valued time t to time features, f. = TE(t). However, TE is optimized based on the time periods in the pre-training data, which can be distinct from those in downstream tasks. To overcome the temporal gap, propose time prompt to adjust time features. Formally, for each downstream task, we define a learnable vector qime as the time prompt. Then, the time feature i, denoted by fr, is reformulated as follows.
Tilee
pime ① fe.
4.4 DUAL CONDITION-NETS
In dynamic graphs, the same node may exhibit distinct behaviors at different times. Meanwhile, different nodes may evolve differently over the same time interval. In other words, nodes and their temporal patterns influence and characterize each other over time, revealing fine-grained node-time patterns. To capture such interplay, we turn to conditional prompt learning (Zhou al., 2022).
Instead of learning a static prompt tailored to a specific task over the entire time span, we design dual condition-nets. comprising a time condition-net that generates a sequence of time-conditioned node prompts for each node and a node condition-net that generates sequence of node-conditioned time prompts at each timestamp, as shown in Fig. Zc). In particular, each condition-net utilizes lightweight network to generate prompts conditioned on the input features, enabling more parameter-efficien approach to prompt tuning compared to directly parameterizing the prompts for each node and timestamp.
Time condition-net. To incorporate time characteristics into node prompts, a straightforward way is to employ a series of learnable vectors prompts for each distinct timestamp. However, this approach does not scale to large time intervals with many timestamps, as it quickly increases the number of tunable parameters. Therefore, propose time condition-ner, which generates series of time-condinee conditiood node prompts. Formally, the node prompt conditioned on time t given by
p TCN( ;K),
where TCN is the time condition-net parameterized by k. This can be viewed as a form of hypernet- work (Ha et al., 2022), where our condition-net TCN serves as a secondary network to generate the node prompts conditioned on the input time features. This formulation enables parameter-efficient prompt generation, especially when using a lightweight condition-net. In our implementation, we opt for simple multi-layer perceptron (MLP) with efficient bottleneck structure (Wu & Leep 2018). Therefore, for a series of events occurring at times {tg. t1......... .. the time condition-net pro- duces their corresponding time-conditioned node prompts (piole 1 ,..... Note that these condi- tional prompts match the dimensionality of the node features. Then, the features node u at time which have been modified by the node prompt in Sect. 4.3 are further updated
(fo..1.....
X X
mode
Pt
node
x
node
Node condition-net. Similarly, to incorporate node characteristies into time prompts, we propose a node condition-ner to generate a series of node-conditioned time prompts, mirroring the time condition-net. The time prompt conditioned on the node features of e at time t is given by
pime [x :)),
(8)
where NCN is the node condition-net parameterized by Φ. which is also implemented as an MLP with bottleneck structure. Consequently, time f, series of node-conditioned time prompts (Piig sime Pior..... . is generated for all nodes, which have the same dimension as the time features. These node-specific time prompts are then applied each node time t, as follows.
Plime
plime
ptime
4.5 PROMPT TUNING
Finally, the features adjusted by the prompts are fed into the dynamic graph encoder, DGE, The embedding of node e at time / is calculated as follows:
b DGE (Fuse(X))) Xx
modee E (Fuse(Epp)) node (din)) (u, tu, No No.)).
(10)

---

Published as a conference paper at ICLR 2025
To tune the prompts and condition-neee for downstream task, we adopt similarity calculation as the task template (Liu et al. 2023b), consistent with the pre-training loss. Specifically, for temporal node classification, consider labeled set Dowwn {(0) y/, t1), (U2, y2, f2).... where each v; denotes a node, and Yi E Y is the class label of on at time 4. Then, the downstream loss is
Loown Diawn made, , pume xr, () = - (..... In exp( 1 sim,he, De, R
(11)
Evey exp( { siiml (b ...,)))
where hy represents class y's prototype embeddings (Liu al. 2023b) at time ty, which is ob- tained by the mean embeddings of examples in class at time t, For temporal link prediction, we utilize the same loss the pre-training loss in Eq. (
During prompt tuning, we only update the lightweight dual prompt vectors (p p and the parameters of the dual condition-nets (K, o), whilst freezing the pre-trained DGNN weights. This parameter-effiiien tuning approach is amenable to label-scarce settings, where Daown comprises only a limited number of training examples. We outline the key steps for prompt tuning Algo- rithm in Appendix A and assess its complexity in Appendix B
5 EXPERIMENTS
In this section, we conduct experiments to evaluate DYGPROMPT and analyze the empirical results.
5.1 EXPERIMENTAL. SETUP
Datasets. We utilize four benchmark datasets for evaluation: Wikipedia, Reddit, MOOC and Genre. We provide further details of the datasets in Appendix
Downstream tasks. We conduct temporal node classification and temporal link prediction to eval- uate the performance of DYGPROMPT. For link prediction, we perform experiments in both trans- ductive and inductive settings, depending on whether nodes in the testing set have appeared in the pre-training or downstream tuning data. In this study, we investigate the pre-training and prompt learning on dynamic graphs, which differs from previous DGNNS (Rossi et al., 2020, Xu et al., 2020). Therefore, we adopt a new data split to accommodate the pre-training and downstream tuning steps. Specifically, given chronologically ordered sequence of events (i.e., edges with timestamps), we use the first 80% for pre-training. Note that we pre-train a DGNN only once for each dataset and subsequently employ the same pre-trained model for all downstream tasks. The remaining 20% of the events are used for downstream tasks. Specifically, they are further split into I 1%/1 %/18% subsets, where the first 1 % is reserved the training pool for downstream prompt tuning, and the next %% as the validation pool, with the last 18% for downstream testing.
To construct a downstream task, start by sampling 30 events (about 0.01% % of the whole dataset) from the training pool, ensuring that there at least one user per class. Then, for node classification, take the user nodes as training instances for downstream adaptation, and their ground-trui labels are given by their labels at the time of the corresponding event. For link prediction, the downstream training pool also helps with adaptation due to its temporal proximity to the test set. Specifically, we take the sampled events as positive instances. while further sampling one negative instance for each positive instance. Specifically, for each event (v, a, 1), we sample one negative instance (v: h, t) such that b is node taken from the training pool that is not linked to v at time f. The validation sets are constructed by repeating the same process on the validation pool. Finally, the test set for node classification simply comprises all user nodes in the testing events, while the test set for link predic- tion expanded to include negative instances following a similar sampling process as in training. Furthermore, for inductive link prediction, appeared in the pre- tre-training or downstream training data. We repeat this sampling process for 100 times construct 100 tasks for node classification and link prediction.
remove instances from the test set if their nodes have
Evaluation. To evaluate the performance on the tasks, we employ AUC-ROC for both node clas- sification (Xu et al. 2020: Rossi al., 2020) and link prediction (Sun et al., 2022a Bei et al. 2024]. following prior work. For each of the 100 tasks, we repeat the experiments with five different random seeds, and report the average and standard deviation over the 500 results.
7

---

Published as a conference paper at ICLR 2025
Table 1: AUC-ROC (%) evaluation of temporal node classification and link prediction.
Methods
Node Classification
Transductive Link Prediction
Inductive Link Predaction
Wikipedia
Rodki
MOOC
Genre
Wikipedia
Reddit
MOOC
Genre
Wikipodia
Reddit
MDOC
Genre
SCN-ROLAND
58.86mms
48.25ex3
49.93ss.e
46. 33
49.61a3.22
50.01a23s
49:82:144
49, 19..5
49.60u231
49,90 49,90a1ee
49.16x24s
27.25
ROT-ROLAND
6281m 47.95.0
50.01+634
47 26.5.
52.34+130
50.044
55.74-311
47.696381
92 vn
49.85ae0s
54.01-2%
49.3 49.38-2.0
TGAT
67.00w5s 53.64.553
59.2744x
11
55.78-mm
62.43.18:
51,49-188
69.11-1
48 s5 57.30
51.42-177
48.38-11
50.6 49.54.131
50.33
50.72.231
72.48u010
67.57
44.60lass
74.38
69.8
54.62.03
TREND
09.92 64.85.m:
66.79
50.344142
63.24am:7
$0.42,045
58.70
52.78am.4
50.15,ase
65.13
57.52at.1
45.31a043
GRAPHMIXER
65.43442
60.21as35
63.7224m
55..59
59,73an3e
61.88aum1
52/42za
60.83a2.2
51,34aa2
57.6/am31
51.16a239
S6.32221e
DDGCL
65.15
55,21as.15
62.34asu
50.9
51.96a146
68.68
55.62.833
68. 39
47.98a1
55,90
55.1882
42. 22.70.33s
CPDG
13.56ms4
65.92ss.2
50.32amm
19.89
52.86anse
59. 39.722233
53.82a8
19.71a2.
47.37
56.40amm
53.58a2
40,01a139
GRAPHPROMPT
73.78ax52
60.899a.3
61.60
$1.
4.228
55.67amcs
67.46pu31
11,07.313
86.78a..4
48//6uass
18.18
50.27auss
87,45:15
PROG
60
68.60e5
63.18
51.
46 s4
42.28
93. 13,328005
S8.73
86.24
89.75=a2
90.69am
56.424195
83.43
TGAT-TIGPROMPT
69 69.3
67. Nesss
73.90mu
51. 72
59,54a1a1
48.45
51,69:s
69.71
49.52s930
66.6
51 51.58e4
48. 28.34
TEIN-TIGPROMPT
44.0ss
63.7545
55,42-300
50.84-775
82.04
83.26w
65.000-171
46.2543.4
81.75-a
79.5
64.98-ss
GPPOMPT
82,09664
73
77.78-sm
69.88-01s
76wnss
53.92-04
72.04.4.1
52.58-a39
25.20
20
80-82-1
TGN-DVGPROMPT
74.47.04
74,00-1 69.064129
94.334012
96.82400s 70.17-075
87.02-166
92.22-0 is
95.694mm
69.77.a
87,63-1
Results are reported in percent. The best mothod is bolded and the runner-up is underlined.
The performance of DYGPROMPT is evaluated against state-of-the-an approaches across four main categories. (1) Conventional DGNNs: We first consider a discrete-time framework ROLAND (You et al., 2022) that adapts static GNNs to DGNNS by treating the node embeddings at each GNN layer as hierarchical node states and updating them recurrently over time. We em- ploy GCN (Kipf & Welling 2017) and GAT (Veličković et al. 2018)) for ROLAND. Additionally, we compare to continuous-tme DGNNs, including TGAT (Rossi et al. 2020). TGN (Xu et al., 2020 TREND (Wen & Fang, 2022), and GraphMixer (Cong et al. (023). For fair comparison, these conventional DGNNs are pre-trained on the first 80% of the events as well, and then contin- ually trained on the same training data in each downstream task following our split. (2) Dynamic graph pre-training: DDGCL TTian et al. 202T) and CPDG (Bei et al., 2024) follow the "pre-train, fine-tune" paradigm. They first pre-train a DGNN, leveraging the intrinsic attributes and dynamic patterns of graphs. Then, they fine-tune the pre-trained model for downstream tasks based on task- specific training data. (3) Static graph prompting: GraphPrompt (Liu et al. 2023b) and ProG (Sun) eet al. 2023) employ a static prompt to adapt pre-trained GNNs to downstream tasks. Specifically, employ DGNNs as the pre-trained Backbones-TGir
Baselines.
for GraphPrompt and TGN for ProG as these combinations perform competitively with their respective prompting approaches. (4) Dynamic graph prompting: TIGPrompt (Chen et al. 2024) proposes prompt generator to generate time- aware prompts. Note that both TIGPrompt and our method DYGPROMPT can be applied to different backbones, such as TGAT and TGN. More details on these baselines are described in Appendix (D] with further implementation details for the baselines and DYGPROMPT in Appendix F
5.2 PERFORMANCE COMPARISON WITH BASELINES
We compare the performance of all methods temporal node classification and link prediction in Table and make the following observations.
First, DYGPROMPT achieves outstanding performance across node classification and link prediction tasks. This observation highlights the effectiveness of our proposed dual prompts and condition-nets To further understand the impact of specific design choices in DYGPROMPT and its robustness to various backbones, we defer a detailed investigation to the ablation studies in Sect. |.3 and the backbone analysis in Sect. 5.4 Second, TIGPrompt, the only other dynamic graph prompt learning method, significantly lags behind DYGPROMPT when using the same backbone. While it employs time-aware prompts, it lacks fine-grained node-time characterization and thus unable to capture complex node-time patterns, where nodes and time mutually influence each other. Third, Graph- Prompt and ProG perform competitively because we employ DGNNs as their backbones in pre- training. rather than static GNNs. However, they still underperform compared to DYGPROMPT, as they only use static prompts.
REMARK. It is worth noting that the results of TGAT and TGN in our experiments are lower than those reported in their original papers. A possible reason is that, in their original setting, these methods were trained on data that immediately precede the test set chronologically. In contrast, in our experiments, to maintain consistency with the pre- pre-training setup, we train them in two stages: first with pre-training, and then downstream training on limited task data where sample only
8

---

Published as a conference paper at ICLR 2025
Table 2: Ablation study reporting AUC-ROC (%). with TGAT the backhone.
Time prompt prompt
Node
Node classificution Wikipedia
Transductive Link Prediction Wikipedia Reddit MOOC
Inductive Link Prediction Wikipedia
Methods
NCN
TCN
Reddit
MOOC
Reddit MOCC
VARIANT
67.00
53.64
$9.27
$5.78
62.43
$1.49
48.21
57.30
$1.42
VARIANT
V
X
72.59
61.82
63.50
68.12
88.59
51.24
51.89
74.84
51.37
VARIANT
x
x
X
73.22
62.51
62.59
66.31
87.06
51.26
50.28
69.71
50.50
63.11
68.36
VARIANT
V
x
X
81.40
73.12
77.15
69.56
90.10
52.16
52.57
75.50
53.31
VARIANT
x V
V
80.34
72.59
76.16
66.62
87.34
51.22
49.05
73.16
52.34
DYGPROMPT
82.09
73.50
77.78
69.88
90.76
53.92
52.58
75.20
53.29
Table 3: AUC-ROC (%) evaluation of DyGPROMPT with different DGNN backbones.
Pre training
Downstream Adaptation
Node classification
Transductive link prediction Wikipedia
Inductive link prediction
Backbone
Wikipedia
Reddit
MOOC
Reddit
MOOC
Wikipedia
Reddit
MOOC
50.61
49.54
50.33
58.45
58.02
50.29
56.87
57.25
50.34
DYREP
DyGPROMPT
$3.52
50.98
51.62
91.64
93.84
72.04
90.28
93.08
72.27
$1.37
49,80
50.53
62.40
59.81
50.71
59.59
61.28
50.57
JODIE
DYGPROMPT
62.84
60.93
67.84
63.56
58.89
52.06
62.80
58.17
52.33
67.00
53.64
59.27
55.78
62.43
51.49
48.21
57.30
51.42
TGAT
DyGPROMPT
82.09
73.50
77.78
69.88
90.76
53.92
52.58
75.20
$3.29
50.61
49.54
50.33
72.48
67.37
54,60
74.38
69.81
54.62
TGN
DYGPROMPT
74.47
74.00
69.06
94.33
96.82
70.17
92.22
95.69
69.77
69.92
64.85
66.79
63.24
80.42
58.70
50.15
65.13
57.52
TREND
DyGPROMPT
70.15
65.24
67.58
64.35
79.62
59.45
51.26
64.88
59.13
GraphMixer
DyGPROMPT
66.39
65.43
60.21
61.42
64.18
59.73
60.25
62.31
61
52.42
52.94
52.19
51.34
57.43
57.64
51.16
52.55
"L" refers to finc-tuning or continually training the backbone downstream task data without our prompt design.
30 events for each task. Moreover, training data that are temporally closer to the test set are more important than earlier data in pre-training. However, due limited data in the downstream train- ing stage, the models converge quickly and fail to effectively learn from these crucial recent time intervals, resulting in lower performance than their original setting. In contrast, prompt-based approach can better handle the temporal gap and adapt well to the test data.
5.3 ABLATION STUDIES
To thoroughly understand the impact of each component within DYGPROMPT, compare DYG- PROMPT with its variants that employ different prompts and condition-nets These variants and their corresponding results, using TGAT as the backbone, presented in Tablee2acre three datasets.
For node classification, both node and time prompts are beneficial, as Variant (with node prompt) and Variant 3 (with time prompt) outperform Variant 1 (without these prompts). Incorporating both prompts in Variant 4 could further boost the performance. Moreover, the node condition-net is advantageous, as integrates node characteristics into time features, since Variant outperforms Variant 2. Similarly, the time condition-net is beneficial for capturing temporal subtleties at dif- ferent times enhance node feature characterization, as Variant 6 outperforms Variant 3. Lastly, DYGPROMPT achieves the best performance, demonstrating the effectiveness of dual prompts and condition-nett nets.
For link prediction, the results generally exhibit similar patterns to those in node classification. notable exception is the MOOC dataset, where the differences between the variants are gener- ally smaller. A potential reason is that interactions in MOOC (online courses) are less dynamic and diverse than interactions Reddit (a major social network) and Wikipedia (a high-traffic site), reducing the effectiveness of prompts designed to bridge the pre-training and downstream gaps. However, our full model DYGPROMPT still achieves significantly better performance than Variant though
1, which does not utilize any prompts. On the other hand, in Reddit and Wikipedia, both pre-training and downstream tasks involve link prediction, node prompts remain useful due to the potential distributional differences in the downstream training data across tasks.

---

Published as a conference paper at ICLR 2025
Wikipidia
Reddit
MOOC
Node dussficcaiion
AUC-0001000
prompis
Transhtive link prodction
DyGPrompl
ADCROC
Suspended user
Active user
Wikipedio
RRddit MOOC
Figure 3: Visualization of output embedding space of nodes.
Figure 4: Sensitivity of
To further visualize the effect of prompt design, sample 200 suspended nodes and 200 active nodes from Wikipedia, Reddit and MOOC, respectively, and show the output embedding space of these nodes, calculated by Variant (w/o prompts) and DYGPROMPT, in Fig- 3 We observe that with both dual prompts and dual condition-nets, node embeddings from different classes are clearly separated, demonstrating the advantages of DYGPROMPT.
5.4 IMPACT BACKBONES AND HYPERPARAMETERS
First, to analyze the flexibility and robustness of DYGPROMPT, we evaluate its performance different DGNN backbones, including DYREP (Trivedi et al. 2019), JODIE (Kumar et al. 2019). TGAT, TGN, TREND, and GRAPHMIXER. The results on three datasets are reported in Table We observe that regardless of the backbone utilized, DYGPROMPT surpasses the original backbone without our prompt design in almost all cases, indicating the robustness of our proposed framework.
Next, we investigate the sensitivity hyperparameters DYGPROMPT. Particularly, we employ two MLPs with bottleneck structure the dual condition-nets nets (Wen & Fang 2023 Wu & Lee 2018 The perceptrons embed the input feature dimension d to and then restore it back d, where d d/o. Here, is hyperparameter that scales the hidden dimension. We evaluate the impact of a, as illustrated Fig. 4 We observe that for both node classification and link prediction tasks, performance generally declines a increases, likely due greater information loss from a "smaller bottleneck." The performance when s is generally competitive across different datasets. Therefore, in our experiments, we set 2.
CONCLUSIONS
In this paper, explored prompt learning dynamic graphs, aiming to narrow the gap between pre-training and downstream applications on such graphs. Our proposed approach, DYGPROMPT, employs dual prompts overcome the divergent objectives and temporal variations across pre- training and downstream tasks. Moreover, we propose dual condition-nets to mutually character- ize node and time patterns. Finally, conducted extensive experiments on four public datasets, demonstrating that DYGPROMPT significantly outperforms various state-of-the-art baselines.
ACKNOWLEDGMENTS
This research project is supported by the Ministry of Education, Singapore, under its Academic Research Fund Tier (Proposal ID: T2EP20122-0041 Any opinions, findings and conclusions or recommendations expressed in this material are those the author(s) and do not reflect the views of the Ministry of Education, Singapore.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Claudio DT Barros, Matheus RF Mendonça, Alex B Vieira, and Artur Ziviani. A survey on embed- ding dynamic graphs. ACM Computing Surveys, 55(1), 2021.
Yuanchen Bei, Hao Xu, Sheng Zhou, Huixuan Chi, Mengdi Zhang, Zhao Li, and Jiajun Bu. Cpdg: A contrastive pre-training method for dynamic graph neural networks. ICDE, pp. 1199-1212, 2024.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. In NeurlPS, pp. 1877-1901, 2020.
Ke-Jia Chen, Jiajun Zhang, Linpu Jiang. Yunyun Wang, and Yuxuan Dai. Pre-training on dynamic graph neural networks. Neurocomputing, 500(C):679-68) 2022.
Xi Chen, Siwei Zhang, Yun Xiong, Xixi Wu, Jiawei Zhang, Xiangguo Sun, Yao Zhang, Ying- long Zhao, and Yulin Kang. Prompt learning temporal interaction graphs. arXiv preprint arXiv:2402.063326, 2024.
Weilin Cong, Si Zhang, Jian Kang, Baichuan Yuan, Hao Wu, Xin Zhou, Hanghang Tong, and Mehrdad Mahdavi. Do we really need complicated model architectures for temporal networks? In ICLR, 2022.
Weilin Cong, Si Zhang, Jian Kang, Baichuan Yuan, Hao Wu, Xin Zhou, Hanghang Tong, and Mehrdad Mahdavi. Do we really need complicated model architectures for temporal networks? In ICLR, 2023.
Yifan Duan, Xinke Jiang, Guibin Zhang. Shilong Wang, Xiaojiang Peng, Wang Ziqi, Junyuan Mao, Hao Wu, and Kun Wang. CaT-GNN: Enhancing credit card fraud detection via causal temporal graph neural networks. arXiv preprint arXiv:2402. 14708, 2024.
Wenqi Fan, Yao Ma, Qing Li, Yuan He, Eric Zhao, Jiliang Tang, and Dawei Yin. Graph neural networks for social recommendation. In WWW, pp. 417-426, 2019.
Taoran Fang, Yunchao Zhang, Yang Yang, Chunping Wang. and Lei Chen. Universal prompt tuning for graph neural networks. In NewrlPS, pp. 52464-224455 2024.
Oliver Ferschke, Torsten Zesch, and Iryna Gurevych. Wikipedia revision toolkit: Efficiently access- ing Wikipedia's ' edit history. Modern Information Technologies and Innovation Methodologies of Education in Professional Training Methodology Theory Experience Problems, 2012.
David Ha, Andrew M Dai, and Quoc V Le. Hypemetworks. In ICLR, 2022.
Maxwell Harper and Joseph A Konstan. The movielens datasets: History and context. TII,, 5(4): 1-19, 2015.
Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay Pande, and Jure Leskovec. Strategies for pre-training graph neural networks. In ICLR, 2020a.
Ziniu Hu, Yuxiao Dong, Kuansan Wang, Kai-Wei Chang, and Yizhou Sun. GPT-GNN: Generative pre-training of graph neural networks. In SIGKDD, pp. 1857-1867, 2020b.
Xuanwen Huang, Yang Yang, Yang Wang, Chunping Wang. Zhisheng Zhang, Jiarong Xu, Lei Chen, and Michalis Vazirgiannis. DGraph: A large-scale financial dataset for graph anomaly detection. In NewrlPS, pp. 22765-22777, 2022.
Takashi Iba, Keichi Nemoto, Bernd Peters, and Peter A Gloor. Analyzing the creative editing behavior of wikipedia editors: Through dynamic social network analysis. Procedia Social and Behavioral Sciences, (4))6644166456 2010.
Chao Jia Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and lanion-language representation learning with noisy text supervision. In ICLRR 2021.

---

Published as a conference paper at ICLR 2025
Xinke Jiang, Zidi Qin, Jiarong Xu, and Xiang Ao. Incomplete graph learning via decoupled variational auto-encoder. In WSDM, pp. 304-312, 2023.
Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional net- works. In ICLR, 2017.
Srijan Kumar, Francesca Spezzano, and VS Subrahmanian. VEWS: A wikipedia vandal early warn- ing system. In SIGKDD, pp. 607-616, 2015.
Srijan Kumar, William L. Hamilton, Jure Leskovec, and Dan Jurafsky. Community interaction and conflict on the web. In WWW, pp. 933-943, 2018.
Srijan Kumar, Xikun Zhang, and Jure Leskovec. Predicting dynamic embedding trajectory in tem- poral interaction networks. In SIGKDD, pp. 1279, 2019.
Phuc H Le-Khac, Graham Healy, and Alan Smeaton. Contrastive representation learning: A framework and review. IEEE Access, 8: 1:199907-19393, 2020.
Hoyeop Lee, Jinbae Im, Seongwon Jang, Hyunsouk Cho, and Schee Chung. McLU: Meta-learned user preference estimator for cold-start recommendation. In SIGKDD, pp. 1073-1082, 2019.
Rongfan Li, Xinke Jiang, Ting Zhong, Goce Trajcevski, Jin Wu, and Fan Zhou. Mining spatio- temporal relations via self-paced graph contrastive learning. In SIGKDD, pp. 936-944, 2022.
Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig, Pre- train, prompt, and predict: A systematic survey of prompting methods in natural language pro- cessing. ACM Computing Surveys, 55(9), 2023a.
Zemin Liu, Xingtong Yu, Yuan Fang, and Xinming Zhang. GraphPrompt: Unifying pre-training and downstream tasks for graph neural networks. In WWW. pp. 417-428, 2023b.
Tharindu Rekha Liyangunawwardnna Andrew Alexandar Adams, and Shirley Ann Williams. MOOCs: A systematic study of the published literature 2008-2012. International Review of Research Open and Distributed Learning, 14(3):202-222) 227, 2013.
Giang Hoang Nguyen, John Boaz Lee, Ryan A Rossi, Nesreen K Ahmed, Eunyee Koh, and Sungchul Kim. Continuous-time dynamic network embeddings. In WWW Companion, pp. 969- 976, 2018
Feiyang Pan, Shuokai Li, Xiang Ao, Pingzhong Tang, and Qing He. Warm up cold-start advertise- ments: Improving ctr predictions via learning to learn id embeddings. In SIGIR, pp. 695-704, 2019.
Aldo Pareja, Giacomo Domeniconi, Jie Chen, Tengfei Ma. Toyotaro Suzumura, Hiroki Kaneza- shi, Tim Kaler, Tao Schardl, and Charles Leiserson. EvolveGCN: Evolving graph convolutional networks for dynamic graphs. In AAAI, pp. 5363-5370, 2020.
Liang Qu, Ningzhi Tang, Ruiqi Zheng, Quoe Viet Hung Nguyen, Zi Huang, Yuhui Shi, and Hongzhi Yin. Semi-decentralized federated ego graph learning for recommendation. In WWW, pp. 339- 348, 2023.
Emanuele Rossi, Ben Chamberlain, Fabrizio Frasca, Davide Eynard, Federico Monti, and Michael Bronstein. Temporal graph networks for deep learning dynamic graphs. arXiv preprint arXiv:2006.1066 10637, 2020.
Joakim Skarding, Bogdan Gabrys, and Katarzyna Musial, Foundations and modeling of dynamic networks using dynamic graph neural networks: survey. IEEE Access, 9:79143-79168. 2021.
Li Sun, Junda Ye, Hao Peng, and Philip S Yu. A self-supervised Riemannian GNN with time varying curvature for temporal graph learning. In CIKM, pp. 1827-1836, 2022a.
Mingchen Sun, Kaixiong Zhou, Xin He, Ying Wang, and Xin Wang. GPPT: Graph pre- training and prompt tuning to generalize graph neural networks. In SIGKDD, pp. 1717-1727, 2022b.
12

---

Published as a conference paper at ICLR 2025
Xiangguo Sun, Hong Cheng, Jia Li. Bo Liu, and Jihong Guan. All in one: Multi-task prompting for graph neural networks. SIGKDD, pp. 2120-2131, 2023.
Zhen Tan, Ruocheng Guo, Kaize Ding, and Huan Liu. Virtual node tuning for few-shot node classi- fication. In SIGKDD, pp. 2177-2188, 2023.
Sheng Tian, Ruofan Wu, Leilei Shi, Liang Zhu, and Tao Xiong. Self-supervised representation learning dynamic graphs. In CIKM, pp. 1814-1823, 2021.
Rakshit Trivedi, Mehrdad Farajtabar, Prasenjeet Biswal, and Hongyuan Zha. DyRep: Learning representations over dynamic graphs. In ICLR, 2019.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. In ICLR, 2018.
Kun Wang, Yuxuan Liang, Xinglin Li, Guohao Li, Bernard Ghanem, Roger Zimmermann, Zhengyang Zhou, Huahui Yi, Yudong Zhang, and Yang Wang. Brave the wind and the waves: Discovering robust and generalizable graph lottery tickets. IEEE TPAMI, 46(5):388-334408 2023.
Yanbang Wang, Yen-Yu Chang, Yunyu Liu, Jure Leskovec, and Pan Li. Inductive representation learning in temporal networks via causal anonymous walks. In ICLR, 2021.
Zhihao Wen and Yuan Fang. TREND: TempoRal event and node dynamics for graph representation learning. WWW, pp. 1159-1169, 2022.
Zhihao Wen and Yuan Fang. Prompt tuning on graph-augmented low-resource text classification. IEEE TKDE, 36(12):9880-9099 2023.
Yuzhong Wu and Tan Lee. Reducing model complexity for dnn based large-scale audio classifica- tion. In ICASSP, pp. 331-335, 2018.
Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and Yu Philip. A comprehensive survey on graph neural networks. JEEE TNNLS, 2211):424,, 2020.
Da Xu, Chuanwei Ruan, Evren Korpeoglu, Sushant Kumar, and Kannan Achan. Inductive represen- tation learning on temporal graphs. In ICLR, 2020.
Weizhi Xu, Junfei Wu, Qiang Liu, Shu Wu, and Liang Wang. Evidence-aware fake news detection with graph neural networks. In WWW, pp. 2501-2510, 2022.
Huaxiu Yao, Chuxu Zhang, Ying Wei, Meng Jiang, Suhang Wang, Junzhou Huang, Nitesh Chawla, and Zhenhui Li. Graph few-shot learning via knowledge transfer. In AAAI, pp. 6656-6663, 2020.
Jiaxuan You, Tianyu Du, and Jure Leskovee. ROLAND: Graph learning framework for dynamic
graphs. In SIGKDD, pp. 2358-2366, 2022.
Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. Graph contrastive learning with augmentations. In NeurlPS, pp. 5812-5823, 2020.
Le Yu, Leilei Sun, Bowen Du, and Weifeng Lv. Towards better dynamic graph learning: New architecture and unified library. NeurIPS, pp. 67686-67700, 2023a.
Xingtong Yu, Zemin Liu, Yuan Fang, and Xinming Zhang. HGPrompt: Bridging homogeneous and heterogeneous graphs for few-shot prompt learning. In AAAI, pp. 16578-16586, 2023b.
Xingtong Yu. Zemin Liu, Yuan Fang, and Xinming Zhang. Learning to count isomorphisms with graph neural networks. In AAAI, pp. 4845-4853, 2023c.
Xingtong Yu, Zhenghao Liu, Yuan Fang, Zemin Liu, Sihong Chen, and Xinming Zhang. Generalized graph prompt: Toward a unification of pre-training and downstream tasks on graphs. IEEE TKDE, 36(11):6237-- 6250, 2023d.
Xingtong Yu, Chang Zhou, Yuan Fang, and Xinming Zhang. MultiGPrompt for multi-task pre- training and prompting on graphs. In WWW, pp. 515-526, 2023e.
13

---

Published as a conference paper at ICLR 2025
Xingtong Yu, Yuan Fang, Zemin Liu, Yuxia Wu, Zhihao Wen, Jianyuan Bo, Xinming Zhang, and Steven CH Hoi. Few-shot learning on graphs: from meta-learning to pre- pre-training and prompting. arXiv preprint 4rX1v:2402.01440, 2024a.
Xingtong Yu, Chang Zhou, Yuan Fang, and Xinming Zhang. Text-free multi-domain graph pre- training: Toward graph foundation models. arXiv preprint arXiv:2405. 13934, 2024b.
Xingtong Yu, Zechuan Gong, Zhou Chang, Yuan Fang, and Zhang Hui. SAMGPT: Text-free graph foundation model for multi-domain pre-training and cross-domain adaptation. www, 2025a.
Xingtong Yu, Jie Zhang, Yuan Fang, and Renhe Jiang. Non-homophilic graph pre-training and prompt learning. In SIGKDD, 2025b.
Xingtong Yu, Chang Zhou, Zhongwei Kuai, Xinming Zhang, and Yuan Fang. GCoT: Chain-of- thought prompt learning for graphs. arXiv preprint arXiv:2502.00992 2025c.
Guibin Zhang, Yanwei Yue, Xiangguo Sun, Guancheng Wan, Miao Yu, Junfeng Fang, Kun Wang, and Dawei Cheng. G-Designer: Architecting multi-agent communication topologies via graph neural networks. arXiv preprint arXiv:2410.1 1782, 2024.
Fan Zhou, Chengtai Cao, Kumpeng Zhang, Goee Trajcevski, Ting Zhong, and Ji Geng. Meta- Meta-GNN: On few-shot node classification in graph meta-learning In CIKM, pp. 2357-2360, 2019.
Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Conditional prompt learning for vision-language models. In CVPR, pp. 16816-16825, 2022.
14

---

Published as a conference paper at ICLR 2025
Algorithm DOWNSTREAM PROMPT TUNING FOR DYGPROMPT
Input: Pre-trained DGNN with Pa Go including those of DGE and TE
Output: Optimized dual prompts (p .p qlime) and optimized parameters (K, of dual condition-nets
1: Encoding dynamic graphs and time via pre-trained DGNN +/
2: "" -P time Φ + initialization
3: while not converged do
4: for each dynamic graph G (V, E) with feature matrix X, time do
Xt.0 f- Xz(u). where is node in G
6:
1e Dual prompt modification by Eqs, %
7:
mode
+
node C Xt,
ptime
+ -p"
jine @ ft
9:
1a Dual conditional prompts generation by Eqs. 6 and 8
10:
Da "f"" :k]
11:
time
+ NCN(x :
12:
1a Dual conditional prompts modification by Eqs. and
13:
XE D Xt,
14:
Hime
Pi, @ Lime
time
15:
1F Temporal node embedding calculation by Eq. 10 $1
h f- DGE FFuse(xt,d plunse) Fuse god glmee
(u,t) No.))
17:
1* Update prototypical nodes #/
18:
for each class do
19:
hr,y + AVERAGE(he.e" instance belongs class at time f)
20:
1 Optimizing dual prompts and the parameters in dual condition-nets */
21:
Calculate Laowa "idee .p _p Ni o) by Eq.
22:
Update n time Φ by backpropagating Lawwn (p mole pime K, ④)
23:
return node lime
APPENDICES
A ALGORITHM
We outline the key steps for downstream prompt tuning in Algorithm 1 In lines 6-8, we use dual prompts to modify node and time features. In lines 9-14, we perform dual conditional prompting. Specifically, we generate node conditioned time prompts and time conditioned node prompts (lines 9-11), and then modify node and time's features using these conditional prompt (lines 12-14). In line 15-16, we calculate node embeddings based on the prompted node and time features. Finally, we update the embeddings for the prototypical nodes based on the few-shot labeled data provided in the task (lines 18-19) and optimize dual prompts and dual condition-nets (lines 20-22). Note that updating prototypical nodes is required exclusively for node classification tasks.
B COMPLEXITY ANALYSIS
For downstream dynamic graph G E,),, the computational process of DYGPROMPT in- volves two main parts: encoding nodes via pre-trained TGN and prompt learning. The first part's complexity is determined by the TGN's architecture, similar to other methods using pre-trained TGN. In a standard TGN, each node aggregates messages from up to neighbors per layer. Assum- ing the aggregation involves most d neighbors, the complexity of calculating node embeddings over L layers per batch time is O(d |V]). The second part, downstream prompt learning, has two stages: dual prompting and dual conditional prompting. For the dual prompting, each node and time feature is adjusted by the node prompt and time prompt, respectively, leading to complexity of O(VV+ + 1) per batch time. For the dual condition-nets with layers, for each timestamp, we first generate the bode-based condition prompt with a complexity of O(K). ), and the time-based condition prompts with a complexity of O(K- Then the condition prompts are used to modify nodes and time features, resulting in complexity of O([V) + 1). Therefore, the total complexity for prompt learning per batch time O((K + 2) ([Vl + 1)). In conclusion, the overall complexity of DYG- PROMPT is O |V| + (K 2) ([V) 1)). The first part dominates the overall complexity, as
15

---

Published as a conference paper at ICLR 2025
Table 4: Summary of datasets.
Nodes
Edges num
Node classes
Dynamic labels
Feature dimension
Time
Dataset
num
span
Wikipedia
9,227
157,474
2
217
172
30 days
Reddit
11,000
672,447
366
172
30 days
MOOC
7.144
411,749
2
4,066
172
30 days
Genre
1,505
17,858,395
474
984
86
1,500 days
O IVI) far exceeds O(KK + 2) ([V]+ 1)). Thus, the additional computational cost introduced by the conditional prompt tuning step is minimal.
C FURTHER DESCRIPTIONS DATASETS
In this section, we provide a summary of these datasets in Table and further comprehensive descriptions of these datasets.
(1) Wikipedid represents a month of modifications made by contributors on Wikipedia pages (Fer- schke et al.) 2012). Following prior studies (Rossi et al. 2020 Xu et al. 2020), we utilize data from the most frequently edited pages and active contributors, obtaining temporal graph contain- ing 9,227 nodes and 457,474 temporal directed edges. Dynamic labels indicate whether contributors temporarily banned from editing.
(2) R represents a dynamic network between posts and users on subreddits. where an edge represents a user writes a post to the subreddit, with about 1 11,000 nodes, about 700,000 temporal edges, and dynamic labels indicating whether user is banned from writing posts.
(3) MOOC is a student-course dataset, representing the actions taken by student on MOOC plat- form. The nodes are users and courses, and edges represent the actions by users on the courses. Dynamic labels indicate whether the student drops-out after the action.
(4) Gen is dynamic network linking users to music genres, with edges represents user listens to a specific genre at a certain time. The dataset includes 1,505 nodes and 17,858,395 temporal edges, and dynamic labels indicate each user's most favored music genre.
D FURTHER DESCRIPTIONS OF BASELINES
In this section, we provide additional details about the baselines used in our experiments.
(1) Conventional DGNNs
)022): ROLAND adapts static GNNs for dynamic graph learning. It offers a new viewpoint for static GNNs, where the node embeddings at different GNN layers are viewed as hierarchical node states, thus capturing temporal evolution.
ROLAND (You et al.
TGAT (Rossi et al. 2020j: TGAT use self-attention mechanism and a time encoding technique based on the classical Bochner's theorem from harmonic analysis. By stacking TGAT layers, the network recognizes the node embeddings as functions of time and is able to inductively infer embeddings for both new and observed nodes as the graph evolves.
TGN (Xu et al. 2020 TGN incorporates a memory mechanism that updates node states based on new events, effectively capturing the historical context and dependencies over time. further propose a novel training strategy that supports efficient parallel processing.
TREND (Wen & Fang 2022): TREND incorporates the Hawkes process into graph neural networks, and adopts event dynamics and node dynamics to capture the individual charac-
http: http://aaa..aattcccceee//bdddddt
htte://aap.starrrocccll///odooo.
16

---

Published as a conference paper at ICLR 2025
teristics of each event and the collective characteristics of events the same node, respec- tively.
GraphMixer (Cong et al. 2023): GraphMixer employs more simple MLP for feature learning, with portion of the parameters dedicated to time encoding being fixed. This approach enhances the model's ability to capture temporal dynamies while maintaining simpler and more flexible architecture.
(2) Graph Pre-training Models
DDGCL (Tian et 2021): DDGCL designs self-supervieed dynamic graph pre-training
method via contrasting two nearby temporal views of the same node identity.
CPDG (Bei et al. 2024): CPDG integrates two types of contrastive pre-training strategies to learn comprehensive node representations that encapsulate both long-term and short- term patterns.
(3) Graph Prompt Models
GraphPrompt (Liu et al. 2023b): GraphPrompt employs subgraph similarity mecha- nism to unify different pretext and downstream tasks, including link prediction, node clas- sification, and graph classification. A learnable prompt is subsequently tuned for each downstream task.
ProG (Sun et al., 2023): ProG reformulate the node- and edge-level tasks as graph-level tasks, and proposes prompt graphs with specific nodes and structures to guide different task.
TIGPrompt propose a prompt generator that generates personalized, dime-award prompts for each node, enhancing the adaptability and expres- siveness of node embeddings for downstream tasks.
TIGPrompt (Chen al. 2024):
IMPLEMENTATION DETAILS
Environment. The environment in which we run experiments is:
Operating system: Windows 11
CPU information: 13th Gen Intel(R) Core(TM) i5-13600KF
GPU information: GeForce RTX 4070Ti (12 GB)
Optimizer. For all experiments, use the Adam optimizer.
Details of baselines. For open-source baselines, we utilize the officially provided code. For the non-open-source CPDG and TIGPrompt, we use own implementations. Each model is tuned according to the settings recommended their respective literature to achieve optimal performance.
For both GCN and GAT in Roland, we employ 2-layer architecture.
For TGAT and TGN, we sample 20 temporal neighbors per node to update their representations. For Trend, after sampling neighboring nodes, apply Hawkes processing to these temporal neighbors using different time decay factors based on the time of each event. For GraphMixer, we employ an MLP to process the input nodes along with their positive and negative examples. The output is then passed through a network composed entirely of linear layers for prediction during training.
For DDGCL, compares two temporally adjacent views of each node, utilizing time-dependent similarity evaluation and a GAN-style contrastive loss function. For CPDG, we employ depth-first- search and
to sample neighbors of each nodes.
For GraphPrompt, we employ 1 h-hop subgraph for similarity calculation. For TIGPrompt, we employ projection prompt generator, it is reported to yield the best performance in their literature.
For all baselines, we set the hidden dimension to 172 for Wikipedia, Reddit, and MOOC, and to 86 for Genre. DDGCL, CPDG, and GraphPrompt leverage TGAT their backbone, while ProG and
17

---

Published as a conference paper at ICLR 2025
Table 5: AUC-ROC (%) evaluation for anomaly detection large-scale dataset DGraph.
Method
DGraph
GRAPHPROMPT
60.18 + 13.06
TIGPROMPT
64.42 + 10.37
DYGPROMPT
73.39 + 5.85
Table 6: Prompt-tuning and testing runtimes (seconds) on node classification.
Wikipedia
Reddit
DGraph
Method
Training
Testing
Training
Testing
Training
Testing
GRAPHPROMPT
0.147
0.230
0.128
0.251
0.436
0.324
TIGPROMPT
0.312
0.397
0.356
0.411
0.847
1.063
DYGPROMPT
0.273
0.233
0.134
0.274
0,574
0.633
TIGPrompt use TGN. For TIGPrompt, we evaluate its performance using both TGAT and TGN as the backbone.
Details of DYGPROMPT. For our proposed DYGPROMPT, conducted experiments using TGN and TGAT backbones. We employ dual-layer perceptrons with bottleneck structure the condition-net and set the hidden dimension of the condition net as 86 for Wikipedia, Reddit and MOOC, while 43 for Genre. We set the hidden dimension to 172 for Wikipedia, Reddit, and MOOC, and to 86 for Genre.
F ADDITIONAL EXPERIMENTS
F. PERFORMANCE ON LARGE-SCALE DATASET
We evaluate DYGPROMPT under the same setting introduced in Sect. 5.1 on a large-scale dataset DGraph (Huang al., 2022), which consists of 3,700,550 nodes, 4,300,999 edges, and 1,225,601 labeled nodes. The dataset is designed for the anomaly detection task, a form of binary node classi- fication. The results are shown Table 5 We observe that DYGPROMPT outperforms competitive baselines, showing the effectiveness of DYGPROMPT on large-scale datasets.
F.2 EFFICIENCY ANALYSIS
We compare the prompt tuning and testing runtimes between DYGPROMPT and competitive base- lines for the node classification task. As shown in Table 6 DYGPROMPT takes only marginally longer than GraphPrompt, while requiring less time than TIGPrompt. The slight overhead over GraphPrompt is acceptable given the substantial improvement in performance. Note that the testing runtimes are measured on the entire test set, which can be much larger than the samples used for prompt tuning, resulting in generally longer runtimes compared prompt tuning.
F.3 PERFORMANCE USING THE SPLITS OF TIGPROMPT
We adopt the splits of TIGPrompt and conduct further experiments. using 50% of the data for pre- training and 20% for prompt tuning, with the remaining 30% equally divided for validation and testing. We report the results in Table 7 where DYGPROMPT still consistently outperforms TIG- Prompt.
F.4 PERFORMANCE ON FIVE-WAY CLASSIFICATION
While most of datasets involve binary classification, also conduct five-way node classifica- tion on the ML-Rating dataset (Harper & Konstan 2013) with 9,995 nodes, 1,000,210 edges, and 5 classes. We adopt the same setting introduced in Sect. 5.T and compare DYGPROMPT with two
18

---

Published as a conference paper at ICLR 2025
Table 7: AUC-ROC (%) evaluation using the splits of TIGPPompt (Chen al.
2024
Node classification Wikipedia
Transductive link prediction Wikipedia
Inductive link prediction Wikipedia
Method
MOOC
MOOC
MOOC
TIGPROMPT
78,85 38.
63 40+2.31
93. 03.95+0.47
78. 78.98+0.52
91.35+0.38
80.26+0.76
DYGPROMPT
81.31
14.58
97.78+0.36
87.15
96.73:0.42
86. 86.14
Table 8: AUC-ROC (%) evaluation for five-way node classification on MI. ML.-Rating.
Method
ML.-Rating
GRAPHPROMPT
52.37 + 1.29
TIGPROMPT
53.26 + 1.33
DYGPROMPT
54.82 + 1.27
competitive baselines in Table strating its effectiveness.
DYGPROMPT consistently outperforms these baselines, demon-
G FURTHER COMPARISON WITH RELATED WORK
We summarize the comparison of DYGPROMPT with representative graph prompt learning methods, including GraphPrompt (Liu et al.) 2023b and ProG (Sun et al. 2023) for static graphs, and the contemporary work TIGPrompt (Chen et al. 2024) for dynamic graphs (using its best-performing variant, Projection TProG, for illustration), as shown in Table 9
More specifically, first, a node prompt modifies the node features to reformulate the task input and bridge the task gap (Eq. 4 while time prompt adjusts the time features to capture the temporal evolution of the dynamic graph (Eq. 5) The dual prompts are motivated by Challenge in Sect. [ Previous static graph prompt learning methods (GraphPrompt and ProG) only utilize node prompt, neglecting the temporal gap between pre-training and downstream tasks. The dual prompts also demonstrate empirical benefits. As shown in the ablation study in Table 2 Variant (using only node prompt) and Variant 3 (using only time prompt) consistently outperform Variant 1. Addition- ally, Variant 4 (incorporating both node and time prompts) generally outperforms Variants 2 and 3, demonstrating the effectiveness of node and time prompts in bridging the task gap and temporal gap between pre-training and downstream tasks.
Second, time-aware node prompts adjust node prompts, and ultimately node features, to reflect tem- poral influence on the node prompts (Eq.() (7) instead of using a fixed node prompt across time. On the other hand, node-aware time prompts adjust the time prompt for each time feature by incor- porating node-specific characteristics (Eq. D instead of using a fixed time prompt across nodes. Consequently, node-time patterns and their dynamic interplay are captured, addressing the Chal- lenge 2 in Section 1. Static graph prompt learning methods lack such designs, failing to capture the mutual characterization between node and time patterns. Moreover, TIGPrompt considers only the temporal impact on nodes by generating time-aware node prompts, but neglects that time prompts for each node can also be influenced by node features, hindering its ability model the scenario where different nodes may exhibit divergent behaviors even at the same time point, as illustrated in Fig. [a). As shown in the ablation study in Table 2, Variant 5 (with node-aware time prompts) out- performs Variant 2 (without node-aware time prompts), Variant 6 (with time- tane-wware node prompts) outperforms Variant (without time-awant node prompts), and DyGPrompt (with both) outperforms Variant 4 (without either), further demonstrating the effectiveness of our designs.
Third, we propose dual condition-nets to effectively generate time-aware node prompts (Eq. 6 and node-aware time prompts (Eq. []] while avoid overfitting in parameter-effiiien way. These condition-nets generate prompts conditioned on the input features rather than directly parameter- izing the prompts, significantly reducing the number of learnalle parameters in the downstream prompt-tuning tuning phase. In our implementation, we use a 2-layer MLP as the condition-net Specifi- cally, our dual condition-ne's contains merely 3,104 learnable parameters for Wikipedia, Reddit and MOOC, 1,554 for Genre and 80100 for DGraph, to generate all the time-aware node prompts and node-aware time prompts In contrast, to generate time-aware node prompts, TIGPrompt requires
19

---

Published as a conference paper at ICLR 2025
Table 9: Comparison with representative prompt learning methods.
Explicit node prompt
Explicit time prompt
Time-aware node prompts
Node-aware prompts
Condition-net
GRAPHPROMPT
X
PROG TIGPROMPT DYGPROMPT
learning specific prompt for each node, leading to a significantly larger number of learnable pa- rameters: 1,609,274 for Wikipedia, 1,911,478 for Reddit, 1,250,998 for MOOC, 130,826 for Genre, and 66,610.156 for DGraph. Moreover, as shown in Table 6 DYGPROMPT requires less training and testing time compared to TIGPrompt, further demonstrating the efficiency of using condition- net. Thus, the condition-nets DYGPROMPT is another significant difference from previous work, addressing Challenge 2 in parametcr-efffiient manner.
20


Reviews:
[{"review_id": "3ZfNPufSjr", "reviewer": "Reviewer_y8dA", "summary": "This paper studies the problem of adapting pre-trained dynamic GNNs. The authors propose DYGPROMPT, a dual prompt mechanism with node and time prompts, supported by dual condition-nets, to bridge the gaps between pre-training and downstream objectives as well as temporal inconsistencies, achieving improved adaptability and performance across dynamic graph tasks.", "questions": "Q1. In previous work, prompts are often implemented as vector concatenations. What are the author’s considerations for using dot-product instead?\n\nQ2. Personally, I am curious whether a pre-trained time encoder can address questions such as “What is the time of the next interaction?”\n\nQ3. How does DYGPROMPT perform on large-scale dynamic graphs?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "S1. I like the idea of using dual prompt mechanism to bridge task and temporal gaps.\n\nS2. Experiments show DYGPROMPT is effective on small real-world graphs.", "weaknesses": "W1. The introduction of dual prompts and dual condition networks enhances adaptability but significantly increases model complexity. I would suggest the authors to add some analysis or experiments to demonstrate DYGPROMPT’s performance in terms of computational efficiency.\n\nW2. I have concerns about the experimental setup. Using the first 80% of interactions for pre-training is uncommon in pre-trained frameworks. For example, TIGPROMPT uses only 20% of the data, yielding surprisingly strong results in comparison.\n\nW3. The discussion of current dynamic graph prompt learning methods, such as TIGPROMPT, is brief and lacks an in-depth analysis of their limitations or how DYGPROMPT specifically addresses these issues. Additionally, Section 4 contains extensive information on previous works, making it difficult to discern the paper’s core contributions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jjTZ2czAll", "reviewer": "Reviewer_BsEP", "summary": "The paper introduces DYGPROMPT, a novel pretraining and prompt learning framework designed for dynamic graph learning. It addresses the challenges of bridging the gap between pre-training objectives and downstream tasks. The authors propose dual prompts -- node prompts and time prompts -- to address the discrepancies in both task objectives and temporal variations between pre-training and downstream tasks. The authors introduce dual condition-nets to model the evolving node-time patterns in downstream tasks. This includes a time condition-net that generates time-conditioned node prompts and a node condition-net that generates node-conditioned time prompts. Experiments on four public datasets demonstrate its superior performance compared to state-of-the-art approaches in tasks such as temporal node classification and temporal link prediction.", "questions": "See above.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- **Relevance to Real-World Applications:** The framework's focus on dynamic graphs makes it highly relevant to real-world applications such as social networks, online learning platforms, and crowdsourcing projects.\n\n- **Parameter Efficiency:** DYGPROMPT's ability to perform well with minimal parameter updates is a significant strength, especially for applications where labeled data is scarce.\n\n- **Open source:** It is commendable that the authors have made their code publicly available, enhancing the transparency and reproducibility of their work.", "weaknesses": "- **Motivation:** The motivation for this work is insufficiently justified. After reading the paper, it remains unclear why temporal patterns would be influenced by node features. Time, as a continuous scale, is inherently objective and independent of specific events.\n\n- **Novelty:** Based on the above, the novelty of this work appears limited. TIGPrompt has already proposed an effective strategy for dynamic graph learning by fusing prompts through the concatenation of node and time embeddings for prompt-based fine-tuning. This contrasts with the authors’ claim that TIGPrompt only considers temporal factors within node features. Thus, the main difference between this work and TIGPrompt seems to lie in the adoption of a more complex information fusion strategy rather than a simple concatenation.\n\n- **Experiments:** The experimental setup and results raise some concerns. First, the reported results are noticeably lower than those of TIGPrompt. Given the relevance of TIGPrompt as a baseline, I recommend that the authors adopt identical experimental settings to ensure comparability. Additionally, the influence of model size on performance is unclear; I suggest that the authors report the number of parameters for both their model and TIGPrompt.\n\n- **Writing:** The writing could be further polished to improve clarity and readability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7FoOo2I3yM", "reviewer": "Reviewer_ngqA", "summary": "To address the challenges of prompt-based learning on dynamic graphs, this paper develops a novel pretraining and prompt learning framework for dynamic graph modeling. The authors propose dual prompts and dual condition networks to model the evolving node-time patterns. Extensive experiments are conducted to evaluate the performance of the proposed node-time solution. Overall, the approach is sound and reasonable.", "questions": "The study considers only two tasks on dynamic graphs. Is it possible to apply this framework to other graph tasks?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1.\tThis paper identifies the issue of pretrained Graph Neural Networks (GNNs)—specifically, the gap between pre-training and task objectives—and proposes novel prompt learning approaches for dynamic graphs. It also considers the evolving interplay patterns between nodes and time points.\n2.\tThe framework addresses temporal variations across time and divergent task objectives by leveraging a node prompt and a time prompt to reduce the gap between the pre-training and downstream phases. The main novelty lies in the introduction of the time prompt, which narrows the inconsistencies that arise from varying priorities at different times. The proposed method is reasonable.\n3.\tExtensive experiments are conducted to evaluate the performance of various methods. The source codes are available anonymously, which adds to the credibility of the results.\n4.\tThe manuscript is well written and easy to follow.", "weaknesses": "1.\tThe concept of temporal prompts for dynamic graphs has been previously investigated (e.g., TIGPrompt), but the differences between approaches are not clearly articulated. In the related work, it is stated that “it only considers the temporal factor in node features, overlooking that temporal patterns are also influences by node features.” More explanations are needed to highlight the technical novelty of this paper.\n2.\tThe overall framework presented in Figure 2 could be improved; it is not clear how to compute the downstream loss (Eq. 11) and why it only links to the time branch.\n3.\tAll evaluated datasets are relatively small, and the paper lacks an efficiency or complexity analysis for larger-scale datasets, which is important for practical applications.\n4.\tThere are minor typos and errors, such as “are also influences by node.”", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6Gvo4nYkdI", "reviewer": "Reviewer_obkg", "summary": "The paper introduces DYGPROMPT, a novel framework for dynamic graph modeling that aims to bridge the gap between pre-training and downstream tasks, such as node classification and link prediction, on dynamic graphs. The authors propose a two-stage approach involving pre-training and prompt learning. In the pre-training phase, a dynamic graph neural network is trained on the task of temporal link prediction. For the downstream tasks, the authors design dual prompts (node and time prompts) to address the differences in objectives and temporal variations between the pre-training and downstream phases. Additionally, they introduce dual condition nets to generate a series of time-conditioned node prompts and node-conditioned time prompts, capturing the evolving node-time patterns. The framework is evaluated on four public datasets, demonstrating superior performance compared to state-of-the-art methods.", "questions": "Q1: Can the authors comment on the scalability of DYGPROMPT to large-scale dynamic graphs, both in terms of computational resources and performance?\n\nQ2: How does DYGPROMPT adapt to long-term temporal changes in dynamic graphs, especially when the graph's structure and node behaviors evolve significantly over time?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "S1. The paper presents a creative solution to the challenge of adapting pre-trained DGNNs to downstream tasks, which is a novel contribution to dynamic graph learning.\n\nS2. The paper is well-written and easy to follow.\n\nS3. Extensive experiments compared with DGNNs, SOTA pre-training models, and graph prompt learning models demonstrate the effectiveness of the proposed model.", "weaknesses": "W1.  While the paper mentions the computational complexity, a deeper analysis comparing the complexity of DYGPROMPT with other methods, especially in terms of training and inference time, is necessary for a prompt-based model.\n\nW2. The used datasets are very small-scale. The largest dataset used only contains about 10K nodes. This makes it difficult to confirm whether the model can be successfully applied to large-scale data and the real world.\n\nW3. The experimental datasets have extreme class numbers. Specifically, 3 of the datasets are binary classification and another one contains 474 classes. It would be better to experiment on other more datasets.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)