Node-Time Conditional Prompt Learning in Dynamic Graphs

## Abstract

Abstract Dynamic graphs capture evolving interactions between entities, such as in social networks, online learning platforms, and crowdsourcing projects. For dynamic graph modeling, dynamic graph neural networks (DGNNs) have emerged as a mainstream technique. However, they are generally pre-trained on the link prediction task, leaving a significant gap from the objectives of downstream tasks such as node classification. To bridge the gap, prompt-based learning has gained traction on graphs, but most existing efforts focus on static graphs, neglecting the evolution of dynamic graphs.
In this paper, we propose DyGPrompt , a novel pre-training and prompt learning framework for dynamic graph modeling. First, we design dual prompts to address the gap in both task objectives and temporal variations across pre-training and downstream tasks. Second, we recognize that node and time features mutually characterize each other,
and propose dual condition-nets to model the evolving node-time patterns in downstream tasks.
Finally, we thoroughly evaluate and analyze DyGPrompt through extensive experiments on four public datasets.

## 1 Introduction

Graph data are pervasive due to their ability to model complex relationships between entities in various applications, including social network analysis (Fan et al., ; Zhou et al., ), Web mining (Xu et al., ; Agarwal et al., ), and content recommendation systems (Qu et al., ; Zhu et al., ).
In many of these applications, the graph structures evolve over time, such as users commenting on posts in a social network (Kumar et al., ; Iba et al., ), editing pages in a crowdsourcing project like Wikipedia (Kumar et al., ), or interacting with courses in an online learning platform (Liyanagunawardena et al., ). Such graphs are termed *dynamic* graphs (Barros et al., ; Skarding et al., ), in contrast to conventional *static* graphs that maintains an unchanging structure.

On dynamic graphs, dynamic graph neural networks (DGNNs) (Pareja et al., ; Xu et al., ) have been widely applied. In a typical design for DGNNs, each node updates its representation by iteratively receiving and aggregating messages from its neighbors in a time-dependent manner.
While DGNNs are often pre-trained on the link prediction task, the downstream application could involve a different task such as node classification, leading to a significant gap between pre-training and task objectives. More advanced pre-training and fine-tuning strategies on dynamic graphs (Bei et al., ; Chen et al., ; Tian et al., ) also suffer from the same limitation. They first pre-train a model for dynamic graphs based on various task-agnostic self-supervised signals on the graph, and then updates the pre-trained model weights in a fine-tuning phase based on task-specific labels pertinent to the downstream application. Likewise, the pre-training and downstream objectives can differ significantly, resulting in a notable gap between them.

To bridge the gap between pre-training and downstream objectives (Liu et al., ), prompt learning have first emerged in language models (Brown et al., ). Fundamentally, a prompt serves to reformulate the input for the downstream task to align with the pre-trained model, while freezing the pre-trained weights. Given much fewer parameters in a prompt than the pre-trained model, prompt learning is parameter-efficient and effective especially in data-scarce scenarios (Yu et al., ). Inspired by the success of prompt in the language domain, recent studies have explored prompt learning on graphs (Liu et al., ; Sun et al., ; ). However, these studies only focus on static graphs, neglecting the unique challenges brought by dynamic graphs.
In this work, we explore Prompt learning for Dynamic Graphs, and propose a framework called DyGPrompt. The problem is non-trivial due to two key challenges.

Figure: Figure 1: Motivation of DyGPrompt. (a) Users comment on different topics over time. (b) Evolving node-time patterns in Wikipedia, as node and time features mutually characterize each other.
Refer to caption: x1.png

First, how do we design prompts to bridge temporal variations across time, in addition to divergent task objectives? Generally, both task objectives and the timing of pre-training and downstream tasks can be inconsistent. For example, consider a user discussion network such as Reddit.com, as illustrated in Fig. 1(a), where users and posts are represented as nodes, and an edge is formed when a user comments on a post. On this dynamic graph, a typical pre-training task involves link prediction—predicting whether a user comments on (i.e., links to) a video game post at time $t_{1}$. In contrast, the downstream task could be the status classification of the user’s account, e.g., determining whether it has been suspended at time $t_{2}$.
While previous work on static graphs (Liu et al., ; Sun et al., ; ) employ prompts to narrow the task objective gap, the temporal gap remains unsolved.
In particular, even when the downstream task also involves link prediction, the evolving graph structure introduces a temporal gap between the pre-training and downstream phases.
In DyGPrompt, we introduce dual prompts, consisting of a *node prompt* and a *time prompt*. On one hand, a node prompt alters the node features to reformulate task input and bridge the task gap, similar to previous work (Liu et al., ; Fang et al., ). On the other hand, a time prompt adjusts the time features that capture the temporal evolution of the dynamic graph, thereby narrowing inconsistencies that arise from varying priorities at different times.

Second, how do we capture evolving patterns across different nodes and time points, driven by the dynamic interplay between them?
Different nodes exhibit unique timelines, evolving distinctively over time as nodes and time mutually influence each other. For example, in the user discussion network, a student may comment on open source codes in the morning, while commenting on their favorite game in the evening. However, as shown in Fig. 1(a), a professor may comment on academic policies at noon, while discussing family topics at night.
Therefore, node and time features mutually characterize each other, exhibiting evolving node-time patterns as shown in Fig. 1(b), which may differ from the patterns observed in the pre-training data. However, previous studies neglect the fine-grained characteristics of downstream nodes, time, and their mutual interactions.
Note that some works (Tan et al., ; Wen & Fang, ) have designed individual prompts or tokens to tailored to each node,
and a contemporary approach (Chen et al., ) has developed temporally-aware prompts. However, they do not account for the mutual characterization between nodes and time, and are thus unable to fully capture the fine-grained, evolving node-time patterns.
In DyGPrompt, inspired by conditional prompt learning (Zhou et al., ), we propose a series of node and time prompts generated by *dual condition-nets*, characterizing the uniqueness of each node and time point. Specifically, at each timestamp, a *time condition-net* generates a sequence of time-conditioned node prompts for each node.
These time-conditioned node prompts modify node features to reflect temporal subtleties at different time points, aligning the node’s pattern at each timestamp with those observed during the pre-training phase.
Likewise, for each node, a *node condition-net* generates a sequence of node-conditioned time prompts at each timestamp.
The node-conditioned time prompts adjust the time features by integrating node characteristics, thereby aligning the temporal patterns of each node with those captured by the pre-trained model.

In summary, the contributions of this work are threefold.
(1) We introduce DyGPrompt for dynamic graph modeling, proposing dual prompts to narrow the gaps arising from both task and dynamic difference between pre-training and downstream objectives.
(2) We recognize that node and time features mutually characterize each other, and further propose dual condition-nets to generate a series of node and time prompts, thereby adapting to downstream node-time patterns.
(3) We conduct extensive experiments on four benchmark datasets, demonstrating the superior performance of DyGPrompt in comparison to the state-of-the-art approaches.

## 2 Related Work

Dynamic graph learning.
Real-world graph structures generally evolve over time. To model such dynamic evolution, various continuous-time dynamic graph learning techniques have been proposed. Typically, these methods update nodes representation by iteratively receiving and aggregating messages from their neighbors in a time-dependent manner (Skarding et al., ).
Some approaches employ dynamic random walks to depict structural changes (Nguyen et al., ; Wang et al., ), while others incorporate a time encoder to integrate temporal context with structural modeling (Xu et al., ; Cong et al., ; Rossi et al., ; Yu et al., ). Furthermore, some researchers utilize temporal point processes to model structural evolution or node communication (Kumar et al., ; Trivedi et al., ; Wen & Fang, ).
However, these methods are typically trained on link prediction tasks, whereas downstream applications may involve different tasks such as node classification, creating a significant gap between training and task objectives. This gap impedes effective knowledge transfer, adversely hurting downstream task performance.

Graph pre-training.
More advanced dynamic graph pre-training methods also face similar problems. Extending from graph pre-training methods (Hu et al., ; ), dynamic graph pre-training captures inherent properties of dynamic graphs using various task-agnostic, self-supervised signals through various strategies such as structural and temporal contrastive learning (Bei et al., ; Tian et al., ), dynamic graph generation (Chen et al., ), and curvature-varying Riemannian graph neural networks (Sun et al., ). They then attempt to transfer the prior knowledge to downstream tasks through fine-tuning with task-specific supervision.
Nonetheless, a gap persists between the objectives of pre-training and fine-tuning (Liu et al., ; Yu et al., ). While pre-training seeks to extract fundamental insights from graphs without supervision, fine-tuning is tailored to specific supervision for a given downstream task.

Graph prompt learning.
First emerging in the language domain, prompt learning effectively bridges the gap between pre-training and downstream objectives (Brown et al., ). With a unified template, prompts are specifically tailored for each downstream task, aligning them more closely with the pre-trained model while keeping the pre-trained parameters frozen. The success of prompt learning has led to its widely adoption on static graphs (Liu et al., ; Sun et al., ; Fang et al., ; Tan et al., ; Yu et al., ; ), yet its application in dynamic graphs remains an open question. A contemporary work, TIGPrompt (Chen et al., ), proposes a prompt generator to learn temporal prompts that address the task and temporal gaps. However, it only considers the temporal factor in node features, overlooking that temporal patterns are also influences by node features. Thus, it falls short in aligning the mutually characterizing node-time patterns in a downstream task with those captured by the pre-trained model.

## 3 Preliminaries

In this section, we present related preliminaries and introduce our scope.

Dynamic graph.
Dynamic graphs are categorized into discrete-time dynamic graphs and continuous-time dynamic graphs (Skarding et al., ). The former consist of a series of static graph snapshots, whereas the latter are depicted as sequences of events on a continuous timeline.
In this work, we adopt the more general continuous-time definition, where a dynamic graph is represented by $G=(V,E,T)$. Here, $V$ denotes the set of nodes, $E$ denotes the set of edges, and $T$ represents the time domain. In particular, each edge $(v_{i},v_{j},t)\in E$ indicates a specific interaction (or event) between nodes $v_{i}$ and $v_{j}$ at time $t$. Each node is associated with a temporal feature vector $\mathbf{x}_{t,v}\in\mathbb{R}^{d}$ evolving over time, represented as a row in the temporal feature matrix $\mathbf{X}_{t}\in\mathbb{R}^{|V|\times d}$.

Dynamic graph neural network.
A popular backbone for static graphs is message-passing graph neural networks (GNNs) (Wu et al., ). Formally, in the $l$-th GNN layer, the embedding of node $v$, represented as $\mathbf{h}^{l}_{v}$, is calculated based on the embeddings from the preceding layer:

$$ $\mathbf{h}^{l}_{v}=\mathtt{Aggr}(\mathbf{h}^{l-1}_{v},\{\mathbf{h}^{l-1}_{u}:u \in\mathcal{N}_{v}\}),$ (1) $$

where $\mathcal{N}_{v}$ denotes the set of neighboring nodes of $v$, and $\mathtt{Aggr}(\cdot)$ is a neighborhood aggregation function.
Extending to dynamic graph neural networks (DGNNs), researchers integrate temporal contexts into neighborhood aggregation (Pareja et al., ; Xu et al., ), often utilizing a *time encoder* ($\mathtt{TE}$) to map timestamps or intervals (Cong et al., ; Rossi et al., ) within a *dynamic graph encoder* ($\mathtt{DGE}$). Consequently, in the $l$-th layer of a dynamic graph encoder, the embedding of node $v$ at time $t$, denoted by $\mathbf{h}^{l}_{t,v}$, is derived as

$$ $\displaystyle\mathbf{h}^{l}_{t,v}=\mathtt{Aggr}(\mathtt{Fuse}(\mathbf{h}^{l-1} _{t,v},\mathtt{TE}(t)),\{\mathtt{Fuse}(\mathbf{h}^{l-1}_{t^{\prime},u},\mathtt {TE}(t^{\prime})):(u,t^{\prime})\in\mathcal{N}_{v}\}),$ (2) $$

where $\mathcal{N}_{v}$ contains the historical neighbors of $v$, with $(u,t^{\prime})\in\mathcal{N}_{v}$ indicating that $u$ interacts with $v$ at time $t^{\prime}$.
$\mathtt{Fuse}(\cdot)$ is a fusion operation that integrates structural and temporal contexts, such as concatenation (Rossi et al., ) or addition (Trivedi et al., ). For simplicity, the output node embedding from the final layer is denoted as $\mathbf{h}_{t,v}$, which is then utilized in a loss function for optimization.
Note that the time encoder maps the time domain to a vector space, $\mathtt{TE}:T\rightarrow\mathbb{R}^{d}$. A common definition is
$\mathtt{TE}(t)=\frac{1}{\sqrt{d}}[\cos(\omega_{1}t),\sin(\omega_{1}t),\ldots,
\cos(\omega_{d}t),\sin(\omega_{d}t)]$,
where $\{\omega_{1},\ldots,\omega_{d}\}$ is a set of learnable parameters (Xu et al., ; Cong et al., ).

Scope of work.
In this work, we pre-train a DGNN with a time encoder on dynamic graphs, employing the task of temporal link prediction (Wang et al., ).
For downstream tasks, we target two popular dynamic graph-based tasks, namely, temporal node classification and temporal link prediction(^1^11As dynamic graphs focus on the evolving structures within a graph, graph classification is rarely evaluated as a task (Skarding et al., ; Pareja et al., ; Xu et al., ; Chen et al., ).).
Specifically, we focus on *data-scarce scenarios* with only limited data for downstream adaptation, as labeled data is often difficult or costly to obtain for node classification (Zhou et al., ; Yao et al., ), while nodes with sparse interactions are common in real-world link prediction tasks (Lee et al., ; Pan et al., ).

In temporal node classification, since node properties (e.g., account status in a social network) can vary as the graph evolves,
the goal is to predict the class of a given node at a specific time $t$ from a predefined set of classes $Y$.
In temporal link prediction, our goal is to predict whether a link exists between two nodes at a specific time in both transductive and inductive settings. The transductive setting involves nodes that have appeared during both pre-training and downstream prompt tuning, whereas the inductive setting exclusively predicts the existence of edges between nodes that have not been seen during both pre-training and downstream tuning.

## 4 Proposed Approach

In this section, we present our proposed model DyGPrompt, starting with its overall framework, followed by its key components.

### 4.1 Overall framework

We illustrate the overall framework of DyGPrompt in Fig. 2, which consists of two phases: pre-training and downstream adaptation.
First, given a dynamic graph in Fig. 2(a), we pre-train a DGNN based on temporal link prediction as shown in Fig. 2(b).
In particular, we employ a universal task template based on similarity calculation (Liu et al., ), which unifies different graph-based tasks, such as link prediction and node classification, across the pre-training and downstream phases.
Second, given a pre-trained DGNN, to enhance its adaptation to downstream tasks, we propose dual prompts, as shown in Fig. 2(c). The dual prompts comprise a node prompt and a time prompt, aiming to bridge the gap caused by different task objectives and dynamic variations, respectively. Moreover, extending from the dual prompts, we propose *dual condition-nets* to generate a series of time-conditioned node prompts and node-conditioned time prompts. These conditional prompts are designed to capture the evolving patterns across nodes and time points, along with their mutual characterization, in a fine-grained manner.

Figure: Figure 2: Overall framework of DyGPrompt.
Refer to caption: x2.png

### 4.2 Pre-training

Pre-training techniques (Le-Khac et al., ) have been widely explored on dynamic graphs (Bei et al., ; Chen et al., ; Tian et al., ). In particular, the most common pre-training task is temporal link prediction (Xu et al., ; Rossi et al., ), which are readily available without the need for manual annotations.
Hence, as illustrated in Fig. 1(b), we adopt link prediction for pre-training. Its simplicity underscores the effectiveness and robustness of our prompt-based learning in downstream tasks.

Specifically, consider an event $(v,a,t)$, i.e., an interaction between nodes $v$ and $a$ at time $t$. Correspondingly, we construct a tuple $(v,a,b,t)$ such that nodes $v$ and $b$ are not linked at $t$, serving as a contrastive signal. For each tuple, the pre-training objective is to maximize the similarity between nodes $v$ and $a$ while minimizing that between $v$ and $b$. Similar to previous work (You et al., ; Yu et al., ), we optimize the following pre-training loss over pre-training data, $\mathcal{D}_{\text{pre}}$.

$$ $\displaystyle\textstyle\mathcal{L}_{\text{pre}}(\mathcal{D}_{\text{pre}}; \Theta)=-\sum_{(v,a,b,t)\in\mathcal{D}_{\text{pre}}}\ln\frac{\exp\left(\frac{1 }{\tau}\text{sim}(\mathbf{h}_{t,v},\mathbf{h}_{t,a})\right)}{\exp\left(\frac{1 }{\tau}\text{sim}(\mathbf{h}_{t,v},\mathbf{h}_{t,b})\right)}.$ (3) $$

In Eq. (3), $\mathbf{h}_{t,v},\mathbf{h}_{t,a},\mathbf{h}_{t,b}$ represent the node embeddings of $v,a,b$ at time $t$, respectively. $\tau>0$ is a temperature hyperparameter. $\Theta$ denotes the set of learnable parameters of the DGNN, including those of the dynamic graph encoder and the time encoder.
The pre-trained parameters, $\Theta_{\text{pre}}=\arg\min_{\Theta}\mathcal{L}_{\text{pre}}(\mathcal{D}_{
\text{pre}};\Theta)$, will be used to initialize the downstream models.

### 4.3 Dual prompts

To reconcile task and temporal variations across pre-training and downstream tasks, we propose dual prompts, comprising a node prompt and a time prompt, as shown in Fig. 2(c).

Node prompt.
In language models, a prompt is a purposely designed textual description that reformulates the input for a downstream task, aligning the task with the pre-trained model (Brown et al., ; Jia et al., ). Likewise, a prompt for static graphs adjust downstream node features (Liu et al., ; Fang et al., ).
Formally, for each downstream task, we define a learnable vector $\mathbf{p}^{\text{node}}$ as the *node prompt* to modify the node features at a given time. Concretely, the input features of node $v$ at time $t$, denoted by $\mathbf{x}_{t,v}$, is modified into the following.

$$ $\displaystyle\mathbf{x}^{\text{node}}_{t,v}=\mathbf{p}^{\text{node}}\odot \mathbf{x}_{t,v},$ (4) $$

where $\odot$ is element-wise multiplication. The node prompt $\mathbf{p}^{\text{node}}$ has the same dimension as node features, altering the importance of various node feature dimensions.

Time prompt.
We adopt a time encoder $\mathtt{TE}$ to map the continuously valued time $t$ to time features, $\mathbf{f}_{t}=\mathtt{TE}(t)$.
However, TE is optimized based on the time periods in the pre-training data, which can be distinct from those in the downstream tasks. To overcome the temporal variations, we utilize a time prompt to adjust time features. Formally, for each downstream task, we define a learnable vector $\mathbf{p}^{\text{time}}$ as the *time prompt*. Then, the time features at $t$, denoted by $\mathbf{f}_{t}$, is adjusted as follows.

$$ $\displaystyle\mathbf{f}^{\text{time}}_{t}=\mathbf{p}^{\text{time}}\odot\mathbf {f}_{t}.$ (5) $$

Therefore, the time prompt reweighs the important of various time feature dimensions to align the downstream temporal patterns with the pre-trained model.

### 4.4 Dual condition-nets

In dynamic graphs, each node may exhibit distinct behaviors at different times. Meanwhile, different nodes may evolve differently over the same time interval. In other words, node and time features influence and characterize each other over time, revealing fine-grained node-time patterns. Given the interdependence of node and time features,
we turn to conditional prompt learning (Zhou et al., ) for finer-grained adaptation.
Instead of learning a static prompt tailored to a specific task over the entire time span, we design dual condition-nets, comprising a time condition-net that generates a sequence of time-conditioned node prompts for each node, and a node condition-net that generates a sequence of node-conditioned time prompts at each timestamp, as shown in Fig. 2(c).

Time condition-net.
To characterize node features based on time, a straightforward way is to employ a series of learnable vectors as prompts for each distinct timestamp. However, this approach does not scale to large time intervals with many timestamps, as it quickly increases the number of tunable parameters.
Therefore, we introduce a time condition-net, which generates a series of time-conditioned node prompts. Formally, the node prompt conditioned on time $t$ is given by

$$ $\displaystyle\tilde{\mathbf{p}}^{\text{time}}_{t}=\mathtt{TCN}(\mathbf{f}^{ \text{time}}_{t};\kappa),$ (6) $$

where $\mathtt{TCN}$ is the time condition-net parameterized by $\kappa$.
This approach is essentially a form of hypernetwork (Ha et al., ), which employs a secondary network—our condition-net $\mathtt{TCN}$—to generate the prompts conditioned on the input feature, rather than directly parameterizing the prompts. This formulation enables parameter-efficient prompt generation, especially when using a lightweight secondary network ($\mathtt{TCN}$).
In our implementation, we opt for a simple multi-layer perceptron (MLP) with an efficient bottleneck structure (Wen & Fang, ; Wu & Lee, ). Therefore, for a series of events occurring at times $\{t_{0},t_{1},\ldots\}$, the time condition-net produces their corresponding time-conditioned node prompts $\{\tilde{\mathbf{p}}^{\text{time}}_{t_{0}},\tilde{\mathbf{p}}^{\text{time}}_{t
_{1}},\ldots\}$. Note that these conditional prompts match the dimensionality of the node features. Then, the features of node $v$ at time $t$, which has been modified by the time prompt in Sect. 4.3, is further updated as

$$ $\displaystyle\tilde{\mathbf{x}}^{\text{node}}_{t,v}=\tilde{\mathbf{p}}^{\text{ time}}_{t}\odot\mathbf{x}^{\text{node}}_{t,v}.$ (7) $$

Node condition-net.
To characterize time features based on nodes, we introduce a node condition-net to generate a series of *node-conditioned time prompts*, mirroring the time condition-net.
Formally, the time prompt conditioned on features of node $v$ at time $t$ is given by

$$ $\displaystyle\tilde{\mathbf{p}}^{\text{node}}_{t,v}=\mathtt{NCN}(\mathbf{x}^{ \text{node}}_{t,v};\phi),$ (8) $$

where $\mathtt{NCN}$ is the node condition-net parameterized by $\phi$, which is also implemented as an MLP with a bottleneck structure. Consequently, for time $t$, a series of node-conditioned time prompts $\{\tilde{\mathbf{p}}^{\text{node}}_{t,v_{0}},\tilde{\mathbf{p}}^{\text{node}}_
{t,v_{1}},\ldots\}$ is generated for each node, which have the same dimension as the time prompts. The time feature for node $v$ at time $t$ is then further modified into

$$ $\displaystyle\tilde{\mathbf{f}}^{\text{time}}_{t,v}=\tilde{\mathbf{p}}^{\text{ node}}_{t,v}\odot\mathbf{f}^{\text{time}}_{t}.$ (9) $$

### 4.5 Prompt tuning

Finally, we feed the modified time features and node features to the dynamic graph encoder, $\mathtt{DGE}$. The embedding of node $v$ at time $t$ is calculated as follows:

$$ $\displaystyle\mathbf{h}_{t,v}=\mathtt{DGE}\left(\mathtt{Fuse}(\tilde{\mathbf{x }}^{\text{node}}_{t,v},\tilde{\mathbf{f}}^{\text{time}}_{t,v}),\left\{\mathtt{ Fuse}(\tilde{\mathbf{x}}^{\text{node}}_{t^{\prime},u},\tilde{\mathbf{f}}^{ \text{time}}_{t^{\prime},v}):(u,t^{\prime})\in\mathcal{N}_{v}\ \right\}\right).$ (10) $$

To tune the prompts and condition-nets for a downstream task, we adopt similarity calculation as the task template (Liu et al., ), consistent with the pre-training loss. Specifically, for temporal node classification, consider a labeled set $\mathcal{D}_{\text{down}}=\{(v_{1},y_{1},t_{1}),(v_{2},y_{2},t_{2}),\ldots\}$, where each $v_{i}$ denotes a node, and $y_{i}\in Y$ is the class label of $v_{i}$ at time $t_{i}$. Then, the downstream loss is

$$ $\displaystyle\textstyle\mathcal{L}_{\text{down}}(\mathcal{D}_{\text{down}}; \mathbf{p}^{\text{node}},\mathbf{p}^{\text{time}},\kappa,\phi)=-\sum_{(v_{i},y _{i},t_{i})\in\mathcal{D}_{\text{down}}}\ln\frac{\exp\left(\frac{1}{\tau}\text {sim}(\mathbf{h}_{t_{i},v_{i}},\bar{\mathbf{h}}_{t_{i},y_{i}})\right)}{\sum_{y \in Y}\exp\left(\frac{1}{\tau}\text{sim}(\mathbf{h}_{t_{i},v_{i}},\bar{\mathbf {h}}_{t_{i},y})\right)},$ (11) $$

where ${\mathbf{h}}_{t_{i},y}$ represents class $y$’s prototype embeddings (Liu et al., ) at time $t_{i}$, which is obtained by aggregating the embeddings of examples in class $y$ at time $t_{i}$.
For temporal link prediction, we utilize the same loss as the pre-training loss in Eq. (3).

During prompt tuning, we only update the lightweight dual prompt vectors ($\mathbf{p}^{\text{node}},\mathbf{p}^{\text{time}}$) and the parameters of the dual condition-nets ($\kappa,\phi$), whilst freezing the pre-trained DGNN weights. This parameter-efficient tuning approach is amenable to label-scarce settings, where $\mathcal{D}_{\text{down}}$ comprises only a limited number of training examples.

We outline the key steps for prompt tuning in Algorithm 1, Appendix A.1 and assess its complexity in Appendix A.2.

## 5 Experiments

In this section, we conduct experiments to evaluate DyGPrompt and analyze the empirical results.

### 5.1 Experimental Setup

Datasets.
We utilize three benchmark datasets for evaluation.
(1) Wikipedia and (2) Reddit are both user–post networks. Each dataset represents a monthly record, with nodes representing users or posts. In Wikipedia, an edge denotes a user editing a post, while in Reddit, an edge represents a user commenting on a post. Dynamic class labels indicate whether a user’s account has been temporarily suspended.
(3) MOOC is a user–course network, where nodes represent users and courses, and edges represent actions by users on the courses. Dynamic class labels indicate whether the user has dropped out of the course following this action.
(4) Genre is a user–music genre network, where a node corresponds to a user or genre, and an edge represents a user listening to a particular genre at a specific moment in time. Dynamic class labels reflect users’ preferred music genre.
We provide further details of the datasets in Appendix A.3.

Downstream tasks.
We conduct temporal node classification and temporal link prediction to evaluate the performance of DyGPrompt. For link prediction, we perform experiments in both transductive and inductive settings, depending on whether nodes in the testing set have appeared in pre-training or downstream tuning data. In this study, we investigate the pre-training and prompt learning on dynamic graphs, which differs from previous DGNNs (Rossi et al., ; Xu et al., ).
Therefore, we adopt a new data split to accommodate the pre-training and downstream tuning steps.
Specifically, given a *chronologically ordered* sequence of events (i.e., edges with timestamps), we use the first 80% for pre-training. Note that we pre-train a DGNN only once for each dataset and subsequently employ the same pre-trained model for all downstream tasks. The remaining 20% the events are used for downstream tasks. Specifically, they are further split into 1%/1%/18% subsets, where the first 1% is reserved as the training pool for downstream prompt tuning, and the next 1% as the validation pool, with the last 18% for downstream testing.

To construct a downstream task,
we start with sampling 30 events (about 0.01% of the whole dataset) from the training pool, under the constraint that there is at least one user per class. Then, for node classification, we take the user nodes as training instances for downstream adaptation.
For link prediction, the downstream training pool also helps with adaptation due to its temporal proximity to the test set. Specifically, we take the sampled events as positive instances, while further sampling one negative instance for each positive instance. Specifically, for each event $(v,a,t)$, we sample one negative instance $(v,b,t)$ such that $b$ is a node taken from the training pool that is not linked to $v$ at time $t$. The validation sets are constructed by repeating the same process on the validation pool.
Finally, the test set for node classification simply comprises all user nodes in the testing events, while the test set for link prediction is expanded to include negative instances following a similar sampling process in training. Furthermore, for inductive link prediction, we remove instances from the test set if their nodes have appeared in pre-training or the downstream training pool.
We repeat this sampling process for 100 times to construct 100 tasks for node classification and link prediction.

Evaluation. To evaluate the performance on the tasks, we employ AUC-ROC for both node classification (Xu et al., ; Rossi et al., ) and link prediction (Sun et al., ; Bei et al., ) following prior work. For each of the 100 tasks, we repeat the experiments with five different random seeds, and report the average and standard deviation over the 500 results.

**Table 1: AUC-ROC (%) evaluation of temporal node classification and link prediction.**
| Methods | Node Classification | Transductive Link Prediction | Inductive Link Prediction |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Wikipedia | Reddit | MOOC | Genre | Wikipedia | Reddit | MOOC | Genre | Wikipedia | Reddit | MOOC | Genre |  |
| GCN-ROLAND | 58.86±10.3 | 48.25±9.57 | 49.93±6.74 | 46.33±3.97 | 49.61±3.12 | 50.01±2.53 | 49.82±1.44 | 49.15±3.74 | 49.60±2.37 | 49.90±1.64 | 49.16±2.48 | 47.25±2.97 |
| GAT-ROLAND | 62.81±9.88 | 47.95±8.42 | 50.01±6.34 | 47.26±3.49 | 52.34±1.82 | 50.04±1.98 | 55.74±3.71 | 47.69±2.81 | 52.29±1.97 | 49.85±2.35 | 54.01±2.16 | 49.38±2.72 |
| TGAT | 67.00±5.35 | 53.64±5.50 | 59.27±4.43 | 51.26±2.31 | 55.78±2.03 | 62.43±1.86 | 51.49±1.30 | 69.11±3.89 | 48.21±1.55 | 57.30±0.70 | 51.42±4.27 | 48.38±4.72 |
| TGN | 50.61±13.6 | 49.54±6.23 | 50.33±4.47 | 50.72±2.31 | 72.48±0.19 | 67.37±0.07 | 54.60±0.80 | 86.46±2.84 | 74.38±0.29 | 69.81±0.08 | 54.62±0.72 | 87.17±2.68 |
| TREND | 69.92±9.27 | 64.85±4.71 | 66.79±5.44 | 50.34±1.62 | 63.24±0.71 | 80.42±0.45 | 58.70±0.78 | 52.78±1.14 | 50.15±0.90 | 65.13±0.54 | 57.52±1.01 | 45.31±0.43 |
| GraphMixer | 65.43±4.21 | 60.21±5.36 | 63.72±4.98 | 50.15±1.49 | 59.73±0.35 | 61.88±0.11 | 52.42±1.38 | 60.83±3.25 | 51.34±0.84 | 57.64±0.31 | 51.16±2.59 | 56.32±3.08 |
| DDGCL | 65.15±4.54 | 55.21±6.19 | 62.34±5.13 | 50.91±2.08 | 54.96±1.46 | 61.68±0.81 | 55.62±0.32 | 68.49±5.31 | 47.98±1.11 | 55.90±1.13 | 55.18±2.73 | 42.70±3.26 |
| CPDG | 43.56±6.41 | 65.92±6.25 | 50.32±5.06 | 49.89±1.34 | 52.86±0.64 | 59.72±2.53 | 53.82±1.50 | 49.71±2.64 | 47.37±2.23 | 56.40±1.17 | 53.58±2.10 | 40.01±3.59 |
| GraphPrompt | 73.78±5.62 | 60.89±6.37 | 64.60±5.76 | 51.28±2.43 | 55.67±0.26 | 67.46±0.31 | 51.07±0.75 | 86.78±3.14 | 48.46±0.28 | 59.18±0.49 | 50.27±0.58 | 87.45±2.57 |
| ProG | 60.86±7.43 | 68.60±5.64 | 63.18±4.79 | 51.46±2.38 | 92.28±0.21 | 93.32±0.06 | 58.73±1.58 | 86.24±2.87 | 89.75±0.28 | 90.69±0.08 | 56.42±1.95 | 85.43±3.16 |
| TGAT-TIGPrompt | 69.21±8.88 | 67.70±9.64 | 73.90±6.68 | 51.38±2.72 | 59.54±1.41 | 78.45±1.44 | 51.69±1.24 | 69.71±4.16 | 49.52±0.85 | 65.66±2.68 | 51.58±4.02 | 48.34±3.28 |
| TGN-TIGPrompt | 44.80±5.45 | 63.75±5.60 | 55.42±3.60 | 50.84±2.75 | 82.04±2.03 | 83.26±2.38 | 65.00±4.73 | 86.25±2.43 | 81.75±1.97 | 79.51±2.58 | 64.98±4.61 | 86.19±3.06 |
| TGAT-DyGPrompt | 82.09±6.43 | 73.50±6.47 | 77.78±5.08 | 52.03±2.24 | 69.88±0.18 | 90.76±0.09 | 53.92±0.97 | 72.04±4.71 | 52.58±0.23 | 75.20±0.17 | 53.29±0.87 | 50.82±3.67 |
| TGN-DyGPrompt | 74.47±3.44 | 74.00±3.10 | 69.06±3.89 | 51.97±2.16 | 94.33±0.12 | 96.82±0.06 | 70.17±0.75 | 87.02±1.63 | 92.22±0.19 | 95.69±0.08 | 69.77±0.66 | 87.63±1.97 |

Baselines.
The performance of DyGPrompt is evaluated against state-of-the-art approaches across four main categories.
(1) *Conventional DGNNs*: We first consider a discrete-time framework ROLAND (You et al., ) that adapts static GNNs to DGNNs by treating the node embeddings at each GNN layer as hierarchical node states and update them recurrently over time. We employ GCN (Kipf & Welling, ) and GAT (Veličković et al., ) for ROLAND. Additionally, we compare to continuous-time DGNNs, including TGAT (Rossi et al., ), TGN (Xu et al., ), TREND (Wen & Fang, ), and GraphMixer (Cong et al., ).
For a fair comparison, these conventional DGNNs are pre-trained on the first 80% events as well, and then continually trained on the same training data in each downstream task following our split.
(2) *Dynamic graph pre-training*: DDGCL (Tian et al., ) and CPDG (Bei et al., ) follow the “pre-train, fine-tune” paradigm. They first pre-train a DGNN, leveraging the intrinsic attributes dynamic patterns of graphs. Then, they fine-tune the pre-trained model for downstream tasks based on task-specific training data.
(3) *Static graph prompting*: GraphPrompt (Liu et al., ) and ProG (Sun et al., )
employ a static prompt to adapt pre-trained GNNs to downstream tasks. Specifically, we employ DGNNs as the pre-trained backbones—TGAT for GraphPrompt and TGN for ProG—as these combinations perform competitively with their respective prompting approaches.
(4) *Dynamic graph prompting*: TIGPrompt (Chen et al., ) proposes a prompt generator to generate time-aware prompts.
Note that both TIGPrompt and our method DyGPrompt can be applied to different backbones, such as TGAT and TGN.
More details of these baselines are described in Appendix A.4, with further implementation details for the baselines and DyGPrompt in Appendix A.5.

### 5.2 Performance Comparison with Baselines

We compare the performance of all methods on temporal node classification and link prediction in Table 1, and make the following observations.

First, DyGPrompt achieves outstanding performance across node classification and link prediction tasks.
This observation highlights the effectiveness of our proposed dual prompts and condition-nets. To further understand the impact of specific design choices in DyGPrompt and its robustness with various backbones, we defer a detailed investigation to the ablation studies in Sect. 5.3 and the backbone analysis in Sect. 5.4, respectively.
Second, TIGPrompt, the only other dynamic graph prompt learning method, significantly lags behind DyGPrompt when using the same backbone. While it employs time-aware prompts, it lacks fine-grained node-time characterization and is thus unable to capture complex node-time patterns where nodes and time mutually influence each other.
Third, GraphPrompt and ProG perform competitively because we employ DGNNs as their backbones in pre-training, rather than static GNNs.
However, they still perform worse than DyGPrompt as they only use static prompts.

Remark. It is worth noting that the results of TGAT and TGN in our experiments is lower than what was reported in their original papers. A possible reason is that, in their original setting, these methods were trained on data that immediately precede the test set chronologically. In contrast, in our experiments, to maintain consistency with the pre-training setup, we train them in two stages: first with pre-training, and then downstream training on limited task data where we only sample 30 events for each task. Moreover, training data that are temporally closer to the test set are more important than earlier data in pre-training. However, due to limited data in the downstream training stage, the models converge quickly and fail to effectively learn from these crucial recent time intervals, resulting in lower performance than their original setting. In contrast, our prompt-based approach can better handle the temporal gap and adapt to the test data well.

**Table 2: Ablation study reporting AUC-ROC (%), with TGAT as the backbone.**
| Methods | Node | Time | $\mathtt{NCN}$ | $\mathtt{TCN}$ | Node classification | Transductive Link Prediction | Inductive Link Prediction |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| prompt | prompt | Wikipedia | Reddit | MOOC | Wikipedia | Reddit | MOOC | Wikipedia | Reddit | MOOC |  |  |  |
| Variant 1 | $\times$ | $\times$ | $\times$ | $\times$ | 67.00 | 53.64 | 59.27 | 55.78 | 62.43 | 51.49 | 48.21 | 57.30 | 51.42 |
| Variant 2 | $\checkmark$ | $\times$ | $\times$ | $\times$ | 72.59 | 61.82 | 63.50 | 68.12 | 88.59 | 51.24 | 51.89 | 74.84 | 51.37 |
| Variant 3 | $\times$ | $\checkmark$ | $\times$ | $\times$ | 73.22 | 62.51 | 62.59 | 66.51 | 87.06 | 51.26 | 50.28 | 69.71 | 50.50 |
| Variant 4 | $\checkmark$ | $\checkmark$ | $\times$ | $\times$ | 72.25 | 63.11 | 62.87 | 68.36 | 90.31 | 52.17 | 52.56 | 75.13 | 51.33 |
| Variant 5 | $\checkmark$ | $\times$ | $\checkmark$ | $\times$ | 81.40 | 73.12 | 77.15 | 69.56 | 90.10 | 52.16 | 52.57 | 75.50 | 53.31 |
| Variant 6 | $\times$ | $\checkmark$ | $\times$ | $\checkmark$ | 80.34 | 72.59 | 76.16 | 66.62 | 87.34 | 51.22 | 49.05 | 73.16 | 52.34 |
| DyGPrompt | $\checkmark$ | $\checkmark$ | $\checkmark$ | $\checkmark$ | 82.09 | 73.50 | 77.78 | 69.88 | 90.76 | 53.92 | 52.58 | 75.20 | 53.29 |

**Table 3: AUC-ROC (%) evaluation of DyGPrompt with different DGNN backbones.**
| Pre-training | Downstream | Node classification | Transductive link prediction | Inductive link prediction |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Backbone | Adaptation | Wikipedia | Reddit | MOOC | Wikipedia | Reddit | MOOC | Wikipedia | Reddit | MOOC |
| DyRep | - | 50.61 | 49.54 | 50.33 | 58.45 | 58.02 | 50.29 | 56.87 | 57.25 | 50.34 |
| DyGPrompt | 53.52 | 50.98 | 51.62 | 91.64 | 93.84 | 72.04 | 90.28 | 93.08 | 72.27 |  |
| JODIE | - | 51.37 | 49.80 | 50.53 | 62.40 | 59.81 | 50.71 | 59.59 | 61.28 | 50.57 |
| DyGPrompt | 62.84 | 60.93 | 67.84 | 63.56 | 58.89 | 52.06 | 62.80 | 58.17 | 52.33 |  |
| TGAT | - | 67.00 | 53.64 | 59.27 | 55.78 | 62.43 | 51.49 | 48.21 | 57.30 | 51.42 |
| DyGPrompt | 82.09 | 73.50 | 77.78 | 69.88 | 90.76 | 53.92 | 52.58 | 75.20 | 53.29 |  |
| TGN | - | 50.61 | 49.54 | 50.33 | 72.48 | 67.37 | 54.60 | 74.38 | 69.81 | 54.62 |
| DyGPrompt | 74.47 | 74.00 | 69.06 | 94.33 | 96.82 | 70.17 | 92.22 | 95.69 | 69.77 |  |
| TREND | - | 69.92 | 64.85 | 66.79 | 63.24 | 80.42 | 58.70 | 50.15 | 65.13 | 57.52 |
| DyGPrompt | 70.15 | 65.24 | 67.58 | 64.35 | 79.62 | 59.45 | 51.26 | 64.88 | 59.13 |  |
| GraphMixer | - | 65.43 | 60.21 | 63.72 | 59.73 | 61.88 | 52.42 | 51.34 | 57.64 | 51.16 |
| DyGPrompt | 66.39 | 61.42 | 64.18 | 60.25 | 62.31 | 52.94 | 52.19 | 57.43 | 52.55 |  |

### 5.3 Ablation studies

To thoroughly understand the impact of each component within DyGPrompt, we compare DyGPrompt with its variants that employ different prompts and condition-nets. These variants and their corresponding results, using TGAT as the backbone, are presented in Table 2 on three datasets.

For node classification, both node and time prompts are beneficial, as Variant 2 (with node prompt) and Variant 3 (with time prompt) outperform Variant 1 (without these prompts). Incorporating both prompts in Variant 4 could further boost the performance.
Moreover, node condition-net is advantageous as it integrates node characteristics into time features, since Variant 5 outperforms Variant 2.
Similarly, time condition-net is beneficial for capturing temporal subtleties at different times to characterize node features, since Variants 6 outperforms Variant 3.
Lastly, DyGPrompt achieves the most optimal performance, demonstrating the effectiveness of dual prompts and condition-nets.

For link prediction, the results generally exhibit similar patterns as in node classification. A notable exception is the MOOC dataset, where the differences between the variants are generally smaller. A potential reason is that interactions in MOOC (online courses) are less dynamic and diverse than interactions in Reddit (a major social network) and Wikipedia (a high-traffic site), reducing the effectiveness of prompts designed to bridge the pre-training and downstream gaps. However, our full model DyGPrompt still achieves significantly better performance than Variant 1, which does not utilize any prompts. On the other hand, on Reddit and Wikipedia, even though both pre-training and downstream tasks involve link prediction, node prompts remain useful due to the potential distributional differences in the downstream training data across tasks.

To further visualize the effect of our prompt design, we sample 200 suspended nodes and 200 active nodes in Wikipedia, Reddit and MOOC, respectively, and show the output embedding space of these nodes calculated by Variant 1 (w/o prompts) and DyGPrompt in Fig. 4. We observe that with both dual prompts and dual-condition nets, node embeddings from different classes are clearly separated, demonstrating the advantages of DyGPrompt.

### 5.4 Impact of backbones and hyperparameters

First, to analyze the flexibility and robustness of DyGPrompt, we evaluate its performance on different DGNN backbones, including DyRep (Trivedi et al., ), JODIE (Kumar et al., ), TGAT, TGN, TREND, and GraphMixer.
The results on three datasets are reported in Table 3.
We observe that regardless of the backbone utilized, DyGPrompt surpasses the original backbone without our prompt design in almost all cases, indicating the robustness of our proposed framework.

Figure: Figure 3: Visualization of output embedding space of nodes.
Refer to caption: x3.png

Next, we investigate the sensitivity of hyperparameters in DyGPrompt. Particularly, we employ two MLPs with a bottleneck structure as the dual condition-nets (Wen & Fang, ; Wu & Lee, ). The perceptrons embed the input feature dimension $d$ to $\tilde{d}$, and then restore it back to $d$, where $\tilde{d}=d/\alpha$. Here, $\alpha$ is a hyperparameter that scales the hidden dimension. We evaluate the impact of $\alpha$, as illustrated in Fig. 4.
We observe that for both node classification and link prediction tasks, performance generally declines as $\alpha$ increases, likely due to greater information loss given a “smaller bottleneck.” The performance when $\alpha$ is around $2$ is generally competitive across different datasets. Therefore, in our experiments, we set $\alpha=2$.

## 6 Conclusions

In this paper, we explored prompt learning on dynamic graphs, aiming to narrow the gap between pre-training and downstream applications on such graphs. Our proposed approach DyGPrompt employs dual prompts to overcome the divergent objectives and temporal variations across pre-training and downstream tasks. Moreover, we propose dual condition-nets to mutually characterize node and time features, thus capturing evolving node-time patterns. Finally, we conducted extensive experiments on four public datasets, demonstrating that DyGPrompt significantly outperforms various state-of-the-art baselines.

## Appendix A Appendices

### A.1 Algorithm

We outline the key steps for downstream prompt tuning in Algorithm 1. In lines 6–8, we use dual prompts to modify node and time features. In lines 9-14, we perform dual conditional prompting. Specifically, we generate node conditioned time prompts and time conditioned node prompts (lines 9-11), and then modify node and time’s features using these conditional prompt (lines 12-14). In line 15-16, we calculate node embeddings based on the prompted node and time features. Finally, we update the embeddings for the prototypical nodes based on the few-shot labeled data provided in the task (lines 18–19) and optimize dual prompts and dual condition-nets (lines 20-22). Note that updating prototypical nodes is required exclusively for node classification tasks.

### A.2 Complexity Analysis

For a downstream dynamic graph $G=(V,E,T)$, the computational process of DyGPrompt involves two main parts: encoding nodes via a pre-trained TGN and prompt learning. The first part’s complexity is determined by the TGN’s architecture, similar to other methods using a pre-trained TGN. In a standard TGN, each node aggregates messages from up to $d$ neighbors per layer. Assuming the aggregation involves at most $d$ neighbors, the complexity of calculating node embeddings over $L$ layers per batch time is $O(d^{L}\cdot|V|)$. The second part, downstream prompt learning, has two stages: dual prompting and dual conditional prompting. For the dual prompting, each node and time feature is adjusted by the node prompt and time prompt, respectively, leading to a complexity of $O(|V|+1)$ per batch time. For the dual condition-nets with $K$ layers, for each timestamp, we first generate the node-based condition prompt with a complexity of $O(K)$, and the time-based condition prompts with a complexity of $O(K\cdot|V|)$. Then the condition prompts are used to modify nodes and time features, resulting in a complexity of $O(|V|+1)$. Therefore, the total complexity for prompt learning per batch time is $O((K+2)\cdot(|V|+1))$. In conclusion, the overall complexity of DyGPrompt is $O(d^{L}\cdot|V|+(K+2)\cdot(|V|+1))$. The first part dominates the overall complexity, as $O(d^{L}\cdot|V|)$ far exceeds $O((K+2)\cdot(|V|+1))$. Thus, the additional computational cost introduced by the conditional prompt tuning step is minimal.

### A.3 Further Descriptions of Datasets

In this section, we provide a summary of these datasets in Table 4 and
a further comprehensive descriptions of these datasets.

(1) Wikipedia(^2^22[http://snap.stanford.edu/jodie/wikipedia.csv](http://snap.stanford.edu/jodie/wikipedia.csv)) represents a month of modifications made by contributors on Wikipedia pages (Ferschke et al., ). Following prior studies (Rossi et al., ; Xu et al., ), we utilize data from the most frequently edited pages and active contributors, obtaining a temporal graph containing 9,227 nodes and 157,474 temporal directed edges. Dynamic labels indicate whether contributors temporarily banned from editing.

(2) Reddit(^3^33[http://snap.stanford.edu/jodie/reddit.csv](http://snap.stanford.edu/jodie/reddit.csv)) represents a dynamic network between posts and users on subreddits, where an edge represents a user writes a post to the subreddit, with about 11,000 nodes, about 700,000 temporal edges, and dynamic labels indicating whether user is banned from writing posts.

(3) MOOC(^4^44[http://snap.stanford.edu/jodie/mooc.csv](http://snap.stanford.edu/jodie/mooc.csv)) is a student-course dataset, representing the actions taken by student on MOOC platform. The nodes are users and courses, and edges represent the actions by users on the courses. Dynamic labels indicate whether the student drops-out after the action.

(4) Genre(^5^55[https://object-arbutus.cloud.computecanada.ca/tgb/tgbn-genre.zip](https://object-arbutus.cloud.computecanada.ca/tgb/tgbn-genre.zip)) is a dynamic network linking users to music genres, with edges represents a user listens to a specific genre at a certain time. The dataset includes 1,505 nodes and 17,858,395 temporal edges, and dynamic labels indicate each user’s most favored music genre.

Figure: Algorithm 1 Downstream Prompt Tuning for DyGPrompt

### A.4 Further Descriptions of Baselines

In this section, we provide additional details about the baselines used in our experiments.

(1) Conventional DGNNs

- •
ROLAND (You et al., ) : ROLAND adapts static GNNs for dynamic graph learning. It offers a new viewpoint for static GNNs, where the node embeddings at different GNN layers are viewed as hierarchical node states, thus capturing temporal evolution.
- •
TGAT (Rossi et al., ): TGAT use a self-attention mechanism and a time encoding technique based on the classical Bochner’s theorem from harmonic analysis. By stacking TGAT layers, the network recognizes the node embeddings as functions of time and is able to inductively infer embeddings for both new and observed nodes as the graph evolves.
- •
TGN (Xu et al., ): TGN incorporates a memory mechanism that updates node states based on new events, effectively capturing the historical context and dependencies over time. It further propose a novel training strategy that supports efficient parallel processing.
- •
Trend (Wen & Fang, ):Trend incorporates the Hawkes process into graph neural networks, and adopts event dynamics and node dynamics to capture the individual characteristics of each event and the collective characteristics of events on the same node, respectively.
- •
GraphMixer (Cong et al., ):GraphMixer employs a more simple MLP for feature learning, with a portion of the parameters dedicated to time encoding being fixed. This approach enhances the model’s ability to capture temporal dynamics while maintaining a simpler and more flexible architecture.

(3) Graph Pre-training Models

- •
DDGCL (Tian et al., ): DDGCL designs a self-supervised dynamic graph pre-training method via contrasting two nearby temporal views of the same node identity.
- •
CPDG (Bei et al., ): CPDG integrates two types of contrastive pre-training strategies to learn comprehensive node representations that encapsulate both long-term and short-term patterns.

**Table 4: Summary of datasets.**
| Dataset | Nodes | Edges | Node | Dynamic | Feature | Time |
| --- | --- | --- | --- | --- | --- | --- |
| num | num | classes | labels | dimension | span |  |
| Wikipedia | 9,227 | 157,474 | 2 | 217 | 172 | 30 days |
| Reddit | 11,000 | 672,447 | 2 | 366 | 172 | 30 days |
| MOOC | 7,144 | 411,749 | 2 | 4,066 | 172 | 30 days |
| Genre | 1505 | 17,858,395 | 474 | 984 | 86 | 1500 days |

(4) Graph Prompt Models

- •
GraphPrompt (Liu et al., ): GraphPrompt employs subgraph similarity as a mechanism to unify different pretext and downstream tasks, including link prediction, node classification, and graph classification. A learnable prompt is subsequently tuned for each downstream task.
- •
ProG (Sun et al., ):ProG reformulate the node- and edge-level tasks as graph-level tasks, and proposes prompt graphs with specific nodes and structures to guide different task.
- •
TIGPrompt (Chen et al., ): TIGPrompt propose a prompt generator that generates personalized, time-aware prompts for each node, enhancing the adaptability and expressiveness of node embeddings for downstream tasks.

### A.5 Implementation Details

General settings
Optimizer: For all experiments, we use the Adam optimizer.

Environment: The environment in which we run experiments is:

- •
Operating system: Windows 11
- •
CPU information: 13th Gen Intel(R) Core(TM) i5-13600KF
- •
GPU information: GeForce RTX 4070Ti (12 GB)

Details of baselines.
For all open-source baselines, we utilize the officially provided code and reproduce the non-open-source CPDG and TIGPrompt. Each model is tuned according to the settings recommended in their respective literature to achieve optimal performance.

For GCN in Roland, we employ a 2-layer architecture, and set the hidden dimensions to 172.
For GAT Roland, we employ a 2-layer architecture and set the hidden dimension to 172.
For TGAT, and TGN, we set the hidden dimension to 172, and sample 20 temporal neighbors for each node to update representation.
For Trend,after sampling neighboring nodes, we apply Hawkes processing to the temporal neighbors of nodes using different time decay factors, based on the time of event occurrence.
For GraphMixer, we employ a MLP to process the input nodes along with their positive and negative examples. The output is then fed into a network structure composed entirely of linear layers for prediction during training.

For DDGCL, we compares two temporally adjacent views of each node, utilizing a time-dependent similarity evaluation and a GAN-style contrastive loss function.
For CPDG, we employ depth-first-search and breadth-first-search to sample neighbors of each nodes. Both DDGCL and CPDG employ TGAT as the backbone, setting the hidden dimension to 172.

For GraphPrompt, we use TGAT as the backbone, and set the hidden dimensions to 172. We employ 1-hop subgraph for similarity calculation.
For ProG,We use TGN as the backbone, and set the hidden dimensions to 172.
For TIGPrompt, we employ TGN and TGAT as the backbone, setting the hidden dimension to 172. We employ projection prompt generator, as it is reported to yield the best performance in their literature.

Details of DyGPrompt.
For our proposed DyGPrompt, We conducted experiments using TGN and TGAT as backbones. We employ a dual-layer perceptrons with bottleneck structure as the condition-net, and set the hidden dimension of the condition net as 86.

## Peer Reviews

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: Q1. In previous work, prompts are often implemented as vector concatenations. What are the author’s considerations for using dot-product instead?

Q2. Personally, I am curious whether a pre-trained time encoder can address questions such as “What is the time of the next interaction?”

Q3. How does DYGPROMPT perform on large-scale dynamic graphs?
soundness: 3
strengths: S1. I like the idea of using dual prompt mechanism to bridge task and temporal gaps.

S2. Experiments show DYGPROMPT is effective on small real-world graphs.
summary: This paper studies the problem of adapting pre-trained dynamic GNNs. The authors propose DYGPROMPT, a dual prompt mechanism with node and time prompts, supported by dual condition-nets, to bridge the gaps between pre-training and downstream objectives as well as temporal inconsistencies, achieving improved adaptability and performance across dynamic graph tasks.
weaknesses: W1. The introduction of dual prompts and dual condition networks enhances adaptability but significantly increases model complexity. I would suggest the authors to add some analysis or experiments to demonstrate DYGPROMPT’s performance in terms of computational efficiency.

W2. I have concerns about the experimental setup. Using the first 80% of interactions for pre-training is uncommon in pre-trained frameworks. For example, TIGPROMPT uses only 20% of the data, yielding surprisingly strong results in comparison.

W3. The discussion of current dynamic graph prompt learning methods, such as TIGPROMPT, is brief and lacks an in-depth analysis of their limitations or how DYGPROMPT specifically addresses these issues. Additionally, Section 4 contains extensive information on previous works, making it difficult to discern the paper’s core contributions.

---

confidence: 4
contribution: 2
initial_rating: 3
presentation: 2
questions: See above.
soundness: 2
strengths: - **Relevance to Real-World Applications:** The framework's focus on dynamic graphs makes it highly relevant to real-world applications such as social networks, online learning platforms, and crowdsourcing projects.

- **Parameter Efficiency:** DYGPROMPT's ability to perform well with minimal parameter updates is a significant strength, especially for applications where labeled data is scarce.

- **Open source:** It is commendable that the authors have made their code publicly available, enhancing the transparency and reproducibility of their work.
summary: The paper introduces DYGPROMPT, a novel pretraining and prompt learning framework designed for dynamic graph learning. It addresses the challenges of bridging the gap between pre-training objectives and downstream tasks. The authors propose dual prompts -- node prompts and time prompts -- to address the discrepancies in both task objectives and temporal variations between pre-training and downstream tasks. The authors introduce dual condition-nets to model the evolving node-time patterns in downstream tasks. This includes a time condition-net that generates time-conditioned node prompts and a node condition-net that generates node-conditioned time prompts. Experiments on four public datasets demonstrate its superior performance compared to state-of-the-art approaches in tasks such as temporal node classification and temporal link prediction.
weaknesses: - **Motivation:** The motivation for this work is insufficiently justified. After reading the paper, it remains unclear why temporal patterns would be influenced by node features. Time, as a continuous scale, is inherently objective and independent of specific events.

- **Novelty:** Based on the above, the novelty of this work appears limited. TIGPrompt has already proposed an effective strategy for dynamic graph learning by fusing prompts through the concatenation of node and time embeddings for prompt-based fine-tuning. This contrasts with the authors’ claim that TIGPrompt only considers temporal factors within node features. Thus, the main difference between this work and TIGPrompt seems to lie in the adoption of a more complex information fusion strategy rather than a simple concatenation.

- **Experiments:** The experimental setup and results raise some concerns. First, the reported results are noticeably lower than those of TIGPrompt. Given the relevance of TIGPrompt as a baseline, I recommend that the authors adopt identical experimental settings to ensure comparability. Additionally, the influence of model size on performance is unclear; I suggest that the authors report the number of parameters for both their model and TIGPrompt.

- **Writing:** The writing could be further polished to improve clarity and readability.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 4
questions: The study considers only two tasks on dynamic graphs. Is it possible to apply this framework to other graph tasks?
soundness: 3
strengths: 1.	This paper identifies the issue of pretrained Graph Neural Networks (GNNs)—specifically, the gap between pre-training and task objectives—and proposes novel prompt learning approaches for dynamic graphs. It also considers the evolving interplay patterns between nodes and time points.
2.	The framework addresses temporal variations across time and divergent task objectives by leveraging a node prompt and a time prompt to reduce the gap between the pre-training and downstream phases. The main novelty lies in the introduction of the time prompt, which narrows the inconsistencies that arise from varying priorities at different times. The proposed method is reasonable.
3.	Extensive experiments are conducted to evaluate the performance of various methods. The source codes are available anonymously, which adds to the credibility of the results.
4.	The manuscript is well written and easy to follow.
summary: To address the challenges of prompt-based learning on dynamic graphs, this paper develops a novel pretraining and prompt learning framework for dynamic graph modeling. The authors propose dual prompts and dual condition networks to model the evolving node-time patterns. Extensive experiments are conducted to evaluate the performance of the proposed node-time solution. Overall, the approach is sound and reasonable.
weaknesses: 1.	The concept of temporal prompts for dynamic graphs has been previously investigated (e.g., TIGPrompt), but the differences between approaches are not clearly articulated. In the related work, it is stated that “it only considers the temporal factor in node features, overlooking that temporal patterns are also influences by node features.” More explanations are needed to highlight the technical novelty of this paper.
2.	The overall framework presented in Figure 2 could be improved; it is not clear how to compute the downstream loss (Eq. 11) and why it only links to the time branch.
3.	All evaluated datasets are relatively small, and the paper lacks an efficiency or complexity analysis for larger-scale datasets, which is important for practical applications.
4.	There are minor typos and errors, such as “are also influences by node.”

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: Q1: Can the authors comment on the scalability of DYGPROMPT to large-scale dynamic graphs, both in terms of computational resources and performance?

Q2: How does DYGPROMPT adapt to long-term temporal changes in dynamic graphs, especially when the graph's structure and node behaviors evolve significantly over time?
soundness: 3
strengths: S1. The paper presents a creative solution to the challenge of adapting pre-trained DGNNs to downstream tasks, which is a novel contribution to dynamic graph learning.

S2. The paper is well-written and easy to follow.

S3. Extensive experiments compared with DGNNs, SOTA pre-training models, and graph prompt learning models demonstrate the effectiveness of the proposed model.
summary: The paper introduces DYGPROMPT, a novel framework for dynamic graph modeling that aims to bridge the gap between pre-training and downstream tasks, such as node classification and link prediction, on dynamic graphs. The authors propose a two-stage approach involving pre-training and prompt learning. In the pre-training phase, a dynamic graph neural network is trained on the task of temporal link prediction. For the downstream tasks, the authors design dual prompts (node and time prompts) to address the differences in objectives and temporal variations between the pre-training and downstream phases. Additionally, they introduce dual condition nets to generate a series of time-conditioned node prompts and node-conditioned time prompts, capturing the evolving node-time patterns. The framework is evaluated on four public datasets, demonstrating superior performance compared to state-of-the-art methods.
weaknesses: W1.  While the paper mentions the computational complexity, a deeper analysis comparing the complexity of DYGPROMPT with other methods, especially in terms of training and inference time, is necessary for a prompt-based model.

W2. The used datasets are very small-scale. The largest dataset used only contains about 10K nodes. This makes it difficult to confirm whether the model can be successfully applied to large-scale data and the real world.

W3. The experimental datasets have extreme class numbers. Specifically, 3 of the datasets are binary classification and another one contains 474 classes. It would be better to experiment on other more datasets.

Meta-review: The paper proposes DYGPROMPT, a dual prompt mechanism with node and time prompts for pre-training dynamic GNNs. The problem is interesting and popular. The proposed model is reasonable. The experiment results show that the proposed method outperforms baseline methods. The paper writing is good. Several issues were raised by reviewers to improve the paper, such as a lack of discussion of related works (e.g., TIGPrompt), relatively small datasets, and complexity analysis. Reviewers are relatively positive about this work.

Decision: Accept (Poster)