Title: Improving Temporal Link Prediction via Temporal Walk Matrix Projection
Venue: NeurIPS 2024
Accepted: yes
Keywords: Temporal Link Prediction

Dynamic Graph Learning

Graph Neural Network
Abstract: Temporal link prediction, aiming at predicting future interactions among entities based on historical interactions, is crucial for a series of real-world applications. Although previous methods have demonstrated the importance of relative encodings for effective temporal link prediction, computational efficiency remains a major concern in constructing these encodings. Moreover, existing relative encodings are usually constructed based on structural connectivity, where temporal information is seldom considered. To address the aforementioned issues, we first analyze existing relative encodings and unify them as a function of temporal walk matrices. This unification establishes a connection between relative encodings and temporal walk matrices, providing a more principled way for analyzing and designing relative encodings. Based on this analysis, we propose a new temporal graph neural network called TPNet, which introduces a temporal walk matrix that incorporates the time decay effect to simultaneously consider both temporal and structural information. Moreover, TPNet designs a random feature propagation mechanism with theoretical guarantees to implicitly maintain the temporal walk matrices, which improves the computation and storage efficiency. Experimental results on 13 benchmark datasets verify the effectiveness and efficiency of TPNet, where TPNet outperforms other baselines on most datasets and achieves a maximum speedup of $33.3 \times$ compared to the SOTA baseline.

Improving Temporal Link Prediction via Temporal Walk Matrix Projection

Abstract

Temporal link prediction, aiming at predicting future interactions among entities based on historical interactions, is crucial for a series of real-world applications. Although previous methods have demonstrated the importance of relative encodings for effective temporal link prediction, computational efficiency remains a major concern in constructing these encodings. Moreover, existing relative encodings are usually constructed based on structural connectivity, where temporal information is seldom considered. To address the aforementioned issues, we first analyze existing relative encodings and unify them as a function of temporal walk matrices. This unification establishes a connection between relative encodings and temporal walk matrices, providing a more principled way for analyzing and designing relative encodings. Based on this analysis, we propose a new temporal graph neural network called TPNet, which introduces a temporal walk matrix that incorporates the time decay effect to simultaneously consider both temporal and structural information. Moreover, TPNet designs a random feature propagation mechanism with theoretical guarantees to implicitly maintain the temporal walk matrices, which improves the computation and storage efficiency. Experimental results on 13 benchmark datasets verify the effectiveness and efficiency of TPNet, where TPNet outperforms other baselines on most datasets and achieves a maximum speedup of 33 . 3 × compared to the SOTA baseline. Our code can be found at https://github.com/lxd99/TPNet .

1 Intorduction

Many real-world dynamic systems can be abstracted as a temporal graph [1], where entities and interactions among them are denoted as nodes and edges with timestamps respectively. Temporal link prediction, aiming at predicting future interactions based on historical interactions, is a fundamental task for temporal graph learning, which can not only help us understand the evolution pattern of the temporal graph but also is crucial for a series of real-world tasks such as recommendations for online platforms [2, 3] and information diffusion prediction [4, 5].

Relative encodings have become an indispensable module for effective temporal link prediction [6-9] where, without them, node representations computed independently by neighbor aggregation will fail

to capture the pairwise information. As the toy example shown in Figure 1, A and F will have the same node representation due to sharing the same local structure. Thus it can not be determined whether D will interact with A or F at t 3 according to their representations. However, by assigning nodes with relative encodings (i.e., additional node features) specific to the target link before computing the node representation, we can highlight the importance of each node and guide the representation learning process to extract pairwise information. For example, in Figure 1, we can infer from the relative encoding of E (in red circle) that D is more likely to interact with F than with A since D and F share a common neighbor, E (detailed discussed in Section 2.2). Although achieving remarkable success, injecting pairwise information based on relative encodings is still far from satisfactory.

1) On Concept , existing ways of constructing relative encodings are fragmented as they are derived from different heuristics. There still lacks a unified view on the connections between different relative encodings, which may allow a more principled way to design new relative encodings. 2) On Method Design , most existing relative encodings are constructed based on structural connectivity between nodes (e.g., whether two nodes are neighbors), while the temporal information is ignored. 3) On Computation , existing methods for relative encodings are inefficient, which usually involve time-consuming graph query operations and need to re-extract the relative encoding from scratch for each target link, making them even become the main computational bottleneck for some methods (shown in Section 4.2).

Link

𝐴

𝐵

𝐶

𝐷

𝐸

𝐹

(𝐷, 𝐹, 𝑡

3 )

[0,0]

[0,0]

[0,0]

[0,0]

[1,1]

[0,0]

(𝐷, 𝐴, 𝑡

3 )

[0,0]

[0,0]

[0,1]

[0,0]

[1,0]

[0,0]

Relative encoding based on adjacency

=

t

0

t

2

t

1

Representation of

𝐹

t

0

t

2

t

1

Representation of

𝐴

𝐴

and

𝐹

have the same node representations

t

2

t

1

t

0

t

1

t

0

𝐵

𝐴

𝐹

𝐸

𝐷

𝐶

Timestamps:

𝑡

0

< 𝑡1

< 𝑡2

< 𝑡3

𝑡

3

?

𝑡

3

?

𝐴

𝐶

𝐸

𝐵

𝐹

𝐸

𝐶

𝐷

To tackle the above issues, this paper makes the following three technical contributions. 1) A Unified View (Concept) . We analyze the construction of existing relative encodings and find that they can be uniformly viewed as a function of temporal random walk matrices, where different relative encodings essentially correspond to the construction of a series of temporal walk matrices based on temporal walk weighting. The presented function provides a unified view to analyze existing methods and may allow a more principal way to design new relative encodings. 2) A Effective and Efficient Method (Method Design and Computation) . Based on our analysis, we propose a Time decay matrix Projection-based graph neural Network for temporal link prediction, named TPNet for short. TPNet introduces a new temporal walk matrix that incorporates the time decay effect of the temporal graph, simultaneously considering both temporal and structural information. Moreover, TPNet encodes the temporal walk matrix into a series of node representations by random feature propagation, which can be efficiently updated when the graph structure changes and is storage-efficient. Importantly, such node representations can be shared by different link likelihood computation processes to decode the pairwise information, reducing the redundancy computation of re-extracting the relative encodings and avoiding time-consuming graph query operations. 3) Theoretical and Empirical Analysis. We conduct a theoretical analysis of the node representations obtained through random feature propagation. Our analysis demonstrates that these representations stem from the random projection of the proposed temporal walk matrix while preserving the inner product of different rows of the matrix. Moreover, we discuss the conditions under which random feature propagation can be applied to improve the computational and storage efficiency of other temporal walk matrices and provide the corresponding propagation mechanisms for temporal walk matrices of existing methods. Empirically, we conduct experiments on 13 benchmark datasets to verify the effectiveness and efficiency of the proposed method, where TPNet outperforms other baselines on most datasets and achieves a maximum speedup of 33 . 3 × compared to the SOTA baseline. Detailed ablation studies also verify the effectiveness of the proposed submodules.

2 Preliminary

In this section, we will first formally define some important concepts in this paper and then give a unified formulation of existing relative encodings.

2.1 Definitons

Definition 1 (Temporal Graph) . Atemporal graph can be considered as a sequence of non-decreasing chronological interactions G = [( { u 1 , v 1 } , t 1 ) , ( { u 2 , v 2 } , t 2 ) , · · · ] with 0 ≤ t 1 ≤ t 2 ≤ · · · , where ( { u i , v i } , t i ) can be considered as a undirected link between u i and v i with timestamp t i . Each node u can be associated with node feature x u ∈ R d N , and each interaction ( { u, v } , t ) has link feature e t u,v ∈ R d E , where d N and d E denote the dimensions of the node feature and link feature.

Definition 2 (Temporal Link Prediction) . The interaction sequence reflects the graph dynamics, and thus the ability of a model to capture the evolution pattern of a dynamic graph can be evaluated by how accurately it predicts the future interactions based on historical interactions. Formally, given the interactions before t (i.e., { ( { u ′ , v ′ } , t ′ ) | t ′ < t } ) and two nodes u , v , the temporal link prediction task aims to predict whether there will be an interaction between u and v at t .

Definition 3 (K-hop Subgraph) . We use the notation G ( t ) = ( V ( t ) , E ( t )) to denote the graph snapshot at t , where E ( t ) includes all the interactions that happen before t and V ( t ) includes all the nodes appear in E ( t ) . Besides, we defined the k-hop subgraph of node u as G k u ( t ) = ( V k u ( t ) , E k u ( t ) , where V k u ( t ) ⊂ N ( t ) is the set of nodes whose shortest path distance to u is less than k on G ( t ) and E k u ( t ) ⊂ E ( t ) is the set of interactions between V k u ( t ) .

Definition 4 (Temporal Walk) . A k-step temporal walk W on G ( t ) is a sequence of node-time pair with decreased temporal order [6], which can be denoted as W = [( w 0 , t 0 ) , · · · , ( w k , t k )] with t = t 0 > t 1 > · · · > t k and ( { w i , w i +1 } , t i +1 ) is an edge on G ( t ) for 0 ≤ i ≤ k -1 . Figure 2 shows a visual illustration of a temporal walk. Here, we stipulate the first timestamp t 0 as the current time t to avoid undefinedness of t 0 . We use the notation M k u,v ( t ) to denote the set of all k-step temporal walks from u to v on G ( t ) . Specially, M 0 u,w ( t ) = { [( u, t )] } if u = w and M 0 u,w ( t ) = ∅ otherwise. When there is no ambiguity, we replace M k u,v ( t ) with M k u,v .

2.2 Relative Encoding for Dynamic Link Prediction

2.2.1 Unified Framework

Given a future link ( u, v, t ) to be predicted, existing temporal link prediction methods (referred as node-wise methods) usually first learn the node representations h u ( t ) and h v ( t ) independently, which are obtained by encoding their k-hop subgraphs,

where X N u,k and X E u,k are the features of nodes and edges in G k u ( t ) respectively 2 . The f enc ( · ) here can be any encoding function that maps a graph into a representation such as a k-layer GNN with a pooling layer Then the link likelihood p t u,v is given p t u,v = f dec ( h u ( t ) , h v ( t )) . The f dec ( · ) is a decoding function that maps the node representations into link likelihood such as an MLP with a Sigmoid output layer. Detailed discussion about existing methods can be found in Appendix F.2.

As mentioned in the Section 1, learning representations independently might fail to capture the pairwise information of the given link. Thus recent methods (referred as link-wise methods) inject the pairwise information by constructing a relative encoding r w | ( u,v ) for each node w ∈ V k u ( t ) ∪ V k v ( t ) as an additional node feature (Detailed construction way for different methods will be introduced in Section 2.2.2). Then Equation (1) will be changed into

where X R u,k is the relative encodings for nodes in G k u ( t ) and X N u,k ⊕ X R u,k indicate the new node features obtained by concatenating X N u,k and X R u,k . Intuitively, the relative encoding r w | ( u,v ) for each node w reflects its importance to predicted link ( u, v, t ) , which can guide the representation learning process to extract the pairwise information specific to the predicted link from the subgraph. For the detailed construction way, the relative encoding r w | ( u,v ) is the concatenation of two similarity features r w | u and r w | v , where r w | u / r w | v encodes some form of similarity between u/v and w (e.g., the number of k-step temporal walks from u to w ). Although the designs of similarity feature r w | u

for different methods are induced from different heuristics, we find that they can be unified in a function about the temporal walk matrix, which is

The s ( · ) here is a score function that maps a temporal walk to a scalar and A ( i ) ( t ) denotes an i-hop temporal walk matrix whose each entry A ( i ) u,w is the sum of the score of all i-step temporal walks from u to w . g ( · ) is a function applied on the vector of [ A (0) u,w ( t ) , A (1) u,w ( t ) , · · · , A ( k ) u,w ( t )] ∈ R k +1 to extract similarity feature. The above equation shows that each relative encoding implies a construction of the temporal walk matrix based on weighting each temporal walk (i.e., s ( · ) ). Next, we will analyze existing relative encodings and show how they can be represented in the form of Equation (3).

2.2.2 Analysis of Existing Methods

Our analysis focuses on the four representative link-wise methods DyGFormer, PINT, NAT, and CAW, which cover all the link-wise methods in the benchmark of dynamic link prediction [9].

DyGFormer [9]. The r w | u of DyGFormer is a one-dimensional vector representing the number of links between w and u . For DyGFormer, we can first set the s ( · ) to be a one-const function (i.e., s ( W ) = 1 in for any W ), which will make A ( k ) u,w be the number of the k-step temporal walks from u to w . Then, setting g ( · ) to be a function that selects the second number of a vector (i.e., g ([ x 0 , x 1 , .., x k ]) = x 1 ) will make Equation (3) generate the similarity feature of DyGFormer.

PINT [8]. The r w | u of PINT is a ( k +1 )-dimensional vector, where r w | u i is the number of ( i -1) -step temporal walks from u to w for 1 ≤ i ≤ k +1 . Setting s ( · ) and g ( · ) can be set to a one-const function and an identity function respectively will make Equation (3) outputs the similarity feature of PINT.

NAT [7]. NAT maintains a series of fix-sized hash maps s (0) u , ..., s ( k ) u and generates the r w | u based on the hash maps. As proved in Appendix A.1, if the size of the hash maps becomes infinite, the r w | u is equivalent to a ( k +1) -dimensional vector, where, for 1 ≤ i ≤ k +1 , r w | u i = 1 if the shortest temporal walk from u to w is less than i ; otherwise, r w | u i = 0 . Let h ( · ) be a binary function where h ( x ) = 1 if x > 0 and h ( x ) = 0 otherwise. Setting the s ( · ) to be a one-const function and g ( · ) to a function of g ([ x 0 , x 1 , ..., x k ]) = [ h ( ∑ 0 i =0 x i ) , h ( ∑ 1 i =0 x i ) , ..., h ( ∑ k i =0 x i )] can make the Equation (3) generate the similarity feature of NAT.

CAWN [6]. The similarity feature r w | u of CAWN reflects the probability of a node w being visited when performing a temporal walk from u . Specifically, CAWN first samples a set of temporal walks beginning from u based on a causal sampling strategy. Then, for each node w , the similarity feature r w | u is extracted based on its occurrence in the sampled walks. As proved in Appendix A.2, the similarity feature r w | u is the estimation of a ( k +1 )-dimensional vector, where, for 1 ≤ i ≤ k +1 , r w | u i is the probability of visiting w through a ( i -1) -step temporal walk matrix based on the sampling strategy of CAWN, which can be represented as r w | u i = ∑ W ∈M i -1 u,w s ′ ( W ) . The value s ′ ( W ) for a given temporal walk W = [( w 0 , t 0 ) , ( w 1 , t 1 ) , ..., ( w k , t k )] is defined as ∏ k -1 i =0 exp ( -α ( t i -t i +1 )) ∑ ( { w ′ ,w } ,t ′ ) ∈E wi ,t i exp ( -α ( t i -t ′ )) , where α is hyperparameter to control the sampling process, E w i ,t i = { ( { w ′ , w } , t ′ ) | t ′ < t i } is the set of interactions attached to w i before t i , and exp ( -α ( t i -t i +1 )) ∑ ( { w ′ ,w } ,t ′ ) ∈E wi ,t i can be considered as the probability of moving from ( w i , t i ) to ( w i +1 , t i +1 ) in the sampling process. Setting s ( · ) to s ′ ( · ) and g ( · ) to be an identity function can make Equation (3) generate the similarity feature of CAWN.

Conclusion . According to the above analysis, we can conclude that (i) Equation (3) provides a unified view to consider the injection of pairwise information from the construction of temporal walk matrix, where different relative encodings (implicitly or explicitly) correspond to a kind of temporal walk matrix. (ii) Examining existing relative encodings from the unified view reveals their limitations. First, the relative encodings of existing methods (except CAWN) ignore the temporal information carried by each temporal walk, where their score function s ( · ) always yield 1 and the entries of the temporal walk matrix is just the count of the temporal walks. Next, although CAWN considers temporal information, they estimate the temporal walk matrix from the sampled temporal

Algorithm 1: Node Representation Maintaining ( G , λ , k , n , d R )

1

Initialize

H

(0)

,

H

(1)

, ...,

H

(

k

)

∈

R

n

×

d

R

as zero matrix ;

2

Fill

H

(0)

with entries independently drawn from

N

(0

,

1

d

R

)

;

3

for

(

u, v, t

)

∈ G

do

4

for

l

=

k

to

1

do

5

H

(

l

)

u

=

H

(

l

)

u

+

e

λt

∗

H

(

l

-

1)

v

;

6

H

(

l

)

v

=

H

(

l

)

v

+

e

λt

∗

H

(

l

-

1)

u

;

7

Return

H

(0)

,

H

(1)

, ..,

H

(

k

)

;

(a) Temporal Graph

𝒢(𝑡)

𝐴

𝑡

3

𝑡

4

𝑡

2

𝑡

1

𝑡

1

𝑡

2

[(𝐵, 𝑡),

𝐶, 𝑡

4

,

𝐴, 𝑡

3

,

𝐷, 𝑡

2

, (𝐹, 𝑡

1

)]

𝐵

𝐶

𝐷

𝐸

𝐹

Timestamps:

𝑡

1

< 𝑡2

< 𝑡3

< 𝑡𝑡

< 𝑡4

𝑡

3

𝑡

2

𝑡

1

𝑡

4

𝐶

𝐴

𝐷

𝐹

𝐵

(b) A temporal walk on

𝒢(𝑡)

walks, which needs time-consuming graph sampling and may introduce large estimation errors. In the next section, we will present a new temporal walk matrix to simultaneously consider the temporal and structural information and show how to efficiently maintain the proposed temporal walk matrix.

3 Methodology

TPNet mainly consists of two modules: Node Representation Maintaining (NRM) and Link Likelihood Computing (LLC). The NRM is responsible for encoding the pairwise information, which maintains a series of node representations. Such representations will be updated when new interaction occurs and can be used to decode the temporal walk information between two nodes. The LLC module is a prediction module, which utilizes the maintained node representations and auxiliary information (e.g., like features) to compute the likelihood of the link to be predicted.

3.1 Node Representation Maintaining

Temporal Matrix Construction . Based on the Equation (3), designing a temporal walk matrix relies on the definition of the score function s ( · ) , where the element of a temporal walk matrix is A ( k ) u,v ( t ) = ∑ W ∈ M k u,v s ( W ) . Unlike most previous methods that only count the number of temporal walks, we consider the temporal information carried by a temporal walk in s ( · ) to simultaneously consider the temporal and structural information. Formally, let t be the current time, given a temporal walk W = [( w 0 , t 0 ) , ( w 1 , t 1 ) , .., ( w k , t k )] , the value of the score function is s ( W ) = ∏ k i =1 e -λ ( t -t i ) , where λ > 0 is a hyperparameter to control the time decay weight. As the current time t goes on, for each interaction ( { w i , w i +1 } , t i +1 ) in the temporal walk W , its weight e -λ ( t -t i ) will decay exponentially. The design of the score function is motivated by the widely observed time decay effect [1, 10] on the temporal graph, where the importance of interactions will decay as time goes on, benefiting better modeling the graph evolution patterns. In the following part of Section 3, the notation of s ( · ) and A ( k ) ( t ) will specifically refer to the score function and temporal walk matrix proposed in this part. Besides, for A (0) ( t ) , we stipulate it as an identity matrix constantly.

Node Representation Maintaining . Directly computing the temporal walk matrices is impractical since we need to enumerate the temporal walks and each matrix needs expensive O ( n × n ) space complexity to store. Thus, we implicitly maintain the temporal walk matrices by maintaining a series of node representations H (0) ( t ) , H (1) ( t ) , ..., H ( k ) ( t ) ∈ R n × d R , where d R ≪ n is the dimension of node representations and H ( l ) u ( t ) ∈ R d R encodes the information about the l -step temporal walks beginning from u for 0 ≤ l ≤ k . The node representations will be updated when a new interaction occurs. Specifically, we first construct a random feature matrix P ∈ R n × d R , where each entry of P is independently drawn from Gaussian distribution with mean 0 and variance 1 d R . Then we initialize H (0) ( t ) as P and H (1) ( t ) , H (2) ( t ) , ..., H ( k ) ( t ) as zero matrix. When a new interaction ( u, v, t ) occurs, for 1 ≤ l ≤ k , we update the representations of u and v by

where the t + denotes the time right after t . A pseudocode for maintaining the node representations is shown in Algorithm 1. The maintaining mechanism here can be considered as a random feature propagation mechanism on the temporal graph, where we initialize the zero layer's representation

of each node as a random feature and repeatedly propagate these features among nodes from the low layer to the high layer as interactions continuously appear. The following theorem shows the relationship between the node representations and the temporal walk matrices.

Theorem 1. If any two interactions on temporal graph G have different timestamps, the obtained representations H (0) ( t ) , H (1) ( t ) , ..., H ( k ) ( t ) satisfy e -λlt ∗ H ( l ) ( t ) = A ( l ) ( t ) P for 0 ≤ l ≤ k .

For simplicity, we assume the timestamps of the interaction are different, and we show how to update the representations when multiple interactions have the same timestamps in Appendix C. We leave the proof in the Appendix B.1. The above theorem shows that the obtained node representation is the projection (i.e. linear transformation) of the temporal walk matrices, where the transform matrix is the initial random feature matrix P . The next theorem shows that the node representations preserve the inner product of the temporal walk matrices.

Theorem 2. Given any ϵ ∈ (0 , 1) , let ∥ · ∥ 2 denote the L2 norm, c l 1 ,l 2 u,v denote 1 2 ( ∥ A ( l 1 ) u ( t ) ∥ 2 2 + ∥ A ( l 2 ) v ( t ) ∥ 2 2 ) , and ¯ H ( l ) ( t ) denote e -λlt ∗ H ( l ) ( t ) , if dimension d R of node representations satisfy d R ≥ 24 ϵ 2 log(4 1 / 3 ( k +1) n ) , then for any 1 ≤ u, v ≤ n , and 0 ≤ l 1 , l 2 ≤ k , we have

where ⟨· , ·⟩ denotes the inner product and | · | denotes taking absolute value.

We leave the proof in Appendix B.2. The above theorem shows that the inner product of the node representations is approximately equal to the inner product of the temporal walk matrices (i.e., ⟨ ¯ H ( l 1 ) u ( t ) , ¯ H ( l 2 ) v ( t ) ⟩ ≈ ⟨ A ( l 1 ) u ( t )) , A ( l 2 ) v ( t ) ⟩ ). Specifically, given any error rate ϵ , if the dimension of the node representations satisfies a certain condition, the difference between ⟨ ¯ H ( l 1 ) u ( t ) , ¯ H ( l 2 ) v ( t ) ⟩ and ⟨ A ( l 1 ) u ( t )) , A ( l 2 ) v ( t ) ⟩ for any u, v, l 1 , l 2 will be less than ϵc l 1 ,l 2 u,v with high probability ( ≥ 1 -1 ( k +1) n ). The error rate can approach 0 infinitely and thus the inner product of the node representations can approach that of temporal walk matrices infinitely, at the cost of increasing the dimension d R . As we will see in Section 4.3, a small dimension ( ≪ n ) in practice is enabled to make the inner product of node representations a good estimation of that of temporal walk matrices. Additionally, due to the i-th row of A (0) ( t ) being the one-hot encoding of i (since it is an identity matrix), we have ⟨ A ( l ) u ( t ) , A (0) w ( t ) ⟩ = A ( l ) u,w ( t ) . Thus, we can obtain [ A (0) u,w ( t ) , · · · , A ( k ) u,w ( t )] in Equation (3) by calculating the inner product between all layers of u 's representation and the zero layer of w 's representation, expressed as [ ⟨ ¯ H (0) u ( t ) , ¯ H (0) w ( t ) ⟩ , · · · , ⟨ ¯ H ( k ) u ( t ) , ¯ H (0) w ( t ) ⟩ ] .

Remark. Compared to directly computing the temporal walk matrices A (0) ( t ) , .., A ( k ) ( t ) , which needs to enumerate the temporal walks and O (( k +1) n 2 ) space complexity to store, maintaining the node representations largely improve the computation and storage efficiency, which only needs O (( k + 1) nd R ) space complexity to store and O ( kd R ) time complexity to update when a new interaction occurs. Actually, Theorem 2 is the direct result of Theorem 1 based on Johnson-Lindenstrauss Lemma [11], where the random projection can preserve the inner product and norm [12]. Notably, the method for implicitly maintaining temporal walk matrices via random feature propagation can be extended to other types of temporal walk matrices, provided they meet a specific condition (i.e., the updating function of the temporal walk matrix can be written as the linear combination of its rows). This condition is not restrictive, and all the temporal walk matrices discussed in Section 2 fulfill it. We show their propagation mechanism and related discussion in Appendix D. In conclusion, the unified function in Equation (3), combined with methods of implicitly maintaining the temporal walk matrices, provides a new way to design a more effective and efficient way to inject pairwise information.

3.2 Link Likelihood Computing

Given an interaction ( u, v, t ) to be predicted, we compute its happening likelihood based on the obtained node representations and auxiliary features. Specifically, we first decode a pairwise feature f u,v ( t ) from the node representations obtained in Section 3.1. Then we compute the node embeddings h u ( t ) and h v ( t ) for node u and v respectively based on their historical interactions. Finally, we give the link likelihood based on h u ( t ) , h v ( t ) , f u,v ( t ) . For notation simplicity, we omit the suffix of h u ( t ) , h v ( t ) , f u,v ( t ) and denote them as h u , h v , f u,v in the following part.

Pairwise Feature Decoding . Although we can obtain the ( k +1) -dimensional feature in Equation (3) by calculating the inner product between the zero-layer representation of one node and all layers of the other node's representation, this method does not consider the correlation between all layers of both nodes. Therefore, we use representations from all layers to decode the pairwise information. Specifically, we first take the node representation of u and v from different layers, which can be denoted as F ∗ = [ e -λ 0 t H (0) ∗ , ..., e -λkt H ( k ) ∗ ] ∈ R ( k +1) × d R with * could be u or v. Then we concatenate them together to get F u,v = [ F u , F v ] ∈ R 2( k +1) × d R and obtain the raw pairwise feature ˜ f u,v by computing the inner product among different rows of F u,v , which is ˜ f u,v = flat ( F u,v F T u,v ) with flat ( · ) means flatten a matrix of R 2( k +1) × 2( k +1) into a vector of R 4( k +1) 2 . Finally, we feed the raw pairwise feature ˜ f u,v into an MLP to get the pairwise feature f u,v , which is f u,v = MLP ( log ( ReLU ( ˜ f u,v ) + 1)) . The ReLU ( · ) here is used to reduce estimation error, where the inner product of temporal walk matrices should be larger than zero and we thus set it to be zero if the inner product of the node representations is negative. The log ( · ) is used to scale the raw pairwise feature, where the range of the inner product between different layers varies greatly and the +1 is the shift term to avoid the undefined value of log (0) . We will see in Section 4.3 that these two operations can improve the training stability.

Auxiliary Feature Learning . The auxiliary features such as link features also provide rich information for modeling the evolution patterns of the temporal graph. In this part, we follow the previous methods [13, 9] and consider the auxiliary feature learning as a sequential learning problem. Specifically, for node u , we take its recent m interactions S u = [( { u, w 1 } , t 1 ) , ..., ( { u, w m } , t m )] before t and learn node embedding h u from this sequence. We first fetch the node features X u,N = [ x w 1 , ..., x w m ] ∈ R m × d N and edge features X u,E = [ e t 1 u,w 1 , .., e t m u,w 1 ] ∈ R m × d E . For timestamps, we map the timestamps into temporal features X u,T = [ ϕ ( t -t 1 ) , .., ϕ ( t -t n )] ∈ R m × d T like [14], where ϕ (∆ t ) = [cos( w 1 ∆ t ) , .., cos( w d T ∆ t )] is a time encoding function to learn the periodic temporal pattern. Besides, we construct a relative encoding sequence X u,F = [ f u,w 1 ⊕ f v,w 1 , ..., f u,w m ⊕ f v,w m ] ∈ R m × 8( k +1) 2 to inject the pairwise features, where f u,w m denote the pairwise feature of u and w m and ⊕ is the concatenation operation. After obtaining the above feature sequence, we concatenate them together and feed it into an MLP to get the final feature sequence Z (0) u = MLP ([ X u,N , X u,E , X u,T , X u,F ]) ∈ R m × d . Subsequently, we stack l layers of MLP-Mixer [15] to capture the temporal and structural dependencies within the feature sequence, which is

Finally, we get the node embedding by mean pooling h u = MEAN ( Z ( l ) u ) . The procedure to get node embedding h v is similar and for the node that does not have m interactions, we pad the feature sequence with zero. Then, the likelihood of the link ( u, v, t ) is given by p t u,v = MLP ([ h u , h v , f u,v ]) , where MLP ( · ) is a 2-layer MLP model with Sigmoid activation function in its output layer.

4 Experiments

4.1 Experimental Settings

Datasets and Baselines . We conduct experiments on 13 benchmark datasets for temporal link prediction, which are Wikipedia, Reddit, MOOC, LastFM, Enron, Social Evo., UCI, Flights, Can. Parl., US Legis., UN Trade, UN Vote and Contact. Eleven popular temporal graph learning methods are selected as baselines including JODIE, DyRep, TGAT, TGN, CAWN, EdgeBank, TCL, GraphMixer, NAT, PINT, and DyGFormer. Details about datasets and baselines can be found in Appendix F.

Task Settings . The task settings strictly follow [9]. Specifically, we conduct experiments under two settings: 1) the transductive setting that predicts links between nodes that have been seen during training and 2) the inductive setting that predicts links between nodes that are not seen during training. Three different negative sampling strategies introduced by [16] are used to sample the negative links and the Average Precision (AP) and Area Under the Receiver Operating Characteristic Curve (AUC-ROC) are adopted as the evaluation metrics. For dataset splitting, we chronologically split

each dataset with 70% / 15% / 15% for training/validating/testing. The training and testing pipeline is the same as that in [9].

Model Configuration . For TPNet, the layer l of node representations, the number of recent interactions m , and dimension d R of the node representations are set to 3, 20 and 10 ∗ log(2E), where E is the number of the interactions. We find the best time decay weight λ via grid search within a range of 10 -4 to 10 -7 . Specifically, the best λ for Contact is 10 -4 , the best λ for MOOC, Can. Parl. and UN Vote is 10 -5 , the best λ for Wikipedia, Reddit, Enron, Social Evo., Flights and US Legis. is 10 -6 , the best λ for LastFM, UCI and UN Trade is 10 -7 .

Implementatoin Details . For baselines, we use the implementation of DyGLib [9], which is a unified temporal graph learning library, and has tuned the best hyperparameters for each baseline. For baselines that are not included in DyGLib (i.e., NAT and PINT), we use their official implementation and find the best hyperparameters via grid search. Experiments are conducted on a Ubuntu server, whose CPU and GPU devices are one Intel(R) Xeon(R) Gold 6226R CPU @ 2.9GHz with 64 CPU cores and four GeForce RTX 3090 GPUs with 24 GB memory respectively. We run each experiment five times and report the average.

4.2 Performance and Efficiency Comparison

Performance comparison among baselines . The performance of TPNet and baselines is shown in Table 1. Due to space limit, Table 1 only shows the results under the transductive setting with random negative sampling strategy and more similar results can be found in Appendix G.1. As shown in Table 1, TPNet achieves the best performance among all the methods on most datasets, verifying its effectiveness. Besides, the link-wise methods (CAWN, NAT, PINT, and DyGFormer) perform better than the node-wise methods, indicating the importance of injecting the pairwise feature. Compared to the baselines, TPNet simultaneously considers the temporal and structural correlations between nodes and encodes the temporal walk matrix into node representations with theoretical guarantees, which contributes to its superior performance.

Efficiency Analysis . We compare the relative inference time of different methods to TPNet to evaluate their efficiency. The results on LastFM and MOOC are shown in Figure 4 and more results can be found in Appendix G.2. As shown in Figure 4, TPNet not only achieves the best performance but is also more efficient than other link-wise methods, where TPNet achieves a 33 × and 71.5 ×

0

1

2

3

4

5

6

7

log2 dimension

91.0

91.5

92.0

92.5

93.0

93.5

94.0

94.5

Average precision

LastFM

TPNet-d

SOTA Baseline

0

1

2

3

4

5

6

7

log2 dimension

89

90

91

92

93

94

95

96

Average precision

MOOC

TPNet-d

SOTA Baseline

speedups compared to the SOTA baseline DyGFormer and CAWN respectively on LastFM. The CAWN and DyGFormer models utilize time-consuming graph queries (e.g., temporal walk sampling) to construct relative encodings, which constitute the main computational bottleneck and consume over 70% of the running time. In contrast, TPNet caches historical interactions into node representations and constructs pairwise features based on these representations, thereby enhancing efficiency.

Scalability Analysis . To further verify the scalability of TPNet, we generated a series of random graphs with an average degree fixed at 100 and the number of edges varying from 1 e 5 to 1 e 8 . Figure 4 shows the change of running time and GPU memory of TPNet, where the growth of the running time and GPU memory is close to and less than the linear growth curve respectively, showing the good scalability of TPNet. In contrast, PINT explicitly stores the temporal walk matrices and encounters out-of-memory error when the edge number reaches 1 e 7 (shown inAppendix G.3), verifying the impracticability of explicitly storing temporal walk matrices.

4.3 Ablation Study

Proposed Components . To verify the effectiveness of the proposed components in TPNet, we compare TPNet with the following variants: 1) w/o NR that remove the node representations and the corresponding features that decoded from them. 2) w/o Time that only considers the structural information by setting the time decay weight λ to be zero. 3) w/o Scale that remove the log( · ) and ReLU ( · ) in the pairwise feature decoding. As shown in Table 2, there is a dramatic performance drop of w/o NR, which shows that the pairwise information carried by the node representations plays a vital role in the performance of TPNet. There is also an obvious performance drop of w/o Time, which confirms the necessity of incorporating temporal information in temporal walk matrix construction. Besides, the unreasonable poor performance of the w/o Scale is due to the various distribution of node representations across different layers, where, without scaling the raw pairwise features, the training will be unstable, and numerical overflow errors may even occur on some datasets. Further details on the distribution of node representations from different layers are provided in Appendix G.5.

Dimension Change . To verify the influence of the node representation dimension. We vary the dimensions from 1 to 128 and report the performance of TPNet (denoted as TPNet-d). As shown in Figure 3, the required dimension of node representations is small, where only 1-dimensional and 16-dimensional node representations can achieve satisfactory performance on MOOC and LastFM respectively. For different datasets, we observe that the average degree may be a main influence of the required dimension, where sparse graphs (like MOOC and Wikipedia) only need a small dimension, and dense graphs (like LastFM and Enron) may require a larger dimension. Empirically, setting the dimension to be 10 ∗ log(2E) is enough to achieve satisfactory performance on all datasets, where E is the number of edges. Results on more datasets can be found in Appendix G.4.

5 Related Work

Temporal link prediction [17] aims at predicting future interactions based on historical topology, which is crucial for a series of real-world applications [2, 3, 18]. Earlier methods considered the temporal graph as a series of graph snapshots that are sampled at regularly-spaced timestamps [19, 20], which will lose the fine-grained temporal information due to ignoring the temporal orders of

10 0

10 1

Relative time per epoch

70

80

90

100

Average precision

JODIE

X0.6

DyRep

TGAT

X7.6

TGN

TCL

X1.8

GraphMixer

X2.4

NAT

(11.8%)

PINT

X6.8

(3.2%)

TPNet

(15.5%)

CAWN

X71.5

(73.5%)

DyGFormer

X33.3

(93.6%)

LastFM

10 0

10 1

Relative time per epoch

75

80

85

90

95

100

Average precision

JODIE

X0.8

DyRep

X1.2

TGAT

X9.2

TGN

X1.2

TCL

X1.9

GraphMixer

X2.6

NAT

(11.6%)

PINT

X4.7

(5.1%)

TPNet

(14.6%)

CAWN

X31.0

(69.5%)

DyGFormer

X18.9

(91.6%)

MOOC

1e5

1e6

1e7

1e8

Number of Edges

10

3

10

2

10

1

10 0

10 1

10 2

10 3

10 4

Running Time

Running Time

Linear Growth

10 1

10 2

10 3

10 4

10 5

10 6

10 7

10 8

interactions in a graph snapshot. Recently, some continuous-time temporal graph learning methods have been proposed [21, 3, 14, 13], which consider the temporal graph as a sequence of interactions with irregular time intervals to fully capture the graph dynamics. For example, TGN [21] maintained a dynamic memory vector for each node and generated node representations by aggregating memory vectors via temporal graph attention to capture the evolution pattern of the temporal graph. However, capturing pairwise information by merely aggregating representations of neighboring nodes [22] is challenging. To address this issue, the link-wise method was proposed, which constructs relative encodings as additional node features to inject the pairwise information into the representation learning process [6, 7, 9, 8]. For example, Souza et al. [8] proposed a relative encoding based on temporal walk counting and theoretically showed that constructing the relative encodings can improve the expressive power of models in distinguishing different links. Despite these advances, existing ways to construct the relative encodings are still far from satisfactory, where computation efficiency is a main concern and temporal information is seldom considered. In this paper, we unify existing relative encodings into a function of temporal walk matrices and explore encoding the pairwise information effectively and efficiently by temporal walk matrix projection.

6 Limitation

One limitation of our method is that the matrix construction approach requires manual predefined settings. Different networks may necessitate distinct construction methods, potentially leading to additional human effort in experimenting with various approaches. For instance, the proposed temporal walk matrix that incorporates the time decay effect may not be optimal for networks characterized by long-term dependencies. Developing an adaptive matrix construction technique will be an interesting direction for future research.

7 Conclusion

In this paper, we study the problem of pairwise information injection for temporal link prediction. We unify existing construction ways of relative encodings into a unified function, which reveals a connection between the relative encoding and temporal walk matrix. Then we propose a new temporal link prediction model, TPNet, to address the computational inefficiencies and the ignorance of temporal information in previous methods. TPNet introduces a new temporal walk matrix to simultaneously consider the temporal and structural information and a random feature propagation mechanism to maintain the temporal walk matrices efficiently. Theoretically, TPNet preserves the inner product of the maintained temporal walk matrices and empirically outperforms other link-wise methods in both effectiveness and efficiency. An interesting future direction may be designing an adaptive feature propagation mechanism.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (No. 62272023 and No. 62276015).

References

Petter Holme and Jari Saramäki. Temporal networks. Physics reports , 519(3):97-125, 2012.

Ziwei Fan, Zhiwei Liu, Jiawei Zhang, Yun Xiong, Lei Zheng, and Philip S. Yu. Continuous-time sequential recommendation with temporal graph collaborative transformer. In International Conference on Information and Knowledge Management , pages 433-442. ACM, 2021.

Srijan Kumar, Xikun Zhang, and Jure Leskovec. Predicting dynamic embedding trajectory in temporal interaction networks. In International Conference on Knowledge Discovery & Data Mining , pages 1269-1278. ACM, 2019.

Fan Zhou, Xovee Xu, Goce Trajcevski, and Kunpeng Zhang. A survey of information cascade analysis: Models, predictions, and recent advances. ACM Comput. Surv. , 54(2):27:1-27:36, 2022.

Xiaodong Lu, Shuo Ji, Le Yu, Leilei Sun, Bowen Du, and Tongyu Zhu. Continuous-time graph learning for cascade popularity prediction. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence , pages 2224-2232, 2023.

Yanbang Wang, Yen-Yu Chang, Yunyu Liu, Jure Leskovec, and Pan Li. Inductive representation learning in temporal networks via causal anonymous walks. In 9th International Conference on Learning Representations , 2021.

Yuhong Luo and Pan Li. Neighborhood-aware scalable temporal network representation learning. In Learning on Graphs Conference , 2022.

Amauri H. Souza, Diego Mesquita, Samuel Kaski, and Vikas K. Garg. Provably expressive temporal graph networks. In Advances in Neural Information Processing Systems , 2022.

Le Yu, Leilei Sun, Bowen Du, and Weifeng Lv. Towards better dynamic graph learning: New architecture and unified library. In Advances in Neural Information Processing Systems , 2023.

Giang Hoang Nguyen, John Boaz Lee, Ryan A. Rossi, Nesreen K. Ahmed, Eunyee Koh, and Sungchul Kim. Continuous-time dynamic network embeddings. In Companion of the The Web Conference 2018 on The Web Conference 2018, WWW 2018, Lyon , France, April 23-27, 2018 , 2018.

D Sivakumar. Algorithmic derandomization via complexity theory. In Proceedings of the thiry-fourth annual ACM symposium on Theory of computing , pages 619-626, 2002.

Santosh S Vempala. The random projection method , volume 65. American Mathematical Soc., 2005.

Weilin Cong, Si Zhang, Jian Kang, Baichuan Yuan, Hao Wu, Xin Zhou, Hanghang Tong, and Mehrdad Mahdavi. Do we really need complicated model architectures for temporal networks? In The Eleventh International Conference on Learning Representations , 2023.

Da Xu, Chuanwei Ruan, Evren Körpeoglu, Sushant Kumar, and Kannan Achan. Inductive representation learning on temporal graphs. In 8th International Conference on Learning Representations . OpenReview.net, 2020.

Ilya O. Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, Mario Lucic, and Alexey Dosovitskiy. Mlp-mixer: An all-mlp architecture for vision. In Advances in Neural Information Processing Systems , 2021.

Farimah Poursafaei, Shenyang Huang, Kellin Pelrine, and Reihaneh Rabbany. Towards better evaluation for dynamic link prediction. In Advances in Neural Information Processing Systems , 2022.

Meng Qin and Dit-Yan Yeung. Temporal link prediction: A unified framework, taxonomy, and review. ACM Comput. Surv. , 56(4):89:1-89:40, 2024. doi: 10.1145/3625820. URL https://doi.org/10.1145/3625820 .

Le Yu, Zihang Liu, Leilei Sun, Bowen Du, Chuanren Liu, and Weifeng Lv. Continuous-time user preference modelling for temporal sets prediction. IEEE Trans. Knowl. Data Eng. , 36(4): 1475-1488, 2024.

Aravind Sankar, Yanhong Wu, Liang Gou, Wei Zhang, and Hao Yang. Dysat: Deep neural representation learning on dynamic graphs via self-attention networks. In WSDM '20: The Thirteenth ACM International Conference on Web Search and Data Mining, Houston, TX, USA, February 3-7, 2020 , pages 519-527. ACM, 2020.

Aldo Pareja, Giacomo Domeniconi, Jie Chen, Tengfei Ma, Toyotaro Suzumura, Hiroki Kanezashi, Tim Kaler, Tao B. Schardl, and Charles E. Leiserson. Evolvegcn: Evolving graph convolutional networks for dynamic graphs. In The Thirty-Fourth AAAI Conference on Artificial Intelligence , 2020.

Emanuele Rossi, Ben Chamberlain, Fabrizio Frasca, Davide Eynard, Federico Monti, and Michael M. Bronstein. Temporal graph networks for deep learning on dynamic graphs. CoRR , abs/2006.10637, 2020.

Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings , 2017.

Roman Vershynin. High-dimensional probability. University of California, Irvine , 10:11, 2020.

Moses Charikar. Similarity estimation techniques from rounding algorithms. In John H. Reif, editor, Proceedings on 34th Annual ACM Symposium on Theory of Computing , 2002.

Ping Li, Trevor Hastie, and Kenneth Ward Church. Very sparse random projections. In Proceedings of the Twelfth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Philadelphia, PA, USA, August 20-23, 2006 , 2006.

John Wright and Yi Ma. High-dimensional data analysis with low-dimensional models: Principles, computation, and applications . Cambridge University Press, 2022.

James W Pennebaker, Martha E Francis, and Roger J Booth. Linguistic inquiry and word count: Liwc 2001. Mahway: Lawrence Erlbaum Associates , 71(2001):2001, 2001.

Rakshit Trivedi, Mehrdad Farajtabar, Prasenjeet Biswal, and Hongyuan Zha. Dyrep: Learning representations over dynamic graphs. In International Conference on Learning Representations , 2019.

Peter J Diggle. Spatio-temporal point processes: methods and applications. Monographs on Statistics and Applied Probability , 107:1, 2006.

Walter Rudin. Fourier analysis on groups . Courier Dover Publications, 2017.

Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings , 2018.

Lu Wang, Xiaofu Chang, Shuang Li, Yunfei Chu, Hui Li, Wei Zhang, Xiaofeng He, Le Song, Jingren Zhou, and Hongxia Yang. TCL: transformer-based dynamic graph modelling via contrastive learning. CoRR , abs/2105.07944, 2021.

Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, and Tie-Yan Liu. Do transformers really perform badly for graph representation? In Annual Conference on Neural Information Processing Systems , 2021.

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Annual Conference on Neural Information Processing Systems , 2017.

A Analysis of Existing Relative Encodings

A.1 NAT

In Algorithm 1 of NAT, it maintains a series of hash maps s (0) u , s (1) u , .., s ( k ) u for each node u , which are sets of node ids. Next, we will first prove the statement holds if the size of the hash maps is infinite.

Statement . s ( i ) is a set of nodes that u can approach through a temporal walk whose length is less than i +1 .

Initially, the s (0) u is set as { u } and s (1) u , .., s ( k ) u are set as empty set. The statement holds. When a new interaction ( { u, v } , t ) occurs and if the size of the hash maps is infinite, for 1 ≤ i ≤ k , the updating function of hash maps can be represented as

where ¯ s ( i ) u and s ( i ) u denote the hash map after and before adding the interaction respectively. Then if the timestamp of ( { u, v } , t ) is larger than that of previous interactions, the newly generated temporal walks beginning from u must first visit v through ( { u, v } , t ) (proved in Section B.1), thus the newly added nodes that u can visit through a temporal walk with length less than i + 1 must belong to s ( i -1) v . So ¯ s ( i ) u will contain all the nodes that u can approach through a temporal walk with length less than i +1 after adding ( { u, v } , t ) . The statement holds.

For a node w , its similarity feature r w | u is a ( k +1) -dimensional vector, where for 1 ≤ i ≤ k +1 , r w | u i = 1 if w ∈ s ( i -1) u and r w | u i = 0 otherwise. Considering that the above statement holds, the similarity feature r w | u is equivalent to a ( k +1) -dimensional vector, where r w | u i = 1 if the shortest temporal walk from u to w is less than i ; otherwise, r w | u i = 0 .

Algorithm 2: Temporal Walk Extraction ( G ( t ) , α , k , u )

1 Initialize W to be { ( u, t ) } ;

2 for i from 1 to k do 3 ( w p , t p ) ← the last (node, time) pair in W ; 4 Sample one ( { w p , w ′ } , t ′ ) ∈ E w p ,t p with prob. ∝ exp( -α ( t p -t ′ )) ; 5 W i ← W i ⊕ ( w ′ , t ′ ) ; 6 Return W;

A.2 CAWN

For constructing r w | u , CAWN first repeatedly samples m temporal walks of length k begging at u according to the sampling strategy in Algorithm 2. Then r w | u is set to be a ( k +1 )-dimensional vector, where r w | u i will be the number of walks whose i-th visited node is w for 1 ≤ i ≤ k +1 , which can be represented as,

where W j is the j-th sampled walks and 1 W j [ i ][0]= w is 1 if W j [ i ][0] = w ; otherwise, 1 W j [ i ][0]= w is 0 . Due to each temporal walk being sampled independently, 1 W 1 [ i ][0]= w , · · · , 1 W m [ i ][0]= w are m independent and identically distributed Bernoulli random variables Ber ( µ i -1 w ) , where µ i -1 w is the probability of reaching w from u after performing a ( i -1) -step temporal walk according to Algorithm 2. And according to the strong law of large numbers (Theorem 1.3.1 of [23]), the mean of these random variables (i.e., 1 m ∑ m j =1 1 W j [ i ][0]= w ⇐⇒ 1 m r w | u i ) will coverage to the mean as the number of sampled walks m → ∞ . The mean of the Ber ( µ i -1 w ) is µ i -1 w . Expand all ( i -1) -step temporal walks from u to w , we have

Timestamps:

𝑡

1

< 𝑡2

< 𝑡3

< 𝑡4

𝐴

Previous 2-step temporal walks

𝑡

2

𝑡

1

𝑡

3

𝑡

4

𝐷

𝐵

𝐶

Historical interaction

New interaction

New 3-step temporal walks

𝑡

4

𝑡

3

𝑡

2

𝐷

𝐶

𝐴

𝐵

𝑡

4

𝑡

3

𝑡

1

𝐷

𝐶

𝐴

𝐷

𝑡

2

𝑡

3

𝐶

𝐴

𝐵

𝑡

1

𝑡

3

𝐶

𝐴

𝐷

𝑡

1

𝑡

2

𝐵

𝐴

𝐷

map

Temporal Graph

where f ( · ) is the sampled probability of a walk according to Algorithm 2. For Algorithm 2, if we are current at ( w i , t i ) , then the probability of moving to ( w i +1 , t i +1 ) through an interaction ( { w i , w i +1 } , t i +1 ) is proportional to exp( -α ( t i -t i +1 )) , which is exp ( -α ( t i -t i +1 )) ∑ ( w ′ ,t ′ ) ∈E wi ,t i exp ( -α ( t i -t ′ )) . Thus the probability of a temporal walk W = [( w 0 , t 0 ) , ( w 1 , t 1 ) , .., ( w k , t k )] is

which is the same as the score function s ′ ( · ) of CAWN we shown in Section 2.2.2. Thus, r w | u i (multiplied with a const 1 m ) is the same as ∑ W ∈M i -1 u,w s ′ ( W ) .

B Proofs

B.1 Proof of Theorem 1

The equation in Theorem 1 can be rewritten as H ( l ) ( t ) = e λlt ∗ A ( l ) ( t ) P . We can consider e λlt ∗ A ( l ) ( t ) as a new temporal walk matrix whose score function for a temporal walk [( w 0 , t 0 ) , ( w 1 , t 1 ) , ..., ( w l , t l )] at time t is e λlt ∗ s ( W ) . Expanding it, we have

We use the notation ¯ A ( l ) ( t ) to denote e λlt ∗ A ( l ) ( t ) and ¯ s ( W ) to denote e λlt ∗ s ( W ) for a l -step temporal walk in the following proof. The original problem is transformed into proving that H ( l ) ( t ) = ¯ A ( l ) ( t ) P . Note that for a given temporal walk [( w 0 , t 0 ) , ( w 1 , t 1 ) , ..., ( w l , t l )] , each term of ¯ s ( W ) (i.e., ∏ l j =1 e λt j ) will no change as time t goes on. Thus the temporal walk matrices ¯ A ( l ) ( t ) will only change when a new interaction occurs. Next, we inspect how the ¯ A ( l ) ( t ) changes when a new interaction ( u, v, t ) occurs. For each element ¯ A ( l ) i,j ( t ) , its change is caused by the newly generated l -step temporal walks from i to j after adding the interaction ( u, v, t ) , which can be written as

where t + denotes the timestamps right after t and ∆ M l i,j denote the newly generated l -step temporal walks from i to j . According to the definition of the temporal walk, the new l -step temporal walks must begin from u or v . Because if there is a temporal walk that does not begin from u or v , it can be represented as [( w 0 , t 0 ) , .., ( w i , t i ) , ( u, t i +1 ) , ( v, t ) , .., ( w l , t l ))] or [( w 0 , t 0 ) , .., ( w i , t i ) , ( v, t i +1 ) , ( u, t ) , .., ( w l , t l ))] , which means that there is an interaction ( { w i , u } , t i +1 ) or ( { w i , v } , t i +1 ) whose timestamp t i is larger than t . (Since the timestamps of a temporal walk are decreasing). This is impossible since ( u, v, t ) is a newly happened interaction. Thus only the u -th row and v -th row of the ¯ A ( l ) ( t ) will change. Besides for each newly generated temporal walk from u , it must be [( u, t ) , ( v, t 1 ) , .., ( w l , t l ))] , where [( v, t 1 ) , .., ( w l , t l ))] corresponds to a ( l -1) -step temporal walk beginning from v . And for any ( l -1) -step temporal walk beginning

from v , we can add a prefix ( u, t ) to make it become a l -step temporal walk beginning from u . Thus for any 1 ≤ i ≤ n , there is a one-to-one mapping between ∆ M l u,i and M l -1 v,i ( t ) with M l -1 v,i ( t ) denote the set of ( l -1) -th temporal walks from v to i before t , and we can rewritten Equation (12) as

The updating function for ¯ A ( l ) v,i ( t ) is also similar. We give a visual illustration of the new temporal walks in Figure 6 for better understanding. Finally, writing the update formula in vector form, for any 1 ≤ l ≤ k we have

which is the same as Equation (14)! Thus if H ( l ) ( t ) = ¯ A ( l ) ( t ) P for 0 ≤ l ≤ k , after adding a new interaction ( u, v, t ) , for 0 ≤ l ≤ k , H ( l ) ( t + ) = ¯ A ( l ) ( t + ) P still holds. Because we have

Note that the equation in Equation (1) holds at initialization, thus the equation always holds and the theorem is proved.

B.2 Proof of Theorem 2

The Theorem 2 can be considered as a special case of the Johnson-Lindenstrauss Lemma [11], where the random projection [12, 24, 25] can preserve the norm and inner product. For the convenience of readers lacking relevant background, we follow [26] to provide the proof under the given conditions of this paper. We begin the proof by the following lemma.

Lemma 1 ((Lemma 3.18 of [26])) . Let x ∈ R d be a d-dimensional random vector whose entries are independent N (0 , 1 d ) . Then for any ϵ ∈ [0 , 1] ,

The following part can be divided into 1) We first give two corollaries and their proofs. 2) Then we give the proof of Theorem 2 based on the corollaries.

B.2.1 Two Corolarries

Based on the above lemma, we can get the following corollaries.

Corollary 1. Given any ϵ ∈ (0 , 1) , x ∈ R m . Let P ∈ R d × m be a random matrix whose entries are independent N (0 , 1 d ) , if d ≥ 8 ϵ 2 log( 1 δ ) , we have

Proof . For the above corollary, we have

Note that each entry of P is an independent N (0 , 1 d ) , thus Px is a random vector whose each entry ( Px ) k = ∑ m i =1 P k,i ∗ x i is an independent N (0 , ∥ x ∥ 2 2 d ) and each entry of Px ∥ x ∥ 2 is an independent N (0 , 1 d ) . Substituting it to Lemma 1 and taking d ≥ 8 ϵ 2 log( 1 δ ) , we have

Based on Corollary 1, we can get the following corollary.

Corollary 2. Given any n m -dimensional vectors x 1 , ..., x n , ϵ ∈ (0 , 1) , let P ∈ R d × m be a random matrix whose entries are independent N (0 , 1 d ) , if d ≥ 24 ϵ 2 log(4 1 / 3 n ) , for any 1 ≤ i, j ≤ n , we have

let E + i,j and E -i,j denote the event of { ∣ ∣ ∥ P ( x i + x j ) ∥ 2 2 -∥ x i + x j ∥ 2 2 ∣ ∣ ≤ ϵ ∥ x i + x j ∥ 2 2 } and { ∣ ∣ ∥ P ( x i -x j ) ∥ 2 2 -∥ x i -x j ∥ 2 2 ∣ ∣ ≤ ϵ ∥ x i -x j ∥ 2 2 } respectively. Since x i + x j and x i -x j can also be consider two m -dimensional vectors. Thus take it and d ≥ 24 ϵ 2 log(4 1 / 3 n ) into Corollary 1, we have P ( E + i,j ) ≥ 1 -1 2 n 3 and P ( E -i,j ) ≥ 1 -1 2 n 3 . Then let C i,j = E + i,j ∩ E -i,j denote that E + i,j and E -i,j hold simultaneously, according to the union bound, we have

where E + i,j denote that E + i,j does not hold and E + i,j ∪ E -i,j denotes that E + i,j or E -i,j holds. According to the union bound, we further have

where ∩ i,j C i,j denote that C i,j holds for any i, j and ∪ i,j C i,j denote that there exist i, j that C i,j holds. If C i,j holds, we have

Multiplying Equation (24) with -1 and adding it to (23), we have

Thus we have

holds for any 1 ≤ i, j ≤ n .

B.2.2 Proof based on Corollaries

The matrix of A ( l ) ( t ) in Theorem 2 can be considered as n vectors with n dimensions, where each row of A ( l ) ( t ) is a n -dimensional vector. Similarly, we can consider A (0) ( t ) , · · · , A ( k ) ( t ) together as n ( k +1) vectors with n dimensions. Considering that P ∈ R n × d R is a random matrix where each entry is an independent N (0 , 1 d R ) and e -λlt ∗ H ( l ) ( t ) is the projection of A ( l ) ( t ) , substitute it into Corollary 2 and taking the number of vectors as ( k +1) n , we can get Theorem 2.

C Batch Updating Mechanism

For the situation where multiple interactions happen simultaneously, we can first compute each interaction's contribution independently and sum them together to update the node representations. The maintaining mechanism is shown in Algorithm 3, where we packed the interactions that happen at the same time into a set B and sum the independent contribution of each interaction in B into ∆ H (1) , · · · , ∆ H ( k ) . For simplicity, we initialize ∆ H (1) , · · · , ∆ H ( k ) each time and it can be replaced by some efficient implementation such as scatter_add operation in pytorch.

D Propagation Mechanism for Other Matrices

For simplicity, here we only consider the situation where the timestamp of each interaction is different and we can adopt a similar batch updating mechanism in Section C to handle the situation where multiple interactions happen simultaneously. For the updating of other types of temporal walk matrices, let A ( t ) = [ A (0) ( t ) , · · · , A ( k ) ( t )] ∈ R n ( k +1) × d denotes the concatenation of the temporal walk matrices. If the following two situations can be satisfied simultaneously, we can apply the random feature propagation mechanism to implicitly maintain the temporal walk matrices.

Condition 1 . After adding a new interaction ( u, v, t ) , the change of each row can be written as the linear combination of other rows, which is for each 1 ≤ i ≤ n ( k +1) , there exists k 1 , ..., k m and l 1 , ..., l m satisfying.

where t + is the time right after t .

Condition 2 . After time moving ∆ t without adding new interactions, the change of each row can be written as the linear combination of other rows, which is for each 1 ≤ i ≤ n ( k +1) , there exists k 1 , ..., k m and l 1 , ..., l m satisfying.

The motivation behind the conditions is that the projection is a linear operation. If the node representations are the projection of the temporal walk matrix at t (i.e., H ( t ) = A ( t ) P ) and the updating function of the temporal walk matrix is the linear combination of other rows, then we can apply the same updating function on H ( t ) , which will make H ( t + ) still be the projection of the temporal walk matrix. For example, applying (27) to H ( t ) will get

Then we can ensure that H ( t ) is always the random projection of A ( t ) and thus preserve the inner product of the A ( t ) .

D.1 Detailed Updating Mechanism

Based on the above analysis, we give the detailed propagation mechanism of methods in Section 2. For NAT, PINT and DyGFormer, their temporal matrix element A ( l ) u,v ( t ) is the number of the l -step temporal walks from u to v and their feature propagation mechanism is shown in Algorithm 4, where the obtained node representations are the projection of the corresponding temporal walk matrix.

For CAWN, its score function for a temporal walk W = [( w 0 , t 0 ) , ( w 1 , t 1 ) , ..., ( w l , t l )] is defined as s ( W ) = ∏ l -1 i =0 exp ( -α ( t i -t i +1 )) ∑ ( { w ′ ,w } ,t ′ ) ∈E wi ,t i exp ( -α ( t i -t ′ )) . As time goes on, its element of temporal walk

matrices will not change. When a new interaction ( u, v, t ) happens, for a temporal walk matrix A ( l ) ( t ) , its u -th row and v -th row will change. Formally, let d ( t ) ∈ R n denotes a time decay degree vector, where for each node u , d u ( t ) is defined as d u ( t ) = ∑ ( { ( u,v ′ } ,t ′ ) ∈E u,t exp( -α ( t ′ -t )) with E u,t denoting the set of interactions attached to u before t , then the updating function of the temporal walk matrix A ( k ) ( t ) can be represented as

where d u ( t ) d u ( t )+1 ∗ A ( l ) u ( t ) corresponds to the change in score of the old l -step temporal walks begging from u and 1 d u ( t )+1 ∗ A ( l -1) v ( t ) correspond to change caused by the newly generated l -step temporal walk from u (the same for v and similar analysis about the updating of temporal walk matrix can be found in Appendix B.1). Then we can give the propagation mechanism for CAWN in Algorithm 5 based on the above analysis, where d corresponds to the time decay degree vector.

E Broader Impact

We proposed an effective and efficient temporal link prediction method, which may advance realworld scenarios that rely on link prediction as a cornerstone, such as recommendation systems. For potential negative impacts, overly accurate link prediction in some contexts may lead to imbalanced outcomes such as reduced diversity in recommendation systems.

F Experimental Settings

F.1 Datasets

Experiments are conducted on the following 13 benchmark datasets 3 collected by [16].

Wikipedia is a bipartite interaction graph that records the edits on Wikipedia pages over a month. Nodes represent users and pages, and links denote the editing behaviors with timestamps. Each link is associated with a 172-dimensional Linguistic Inquiry and Word Count (LIWC) feature [27].

Reddit is a bipartite graph capturing user posts under subreddits over a month. Nodes represent users and subreddits, while links denote timestamped posts. Each link carries a 172-dimensional LIWC feature.

MOOC is a bipartite interaction network of online courses, where nodes represent students and course content units (e.g., videos, problem sets). Links indicate students' access to specific content units and have a 4-dimensional feature.

LastFM is a bipartite network detailing song-listening behaviors of users over one month. Nodes are users and songs, and links represent listening activities.

Enron records email communications between employees of the Enron Energy Corporation over three years.

Social Evo. is a mobile phone proximity network tracking daily activities within an undergraduate dormitory for eight months, with each link having a 2-dimensional feature.

UCI is an online communication network with nodes representing university students and links representing posted messages.

Flights is a dynamic flight network showing air traffic development during the COVID-19 pandemic. Nodes represent airports, and links denote tracked flights. Each link has a weight indicating the number of flights between two airports per day.

Can. Parl. is a dynamic political network recording interactions between Canadian Members of Parliament (MPs) from 2006 to 2019. Nodes represent MPs from electoral districts, and links are formed when two MPs vote "yes" on a bill. The weight of each link represents the number of times one MP voted "yes" in support of another MP in a year.

US Legis. is a senate co-sponsorship network tracking social interactions between US Senators. The weight of each link indicates the number of times two senators co-sponsored a bill in a given congress.

UN Trade captures food and agriculture trade between 181 nations over more than 30 years. The weight of each link represents the total normalized agriculture import or export values between two countries.

UN Vote records roll-call votes in the United Nations General Assembly. A link between two nations increases in weight each time both vote "yes" on an item.

Contact tracks the evolution of physical proximity among approximately 700 university students over a month. Each student has a unique identifier, and links indicate close proximity, with weights revealing the extent of physical closeness between students.

The statistics of the datasets are shown in Table 3, where #N&L Feat stands for the dimensions of node and link features.

F.2 Baselines

We select the following eleven popular baselines:

JODIE [3] designs a recurrent architecture to maintain a memory vector for each node and a projection layer to map the node memories into future representation trajectories.

DyRep [28] considers each link as a temporal point process [29] and designs a deep temporal point process model to capture the dynamics of the observed process.

TGAT [14] proposes a time encoding function based on Bochner's Theorem [30] and combines it with the graph attention mechanism [31] to learn dynamic node representations.

TGN [21] proposes a general framework for temporal graph learning, which includes a memory module to maintain node memories and an embedding module to aggregate node memories.

CAWN [6] captures the evolution pattern of the temporal graph by causal anonymous walks. For a given link, CAWN first sampled a set of temporal walks beginning from the two end nodes respectively and constructs relative encodings. The sampled walks together with relative encodings are then mapped into node representations via a sequential model.

EdgeBank [16] is a statistical method, that gives the likelihood of a link based on historical interactions between the two end nodes.

TCL [32] propose a graph transformer architecture [33] to learning node representations, which samples neighbor nodes based on BFS and maps them into the node representation.

NAT [7] propose a dictionary-type neighborhood representation to efficiently capture the correlation information between nodes, which maintains a series of N-caches to store the neighborhood information and use them to decode the pairwise information between nodes.

PINT [8] proposes an injective temporal message passing mechanism to learn node representations and relative positional features constructed based on temporal walk counting to inject pairwise information between nodes.

GraphMixer [13] proposes a simplified temporal graph learning architecture, which employs MLP-Mixer to learn the representation of a node from its historical interaction sequences.

DyGFormer [9] proposes a transformer architecture [34] to learning node representations, which include a patching technique to capture the long-term histories of a node to and a neighbor co-occurrence encoding scheme to capture the correlation between nodes.

G Additional Experimental Results

G.1 Performance Comparison

The results under other settings are shown in Table 6, Table 7, Table 8, Table 9 and Table 10.

G.2 Efficicency Analysis

Efficiency analysis results on Reddit, UCI, and Wikipedia are shown in Figure 7.

G.3 Scalability Analysis

The scalability analysis result of PINT is shown in Table 4, where OOM indicates the out-of-memory error.

10 0

10 1

Relative time per epoch

97

98

99

100

Average precision

JODIE

X1.3

DyRep

X2.0

TGAT

X30.8

TGN

X1.8

TCL

X2.5

GraphMixer

X3.0

NAT

(11.8%)

PINT

X10.1

(2.9%)

TPNet

(14.1%)

CAWN

X24.6

(57.9%)

DyGFormer

X6.9

(83.2%)

Reddit

10 0

10 1

Relative time per epoch

80

85

90

95

100

Average precision

JODIE

X0.5

TGAT

TGN

X0.9

TCL

X1.7

GraphMixer

X2.1

NAT

(12.7%)

PINT

(3.6%)

TPNet

(13.8%)

CAWN

X26.6

(71.3%)

DyGFormer

X5.4

(81.3%)

UCI

10 0

10 1

Relative time per epoch

95

100

Average precision

JODIE

X0.8

DyRep

X1.2

TGAT

X8.7

TGN

X1.3

TCL

X1.8

GraphMixer

X2.4

NAT

(12.7%)

PINT

X8.2

(3.2%)

TPNet

(13.5%)

CAWN

(63.9%)

DyGFormer

X5.9

(80.8%)

Wikipedia

DyRep

G.4 Influence of Node Representation Dimension

Results on Wikipedia, Enron, and UCI are shown in Figure 8.

G.5 Statistic Analysis of Node Representations

Table 5 shows the mean of node representation norm at different layers, where aEb indicates a × 10 b .

X0.9

0

1

2

3

4

5

6

7

log2 dimension

99.05

99.10

99.15

99.20

99.25

99.30

Average precision

Wikipedia

TPNet-d

SOTA Baseline

0

1

2

3

4

5

6

7

log2 dimension

92.0

92.2

92.4

92.6

92.8

93.0

Average precision

Enron

TPNet-d

SOTA Baseline

0

1

2

3

4

5

6

7

log2 dimension

96.6

96.8

97.0

97.2

Average precision

UCI

TPNet-d

SOTA Baseline

Metric

Dataset

JODIE

DyRep

TGAT

TGN

CAWN

EdgeBank

TCL

GraphMixer

NAT

PINT

DyGFormer

TPNet

AP

Wikipedia

83

.

01

±

0

.

66

79

.

93

±

0

.

56

87

.

38

±

0

.

22

86

.

86

±

0

.

33

71

.

21

±

1

.

67

73

.

35

±

0

.

00

89

.

05

±

0

.

39

90

.

90

±

0

.

10

68

.

36

±

4

.

47

91

.

33

±

0

.

76

82

.

23

±

2

.

54

81

.

55

±

4

.

10

Reddit

80

.

03

±

0

.

36

79

.

83

±

0

.

31

79

.

55

±

0

.

20

81

.

22

±

0

.

61

80

.

82

±

0

.

45

73

.

59

±

0

.

00

77

.

14

±

0

.

16

78

.

44

±

0

.

18

80

.

58

±

0

.

91

83

.

93

±

1

.

11

81

.

57

±

0

.

67

81

.

02

±

1

.

31

MOOC

78

.

94

±

1

.

25

75

.

60

±

1

.

12

82

.

19

±

0

.

62

87

.

06

±

1

.

93

74

.

05

±

0

.

95

60

.

71

±

0

.

00

77

.

06

±

0

.

41

77

.

77

±

0

.

92

85

.

42

±

3

.

26

84

.

80

±

1

.

57

85

.

85

±

0

.

66

92

.

69

±

0

.

95

LastFM

74

.

35

±

3

.

81

74

.

92

±

2

.

46

71

.

59

±

0

.

24

76

.

87

±

4

.

64

69

.

86

±

0

.

43

73

.

03

±

0

.

00

59

.

30

±

2

.

31

72

.

47

±

0

.

49

78

.

75

±

1

.

63

84

.

95

±

0

.

43

81

.

57

±

0

.

48

87

.

74

±

0

.

50

Enron

69

.

85

±

2

.

70

71

.

19

±

2

.

76

64

.

07

±

1

.

05

73

.

91

±

1

.

76

64

.

73

±

0

.

36

76

.

53

±

0

.

00

70

.

66

±

0

.

39

77

.

98

±

0

.

92

72

.

90

±

1

.

38

85

.

41

±

0

.

84

75

.

63

±

0

.

73

80

.

79

±

1

.

68

Social Evo.

87

.

44

±

6

.

78

93

.

29

±

0

.

43

95

.

01

±

0

.

44

94

.

45

±

0

.

56

85

.

53

±

0

.

38

80

.

57

±

0

.

00

94

.

74

±

0

.

31

94

.

93

±

0

.

31

91

.

29

±

4

.

89

96

.

73

±

0

.

29

97

.

38

±

0

.

14

96

.

80

±

0

.

41

UCI

75

.

24

±

5

.

80

55

.

10

±

3

.

14

68

.

27

±

1

.

37

80

.

43

±

2

.

12

65

.

30

±

0

.

43

65

.

50

±

0

.

00

80

.

25

±

2

.

74

84

.

11

±

1

.

35

75

.

38

±

1

.

28

93

.

96

±

0

.

20

82

.

17

±

0

.

82

86

.

34

±

0

.

80

Flights

66

.

48

±

2

.

59

67

.

61

±

0

.

99

72

.

38

±

0

.

18

66

.

70

±

1

.

64

64

.

72

±

0

.

97

70

.

53

±

0

.

00

70

.

68

±

0

.

24

71

.

47

±

0

.

26

64

.

74

±

1

.

83

66

.

82

±

1

.

44

66

.

59

±

0

.

49

69

.

10

±

1

.

27

51

.

79

±

0

.

63

63

.

31

±

1

.

23

67

.

13

±

0

.

84

68

.

42

±

3

.

07

66

.

53

±

2

.

77

63

.

84

±

0

.

00

65

.

93

±

3

.

00

74

.

34

±

0

.

87

77

.

72

±

1

.

78

63

.

84

±

5

.

36

97

.

00

±

0

.

31

86

.

61

±

3

.

57

US Legis.

51

.

71

±

5

.

76

86

.

88

±

2

.

25

62

.

14

±

6

.

60

74

.

00

±

7

.

57

68

.

82

±

8

.

23

63

.

22

±

0

.

00

80

.

53

±

3

.

95

81

.

65

±

1

.

02

91

.

12

±

1

.

97

73

.

58

±

4

.

16

85

.

30

±

3

.

88

94

.

55

±

0

.

62

UN Trade

61

.

39

±

1

.

83

59

.

19

±

1

.

07

55

.

74

±

0

.

91

58

.

44

±

5

.

51

55

.

71

±

0

.

38

81

.

32

±

0

.

00

55

.

90

±

1

.

17

57

.

05

±

1

.

22

78

.

65

±

1

.

16

70

.

07

±

2

.

28

64

.

41

±

1

.

40

85

.

22

±

1

.

22

UN Vote

70

.

02

±

0

.

81

69

.

30

±

1

.

12

52

.

96

±

2

.

14

69

.

37

±

3

.

93

51

.

26

±

0

.

04

84

.

89

±

0

.

00

52

.

30

±

2

.

35

51

.

20

±

1

.

60

71

.

39

±

2

.

68

71

.

79

±

2

.

53

60

.

84

±

1

.

58

74

.

68

±

1

.

38

Contact

95

.

31

±

2

.

13

96

.

39

±

0

.

20

96

.

05

±

0

.

52

93

.

05

±

2

.

35

84

.

16

±

0

.

49

88

.

81

±

0

.

00

93

.

86

±

0

.

21

93

.

36

±

0

.

41

96

.

84

±

0

.

57

97

.

61

±

0

.

18

97

.

57

±

0

.

06

98

.

02

±

0

.

15

Avg. Rank

8.23

7.69

7.54

5.92

10.31

8.08

7.77

6.23

5.85

3.62

4.23

2.46

AUC

Wikipedia

80

.

77

±

0

.

73

77

.

74

±

0

.

33

82

.

87

±

0

.

22

82

.

74

±

0

.

32

67

.

84

±

0

.

64

77

.

27

±

0

.

00

85

.

76

±

0

.

46

87

.

68

±

0

.

17

69

.

32

±

2

.

78

89

.

25

±

0

.

49

78

.

80

±

1

.

95

79

.

89

±

2

.

47

Reddit

80

.

52

±

0

.

32

80

.

15

±

0

.

18

79

.

33

±

0

.

16

81

.

11

±

0

.

19

80

.

27

±

0

.

30

78

.

58

±

0

.

00

76

.

49

±

0

.

16

77

.

80

±

0

.

12

79

.

36

±

0

.

31

82

.

98

±

0

.

63

80

.

54

±

0

.

29

81

.

87

±

0

.

49

MOOC

82

.

75

±

0

.

83

81

.

06

±

0

.

94

80

.

81

±

0

.

67

88

.

00

±

1

.

80

71

.

57

±

1

.

07

61

.

90

±

0

.

00

72

.

09

±

0

.

56

76

.

68

±

1

.

40

84

.

93

±

2

.

89

87

.

44

±

1

.

74

87

.

04

±

0

.

35

93

.

45

±

0

.

67

LastFM

75

.

22

±

2

.

36

74

.

65

±

1

.

98

64

.

27

±

0

.

26

77

.

97

±

3

.

04

67

.

88

±

0

.

24

78

.

09

±

0

.

00

47

.

24

±

3

.

13

64

.

21

±

0

.

73

75

.

89

±

2

.

21

81

.

89

±

0

.

96

78

.

78

±

0

.

35

84

.

64

±

0

.

45

Enron

75

.

39

±

2

.

37

74

.

69

±

3

.

55

61

.

85

±

1

.

43

77

.

09

±

2

.

22

65

.

10

±

0

.

34

79

.

59

±

0

.

00

67

.

95

±

0

.

88

75

.

27

±

1

.

14

73

.

22

±

2

.

18

83

.

80

±

0

.

68

76

.

55

±

0

.

52

81

.

16

±

1

.

28

Social Evo.

90

.

06

±

3

.

15

93

.

12

±

0

.

34

93

.

08

±

0

.

59

94

.

71

±

0

.

53

87

.

43

±

0

.

15

85

.

81

±

0

.

00

93

.

44

±

0

.

68

94

.

39

±

0

.

31

91

.

87

±

4

.

52

96

.

82

±

0

.

24

97

.

28

±

0

.

07

97

.

22

±

0

.

30

UCI

78

.

64

±

3

.

50

57

.

91

±

3

.

12

58

.

89

±

1

.

57

77

.

25

±

2

.

68

57

.

86

±

0

.

15

69

.

56

±

0

.

00

72

.

25

±

3

.

46

77

.

54

±

2

.

02

71

.

52

±

1

.

61

92

.

05

±

0

.

36

76

.

97

±

0

.

24

80

.

42

±

0

.

64

Flights

68

.

97

±

1

.

87

69

.

43

±

0

.

90

72

.

20

±

0

.

16

68

.

39

±

0

.

95

66

.

11

±

0

.

71

74

.

64

±

0

.

00

70

.

57

±

0

.

18

70

.

37

±

0

.

23

67

.

11

±

2

.

16

69

.

52

±

0

.

90

68

.

09

±

0

.

43

71

.

82

±

0

.

82

62

.

44

±

1

.

11

70

.

16

±

1

.

70

70

.

86

±

0

.

94

73

.

23

±

3

.

08

72

.

06

±

3

.

94

63

.

04

±

0

.

00

69

.

95

±

3

.

70

79

.

03

±

1

.

01

81

.

47

±

2

.

58

73

.

89

±

4

.

50

97

.

61

±

0

.

40

86

.

39

±

3

.

73

US Legis.

67

.

47

±

6

.

40

91

.

44

±

1

.

18

73

.

47

±

5

.

25

83

.

53

±

4

.

53

78

.

62

±

7

.

46

67

.

41

±

0

.

00

83

.

97

±

3

.

71

85

.

17

±

0

.

70

94

.

63

±

1

.

16

83

.

45

±

2

.

85

90

.

77

±

1

.

96

96

.

28

±

0

.

44

UN Trade

68

.

92

±

1

.

40

64

.

36

±

1

.

40

60

.

37

±

0

.

68

63

.

93

±

5

.

41

63

.

09

±

0

.

74

86

.

61

±

0

.

00

61

.

43

±

1

.

04

63

.

20

±

1

.

54

80

.

68

±

0

.

98

76

.

95

±

1

.

92

73

.

86

±

1

.

13

88

.

90

±

1

.

00

UN Vote

76

.

84

±

1

.

01

74

.

72

±

1

.

43

53

.

95

±

3

.

15

73

.

40

±

5

.

20

51

.

27

±

0

.

33

89

.

62

±

0

.

00

52

.

29

±

2

.

39

52

.

61

±

1

.

44

76

.

47

±

2

.

15

76

.

61

±

3

.

05

64

.

27

±

1

.

78

78

.

43

±

1

.

09

Contact

96

.

35

±

0

.

92

96

.

00

±

0

.

23

95

.

39

±

0

.

43

93

.

76

±

1

.

29

83

.

06

±

0

.

32

92

.

17

±

0

.

00

93

.

34

±

0

.

19

93

.

14

±

0

.

34

96

.

79

±

0

.

50

97

.

34

±

0

.

16

97

.

17

±

0

.

05

97

.

73

±

0

.

11

Avg. Rank

6.77

7.31

8.38

5.62

10.31

7.54

8.54

7.08

6.46

3.15

4.77

2.08

Metric

Dataset

JODIE

DyRep

TGAT

TGN

CAWN

EdgeBank

TCL

GraphMixer

NAT

PINT

DyGFormer

TPNet

AP

Wikipedia

75

.

65

±

0

.

79

70

.

21

±

1

.

58

87

.

00

±

0

.

16

85

.

62

±

0

.

44

74

.

06

±

2

.

62

80

.

63

±

0

.

00

86

.

76

±

0

.

72

88

.

59

±

0

.

17

53

.

85

±

6

.

25

82

.

39

±

2

.

70

78

.

29

±

5

.

38

79

.

35

±

5

.

52

Reddit

86

.

98

±

0

.

16

86

.

30

±

0

.

26

89

.

59

±

0

.

24

88

.

10

±

0

.

24

91

.

67

±

0

.

24

85

.

48

±

0

.

00

87

.

45

±

0

.

29

85

.

26

±

0

.

11

81

.

71

±

1

.

03

87

.

59

±

0

.

65

91

.

11

±

0

.

40

88

.

19

±

0

.

33

MOOC

65

.

23

±

2

.

19

61

.

66

±

0

.

95

75

.

95

±

0

.

64

77

.

50

±

2

.

91

73

.

51

±

0

.

94

49

.

43

±

0

.

00

74

.

65

±

0

.

54

74

.

27

±

0

.

92

77

.

28

±

1

.

94

75

.

70

±

1

.

43

81

.

24

±

0

.

69

88

.

18

±

0

.

97

LastFM

62

.

67

±

4

.

49

64

.

41

±

2

.

70

71

.

13

±

0

.

17

65

.

95

±

5

.

98

67

.

48

±

0

.

77

75

.

49

±

0

.

00

58

.

21

±

0

.

89

68

.

12

±

0

.

33

64

.

02

±

1

.

48

72

.

24

±

0

.

92

73

.

97

±

0

.

50

77

.

99

±

1

.

30

Enron

68

.

96

±

0

.

98

67

.

79

±

1

.

53

63

.

94

±

1

.

36

70

.

89

±

2

.

72

75

.

15

±

0

.

58

73

.

89

±

0

.

00

71

.

29

±

0

.

32

75

.

01

±

0

.

79

70

.

34

±

1

.

73

80

.

23

±

1

.

37

77

.

41

±

0

.

89

75

.

36

±

1

.

81

Social Evo.

89

.

82

±

4

.

11

93

.

28

±

0

.

48

94

.

84

±

0

.

44

95

.

13

±

0

.

56

88

.

32

±

0

.

27

83

.

69

±

0

.

00

94

.

90

±

0

.

36

94

.

72

±

0

.

33

92

.

12

±

4

.

53

97

.

15

±

0

.

25

97

.

68

±

0

.

10

97

.

35

±

0

.

32

UCI

65

.

99

±

1

.

40

54

.

79

±

1

.

76

68

.

67

±

0

.

84

70

.

94

±

0

.

71

64

.

61

±

0

.

48

57

.

43

±

0

.

00

76

.

01

±

1

.

11

80

.

10

±

0

.

51

57

.

55

±

0

.

81

83

.

61

±

0

.

23

72

.

25

±

1

.

71

77

.

26

±

1

.

57

Flights

69

.

07

±

4

.

02

70

.

57

±

1

.

82

75

.

48

±

0

.

26

71

.

09

±

2

.

72

69

.

18

±

1

.

52

81

.

08

±

0

.

00

74

.

62

±

0

.

18

74

.

87

±

0

.

21

59

.

16

±

1

.

48

61

.

99

±

1

.

70

70

.

92

±

1

.

78

64

.

78

±

1

.

50

48

.

42

±

0

.

66

58

.

61

±

0

.

86

68

.

82

±

1

.

21

65

.

34

±

2

.

87

67

.

75

±

1

.

00

62

.

16

±

0

.

00

65

.

85

±

1

.

75

69

.

48

±

0

.

63

78

.

03

±

1

.

27

59

.

15

±

6

.

20

95

.

44

±

0

.

57

85

.

59

±

3

.

08

US Legis.

50

.

27

±

5

.

13

83

.

44

±

1

.

16

61

.

91

±

5

.

82

67

.

57

±

6

.

47

65

.

81

±

8

.

52

64

.

74

±

0

.

00

78

.

15

±

3

.

34

79

.

63

±

0

.

84

88

.

40

±

2

.

34

67

.

78

±

4

.

29

81

.

25

±

3

.

62

91

.

05

±

1

.

21

UN Trade

60

.

42

±

1

.

48

60

.

19

±

1

.

24

60

.

61

±

1

.

24

61

.

04

±

6

.

01

62

.

54

±

0

.

67

72

.

97

±

0

.

00

61

.

06

±

1

.

74

60

.

15

±

1

.

29

78

.

38

±

2

.

24

69

.

72

±

1

.

84

55

.

79

±

1

.

02

86

.

61

±

0

.

99

UN Vote

67

.

79

±

1

.

46

67

.

53

±

1

.

98

52

.

89

±

1

.

61

67

.

63

±

2

.

67

52

.

19

±

0

.

34

66

.

30

±

0

.

00

50

.

62

±

0

.

82

51

.

60

±

0

.

73

72

.

53

±

1

.

94

68

.

37

±

1

.

92

51

.

91

±

0

.

84

75

.

05

±

1

.

41

Contact

93

.

43

±

1

.

78

94

.

18

±

0

.

10

94

.

35

±

0

.

48

90

.

18

±

3

.

28

89

.

31

±

0

.

27

85

.

20

±

0

.

00

91

.

35

±

0

.

21

90

.

87

±

0

.

35

91

.

13

±

1

.

48

94

.

05

±

0

.

52

94

.

75

±

0

.

28

95

.

84

±

0

.

32

Avg. Rank

9.08

8.62

5.92

6.23

7.62

7.77

6.69

6.38

7.31

5.08

4.46

2.85

AUC

Wikipedia

70

.

96

±

0

.

78

67

.

36

±

0

.

96

81

.

93

±

0

.

22

80

.

97

±

0

.

31

70

.

95

±

0

.

95

81

.

73

±

0

.

00

82

.

19

±

0

.

48

84

.

28

±

0

.

30

52

.

11

±

5

.

83

77

.

03

±

1

.

93

75

.

09

±

3

.

70

75

.

36

±

3

.

41

Reddit

83

.

51

±

0

.

15

82

.

90

±

0

.

31

87

.

13

±

0

.

20

84

.

56

±

0

.

24

88

.

04

±

0

.

29

85

.

93

±

0

.

00

84

.

67

±

0

.

29

82

.

21

±

0

.

13

76

.

01

±

1

.

04

81

.

02

±

0

.

68

86

.

23

±

0

.

51

81

.

64

±

0

.

42

MOOC

66

.

63

±

2

.

30

63

.

26

±

1

.

01

73

.

18

±

0

.

33

77

.

44

±

2

.

86

70

.

32

±

1

.

43

48

.

18

±

0

.

00

70

.

36

±

0

.

37

72

.

45

±

0

.

72

75

.

56

±

2

.

39

77

.

00

±

2

.

24

80

.

76

±

0

.

76

89

.

07

±

0

.

63

LastFM

61

.

32

±

3

.

49

62

.

15

±

2

.

12

63

.

99

±

0

.

21

65

.

46

±

4

.

27

67

.

92

±

0

.

44

77

.

37

±

0

.

00

46

.

93

±

2

.

59

60

.

22

±

0

.

32

58

.

54

±

2

.

37

69

.

51

±

0

.

61

69

.

25

±

0

.

36

72

.

48

±

0

.

90

Enron

70

.

92

±

1

.

05

68

.

73

±

1

.

34

60

.

45

±

2

.

12

71

.

34

±

2

.

46

75

.

17

±

0

.

50

75

.

00

±

0

.

00

67

.

64

±

0

.

86

71

.

53

±

0

.

85

69

.

62

±

1

.

12

78

.

91

±

0

.

96

74

.

07

±

0

.

64

75

.

44

±

1

.

38

Social Evo.

90

.

01

±

3

.

19

93

.

07

±

0

.

38

92

.

94

±

0

.

61

95

.

24

±

0

.

56

89

.

93

±

0

.

15

87

.

88

±

0

.

00

93

.

44

±

0

.

72

94

.

22

±

0

.

32

92

.

53

±

4

.

20

97

.

10

±

0

.

22

97

.

51

±

0

.

06

97

.

59

±

0

.

25

UCI

64

.

14

±

1

.

26

54

.

25

±

2

.

01

60

.

80

±

1

.

01

64

.

11

±

1

.

04

58

.

06

±

0

.

26

58

.

03

±

0

.

00

70

.

05

±

1

.

86

74

.

59

±

0

.

74

49

.

73

±

1

.

20

79

.

42

±

0

.

31

65

.

96

±

1

.

18

70

.

85

±

0

.

96

Flights

69

.

99

±

3

.

10

71

.

13

±

1

.

55

73

.

47

±

0

.

18

71

.

63

±

1

.

72

69

.

70

±

0

.

75

81

.

10

±

0

.

00

72

.

54

±

0

.

19

72

.

21

±

0

.

21

59

.

73

±

1

.

61

59

.

61

±

1

.

65

69

.

53

±

1

.

17

64

.

21

±

1

.

30

52

.

88

±

0

.

80

63

.

53

±

0

.

65

72

.

47

±

1

.

18

69

.

57

±

2

.

81

72

.

93

±

1

.

78

61

.

41

±

0

.

00

69

.

47

±

2

.

12

70

.

52

±

0

.

94

80

.

03

±

1

.

75

65

.

30

±

6

.

98

96

.

70

±

0

.

59

85

.

05

±

2

.

71

US Legis.

59

.

05

±

5

.

52

89

.

44

±

0

.

71

71

.

62

±

5

.

42

78

.

12

±

4

.

46

76

.

45

±

7

.

02

68

.

66

±

0

.

00

82

.

54

±

3

.

91

84

.

22

±

0

.

91

93

.

04

±

1

.

35

79

.

09

±

3

.

45

87

.

96

±

1

.

80

94

.

48

±

0

.

50

UN Trade

66

.

82

±

1

.

27

65

.

60

±

1

.

28

66

.

13

±

0

.

78

66

.

37

±

5

.

39

71

.

73

±

0

.

74

74

.

20

±

0

.

00

67

.

80

±

1

.

21

66

.

53

±

1

.

22

80

.

77

±

1

.

31

75

.

76

±

1

.

54

62

.

56

±

1

.

51

89

.

56

±

0

.

87

UN Vote

73

.

73

±

1

.

61

72

.

80

±

2

.

16

53

.

04

±

2

.

58

72

.

69

±

3

.

72

52

.

75

±

0

.

90

72

.

85

±

0

.

00

52

.

02

±

1

.

64

51

.

89

±

0

.

74

77

.

36

±

1

.

52

74

.

11

±

2

.

51

53

.

37

±

1

.

26

79

.

13

±

1

.

16

Contact

94

.

47

±

1

.

08

94

.

23

±

0

.

18

94

.

10

±

0

.

41

91

.

64

±

1

.

72

87

.

68

±

0

.

24

85

.

87

±

0

.

00

91

.

23

±

0

.

19

90

.

96

±

0

.

27

92

.

51

±

1

.

16

94

.

71

±

0

.

33

95

.

01

±

0

.

15

95

.

93

±

0

.

23

Avg. Rank

8.08

8.23

6.77

6.31

7.31

7.00

7.00

6.54

7.46

5.08

5.00

3.23

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The abstract and introduction list the limitations of existing relative encodings, and the goal of this paper is to tackle such limitations.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We discuss the limitation in the appendix.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: All theorems include the full sets of assumptions and proof.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We provide the implementation details of our method in the appendix.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We include our codes in the supplemental material.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Implementation details are shown in the appendix.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in the appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We report the mean and standard deviation in tables that record the experimental results.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We list the compute resources of our experiments in the appendix.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: Our paper follows the NeurIPS Code of Ethics in every respect.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We discuss the potential societal impacts in the appendix.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [Yes]

Justification: The proposed method does not have a high risk for misuse.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We follow the license of each asset and cite them properly.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [No]

Justification: We do not provide new assets in this paper.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: This paper does not include crowdsourcing experiments and research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: This paper does not include crowdsourcing experiments and research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a framework for analysis of relative encodings as a function of random walk matrices, and a new model for temporal link prediction. The new model offers SOTA performance on multiple link prediction datasets, and achieves this performance more efficiently than current best models. The design is of the model is explained in detail.\n\nSTRENGTHS:\nThis is an overall good paper which proposes and evaluates a new SOTA approach for temporal link prediction. It also introduces an original framework for unifying a range of existing methods.\n\n1. The provided code is sufficiently commented, and detailed instructions for setup are provided, along with utility functions to start the training process.\n2. Extensive evaluation has been done of the proposed model, and SOTA performance across a range of datasets has been demonstrated.\n3. Current methods have been posed in the newly proposed framework, which is an interesting analytical contribution.\n\nClearly presented and significant contribution to the field of temporal link prediction.\n\nWEAKNESSES:\n1. The limitations of the approach are only briefly discussed in the appendix.\n2. Although the code is well presented, it did not run out of the box for me. Some effort was required to install additional dependencies not listed in the requirements.txt, and to fix a runtime error in the sampling algorithm of the DataLoader.\n\nMinor points / text linting suggestions:\n\nLine 81: What is a \"unified formation\", did you mean \"formulation\"?\n\nLine 94: Style \"all the interactions happen\" -> \"all the interactions that happen\"\n\nLine 128: Doesn't A^(i) aggregate the i-step temporal walks, instead of k-step? Typo?\n\nLine 187: \"The value of score function..\" -> \"The value of the score function\".\n\nLine 202: What is d? The node degree or dimensionality? Please define. Also in line 2, Algorithm 1.\n\nLine 203: Typo enumerating the H vectors. The index 1 is repeated, whereas it should be followed by 2.\n\nLine 205: Style \"Pseudocode code\" -> \"Pseudocode\"\n\nLine 211: Having the integer \"i\" in the exponent of e could be misleading for a reader who is skimming to get an overview of the paper. This letter is usually reserved as the imaginary unit when exponentiating e, so ideally pick another letter as the index.\n\nLine 261: \"and obtained\" -> \"and obtain\"\n\nQUESTIONS:\n1. Line 265: Why does ReLU reduce estimation error? Is this an empirical result, or an architectural consideration specific to your data? Please clarify.\n\n2. Line 43: The second argument sounds a bit vague. Could specific examples be mentioned for methods that use and don't use temporal information in constructing their embeddings?\n\n3. Minor point on code: distutils has been removed in python 3.12 and above. To run the code I've had to install the following dependencies in addition to the ones listed: setuptools, numba, sklearn. I've had to fix the following issue.\n\n```{python}\nset(random.sample(test_node_set, int(0.1 * num_total_unique_node_ids)))\n```\nHad to be changed to:\n```{python}\nset(random.sample(sorted(test_node_set), int(0.1 * num_total_unique_node_ids)))\n```\n\t\t\nThis happened installing your code in a new python 3.12 environment.\n\nLIMITATIONS:\nThe limitations are briefly discussed in the appendix.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-10"}

comments: SUMMARY:
Based on the analysis of traditional methods, this paper proposes a unified framework for relative encoding and introduces a new temporal neural network, TPNet. This model not only addresses the high time complexity issues of traditional methods but also enhances relative encoding by incorporating factors such as time decay effects.

STRENGTHS:
1、The authors propose a unified framework for relative encoding, treating them as a function of temporal random walk matrices.

2、The temporal random walk matrix not only takes into account temporal and structural information but also incorporates the effects of time decay.

3、To reduce computational complexity, the authors proposed an approximation scheme for the temporal random walk matrix, detailed in Algorithm 1.

4、The authors conducted thorough axiomatic proofs, providing solid theoretical support for the effectiveness of the TPNet network

WEAKNESSES:
1、If the authors could provide a comparison of memory usage, it would make the model more convincing.

2、This paper does not provide an analysis of the impact of different functions g on relative encoding.

3、The author did not compare with the latest models (published in AAAI 2024 or WWW 2024) in Table 1.

QUESTIONS:
please see weaknesses

LIMITATIONS:
n/a
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-11

comments: SUMMARY:
The article investigates the application of relative encodings in the task of link prediction over temporal networks. Initially, the authors formally unify previously used relative encodings, such as DyGFormer, PINT, NAT, and CAWN, within a unique framework. Subsequently, they propose a novel model, TPNet, and conduct comparative evaluations using several datasets and competitors.

STRENGTHS:
The provided framework offers a clearer and broader perspective on existing approaches based on temporal walks. The source code is well-commented and likely to be highly usable. The experimental setup investigates various competitors, datasets, and negative sampling techniques.

WEAKNESSES:
A fundamental concept in the article is "relative encoding." However, the intuition behind this concept is not introduced until line 119. I recommend presenting the meaning of "relative encoding" earlier in the article for better clarity and understanding.

QUESTIONS:
If I understand correctly, the authors introduce a weight decay mechanism to exponentially decrease the weight of older interactions. While this decision is well-motivated by previous studies, it is well known that the recurrence of interactions is fundamentally important in social networks (e.g., sociopatterns.org). I am curious whether the temporal periodicity of interactions is maintained with this approach. Could the authors discuss this aspect in more detail?

LIMITATIONS:
The authors discussed the limitations of the model in the appendix.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-11