Title: Recurrent Temporal Revision Graph Networks
Venue: NeurIPS 2023
Accepted: yes
Keywords: temporal graph

temporal graph network

temporal graph model expressiveness

continuous-time dynamic graph
Abstract: Temporal graphs offer more accurate modeling of many real-world scenarios than static graphs. However, neighbor aggregation, a critical building block of graph networks, for temporal graphs, is currently straightforwardly extended from that of static graphs. It can be computationally expensive when involving all historical neighbors during such aggregation. In practice, typically only a subset of the most recent neighbors are involved. However, such subsampling leads to incomplete and biased neighbor information. To address this limitation, we propose a novel framework for temporal neighbor aggregation that uses the recurrent neural network with node-wise hidden states to integrate information from all historical neighbors for each node to acquire the complete neighbor information. We demonstrate the superior theoretical expressiveness of the proposed framework as well as its state-of-the-art performance in real-world applications. Notably, it achieves a significant +9.4% improvement on averaged precision in a real-world Ecommerce dataset over existing methods on 2-layer models.

Recurrent Temporal Revision Graph Networks

Abstract

Temporal graphs offer more accurate modeling of many real-world scenarios than static graphs. However, neighbor aggregation, a critical building block of graph networks, for temporal graphs, is currently straightforwardly extended from that of static graphs. It can be computationally expensive when involving all historical neighbors during such aggregation. In practice, typically only a subset of the most recent neighbors are involved. However, such subsampling leads to incomplete and biased neighbor information. To address this limitation, we propose a novel framework for temporal neighbor aggregation that uses the recurrent neural network with node-wise hidden states to integrate information from all historical neighbors for each node to acquire the complete neighbor information. We demonstrate the superior theoretical expressiveness of the proposed framework as well as its stateof-the-art performance in real-world applications. Notably, it achieves a significant + 9 . 6% improvement on averaged precision in a real-world Ecommerce dataset over existing methods on 2-layer models.

1 Introduction

Many real-world applications (such as social networks [28, 41], recommender systems [3, 29], traffic forecasting [18, 45] and crime analysis [11]) involve dynamically changing entities and increasing interactions, which can be naturally modeled as temporal graphs (also known as dynamic graphs) where nodes and edges appear and evolve over time. Compared with static graph solutions [13, 35, 9], dynamic graphs can be more accurate in modeling node states and inferring future events.

In modern graph representation learning, neighbor aggregation, i.e., generating higher-level node embeddings via referring to the embeddings of their neighbors [13, 35, 17], is a critical common ingredient that helps more effectively leverage the connectivity information in the graphs. Practically, however, relatively hot nodes may have rapidly growing lists of neighbors, which makes involving all neighbors in neighbor aggregation computationally expensive. In static graphs, SAGE [8] suggests randomly sampling d neighbors to approximate all neighbors. For temporal graphs, the prevalent approach is to consider only the most recent d neighbors [28, 33]. But such subsampling of neighbors

could lead to incomplete and biased information, as the neighbor information preceding the most recent d neighbors is permanently lost.

To mitigate the problem caused by neighbor subsampling, we propose an alternative solution to classic neighbor aggregation for temporal graphs, called recurrent temporal revision (RTR), which can be used as a standard building block (a network layer) for any temporal graph network. Each RTR layer incorporates a hidden state for each node to integrate the information from all its neighbors using a recurrent neural network (RNN). The hidden state will serve as the embedding of each node at that layer, embracing the complete neighbor information and having the flexibility to focus more on long-term or short-term behaviors depending on the characteristics of the problem / dataset.

The existence of such a hidden state implies that all historical neighbors have already been integrated through previous updates. So when a new edge event comes, we may build a message with the latest neighbor to update the hidden state through the RNN. On the other hand, taking the state updates of previous neighbors into account can also be beneficial, which could provide additional information including new interactions of these neighbors [28] as well as updates in the neighbor states due to the influence of their own neighbors. In other words, re-inputting neighbors would enable revising their previously provided information. Therefore, we propose a module called temporal revision (TR) to collect state updates of previous neighbors to update the hidden state together with its latest interaction. To better capture the state updates, besides their current states, we additionally input the revisions and state changes of neighbors into the temporal revision module. Subsampling of such state updates of neighbors can be less problematic, since the state of each neighbor has already been observed and integrated by the hidden state in the past.

Towards further enhancement, we propose to introduce heterogeneity in temporal revision, i.e., treating involved nodes non-uniformly, to boost its theoretical expressiveness. Heterogeneity can be naturally incorporated, since when a node collects revisions from its neighbors, the neighbors may, the other way around, collect the revision from the node (at a lower level), which is, however, directly accessible to the node itself. We thus introduce heterogeneity by recursively identifying and marking such self-probing nodes as specialties in revision calculation. We theoretically show that such heterogeneous revision leads to improved expressiveness beyond Temporal-1WL [33], and practically verify its potential to promote effective information exchange among nodes.

Our contributions can be summarized as follows:

We propose a novel neighbor aggregation framework for temporal graphs to tackle the challenges arising from the necessary subsampling of neighbors.

We propose two expressiveness measures for temporal graph models in terms of temporal link prediction and temporal graph isomorphism test, and characterize their theoretical connection. We prove theoretically the superior expressiveness of our proposed framework, as well as provide several new results on the expressiveness of other baselines.

We demonstrate in experiments that the proposed framework can significantly outperform state-ofthe-art methods. In particular, we can consistently get improvements as the number of aggregation layers increases, while previous methods typically exhibit only marginal improvements [28, 38].

An Ecommerce dataset is provided along with this submission, which can be valuable for evaluating temporal graph models in real-world settings.

2 Preliminaries

2.1 Temporal Graph Modeling

A temporal graph can be represented as an ordered list of timed events, which may include the addition, deletion, and update of nodes and edges. For simplicity, we will only discuss the handling of edge addition events in this paper, where newly involved nodes will be accordingly added and initialized. It can be easily extended to handle other events [28]. We denote the set of nodes and edges up to time t as V ( t ) and E ( t ) , respectively.

Modeling of temporal graph typically generates node embeddings that evolve over time as graph events occur, which can be used by downstream tasks such as link prediction [2, 29] and node classification [20, 4]. The base node embeddings can have various sources, including direct node

features [8, 35, 41], learnable parameters [9, 36], generated by other modules such as an RNN [28, 38, 33] or specifically designed module [3, 16, 22]). Modern temporal graph methods [28, 5, 33] typically employ additional layers on top of the base embeddings to (further) leverage the connectivity information within the graph, generating higher-level node embeddings with a wider receptive field that encompasses relevant edges and nodes.

2.2 (Temporal) Neighbor Aggregation

Among various approaches, neighbor aggregation is one of the commonly used techniques in various graph models as a critical ingredient, where the embedding of each node is updated by considering the embeddings of its neighbors. In the context of temporal graphs, the neighbor aggregation layer can be defined as follows [28, 38, 33]:

where t denotes the current time. e uv,t ′ denotes the event between node u and node v at time t ′ ( ≤ t ) . Φ ( e uv,t ′ ) encodes the event feature, as well as its related time and sequence information, which may include the current time, the event time, the current number of interactions of u and v , etc. AGG k and COMBINE k denote arbitrary learnable functions, which are typically shared at layer k . a k u ( t ) is the aggregated information from neighbors at the k -1 layer, while h k u ( t ) is the new embedding of node u at layer k , combining its current embedding at the k -1 layer with the aggregated information a k u ( t ) . This can be viewed as a straightforward extension of classic graph convolution layers [13, 35, 17] with additional time or sequence information.

2.3 Expressiveness of Temporal Graph Models

Expressiveness measure is crucial for identifying and comparing the capabilities of different models. While several studies have explored the expressiveness of (temporal) graph models [40, 5, 33, 39, 43], a widely accepted definition for the expressiveness of temporal graph models is currently lacking.

Towards addressing this gap and establishing a solid groundwork, we propose the following formal definition of expressiveness in terms of temporal graph isomorphism test and temporal link prediction:

Definition 1. (Temporal graph isomorphism indistinguishability). Given a pair of temporal graphs ⟨ G 1 ( t ) , G 2 ( t ) ⟩ , we say they are indistinguishable w.r.t.a model f , if and only if there exists a bijective mapping between nodes of G 1 ( t ) and G 2 ( t ) , such that, for each pair of nodes in the mapping, their embeddings generated by f are identical at any time t ′ ≤ t .

Definition 3. (Temporal graph isomorphism expressiveness). If a model f A can distinguish any pair of temporal graphs that model f B can distinguish, while being able to distinguish certain graph pair that f B fails, we say f A is strictly more expressive than f B in temporal graph isomorphism test.

Definition 2. (Temporal link prediction indistinguishability). Given a temporal graph G ( t ) and two pairs of nodes ⟨ ( u 1 , v 1 ) , ( u 2 , v 2 ) ⟩ therein representing two temporal link prediction problems, we say they are indistinguishable w.r.t. a model f , if and only if the embedding of u 1 and u 2 , v 1 and v 2 generated by f are identical at any t ′ ≤ t .

Definition 4. (Temporal link prediction expressiveness). If a model f A can distinguish any pair of temporal link prediction problems that model f B can distinguish, while being able to distinguish certain pair of temporal link prediction problems that f B fails, we say f A is strictly more expressive than f B in temporal link prediction.

We can similarly define equally expressive, more or equally expressive, and so on.

Through above, we have, for the first time, provided a consistent definition of expressiveness for both temporal graph isomorphism test and temporal link prediction. Besides, compared with existing definitions, ours is more concise and straightforward, avoiding the introduction of concepts such as the Identifiable Set as in [5] and Temporal Computation Tree as in [33].

Most existing results on the expressiveness of temporal graph models can be easily transferred to our definition. For example, message-passing temporal graph networks (MP-TGN) are strictly

(a)

(b)

h6(t)

O

RRRRR

o

RTRk

Classic

1

2

3

2

3

4

2

3

4

5

4

RMSGk

RRR

Round 1embedding

Round2embedding

Round3embedding

RRRRh(t-)

?

TR

0

0

o

3

4

5

+

HR

Ours

二

1

2

3

4

2

3

4

5

1

2

3

4

5

RTRk-1

:

Hidden state (cover all previous neighbors)

RMSG(state revising message)

Round3embedding

Base embedding h°(t)

more expressive in isomorphism test when the (temporal) neighbor aggregation layers are injective (referred to as IMP-TGN) [33]; and IMP-TGN would be as expressive as temporal 1-WeisfeilerLehman (Temporal-1WL) test [33, 39], if with sufficient many layers.

3 The Proposed Method

3.1 Recurrent Temporal Revision

In large graphs, having too many neighbors can pose practical computation challenges in neighbor aggregation. For static graphs, SAGE [8] suggests approximating neighbor information by randomly subsampling a fixed number ( d ) of neighbors. For temporal graphs, recent models [28, 33] typically involve the most recent d neighbors to reduce the computation cost of temporal neighbor aggregation, which often yields better performance compared with uniform sampling [28]. But such subsampling of neighbors fails to provide complete neighbor information and biases it towards short-term behaviors. The effects are illustrated in Figure 1(a).

Towards addressing this problem, we propose to introduce a hidden state h k u ( t ) in each aggregation layer k > 0 for each node to integrate information from all its historical neighbors under a state transition model, typically recurrent neural network (RNN). Since events in the temporal graph, and hence the addition of neighbors, occur sequentially, such integration can be done naturally.

Formally, when an event e uv,t occurs, we will need to update the hidden state h k u of u (and also v ). A straightforward implementation would be updating the hidden state with a message of the new event. However, taking the state updates of previous neighbors into account can also be beneficial, which can provide additional information including: the new interactions of neighbors, their state evolutionary directions, updates in the neighbor states due to the influence of their own neighbors. To achieve this, we introduce a module called temporal revision (TR) to collect state updates of previous neighbors, which will be later fused with the latest event to update the hidden state:

where TR k is an arbitrary learnable function shared at layer k , t -represents the time right before t . We collect not only the newest states of neighbors h k -1 v ( t ) at the k -1 layer, but also their state changes ∆ h k -1 v ( t ) = h k -1 v ( t ) -h k -1 v ( t -) and their revisions r k -1 v ( t ) , as well as the event information Φ ( e uv,t ′ ) , to help the model better extract the state update information. We set r 0 u ( t ) as zero vectors. When requesting the embedding of a node at the 0 -th layer, we refer to its base embedding. Please refer to Appendix A for more detailed discussions of the base embedding.

In subsequent ablation studies, we show that retaining ∆ h k -1 v ( t ) alone, while removing h k -1 v ( t ) , will not lead to degeneration of performance, which means the effectiveness in aggregation can rely

solely on the changes of states, while not necessarily on the neighbor states themselves. Temporal revision is hence substantially different from classic temporal aggregations [38, 28, 5, 33].

To encode all neighbor update information, we build a state revising message (RMSG) via:

where RMSG k is an arbitrary learnable function shared at layer k , integrating the node embedding at the k -1 layer h k -1 u ( t ) , the embedding of the new neighbor at the k -1 layer h k -1 v ( t ) , the collected neighbor state update information r k u ( t ) , as well as the event information Φ ( e uv,t ) . 3 Then the proposed recurrent temporal revision (RTR) layer can be formulated as:

3.2 Heterogeneous Revision

where the state revising message is consumed to update the hidden state h k u from time t -to t via UPDATE k , a learnable function shared at layer k responsible for hidden state transition. For all RTR layers, we initialize h k u ( 0 ) as zero vectors for all nodes.

Heterogeneous message passing can improve the expressiveness of static GNN [43], which can be naturally integrated into our revision computation.

The term r k u ( t ) recursively gathers revision from previously interacted nodes, and they may in turn gather the revision of node u (at a lower level), which is, however, directly accessible to u itself. Therefore, we may adopt a different treatment for such self-probing calculations during recursive aggregation. To achieve this, we may mark all nodes in the recursive computation path as specialties.

Formally, we define the heterogeneous revision as:

We show in the next section that incorporating such heterogeneous revision theoretically leads to enhanced expressiveness, which will be subsequently validated in our later experiments.

where we introduce an extra argument S for revision, which is a set that is initialized as empty ∅ and recursively adds each u encountered along the computation path. stands for the indicator function, which triggers the difference between nodes in S and the rest, bringing about heterogeneity in the revisions. To switch to heterogeneous revision, we replace r k u ( t ) in (4) with r k u ( t, ∅ ) . We set the base case r 0 u ( t, S ) as zero vectors for any u and S .

We illustrate the complete inner structure of an RTR layer in Figure 1(b). The implementation details, including the design of all these learnable functions, are provided in Appendix A.

3.3 Model Expressiveness

For model expressiveness, we assume our RTR layers are built on top of a base RNN. For a fair comparison, it is assumed the base RNNs for all models are the same and of the best expressiveness, as suggested in [5, 33]. We focus on RTR layers whose (learnable) functions in (4) - (6) are injective, which can be achieved by adopting similar approaches as in [33, 39]. We refer to such a model as RTRGN. Besides, we assume node and edge attributes come from a finite set as [25, 5, 33].

We will mainly compare RTRGN with PINT [33], which is an instance of IMP-TGN that can be equally expressive as Temporal 1-WL in isomorphism test. PINT also offers augmented positional features (referred to as PINT-pos) designed to enhance its expressiveness.

Our first theoretical result demonstrates that the temporal graph isomorphism test is inherently more challenging than temporal link prediction. All proofs are deferred to Appendix B.

Proposition 1. Being more expressive in temporal graph isomorphism test implies being more expressive in temporal link prediction ( f A ≻ is f B ⇒ f A ≻ lp f B ) but not vice versa ( f A ≻ lp f B ⇏ f A ≻ is f B ).

Expressiveness (

Link Prediction

)

Expressiveness (

Isomorphism Test

)

f

A

≻

is

f

B

f

A

≻

lp

f

B

(Prop. 1)

f

A

≅

is

f

B

f

A

≅

lp

f

B

(Corollary 1)

PINT-pos

≻

lp

PINT

≻

lp

MP-TGNs [33]

PINT

≅

is

Temporal-1WL [33]

Time-then-IMP

≅

lp

PINT

≅

lp

Temporal-1WL (Prop. 2)

Time-then-IMP

≅

is

PINT

≅

is

Temporal-1WL (Prop. 2)

RTRGN

≻

lp

PINT (Prop. 3)

RTRGN

≻

is

PINT (Prop. 3)

PINT-pos can be false positive (Prop. 4)

PINT-pos can be false positive (Prop. 4).

Most expressive: RNN + Temporal Relational Pooling (Prop. 5 in Appendix B)

The first part of this proposition can be proved by reducing the temporal link prediction problem into a temporal graph isomorphism test of the current graph adding the two temporal links separately. The second part of this proposition is proved by constructing a pair of two models: one is more expressive than the other in temporal link prediction, while being not more expressive in temporal graph isomorphism test. The proof and the construction of the counter-examples inside suggest that temporal graph isomorphism test is a more challenging task than temporal link prediction, as it requires global distinguishability rather than just local distinguishability of the involved nodes.

Next, we demonstrate that Time-then-IMP, the strongest among the three classes of temporal graph models suggested in [5] with the graph module of its time-then-graph scheme being injective message passing (IMP) to ensure 1WL expressiveness [39], is equally expressive as PINT and Temporal-1WL in both temporal graph isomorphism test and temporal link prediction.

Proposition 2. Time-then-IMP is equally expressive as PINT and Temporal-1WL in both temporal graph isomorphism test and temporal link prediction.

The proof establishes that all these models are equally expressive as Temporal-1WL in both temporal link prediction and temporal graph isomorphism test. The proposition suggests that Time-then-IMP, although utilizes a distinct form of base RNN to encode all historical features of nodes and edges (which comes at the cost of significantly more space needed to embed each edge), its expressiveness 4 is ultimately equivalent to the Temporal-1WL class, which includes models like PINT.

We next show that RTRGN is more expressive than PINT, as well as the Temporal-1WL class:

Proposition 3. RTRGN is strictly more expressive than the Temporal-1WL class represented by PINT in both temporal graph isomorphism test and temporal link prediction.

As we have shown, the expressiveness of all the previously mentioned models is bounded by the Temporal-1WL, which, although powerful, can fail in certain cases. To illustrate this, we construct a temporal graph isomorphism test task called Oscillating CSL (Figure 2) where all Temporal-1WL models, including Time-then-IMP, fail. The difficulty of the task stems from the regular structure of the graph snapshots and stacked graphs at each timestamp, which cannot be distinguished by Temporal-1WL tests. RTRGN excels in this task due to the heterogeneity introduced in its revision computation, which enables its expressiveness to go beyond the limit of Temporal-1WL.

Proposition 4. PINT-pos can be false positive in both temporal graph isomorphism test and temporal link prediction, e.g., may incorrectly classify isomorphic temporal graphs as non-isomorphic.

On the other hand, we observe that PINT-pos can produce false positives (unlike the other models). Specifically, there are cases where PINT-pos mistakenly categorizes isomorphic temporal graphs as non-isomorphic, and incorrectly distinguishes two temporal links that should not be distinguishable. It fails essentially because its positional feature is permutation-sensitive, while all other mentioned models are permutation-invariant, which is a desirable property in graph model design [5, 25]. This further highlights the superiority of heterogeneous revision over augmented positional features.

Finally, we formulate a most expressive model that can always output the ground truth (Appendix B).

t

t

= {

t

1

,

t

2

,

t

3

,

t

4

, …}

t

= {

t

1

,

t

3

, …}

t

= {

t

2

,

t

4

, …}

1

t

2

C

7,2

3

3

4

5

6

7

1

2

3

4

5

6

7

1

2

C

7,2

…

3

4

5

6

7

1

2

C

7,3

t

1

t

2

C

7,3

t

3

3

4

5

6

7

1

2

3

4

5

6

7

1

2

C

7,3

…

3

4

5

6

7

1

2

C

7,2

3

4

5

6

7

1

2

3

4

5

6

7

1

2

t

4 Related Works

Early approaches in dynamic graphs learning focused on discrete-time dynamic graphs (DTDGs), which consist of a sequence of graph snapshots usually sampled at regular intervals [27, 6, 42]. They employed techniques such as stacking graph snapshots and then applying static methods [19, 1, 10, 31] and assembling snapshots into tensors for factorization [44, 23]. Among them, ROLAND [42] introduced hierarchical hidden states, but each state update is followed by a heavy finetuning of the entire model, which can be computationally expensive when applied to continuous-time dynamic graphs (CTDGs). In comparison, our framework is directly applied to CTDGs, handling the lists of timed events, and can be much more efficient.

DTDGs can always be converted to CTDGs without information loss. However, the high computation cost poses a major obstacle for applying most DTDG methods to CTDGs. More and more research focuses on CTDGs. Sequence-based methods such as DeepCoevolve [3], Jodie [16], and Dyrep [34] utilize RNN to capture the evolution of node representations, while some other works [26, 24] employ random walks to model CTDG, including CAWN [37]. NAT [21] caches the high-order neighbor information for computing joint neighbor features.

TGAT [38] and TGN [28] were the first to introduce GNN for CTDGs. Subsequently, combining GNNs with RNNs for temporal graph embedding generation became a mainstream idea. PINT [33] introduced MLP-based injective aggregators to replace attention-based aggregators in TGN. GRUGCN[5] analyzed three strategies to combine GNN with RNN and classified them into time-and-graph (e.g., [30, 17, 2]), time-then-graph (e.g., [38, 28]), and graph-then-time frameworks (e.g., [27, 7, 29]).

Our heterogeneous revision leads to enhanced expressiveness. Similar ideas were adopted in static GNNs. For example, [43] introduced heterogeneity in the root node during message passing.

5 Experiments

5.1 Temporal Link Prediction on Benchmark Datasets

We evaluate the proposed RTRGN with the temporal link prediction task on 8 real-world datasets (7 public datasets as well as a private Ecommerce dataset that will be released along with this submission), and compare it against related baselines, including Dyrep [34], Jodie [16], TGAT [38], TGN [28], CAWN [37], NAT [21], GRU-GCN [5] and PINT [33]. The evaluation is conducted in two settings: the transductive which predicts links involving nodes observed during training, and the inductive which predicts links involving nodes that are never observed during training.

The official implementations of different baselines can vary in their experiment settings, especially on the dataset preparation and the negative edge sampling . To ensure a fair and consistent comparison, we calibrate the experiment settings of all baselines with the one used by TGN, which is the same as TGAT and PINT, but differs somewhat from other works such as CAWN and NAT. The latter may

±

±

±

±

±

±

±

±

±

±

±

±

±

±

±

±

lead to inflated results, with inductive performance surpassing transductive performance, potentially reaching over 99% accuracy in CAWN and NAT. We set d = 10 for our experiments. More details about the datasets, the baselines, and the experiment settings can be found in Appendix C.

This indicates that layer design is important for temporal graph networks to more effectively leverage the power of multiple layers. Another advantage, as shown in Figure 3, we found in experiments is that the training converges faster for larger k .

The results for the 1-layer setting ( k = 1 ) are presented in Table 2. It is evident that our proposed RTRGN surpasses all baselines by a significant margin, suggesting a substantial improvement. To delve deeper into the problem, we further examine the results for the 2-layer setting (Table 3), where we compare RTRGN against baselines that have demonstrated potentially better performance than other baselines in the 1-layer setting. As we can see, the performances of the baselines show limited improvement when transitioning from the 1-layer to 2-layer setting, aligning with previous findings [38, 28]. By contrast, RTRGN consistently exhibits significant improvements in the 2-layer setting: the average gap of transductive AP between RTRGN and the second best method is + 4 . 4% . The largest gap appears in our Ecommerce dataset, which is + 9 . 6% . The gap on Wikipedia is small probably due to a 1-layer model is already sufficient to give strong performance.

5.2 Temporal Graph Isomorphism Test

To validate the superior expressiveness of the proposed RTRGN against existing methods, we conduct experiments of temporal graph isomorphism tests on two synthetic datasets. We run 4-layer models on testing graphs, where at each time step, we compute and store the embedding for each node. To check the isomorphism between two temporal graphs under a model, we gather the lists of generated embeddings (stacking in time for each node) of the two graphs and verify whether there exists a bijective mapping such that the two lists are identical (via sort and scan). If the two lists are identical, we conclude that the model outputs isomorphic (negative); otherwise, non-isomorphic (positive).

Oscillating CSL is a synthetic temporal graph isomorphism test task, which is a commonly used benchmark for assessing the expressiveness of GNNs [25, 5]. Each graph in this task is composed of a sequence of graph snapshots (of length 6 ) that correspond to the oscillation between 2 different prototypes randomly selected from { C 11 , 2 , . . . , C 11 , 5 } . Here C N,s denotes a circular skip link (CSL) graph with N nodes and skip length s . The goal is to determine whether the temporal graph follows the pattern C N,s 1 → C N,s 2 is isomorphic to its reversed counterpart C N,s 2 → C N,s 1 (Figure 2).

W

Depth node

Wedge edge

Arrangement edge

Cuneiform Writing is a modified version of the Cuneiform graph dataset [14] (Figure 4), which consists of graphs representing Hittite Cuneiform signs. The dataset was originally derived from tablets, where individual wedges along with their arrangements were automatically extracted. We draw out arrangement edges with short spatial distances, while ensuring that each depth node has at most 2 arrangement edges. To transform static graphs into temporal graphs, we adopt sequential edge generation that adds arrangement edges first, then the wedge edges. For each task, we randomly select 2 cuneiform signs with an equal number of nodes. Refer to Appendix C for more details.

The results are shown in Table 4. RTRGN achieves perfect accuracy in both experiments, attributed to its heterogeneous revision (HR). Notably, PINT-pos fails entirely to correctly identify true negatives in the Oscillating CSL task. TGN and GRU-GCN on the other hand exhibit partial failures in accurately distinguishing challenging non-isomorphic graphs. One such failure case is illustrated in Figure 4.

5.3 Ablation Studies

We conduct a set of ablation studies to dissect the impact of each proposed component in our RTRGN. The experiments are conducted on three representative datasets: MovieLens, Wikipedia, and UCI. The results for MovieLens are presented in Figure 5, while the results for the remaining two datasets, which validate the consistency of our findings, are included in Appendix D.

<latexi sh

1_b

64="zWHcCB

O

J+

y

2YumG5

FQ

>A

VDLS

N

3U

vq

X

g

R

w

r

f0KZ

Mp

P

E

9

n

I8

d

/

T

7k

j

o

<latexi sh

1_b

64="JB0

X

3FK7

9

YpQ

>A

/H

c

VD

S

N

2

r

f

d

v

W

5

M

+

y

OZ

R

C

U

8

G

j

E

T

q

I

m

g

P

<latexi sh

1_b

64="Q+TGY/A

oru

U3

9

S

cW

M

>

B

n

VDL

N

F

2p

f

XH

J

E

k

m

Z

d

IO

zC

KR

v

y

P

j

q

7

5

0

g

8

w

<latexi sh

1_b

64="

UZd8q2F3PmQKS

f

I

Lg

>A

CBH

c

VD

N

J

v

M

u

kWo

0

pY

X

R

wT

yn

9

O

G

j

7/

r

5

z

E

+

<latexi sh

1_b

64="WKTMpwr

R

L+Zm

3f5

Sv

N8

>A

B/n

c

VD

F

2

UXH

J

Eu

k

o

d

IO

zC

Q

Y

G

y

P

j

q

7

0

9

g

<latexi sh

1_b

64="zWHcCB

O

J+

y

2YumG5

FQ

>A

VDLS

N

3U

vq

X

g

R

w

r

f0KZ

Mp

P

E

9

n

I8

d

/

T

7k

j

o

<latexi sh

1_b

64="JB0

X

3FK7

9

YpQ

>A

/H

c

VD

S

N

2

r

f

d

v

W

5

M

+

y

OZ

R

C

U

8

G

j

E

T

q

I

m

g

P

<latexi sh

1_b

64="Q+TGY/A

oru

U3

9

S

cW

M

>

B

n

VDL

N

F

2p

f

XH

J

E

k

m

Z

d

IO

zC

KR

v

y

P

j

q

7

5

0

g

8

w

<latexi sh

1_b

64="

UZd8q2F3PmQKS

f

I

Lg

>A

CBH

c

VD

N

J

v

M

u

kWo

0

pY

X

R

wT

yn

9

O

G

j

7/

r

5

z

E

+

<latexi sh

1_b

64="WKTMpwr

R

L+Zm

3f5

Sv

N8

>A

B/n

c

VD

F

2

UXH

J

Eu

k

o

d

IO

zC

Q

Y

G

y

P

j

q

7

0

9

g

<latexi sh

1_b

64="zWHcCB

O

J+

y

2YumG5

FQ

>A

VDLS

N

3U

vq

X

g

R

w

r

f0KZ

Mp

P

E

9

n

I8

d

/

T

7k

j

o

<latexi sh

1_b

64="JB0

X

3FK7

9

YpQ

>A

/H

c

VD

S

N

2

r

f

d

v

W

5

M

+

y

OZ

R

C

U

8

G

j

E

T

q

I

m

g

P

<latexi sh

1_b

64="Q+TGY/A

oru

U3

9

S

cW

M

>

B

n

VDL

N

F

2p

f

XH

J

E

k

m

Z

d

IO

zC

KR

v

y

P

j

q

7

5

0

g

8

w

<latexi sh

1_b

64="

UZd8q2F3PmQKS

f

I

Lg

>A

CBH

c

VD

N

J

v

M

u

kWo

0

pY

X

R

wT

yn

9

O

G

j

7/

r

5

z

E

+

<latexi sh

1_b

64="WKTMpwr

R

L+Zm

3f5

Sv

N8

>A

B/n

c

VD

F

2

UXH

J

Eu

k

o

d

IO

zC

Q

Y

G

y

P

j

q

7

0

9

g

Baselines and ours

Ablation on

state update

Ablation on revision

h

k

u

(

t

-

)

r

k

u

(

t

)

h

k

-

1

v

(

t

)

∆

h

k

-

1

v

(

t

)

r

k

-

1

v

(

t

)

Baselines and ours

Ablation on

state update

Ablation on revision

Baselines and ours

Ablation on

state update

Ablation on revision

h

k

u

(

t

-

)

r

k

u

(

t

)

h

k

-

1

v

(

t

)

∆

h

k

-

1

v

(

t

)

r

k

-

1

v

(

t

)

h

k

u

(

t

-

)

r

k

u

(

t

)

h

k

-

1

v

(

t

)

∆

h

k

-

1

v

(

t

)

r

k

-

1

v

(

t

)

k=2

100

100

100

一

AUC

95

95

95

TransductiveA

90

90

90

85

85

85

80

80

80

TGN

PINT

RTRGN

w/o

w/o

TGN

PINT

RTRGN

w/o

w/o

TGN

PINT

RTRGN

w/o

w/o

w/o

w/o

Ablation on state update. Our first ablation study is on h k u ( t -) , where we set h k u ( t -) = 0 in (5) for k > 0 , hence do not retain the hidden state but compute aggregation dynamically, denoted as w/o h k u ( t -) . The results suggest that including the hidden state is crucial for further improving performance when k > 1 . As the second ablation study, we set r k u ( t ) = 0 in (4), which leads to a pure RNNdesign without neighbor aggregation, denoted as w/o r k u ( t ) . It results in significant performance drops, indicating the criticality of integrating revisionary neighbor information in the message.

Ablation on TR module. The rest ablation studies are performed on the temporal revision module. As the first, we remove h k -1 v ( t ) in (6), denoted as w/o h k -1 v ( t ) . Interestingly, this setting exhibits only a marginal performance drop and even slightly outperforms the setting denoted as w/o ∆ h k -1 v ( t ) where ∆ h k -1 v ( t ) is removed instead. It suggests that the effectiveness of revision can primarily rely on the state change of neighbors, not necessarily the neighbor states. This may be attributed to that the hidden states have already incorporated such information through previous messages. As the last two settings, we remove r k -1 v ( t ) in (6), denoted as w/o r k -1 v ( t ) , and use the homogeneous version of revision (3) instead of the heterogeneous version (6), denoted as w/o HR. Both settings result in noticeable performance drops, indicating the benefits of incorporating revisions of neighbors in the TR module, as well as the introduction of heterogeneity in revision.

We provide the theoretical analysis of the time and space complexity of RTRGN in Appendix A. More experiments, e.g., ablation studies on subsampling amount d and practical run-time comparisons, can be found in Appendix D.

6 Conclusion

We have proposed a new aggregation layer, named recurrent temporal revision (RTR) for temporal graphs. We have shown that it leads to superior expressiveness that goes beyond Temporal-1WL. We have also demonstrated that it can lead to significant improvements over current state-of-the-art methods. As an important breakthrough, we have demonstrated that it enables effective utilization of multiple aggregation layers, while existing methods have difficulties in effectively harnessing the power of multiple layers.

References

Nahla Mohamed Ahmed and Ling Chen. An efficient algorithm for link prediction in temporal uncertain social networks. Information Sciences , 331:120-136, 2016.

Jinyin Chen, Xueke Wang, and Xuanheng Xu. Gc-lstm: Graph convolution embedded lstm for dynamic network link prediction. Applied Intelligence , pages 1-16, 2022.

Hanjun Dai, Yichen Wang, Rakshit Trivedi, and Le Song. Deep coevolutionary network: Embedding user and item features for recommendation. arXiv preprint arXiv:1609.03675 , 2016.

Songgaojun Deng, Shusen Wang, Huzefa Rangwala, Lijing Wang, and Yue Ning. Graph message passing with cross-location attentions for long-term ili prediction. arXiv preprint arXiv:1912.10202 , 2019.

Jianfei Gao and Bruno Ribeiro. On the equivalence between temporal and static equivariant graph representations. In Proc. ICML , pages 7052-7076. PMLR, 2022.

Palash Goyal, Sujit Rokka Chhetri, and Arquimedes Canedo. dyngraph2vec: Capturing network dynamics using dynamic graph representation learning. Knowledge-Based Systems , 187:104816, 2020.

Palash Goyal, Nitin Kamra, Xinran He, and Yan Liu. Dyngem: Deep embedding method for dynamic graphs. arXiv preprint arXiv:1805.11273 , 2018.

Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in NeurIPS , 30, 2017.

Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. Lightgcn: Simplifying and powering graph convolution network for recommendation. In Proc. SIGIR , pages 639-648, 2020.

Nahla Mohamed Ahmed Ibrahim and Ling Chen. Link prediction in dynamic social networks by integrating different types of information. Applied Intelligence , 42:738-750, 2015.

Guangyin Jin, Qi Wang, Cunchao Zhu, Yanghe Feng, Jincai Huang, and Jiangping Zhou. Addressing crime situation forecasting task with temporal graph convolutional neural network approach. In Proc. ICMTMA , pages 474-478. IEEE, 2020.

Seyed Mehran Kazemi, Rishab Goel, Sepehr Eghbali, Janahan Ramanan, Jaspreet Sahota, Sanjay Thakur, Stella Wu, Cathal Smyth, Pascal Poupart, and Marcus Brubaker. Time2vec: Learning a vector representation of time. arXiv preprint arXiv:1907.05321 , 2019.

Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 , 2016.

Nils M Kriege, Matthias Fey, Denis Fisseler, Petra Mutzel, and Frank Weichert. Recognizing cuneiform signs using graph based methods. In International Workshop on Cost-Sensitive Learning , pages 31-44. PMLR, 2018.

Nils M Kriege, Matthias Fey, Denis Fisseler, Petra Mutzel, and Frank Weichert. Recognizing cuneiform signs using graph based methods. In International Workshop on Cost-Sensitive Learning , pages 31-44. PMLR, 2018.

Srijan Kumar, Xikun Zhang, and Jure Leskovec. Predicting dynamic embedding trajectory in temporal interaction networks. In Proc. SIGKDD , pages 1269-1278, 2019.

Jia Li, Zhichao Han, Hong Cheng, Jiao Su, Pengyun Wang, Jianfeng Zhang, and Lujia Pan. Predicting path failure in time-evolving graphs. In Proc. SIGKDD , pages 1279-1289, 2019.

Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. arXiv preprint arXiv:1707.01926 , 2017.

David Liben-Nowell and Jon Kleinberg. The link prediction problem for social networks. In Proc. CIKM , pages 556-559, 2003.

Mingxuan Lu, Zhichao Han, Zitao Zhang, Yang Zhao, and Yinan Shan. Graph neural networks in real-time fraud detection with lambda architecture. arXiv preprint arXiv:2110.04559 , 2021.

Yuhong Luo and Pan Li. Neighborhood-aware scalable temporal network representation learning. In Proc. LoG , 2022.

Yao Ma, Ziyi Guo, Zhaocun Ren, Jiliang Tang, and Dawei Yin. Streaming graph neural networks. In Proc. SIGIR , pages 719-728, 2020.

Yunpu Ma, Volker Tresp, and Erik A Daxberger. Embedding models for episodic knowledge graphs. Journal of Web Semantics , 59:100490, 2019.

Sedigheh Mahdavi, Shima Khoshraftar, and Aijun An. dynnode2vec: Scalable dynamic network embedding. In 2018 IEEE International Conference on Big Data , pages 3762-3765. IEEE, 2018.

Ryan Murphy, Balasubramaniam Srinivasan, Vinayak Rao, and Bruno Ribeiro. Relational pooling for graph representations. In Proc. ICML , pages 4663-4673. PMLR, 2019.

Giang Hoang Nguyen, John Boaz Lee, Ryan A Rossi, Nesreen K Ahmed, Eunyee Koh, and Sungchul Kim. Continuous-time dynamic network embeddings. In Companion Proc. TheWebConf , pages 969-976, 2018.

Aldo Pareja, Giacomo Domeniconi, Jie Chen, Tengfei Ma, Toyotaro Suzumura, Hiroki Kanezashi, Tim Kaler, Tao Schardl, and Charles Leiserson. Evolvegcn: Evolving graph convolutional networks for dynamic graphs. In Proc. AAAI , volume 34, pages 5363-5370, 2020.

Emanuele Rossi, Ben Chamberlain, Fabrizio Frasca, Davide Eynard, Federico Monti, and Michael Bronstein. Temporal graph networks for deep learning on dynamic graphs. In ICML 2020 Workshop on Graph Representation Learning , 2020.

Aravind Sankar, Yanhong Wu, Liang Gou, Wei Zhang, and Hao Yang. Dysat: Deep neural representation learning on dynamic graphs via self-attention networks. In Proc. WSDM , pages 519-527, 2020.

Youngjoo Seo, Michaël Defferrard, Pierre Vandergheynst, and Xavier Bresson. Structured sequence modeling with graph convolutional recurrent networks. In Proc. ICONIP , pages 362-373. Springer, 2018.

Umang Sharan and Jennifer Neville. Temporal-relational classifiers for prediction in evolving domains. In Proc. ICDM , pages 540-549. IEEE, 2008.

Hava T Siegelmann and Eduardo D Sontag. On the computational power of neural nets. In Proc. COLT , pages 440-449, 1992.

Amauri Souza, Diego Mesquita, Samuel Kaski, and Vikas Garg. Provably expressive temporal graph networks. Proc. NeurIPS , 35:32257-32269, 2022.

Rakshit Trivedi, Mehrdad Farajtabar, Prasenjeet Biswal, and Hongyuan Zha. Dyrep: Learning representations over dynamic graphs. In Proc. ICLR , 2019.

Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, Yoshua Bengio, et al. Graph attention networks. stat , 1050(20):10-48550, 2017.

Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. Neural graph collaborative filtering. In Proc. SIGIR , pages 165-174, 2019.

Yanbang Wang, Yen-Yu Chang, Yunyu Liu, Jure Leskovec, and Pan Li. Inductive representation learning in temporal networks via causal anonymous walks. arXiv preprint arXiv:2101.05974 , 2021.

Da Xu, Chuanwei Ruan, Evren Korpeoglu, Sushant Kumar, and Kannan Achan. Inductive representation learning on temporal graphs. In Proc. ICLR , 2020.

Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In Proc. ICLR , 2019.

Keyulu Xu, Chengtao Li, Yonglong Tian, Tomohiro Sonobe, Ken-ichi Kawarabayashi, and Stefanie Jegelka. Representation learning on graphs with jumping knowledge networks. In Proc. ICML , pages 5453-5462. PMLR, 2018.

Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, and Jure Leskovec. Graph convolutional neural networks for web-scale recommender systems. In Proc. SIGKDD , pages 974-983, 2018.

Jiaxuan You, Tianyu Du, and Jure Leskovec. Roland: graph learning framework for dynamic graphs. In Proc. SIGKDD , pages 2358-2366, 2022.

Jiaxuan You, Jonathan M Gomes-Selman, Rex Ying, and Jure Leskovec. Identity-aware graph neural networks. In Proc. AAAI , volume 35, pages 10737-10745, 2021.

Wenchao Yu, Wei Cheng, Charu C Aggarwal, Haifeng Chen, and Wei Wang. Link prediction with spatial and temporal consistency in dynamic networks. In Proc. IJCAI , pages 3343-3349, 2017.

Jiani Zhang, Xingjian Shi, Junyuan Xie, Hao Ma, Irwin King, and Dit-Yan Yeung. Gaan: Gated attention networks for learning on large and spatiotemporal graphs. arXiv preprint arXiv:1803.07294 , 2018.

Peer reviews:
comments: SUMMARY:
Temporal graphs are more accurate for modeling real-world scenarios compared to static graphs, but the current approach of extending neighbor aggregation from static graphs to temporal graphs is computationally expensive when considering all historical neighbors. To address this, the authors propose a novel framework that uses recurrent neural networks with node-wise hidden states to integrate information from all historical neighbors for each node, ensuring complete neighbor information without subsampling biases.

STRENGTHS:
1. The model proposed by the authors looks reasonable, at least  their experiments suggest so.

2. The authors do a good job in describing the model. They propose two measures of expressiveness for temporal graph models, specifically in the context of temporal link prediction and temporal graph isomorphism tests. They establish a theoretical connection between these measures and demonstrate the superior expressiveness of their framework, while also presenting new findings on the expressiveness of other baseline methods.

3. The ablation studies display the positive effect of each mechanism.

WEAKNESSES:
Some concerns in this paper should be discussed precisely. 

1. In introduction, the heterogeneity in temporal revision is not explained.

2. Figure 1 lacks a specific description. The other baselines and the framework in this paper can be specified in the legend.

3. This paper lacks a description of Table 1. The authors can add a description of Table 1.

QUESTIONS:
1. Eq.3, 4 and 5 all refer to arbitrary learnable function, could you please specify what function is used in this paper?

2. In the experiment shown in Table 3, is it possible to display the results of all datasets?

LIMITATIONS:
The authors adequately addressed the limitations.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2023-06-24

comments: SUMMARY:
The paper studies the problem of temporal graph learning. The authors propose the recurrent temporal revision (RTR) layer, which involves learning a hidden state for each node and utilizing recursive neural networks to integrate information from all neighbors. Additionally, the authors introduce heterogeneity in RTR and theoretically demonstrate that this heterogeneous revision enhances expressiveness beyond Temporal-1WL.

STRENGTHS:
S1. The majority of the paper is easy to follow.

S2. Experimental results show that RTR outperforms state-of-the-art methods.

WEAKNESSES:
W1. My main concern is with the motivation of paper. Why do we need to make each node embrace complete neighbor information? In real-world recommendation system data, the browsing/purchasing tendencies of users are often more closely related to their short-term interests. The inclusion of excessive and outdated information may introduce more noise to the model instead.

W2. The proposed RTR can be understood as weight learning and important neighbor selection based on complete historical information. Could changing the sampling strategy from selecting k nearest neighbors to selecting k most important historical neighbors for methods like TGN also result in similar performance improvements?

W3. The paper lacks an analysis of time complexity as well as a comparison of the model's parameter count.

Minor comments

M1. Page 1, Line 16: social network, recommender system -> social networks, recommender systems

M2. Page 4, Line 160: an learnable function -> a learnable function

M3. Page 8, Figure 4 caption: TGN and GRU-GCN fails to -> TGN and GRU-GCN fail to

QUESTIONS:
Please refer to W1-W3.

LIMITATIONS:
I have some concerns about the scalability of the proposed method.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-07

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors propose a novel aggregation layer based on a recurrent neural network dubbed recurrent temporal revision (RTR) for temporal graph networks to solve the dynamic temporal graph embedding problem. Specifically, a new aggregation function is proposed, which encodes neighbor state information and event information. This paper also proposes two expressive measures and uses them to demonstrate theoretically the difference from previous work. Empirical results on several well-known datasets show the effectiveness of the proposed method.\n\nSTRENGTHS:\n1．\tThe explanations of the components of the approach are clear and detailed.\n2．\tThis work uses the proposed two measurements, and theoretically, fully justifies the difference from the previous methods. It also includes algorithm time complexity and running time analysis\n3．\tAblation settings are very detailed.\n\nWEAKNESSES:\n1.\tIt is unclear which part of the ablation experiment corresponds to which of the two proposed measurements. It would be good to clarify that.\n2.\tIt’s better to have an Algorithm line by line to clearly correspond to specific formulae and modules for Figure 1b.\n\nQUESTIONS:\n1.\tSome symbols and formulations need to be clarified:\n    a)\tEq. (4) should be accurately formulated.\n    b)\tr_t^0(t) in Line 148 should be a vector. \n2.\tWhich variance or function in Eq3 and Eq4 can reflect the two proposed measurements?\n3.\tThe organization of Section 3 should be improved. Each subsection ends abruptly without a clear takeaway message. \n4.\tTable 1 is hard to read.\n5.\tThe differentiation between the current work and previous works (especially TGAT, and TGS) need to be explained more clearly.\n6.\tWhy do the 2-layer experiments only run in selected datasets? What’s the selection criterion?\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-08"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a novel framework called Recurrent Temporal Revision (RTR) to address the challenge of neighbor subsampling in temporal graphs, an issue commonly encountered in real-world applications like social networking, recommender systems, traffic forecasting, and crime analysis. RTR serves as a standard building block for any temporal graph network, utilizing a recurrent neural network to integrate all neighbor information through a hidden node state. This hidden state mitigates the problem caused by neighbor subsampling and aims to capture complete neighbor information.\n\nThe paper also introduces a new concept of \"temporal revision\" to update the hidden state by capturing the state changes of neighbors. To further boost the theoretical expressiveness of this technique, the authors propose incorporating heterogeneity into temporal revision by recursively identifying and marking certain nodes as specialties in the revision calculation process.\n\nThe authors suggest that this new approach offers superior expressiveness in terms of temporal link prediction and temporal graph isomorphism test, a claim that is backed by theoretical proofs and experimental results. Additionally, they provide a new Ecommerce dataset for evaluating temporal graph models in real-world settings. Their experimental results indicate the proposed framework's significant improvement over state-of-the-art methods, particularly as the number of aggregation layers increases.\n\nSTRENGTHS:\n- The authors introduce innovative ways of evaluating the expressiveness of graph algorithms by defining two novel metrics: the temporal graph isomorphism test and the temporal graph link prediction. They apply these new measures to several state-of-the-art models, successfully establishing an equivalence between Temporal-1WL and these baseline models which implies the shared drawbacks of these models.\n- The paper presents a unique temporal learning model grounded on two inventive ideas: 1) Utilizing hidden states to retain aggregated information which is subsequently updated through a temporal revision layer, and 2) Embedding heterogeneity by giving special treatment to the central node during the recursive revisioning process. The authors compellingly argue that their proposed method outperforms state-of-the-art models in terms of expressiveness, as defined within the context of this paper.\n- The proposed method demonstrates promising performance in real-world temporal graph tasks, particularly in the temporal link prediction task. Additionally, it excels in the graph isomorphism test examples, reinforcing its potential for practical applications in various fields.\n\nWEAKNESSES:\n- The rationale behind the new definition of expressiveness isn't effectively conveyed in the paper. While it's true that there isn't a universally accepted definition for the expressiveness of a temporal graph model at present, the paper falls short in clarifying why its proposed isomorphism test improves upon existing ones. The lack of a clear explanation leaves a gap in understanding the superiority of the proposed metrics.\n\nQUESTIONS:\nFrom the proof of proposition 3, the extra expressiveness of the proposed model comes from the heterogenous revision which could be in principle added to other IMP-TGN representation model. Can we conclude that heterogeneity help improve expressiveness of temporal graph model?\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-08"}