SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\pi_ {i} = \operatorname {s o f t m a x} _ {i} \left(\left(\mathcal {T} \left(G, G _ {u v} ^ {\prime}\right) \| S _ {V N} (G)\right) \| S _ {V N} \left(G _ {u v} ^ {\prime}\right)\right) W _ {\pi}), \tag {17}
$$
where  $Y$  is the total number of experts,  $\pi_i$  is the mixing coefficient of expert  $i$ . Additionally,  $W_i \in \mathbb{R}^{4d \times 1}$  is the weight matrices for expert  $i$  and  $W_{\pi} \in \mathbb{R}^{3 \times Y}$  is the weight matrix of the gate unit. Finally,  $h_v$  and  $h_v'$  are the embeddings of node  $v$  in the original graph and the virtual time-evolved graph generated by the encoder.
3.4 Optimization
During training, we evaluated the convergence behavior of our model by monitoring the training and validation loss, ensuring that the model was not underfitting or overfitting. The loss function is:
$$
\ell = \sum_ {\left(v _ {i}, v _ {j}, t _ {i j}\right) \in \mathcal {E}} \log P \left(v _ {i}, v _ {j} \mid t _ {i j}\right) - \mathcal {Q} \cdot \mathbf {E} _ {\tilde {v} \sim P (\tilde {v})} \log P \left(v _ {i}, \tilde {v} \mid t _ {i j}\right), \tag {18}
$$
where  $(v_{i}, v_{j}, t_{ij})$  is an observed edge on the temporal network,  $\mathcal{Q}$  denotes the number of negative samples, and  $P(\tilde{v})$  is the negative sampling distribution over the node space  $\mathbf{E}$ .
3.5 Computational Complexity Analysis
This section highlights the efficiency of our approach in calculating approximate thermodynamic quantities of temporal networks. Based on Eq.3 and Eq. 5, the time complexity of computing approximate von Neumann entropy and the approximate temperature is  $O(|V|^2)$ , where  $|V|$  is the number of nodes in the network. Moreover, we compute the approximate thermodynamic quantities in the neighborhood for the large networks and set the sampled neighborhood size  $N$ . The computational complexity can be reduced to  $O(N^2)$  in this setting. Therefore, the time complexity demonstrates scalability and establishes the feasibility of our method for moderate or large networks. Since the time complexity is controllable this means we can achieve efficient computation.
Datasets. The temporal network datasets used in our experiments are divided into three categories: (a) QA: The "answers to questions"
Table 2: Statistics of the datasets.
Dataset	Nodes	Edges	Timespan
MathOverflow	21,688	107,581	2350 days
BitcoinOTC	5,881	35,592	1903 days
BitcoinAlpha	3,783	24,186	1901 days
Wikipedia	9,227	157,474	30 days
Table 1: Performance of AUC(%) for link prediction. The best results in each column are highlighted in bold font and the second-best results are underlined. We report the AP results in Appendix B.
Task	Methods	MathOverflow	BitcoinAlpha	BitcoinOTC	Wikipedia
Transductive	JODIE	86.07 ±0.48	91.14 ±0.18	92.29 ±0.11	93.58 ±2.00
DyRep	80.77 ±0.65	79.39 ±3.17	79.21 ±4.10	94.22 ±0.27
TGN	80.47 ±3.24	86.71 ±1.00	86.78 ±2.29	98.46 ±0.10
TGAT	71.80 ±0.91	78.99 ±0.50	79.53 ±0.67	95.34 ±0.10
CAW	53.82 ±0.28	64.70 ±0.93	73.95 ±1.22	98.96 ±0.10
TDLG	84.02 ±0.16	92.83 ±0.22	93.48 ±0.22	88.93 ±0.09
NeurTWs	92.56 ±0.51	93.95 ±0.41	95.75 ±0.01	94.54 ±0.87
ESSEN	98.60 ±0.40	99.10 ±0.16	98.88 ±0.42	99.03 ±0.33
Inductive	JODIE	67.06 ±0.42	74.47 ±0.16	76.21 ±0.47	91.44 ±1.99
DyRep	63.50 ±0.66	66.27 ±0.73	65.09 ±0.86	91.03 ±0.34
TGN	64.50 ±1.17	69.36 ±0.94	76.52 ±1.25	97.70 ±0.18
TGAT	60.02 ±0.75	66.42 ±1.17	66.62 ±1.99	93.99 ±0.30
CAW	57.67 ±0.33	64.38 ±1.01	72.99 ±0.46	98.75 ±0.14
TDLG	74.31 ±1.58	83.85 ±1.65	85.22 ±3.89	45.77 ±3.06
NeurTWs	91.83 ±0.13	94.20 ±0.26	96.08 ±0.38	94.63 ±0.47
ESSEN	98.33 ±0.28	98.07 ±0.64	98.67 ±0.31	98.80 ±0.10
dataset of MathOverflow. (b) Bitcoin trading data: BitcoinAlpha Dataset and BitcoinOTC Dataset [9; 8]. (c) Social networks: Wikipedia Dataset[10]. Table 2 gives more details concerning the properties of these datasets.
Baselines. In addition to reporting the performance of our ESSEN method, we report results for several popular dynamic graph learning methods, namely a) JODIE [10]; b) DyRep [21]; c) TGAT [25]; d) TGN [19]; e) CAW [24]; f) TDLG[4]; and g) NeurTWs[6]. We give more details concerning these baselines in Appendix C.
Link Prediction Task Settings. We evaluate our model on the link prediction task with two different settings:
- Transductive Setting. The model under the transductive setting is trained on the available nodes and their connections to predict future links between the nodes. The setting assumes that the network will not add unseen nodes at future testing times. This mainly evaluates the transductive ability of the model.
- Inductive Setting. The inductive setting predicts missing links for existing nodes together with potential new nodes that may be added at future times. This generalizes link prediction beyond the known nodes, considering the possibility of the addition of new nodes. In this way, it learns network patterns and characteristics to make predictions applicable to both known and unknown nodes.
Implementation Training Details. For each dataset, we used the training time points  $T_{tr} = 70\%$  to split the dataset results in approximately  $70\% - 15\% - 15\%$  of the total edges [25]. The principal hyperparameters are set as follows: a) the number of attention heads  $\mathcal{U} = \{2,3\}$ , b) the number of the GNN layers  $\mathcal{L} = 2$ , c) the maximum number of aggregated neighbors  $n \in \{60,80,100\}$ , d) the total number of experts in MoTE  $Y = \{4,6,8,10\}$ , and e) the dimension of the node embedding  $d_{n} = 172$ . We use the ADAM optimization algorithm for model training with a learning rate 1e-3 and batch size of 128. All the models are implemented in PyTorch and evaluated on a single Tesla A100 GPU.
Table 1 reports the transductive and inductive link prediction task results on four datasets, demonstrating the state-of-the-art performance of our method on link prediction tasks. Our method significantly outperforms all baselines on all datasets. In particular, in the MathOverflow dataset, compared with NeurTWs, the second strongest baseline, ESSEN improved the AUC(\%) by  $5.04\%$  and  $6.50\%$  on average in the transductive and inductive setting. The results demonstrate that our method has a
clear advantage for temporal networks. Specifically, our method performs well on both long and short-evolution time networks, while the effectiveness of baseline models varies significantly. CAW and TGAT have significant gaps in performance with MathOverflow and Wikipedia for all tasks. This indicates that our framework better represents networks with ever-changing evolution states. This superiority in performance can be attributed to a) our von Neumann entropy aware mechanism, b) the virtual evolution node representation learning, and c) the MoTE decoder. In addition, our method is effective under both transductive and inductive settings. By contrast, the baseline JODIE method cannot predict interactions well between unseen nodes because it pays more attention to node identities rather than the evolution states of temporal networks.
To validate the effectiveness of the novel elements comprising ESSEN, we conduct a series of ablation studies and report the AUC results. We investigate the performance of the proposed modules with three ablated models on the MathOverflow and BitcoinAlpha dataset: a) ESSEN-  $E$ , here we remove the von Neumann edge entropy bias in the attention mechanism of ESSEN. b) ESSEN-  $V$ , here we remove the virtual evolution node representation and only use the node embeddings of the original graph for decoding. c) ESSEN-  $D$ , here we replace the MoTE decoder with a simple MLP decoder. In Fig. 3, we can see that the performance degrades without considering the von
Figure 3: Ablation Study
Neumann entropy information for edges. This demonstrates the effectiveness of the proposed von Neumann entropy aware attention mechanism. Disabling the virtual evolution node representation also degrades performance. Furthermore, when the MoTE decoder is removed, the MathOverflow and BitcoinAlpha datasets exhibit more severe drops in performance, demonstrating that the MoTE decoder excels on temporal networks with a long time span and a greater number of evolution states.
4.4 Qualitative Analysis of Evolution States
In Fig. 4, we show curves of the von Neumann entropy change for different networks over the same time interval. Compared with the BitcoinOTC dataset, the MathOverflow leads to the generation of more nodes with high degrees, and these have higher entropy. The von Neumann entropy increases at varying rates, depending on the evolution states of the network in question. This reveals the potential relationship between von Neumann entropy and evolution states. Moreover, we can further understand network structure using von Neumann entropy. For evolution states with low entropy, networks tend to be tree-like or string-like and have more low-degree nodes. For networks with evolution states with high entropy there are more high-degree nodes and the networks tend to be fully connected.
Figure 4: Von Neumann entropy changes on temporal networks.
4.5 Parametric Sensitivity
We investigate the sensitivity of our ESSEN to various parameters and evaluate their impact on model performance. In Fig. 5, we report the results of this study and draw the following observations: a) By exploring different training time split points, we evaluate ESSEN's performance with lower training samples and more testing samples. Our model sustains excellent performance even when the number of samples in the training set is reduced. The results clearly show the robustness of ESSEN for complex evolution states. b) Considering the number of experts  $Y$ , both datasets have "sweet spots". This finding indicates that different datasets exhibit a preference for specific numbers of experts in the MoTE decoder. This can be attributed to the varying complexity of evolution states in the temporal network. c) We observe a strong correlation between the evolution-aware neighborhood size  $N$  and the number of nodes in the temporal networks. The link prediction performance decreases when  $N \ll |V|$ . For example, the total number of nodes in the MathOverflow dataset is 21688, which is also the maximum value of  $|V|$  because nodes will add and deleted over time. The AUC results for the MathOverflow dataset reduced from  $98.56\%$  to  $95.43\%$  when varying  $N$  from 200 to 50. This
(a) Training Time Point  $T_{tr}(\%)$
(b) Number of Experts  $Y$
Figure 5: Study on important settings. We report the results of the inductive link prediction.
(c) Sampled Neighborhood Size  $N$
result shows a promising trade-off between computational efficiency and the loss of information due to sampling.
Temporal Network Representation Learning. Network representation learning is often used to transform large networks into lower-dimensional vectors. For instance, CTDNE [15] learns node embedding from a continuous-time dynamic network instead of a sequence of snapshots. Besides, JODIE [10] uses two recurrent neural networks (RNNs) to learn trajectories of users and items and updates the embedding when interaction occurs. Expanding on this, TGAT [25] utilizes a self-attention mechanism and presents an encoding method to learn inductively. Additionally, CAW [24] and NeuralTWs [6] learn temporal structure using random walk. Specifically, CAW proposes a new anonymization strategy, and NeuralTWs considers structural and tree traversal properties. TDLG[4] aims to model the edges in temporal networks directly instead of calculating from node embedding. Despite these advancements, capturing the global network evolution states within acceptable time complexities remains a formidable challenge.
Von Neumann Entropy of Static Graph. Von Neumann graph entropy serves as a pivotal descriptor of a network system's statistical state [3; 16; 17]. Researchers like De Domenico et al. [5] have employed von Neumann graph entropy for structural reduction in multiplex networks, showcasing its versatility. Li et al. [11] explored convergence using von Neumann entropy in the context of network-ensemble comparison, illuminating its potential applications. Moreover, Liu et al. [13] delved into universal patterns of the dynamic genome through von Neumann graph entropy, broadening the scope of its utility. Wang [23] took a different route by approximating von Neumann graph entropy with node degrees, effectively modeling network evolution. Despite these endeavors, the application of von Neumann entropy in representing temporal structures has remained largely unexplored. To the best of our knowledge, our work pioneers the incorporation of von Neumann entropy in temporal network representation learning, opening new avenues for research and exploration in this domain.
In this paper, we propose ESSEN, an evolution states aware network for recognizing and analyzing the evolution states in temporal networks. We addressed the limitations of existing methods in capturing the time-varying nature of network structures, especially in complex evolution states. Our framework incorporates a von Neumann entropy aware attention mechanism and network evolution state contrastive learning for graph encoding. The decoding stage utilizes a unique decoder referred to as the Mixture of Thermodynamic Experts (MoTE). We evaluated ESSEN on link prediction tasks under transductive and inductive settings and compared it to state-of-the-art baselines. The experimental results demonstrate the effectiveness of our proposed method in capturing temporal dynamics where it outperforms existing approaches. Our work contributes to advancing the field of temporal network analysis and opens up possibilities for future research in both alternative domains and under additional network dynamics. In the future, we will focus on improving ESSEN's efficiency and scalability, allowing it to handle larger datasets and real-time analysis.
This work is supported by the National Natural Science Foundation of China (62176227, U2066213); the Fundamental Research Funds for the Central Universities (20720210047). We would like to thank the anonymous reviewers for their kind comments and valuable suggestions.
[20] Amauri Souza, Diego Mesquita, Samuel Kaski, and Vikas Garg. Provably expressive temporal graph networks. Advances in Neural Information Processing Systems, 35:32257-32269, 2022.
[21] Rakshit Trivedi, Mehrdad Farajtabar, Prasenjeet Biswal, and Hongyuan Zha. Dyrep: Learning representations over dynamic graphs. In ICLR, 2019.
[22] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[23] Jianjia Wang. Statistical Mechanics for Network Structure and Evolution. PhD thesis, University of York, 2018.
[24] Yanbang Wang, Yen-Yu Chang, Yunyu Liu, Jure Leskovec, and Pan Li. Inductive representation learning in temporal networks via causal anonymous walks. In ICLR, 2021.
[25] Da Xu, Chuanwei Ruan, Evren Korpeoglu, Sushant Kumar, and Kannan Achan. Inductive representation learning on temporal graphs. In ICLR, 2020.
[26] Cheng Ye. Entropic characterization and time evolution of complex networks. PhD thesis, University of York, 2016.
[27] Cheng Ye, Richard C Wilson, César H Comin, Luciano da F Costa, and Edwin R Hancock. Approximate von neumann entropy for directed graphs. Physical Review E, 89(5):052804, 2014.
[28] Muhan Zhang and Yixin Chen. Link prediction based on graph neural networks. Advances in neural information processing systems, 31, 2018.
[29] Lekui Zhou, Yang Yang, Xiang Ren, Fei Wu, and Yueting Zhuang. Dynamic network embedding by modeling triadic closure process. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
A Additional Algorithm Details
A.1 The Derivation of Approximate Von Neumann Entropy on Temporal Network
In the following, We commence by summarizing the approximation of the undirected graph von Neumann entropy presented by [27]. First, we introduce the von Neumann entropy, which can be computed from the normalized Laplacian spectrum as follows:
$$
S _ {V N} (G) = - \operatorname {T r} (P \log P) = - \sum_ {i = 1} ^ {| V |} \frac {\lambda_ {i}}{| V |} \log \frac {\lambda_ {i}}{| V |}, \tag {19}
$$
where  $\lambda_1, \ldots, \lambda_{|V|}$  are the eigenvalues combinatorial Laplacian matrix. Scaling the normalized Laplacian matrix by the reciprocal of its trace, we obtain a density matrix  $\frac{\hat{L}}{|V|}$ . The eigenvalues of the density matrix is  $\left(\frac{\hat{\lambda}_1}{|V|}, \frac{\hat{\lambda}_2}{|V|}, \ldots, \frac{\hat{\lambda}_{|V|}}{|V|}\right)$  and thus the von Neumann entropy of density matrix associated with the normalized Laplacian matrix of the graph is defined as
$$
S _ {V N} (G) = - \sum_ {j = 1} ^ {| V |} \frac {\hat {\lambda} _ {j}}{| V |} \ln \frac {\hat {\lambda} _ {j}}{| V |}. \tag {20}
$$
The von Neumann entropy above relies on the computation of the normalized Laplacian spectrum, therefore its computational complexity is cubic in the number of nodes. The Taylor expansion for  $\ln \frac{\lambda_i}{|V|}$  is
$$
\left(\frac {\hat {\lambda} _ {j}}{| V |} - 1\right) - \frac {1}{2} \left(\frac {\hat {\lambda} _ {j}}{| V |} - 1\right) ^ {2} + \frac {1}{3} \left(\frac {\hat {\lambda} _ {j}}{| V |} - 1\right) ^ {3} - \frac {1}{4} \left(\frac {\hat {\lambda} _ {j}}{| V |} - 1\right) ^ {4} + \dots . \tag {21}
$$
If we keep the first item of the expansion and discard the remaining that contribute to a small amount,  $\ln \frac{\hat{\lambda}_i}{|V|}$  is approximated using  $\left(\frac{\hat{\lambda}_j}{|V|} - 1\right)$ . Then the entropy  $S_{VN}(G)$  can be replaced by the quadratic entropy  $\sum_j \frac{\hat{\lambda}_j}{|V|} \left(1 - \frac{\hat{\lambda}_j}{|V|}\right)$ , then we obtain
$$
\begin{array}{l} S _ {V N} (G) = - \sum_ {j} \frac {\hat {\lambda} _ {j}}{| V |} \ln \frac {\hat {\lambda} _ {j}}{| V |} \simeq \sum_ {j} \frac {\hat {\lambda} _ {j}}{| V |} \left(1 - \frac {\hat {\lambda} _ {j}}{| V |}\right) \tag {22} \\ = \frac {1}{| V |} \sum_ {j} \lambda_ {j} - \frac {1}{| V | ^ {2}} \sum_ {j} \lambda_ {j} ^ {2}. \\ \end{array}
$$
Using the fact that  $\mathrm{Tr}\left[\hat{\mathbf{L}}^k\right] = \sum_j\hat{\lambda}_j^k$  , the quadratic entropy can be rewritten as
$$
S _ {V N} (G) = \frac {\operatorname {T r} [ \hat {L} ]}{| V |} - \frac {\operatorname {T r} \left[ \hat {L} ^ {2} \right]}{| V | ^ {2}}. \tag {23}
$$
The normalized Laplacian matrix  $\hat{L}$  has unit diagonal elements. For the trace of the normalized Laplacian matrix we have
$$
\operatorname {T r} \left[ \hat {L} \right] = | V |. \tag {24}
$$
Similarly, for the trace of the square of the normalized Laplacian, we have
$$
\begin{array}{l} \operatorname {T r} \left[ \hat {L} ^ {2} \right] = \sum_ {u \in V} \sum_ {v \in V} \hat {L} _ {u v} \hat {L} _ {u v} = \sum_ {u \in V} \sum_ {v \in V} \left(\hat {L} _ {u v}\right) ^ {2} \\ = \sum_ {u, v \in V u = v} \left(\hat {L} _ {u v}\right) ^ {2} + \sum_ {\substack {u, v \in V \\ u \neq v}} \left(\hat {L} _ {u v}\right) ^ {2} = | V | + \sum_ {(u, v) \in e} \frac {1}{d _ {u} d _ {v}}. \tag{25} \\ \end{array}
$$
Substituting Eq.24 and Eq.25 into Eq.23, the entropy becomes
$$
S _ {V N} (G) = \frac {\operatorname {T r} [ \hat {L} ]}{| V |} - \frac {\operatorname {T r} \left[ \hat {L} ^ {2} \right]}{| V | ^ {2}} = \frac {| V |}{| V |} - \frac {| V |}{| V | ^ {2}} - \sum_ {(u, v) \in e} \frac {1}{| V | ^ {2} d _ {u} d _ {v}} = 1 - \frac {1}{| V |} - \frac {1}{| V | ^ {2}} \sum_ {(u, v) \in e} \frac {1}{d _ {u} d _ {v}}. \tag {26}
$$
We project the temporal network to the time-independent 2-D plane as an edge-weighted graph, resulting in a simplified depiction of the underlying network structure at a given time. As a result, The expression of the approximate entropy is quadratic in the number of nodes.
Table 3: Performance of  $\mathrm{AP}(\%)$  for link prediction. The best results in each column are highlighted in bold font and the second-best results are underlined.
Task	Methods	MathOverflow	BitcoinAlpha	BitcoinOtc	Wikipedia
Transductive	JODIE	84.95 ±0.43	90.32 ±0.19	91.50 ±0.19	92.95 ±2.27
DyRep	80.97 ±0.25	79.42 ±2.23	78.95 ±2.76	94.63 ±0.20
TGN	81.51 ±1.73	86.47 ±0.42	88.76 ±1.70	98.52 ±0.09
TGAT	74.35 ±0.29	79.18 ±0.54	79.53 ±0.84	93.18 ±0.13
CAW	61.40 ±0.28	71.27 ±0.87	79.29 ±0.76	98.82 ±0.12
TDLG	82.87 ±0.16	91.19 ±0.24	92.24 ±0.25	87.25 ±0.15
NeurTWs	93.07 ±0.54	94.14 ±0.24	96.17 ±0.08	96.01 ±0.52
Ours	98.60 ±0.26	99.06 ±0.20	98.83 ±0.47	99.04 ±0.26
Inductive	JODIE	68.58 ±0.49	75.02 ±0.20	77.44 ±0.14	89.33 ±5.04
DyRep	65.65 ±0.44	66.54 ±1.04	65.94 ±0.86	91.94 ±0.27
TGN	67.04 ±1.42	70.52 ±1.06	79.74 ±1.21	97.83 ±0.16
TGAT	62.77 ±0.64	67.09 ±0.88	68.32 ±1.84	94.18 ±0.43
CAW	64.79 ±0.31	70.70 ±0.93	78.21 ±0.29	99.11 ±0.13
TDLG	70.18 ±2.16	79.53 ±3.19	80.95 ±6.88	53.47 ±2.41
NeurTWs	92.68 ±0.40	94.16 ±0.27	96.44 ±0.34	96.12 ±0.22
Ours	98.41 ±0.17	97.91 ±0.69	98.55 ±0.33	98.83 ±0.10
B.1 Performances in Average Precision
Table 3 reports the detailed transductive and inductive link prediction results of AP.
B.2 Time Comparison
Fig. 6 compares the training times of ESSEN against the second-strongest baseline NeurTWs. For fairness, we use the same batch size for both models and experiment in the same environment. Note that the running time of ESSEN is down quickly because the approximate thermodynamic quantities have been computed at the first epoch and use cache after that.
Figure 6: Time Comparison
C Experimental Setting
We introduce the datasets used in this paper as follows. In Fig. 7, we report the degree distribution on the 30th Day and 270th Day in datasets BitcoinOTC and MathOverflow. In order to align the timestamps, we shift the time so that they begin with zero. Additionally, we renumber the nodes to optimize space usage. Further details regarding the preprocessing steps undertaken to prepare these datasets for our method are discussed below.
(a) MathOverflow: 30th Day
(b) MathOverflow: 270th Day
(c) BitcoinOTC: 30th Day
(d) BitcoinOTC: 270th Day
Figure 7: Degree distribution at the 30th Day and 270th Day in different networks.
- MathOverflow dataset. It is a temporal network of interactions on the stack exchange website Math Overflow. The nodes represent users, and the edges represent the answers to questions. For example, a directed edge  $(u,v,t)$  represents user  $u$  answered user  $v$ 's question at time  $t$ . Since edge features and node features are not provided, we use the all-zero vectors instead.
- BitcoinOTC dataset.3 It is a who-trusts-whom network of people trade using Bitcoin on the BitcoinOTC platform. Each line records a trade from rater to rate and the rating ranges from -10 to +10 in step 1.
- Bitcoin Alpha dataset. It is a similar network to the Bitcoin Alpha platform. Both of the datasets have no edge features or node features, so we initialize them as all-zero vectors.
- Wikipedia dataset. It is a dataset of edited records from Wiki pages over a month. We use the top-edited pages and active users as nodes, and each row in our data represents a user editing a page. This dataset records user editing of pages over the course of a month. The timestamp indicates the time when the user edited the page. As with the Reddit dataset, the features of these nodes were processed through LIWC. The user labels indicate if users are temporarily banned from editing.
The introduction of baselines and their setting details are shown as follows. Baselines not specially mentioned use the default settings of the cited paper.
- JODIE uses two recurrent neural networks (RNNs) to learn trajectories of users and items, and updates the embedding when the interaction occurs. We set the number of epochs to 50 and the dimensions of node and time embedding to 100.
- DyRep is a temporal point process model capturing both topological evolution and nodes' activities. we set the number of epochs to 50 and the patience for early stopping to 5.
- TGAT utilizes a self-attention mechanism and presents a novel encoding method to learn graph embedding inductively. The batch size is 200 and the number of epochs is 50. We set the dimensions of node and time embedding to 100 and 20 neighbors are sampled in aggregation.
2https://snap.stanford.edu/data/sx-mathoverflow.html