# Predicting Dynamic User-Item Interaction with Meta-Path Guided Recursive RNN
## 5 Experiment
### Baselines and Evaluation Metrics  
We compare the proposed method with the following baseline models.  
* LSTM [45] is an important ingredient of RNN architectures. Here we simply record the sequence of items, dropping of the time information.
* Time-LSTM [45] is a new LSTM variant, which equips LSTM with time gates to model the time intervals.
* Jodie [36] is a coupled recurrent neural network model to learn the dynamic embeddings of users and items. Here we ignore the one-hot embedding for item in Jodie, because it cannot be utilized in a large number of items.
* NGCF [28] is a recommendation framework based on a graph neural network, which explicitly encodes the collaborative signal in user-item bipartite graph by performing embedding propagation.
* LightGCN [31] is a state-of-the-art collaborative filtering based method. It simplifies the design of GCN to make it more concise and appropriate for recommendation.
* RRNN-S [1] is a recent state-of-the-art recursive RNN based shift embedding model for predicting dynamic user-item interaction.  
We use 80% data for training, 10% for validation, and the remaining 10% for testing. We adopt mean reciprocal rank (MRR) and Recall@K defined as follows as the evaluation metric. **MRR** is a statistic measure for evaluating any process that produces a list of possible responses to a sample of queries, ordered by the probability of correctness. It is the average of the reciprocal ranks of results for a sample of queries Q: \(MRR=\frac{1}{|\mathcal{O}|}\sum_{i=1}^{|\mathcal{O}|}\frac{1}{rank_{i}}\), where \(rank_{i}\) refers to the rank position of the first relevant document for the \(i-th\) query. **Recall@K** measures the fraction of the total amount of relevant instances that are actually retrieved.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Data** & **\#Users** & **\#Items** & **\#Interactions** \\ \hline Wikipedia & 8227 & 1000 & 157,474 \\ Reddit & 10,000 & 1000 & 672,447 \\ JingDong & 10,692 & 303,150 & 1,198,735 \\ \hline \hline \end{tabular}  
* **Wikipedia editing dataset:** This dataset contains one month of edits made on Wikipedia pages. We select the 1000 pages that get the most edits as items and editors who made at least 5 edits as users. In total, we have 8227 users. There are 157,474 interactions between the selected users and pages in total, and the edited text is considered as features.
* **Reddit post dataset:** This dataset includes one month of posts made by users on subreddits. We select the 1000 most active subreddits as items and the 10,000 most active users. There are 672,447 interactions and the text of each post is converted to a feature vector.
* **JingDong dataset:** This dataset is extracted from JD.com, which contains the records of users’ online behaviors on the JingDong website. It contains 1,198,735 interactions between 10,692 users and 303,150 items from March 2020 to April 2020.  
\end{table}
Table 2: Statistics of the two datasets.  
The embedding dimension is set to 128, the learning rate is 0.001, and the model is trained with Adam optimizer with a weight decay of 0.00001. The loss curves of the training process for the two datasets are shown in Figure 5. It shows that the model converges quickly. Within around 10 epochs, the training loss first drops quickly and then becomes stable.