# Predicting Dynamic User-Item Interaction with Meta-Path Guided Recursive RNN
## 5 Experiment
### Results  
#### 5.3.1 Comparison with Baselines  
Table 3 shows the results of our model and the baseline models. We observe that MRRNN-S outperforms the baselines in most cases on the three datasets. It is also worth noting that our model performs significantly better than other models on the JingDong dataset, which is a relatively larger dataset. The best results are highlighted with bold font, and the best results achieved by baselines are underlined.  
Among the baselines, LSTM only uses the interaction order information in the item sequence, and it does not take the time interval length of two successive interactions into consideration. As a variant of LSTM, Time-LSTM incorporates the time interval information into sequence data learning. Thus, Time-LSTM outperforms LSTM by 12.7% in the Wikipedia dataset and 12.2% in the JingDong dataset. Compared with LSTM and Time-LSTM, NGCF and LightGCN are collaborative filtering-based methods, which are not able to capture the time information in the temporal user-item interaction network. Thus the two methods are not suitable for the dynamic user-item interaction prediction and their performance is the worst among all the methods. Therefore, LSTM and Time-LSTM perform better than NGCF and LightGCN. Jodie considers dynamic and sequentially dependence between user-item interactions, which means richer auxiliary information can be absorbed into the model for a more accurate prediction. RRNN-S is able to capture the user-user and item-item relationships which is helpful to learn the similarity between nodes from high-order proximity. It improves the performance by 4.2%, 2.1%, and 9.2% on the three datasets compared with Jodie, respectively. One can also observe that MRRNN-S not only obtains sequence patterns from sequence data but also explores semantic information and structural information from heterogeneous graph data. Thus MRRNN-S can acquire more  
Figure 5: The training loss curves of MRRNN-S over the three datasets.  
\begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline
**Datasets** & **Metric** & **LSTM** & **Time-LSTM** & **Jodie** & **NGCF** & **LightGCN** & **RRNN-S** & **MRRNN-S** \\ \hline \multirow{3}{*}{Wikipedia} & MRR & 0.332 \({}^{*}\) & 0.351 \({}^{*}\) & 0.741 \({}^{*}\) & - & - & 0.756 \({}^{*}\) & **0.779 \({}^{*}\)** \\  & Recall@10 & 0.401 \({}^{*}\) & 0.452 \({}^{*}\) & 0.803 \({}^{*}\) & 0.198 \({}^{*}\) & 0.248 \({}^{*}\) & **0.837 \({}^{*}\)** & 0.801 \({}^{*}\) \\ \multirow{3}{*}{Reddit} & MRR & 0.343 \({}^{*}\) & 0.355 \({}^{*}\) & 0.718 \({}^{*}\) & - & - & 0.733 \({}^{*}\) & **0.766 \({}^{*}\)** \\  & Recall@10 & 0.530 \({}^{*}\) & 0.575 \({}^{*}\) & 0.832 \({}^{*}\) & 0.254 \({}^{*}\) & 0.287 \({}^{*}\) & 0.842 \({}^{*}\) & **0.856 \({}^{*}\)** \\ \multirow{3}{*}{JingDong} & MRR & 0.039 \({}^{*}\) & 0.047 \({}^{*}\) & 0.080 & - & 0.089 \({}^{*}\) & **0.264 \({}^{*}\)** \\  & Recall@10 & 0.057 \({}^{*}\) & 0.064 \({}^{*}\) & 0.131 \({}^{*}\) & 0.010 \({}^{*}\) & 0.013 \({}^{*}\) & 0.142 \({}^{*}\) & **0.343 \({}^{*}\)** \\ \hline \hline \end{tabular}
\end{table}
Table 3: Overall performance w.r.t. Recall@10 and MRR. “\({}^{*}\)” indicates the improvement of the MRRNN-S over the baseline is significant at the level of 0.05.  
detailed auxiliary information which results in MRRNN-S outperforming RRNN-S. As the size of the dataset increases (the number of interactions in the dataset), the performance improvement of model MRRNN-S becomes more and more obvious, especially in the JingDong dataset.