# Exploring the Performance of Continuous-Time Dynamic Link Prediction Algorithms
## 6 Results
### Which Competing Negative Sampling Strategy Misleads the Model More over Time?  
In the previous experiment, we observed that using a different NS strategy can dramatically alter the prediction results. However, it is crucial to note that performance evolves over time in a temporal graph. In this section, we demonstrate how distinguishing different NS techniques can help provide a more nuanced insight into the performance of each method over time.  
In Figure 5, we observe the performance over time against edge-NS. For all methods and datasets, we note that the rank of the positive events increases over time while the rank of the other edges decreases. Indeed, as memory is filled over time, more events will appear to be likely to occur. In particular, the score of randomly sampled historical and overlap edges will rise in general as compared to the score of the positive event.  
A general trend is that the rank of inductive edge/destination NS (green and red lines, respectively) increases on the training period, before dropping on the test set. These types of negative samples will eventually be the ones which will confuse the model during test time, as their rank becomes increasingly closer to the rank of the positive event after \(t_{split}\). In general, historical and overlap items follow the opposite trend: they become more likely up to \(t_{split}\), and their rank then starts increasing again either at \(t_{split}\), or shortly before (notably for historical edges/destinations).  
These plots make it clear that the _type_ of error made by the models changes over time. Indeed, while on the train set, inductive edges are likely to be scored low (less likely) since they have not been observed yet; they become much more likely on the test set. Evenduring the test period, for instance, looking at EdgeBank on the UCI datasets in Figure 5, shortly after \(t_{split}\), the model will tend to rank the positive events lower than the overlap and historical edges. After some time, however, the inductive edge negative samples will tend to be ranked similarly to the historical and overlap edges.  
These visualizations give profound insights on the differences in performance between TGN and DyRep. For instance, on the Lastfm dataset, TGN seems to be more consistent in ranking the inductive destination as high compared with the other destination nodes. However, its overall performance on the test set is not clearly better than DyRep. On the other hand, in Figure 6, the performances on the UCI dataset indicate that, while TGN is clearly able to push historical destinations further than the inductive destination, DyRep tends to assign them similar rankings across the test set.  
Figure 5: MAR of each method over time. The positive event “Pos” is ranked against negative events resulting from different negative edge sampling strategies as defined in Section 4.2: historical edge, overlap edge, and inductive edge.  
In both Figures 5 and 6, the examples on the UCI data allow us to visualize the effect of seasonality on the performance. Indeed, for both methods, the rank of the positive event peaks just before the split time. This time corresponds roughly to the lower activity that is also observed in Figure 2g.  
To conclude, the results shown in Figures 4-6 demonstrate that an overly optimistic AUC may hide a more intricate reality, especially when taking simple baselines as a reference point. While more targeted NS strategies help to identify settings where heuristics outperform proposed heavier models, plotting the performance over time is important to obtain an idea of how a model's performance will react to domain-specific changes in the data over time.  
Figure 6: MAR of each method over time. The positive event “Pos” is ranked against negative events resulting from different negative destination sampling strategies as defined in Section 4.2: historical destination, overlap destination, inductive destination.