Paper: Deep graph kernel point processes (ICLR-2024)

Reviewer comment (Novelty):
The main concern is the contribution of this article. Many previous works have tried to study point processes in the context of GNN (for example, see [1]). From this perspective, this article is not innovative enough.

Author response:
We thank the reviewer for the positive comments and confirmation of our experiments. 

Regarding weakness, we have now provided a comprehensive comparison with related works in the literature, in Table A.1, and have explained the contribution/novelty of our paper in detail in Appendix A. 

We want to point out that the extension to the graph kernel is non-trivial, and a lot of modeling and experiment validation development needs to be done. We have considered various types of graph kernels based on various GNNs and also consider the case when the graph topology is known, and when the graph topology is unknown, then we use the GAT. We provided extensive numerical experiments and compared them with multiple baselines and SOTA. In particular, since point process data over graph is so widely available, and there is still a lack of principled graph kernel design, we believe this work will be a valuable addition to the literature.

Reviewer comment (Novelty):
Based on comparison with existing work, would you mind further elaborating on the contribution of this article?

Author response:
We thank the reviewer for the positive comments and confirmation of our experiments. 

Regarding weakness, we have now provided a comprehensive comparison with related works in the literature, in Table A.1, and have explained the contribution/novelty of our paper in detail in Appendix A. 

We want to point out that the extension to the graph kernel is non-trivial, and a lot of modeling and experiment validation development needs to be done. We have considered various types of graph kernels based on various GNNs and also consider the case when the graph topology is known, and when the graph topology is unknown, then we use the GAT. We provided extensive numerical experiments and compared them with multiple baselines and SOTA. In particular, since point process data over graph is so widely available, and there is still a lack of principled graph kernel design, we believe this work will be a valuable addition to the literature.

Reviewer comment (Novelty):
The good accuracy achieved by the proposed model is practically important, but the technical contribution of the model seems to be somewhat marginal because it is a reasonable but mediocre idea to use a GNN to model the inter-mark relationship in inference kernels, and any technical difficulty is not seen in the training algorithm.

Author response:
We thank the reviewer for the positive comments. We now have clarified in Appendix A that incorporating GNN in the graph-based influence kernel is crucial and non-trivial, requiring a carefully designed model. Our paper is the first one to incorporate GNN in the influence kernel without any parametric constraints.

Reviewer comment (Theory):
Any limitations of the proposed model are not discussed. For example, it seems that the intensity function of the proposed model could be negative even if the log-barrier method is adopted, while the conventional methods (e.g., RMTTP) are designed not to worry about it.

Author response:
We now have discussed the potential limitation in the Conclusion of the paper. As we have explained to the above reviewer, current kernel representation still assumes that the influence over events is "additive," which is common in the kernel-based type of methods for point processes. To be more general, one can potentially adopt a non-additive model over events that can be more complex, and we leave it for future work.

Reviewer comment (Reproducibility):
To the best of my knowledge, (Omi et al., 2019) doesn’t consider marks of each event in the model. How was FullyNN implemented in the experiment as a marked point process model?

Author response:
Indeed, the reference (Omi et al. 2019) did not consider marks, and their model can only be used for temporal point process. In order to compare with this method for the graph setting, we further extend their method to model the mark distribution on top of FullyNN. We have now provided complete details of the extended FullyNN in Appendix H.2.

Reviewer comment (Reproducibility):
Graph construction, including the definition of nodes and edges, is pivotal in GNN-based methods. The paper needs to provide a detailed explanation of how edges between nodes are established. Are the edge construction methods applicable universally across different scenarios, or do they require case-specific adaptations? While the process is well-defined for synthetic datasets, it is less clear in real-world data scenarios. The assertion that edges represent potential interactions should be elaborated upon to ensure clarity.

Author response:
We would clarify that for real-world datasets, we adopt GAT in our framework to infer the underlying graph structure and node interactions based on the learned attention weights from data, since the underlying graph structure is unobserved in real-world datasets and we have no prior knowledge about the graphs. The latent graph structure is constructed upfront in synthetic datasets.

Reviewer comment (Theory):
The authors use THP-S and SHAP-G as baselines. However, these baselines employ powerful transformers and graph attention mechanisms. It would be expected that they offer significant flexibility. This work should provide a robust justification for why the proposed model outperforms these baselines. Is there a theoretical basis for the superior performance?

Author response:
We first would point out that the model performance of neural point processes is affected by various factors (*e.g.*, deep neural network architectures, model formulation of the point process, specific problems), and so far, a general theoretical basis can hardly be established.

Meanwhile, we would clarify there is no GNN architecture incorporated in \texttt{THP-S} and \texttt{SHAP-G}, and they model the conditional intensity function instead of the influence kernel, which may not be able to capture the repeated influence patterns effectively like our method.

Reviewer comment (Evaluation):
The discrepancy between the log-likelihood, which is similar to the baseline, and the substantially improved time prediction MAE is perplexing. I am suspicious about the accuracy of the MAE calculation. I recommend the authors clarify how this probabilistic prediction model is employed for predictions – whether it is based on random sampling or mean predictions.

Author response:
The predictability of the proposed model is evaluated through sampling. As we clarify in the paper, we use the learned model on the training set to generate new event sequences over the entire time horizon, and compare the generated sequences with the testing sequences, in terms of the predicted event frequency and the predicted distribution of event types. The time MAE is calculated as the mean absolute error between the temporal frequency of generated events and testing events.

Reviewer comment (Evaluation):
In the evaluation using real-world data, Table 3 indicates that the advantage of the graph-based approach diminishes as the number of nodes increases. This observation runs counter to my expectation that graph modeling should excel in complex graph structures. The authors should provide a detailed analysis and potential explanations for this outcome.

Author response:
We would point out that the comparison of the absolute difference of performance metrics across different point process datasets has little practical meaning, for they have diverse problem settings and data features. More importantly, our proposed method enjoys superior performance advantages against baselines consistently across different datasets.

Reviewer comment (Experiments):
Lack of crucial baseline comparisons. A robust baseline comparison is essential for a comprehensive evaluation of the proposed method. The absence of such comparisons is a notable limitation in the paper. For instance, even without incorporating a graph, models like THP [5], and DAPP [6] possess the inherent capability to capture inter-event interactions. Additionally, there are intensity-free methods [7] available in the literature. Hence, it is necessary to compare the proposed model against state-of-the-art point process models [1,2,3,4,5,6,7].  
- [5] Zuo, Simiao, et al. "Transformer hawkes process." International conference on machine learning. PMLR, 2020.  
- [6] Zhu, Shixiang, et al. "Deep fourier kernel for self-attentive point processes." International Conference on Artificial Intelligence and Statistics. PMLR, 2021.  
- [7] Shchur, Oleksandr, Marin Biloš, and Stephan Günnemann. "Intensity-Free Learning of Temporal Point Processes." International Conference on Learning Representations. 2019.

Author response:
We would highlight that we have already compared with several state-of-the-arts that the reviewer listed, and have highlighted the results in the paper's experiments (Section 4).

Reviewer comment (Writing):
It will be good to add some background information on the graph-based kernels so that the paper is more self-contained.

Author response:
We thank the reviewer for the positive comment and acknowledge the value of our proposed model. We have added additional background information on the graph-based kernels to make the paper more self-contained.

Reviewer comment (Theory):
It will also be helpful if the authors can comment on what types of network interactions can be modeled and what the limitations of such representation are, so that readers will have a better idea of the advantages and limitations of the proposed model.

Author response:
Regarding "what types of network interactions can be modeled and what the limitations of such representation are." Currently, we assume the kernel is not "rank-one," with more than one pair of basis functions (over time and graph), the temporal kernel can allow non-stationarity over time, and the graph basis is completely general to enable efficient computational; we assume each basis function is decoupled over "time" and "graph" which is also intuitive and commonly used in the literature; compared with prior models, which usually is "rank-one" our model can be much more expressive while striking similar computational complexity. 

One potential limitation is that the current kernel representation still assumes that the influence over events is "additive," which is common in the kernel-based type of methods for point processes. To be more general, one can potentially adopt a non-additive model over events that can be more complex, and we leave it for future work. We have now discussed the limitation in the Conclusion section.

Reviewer comment (Theory):
On page 4, the basis functions of $g_d(\cdot,\cdot)$ typically need to be orthogonal. Can you comment on how to ensure the orthogonality of basis functions using the fully connected neural network? If orthogonality is not imposed, how do you ensure the uniqueness of the decomposition?

Author response:
Although the design of the kernel is motivated by Mercer's theorem explained in Sec. 3.2, and the basis function is represented using neural networks. In practice, we do not need orthogonality for the basis function. The reason can be explained potentially by the fact that as long as the basis function spans the space, it will be a good representation of the kernel; the training of the neural networks is done using stochastic gradient descent, which rarely leads to any nearly linearly dependent basis functions. In our experiment, we have chosen the rank $r$ to be large enough such that the empirical performance is good and validated in Appendix G.