Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: This work proposes a new marked temporal point process model that manages to leverage neural networks in such a way to recover and identify the influence that certain events have on future ones. This is accomplished by modeling the overall intensity function of the model as a summation of individual decaying effects that prior events have on the rate of occurrence for future events.
main_review: There is a definite lack of interpretability in the methods proposed by the neural point process literature. Point process models are very powerful predictive tools; however, many of the desired use cases for them require interpreting why certain events are more likely to occur given a particular history (e.g., within healthcare). From my perspective, this work makes a solid step towards addressing this issue. The proposal is well founded, has a clear motivation, and adequately demonstrates the effectiveness via empirical results. 

 As for clarity, the main thrust of the paper is very clear. I do think that from about lines 180 to 200 and the entire synthetic data experiment section is a little dense and could stand to be edited for readability (I do understand that this is most likely a byproduct of the submission page limit).

I am curious about a few things that weren't directly answered in the paper:
1. What is the added effect of treating the embeddings probabilistically? Do the results differ significantly if, for instance, all of the parameters are learned deterministically without any priors? I would imagine that the results of at least 6.3 would be affected, but I wonder if this is actually the case empirically. Note that I do think the proposal is significant due to the formulation of the intensity function alone. 
2. What sort of structure do $\phi_g$ and $\phi_\eta$ take on? I understand that they are neural networks, but I cannot find any details about their hyper parameters for the experiments (i.e., the number of layers they have, activation function, initial values, etc.).
3. For the experiments using real data, were the sequences truncated at all or were the entire sequences modeled during training and inference?
limitations_and_societal_impact: I think that there could be a bit more discussion surrounding the societal impact of this model.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper proposes a semi-parametric class of point processes which is intensity-based while using kernel-based,non-parametric functions to model the effects between types of events in the intensity function. This reduces the number of the learnable parameters hence making the model applicable to datasets with a large number of event types.  In particular, it 

(1) formulates in Lemma 3.1 the property of such kernel functions that guarantee a time-decaying effect of the influence functions when there is present either an excitatory effect or an inhibitory effect 

(2) proposes a class of kernel functions in corollary 3.1.1 . that respects the property in (1)

(3) suggests a training algorithm that leverages a bayesian linear regression model for constructing the kernel functions and stochastic variational inference on batches of types and event sequences for accelerating inference.
main_review: In general, I think the paper is well-written while it addresses an important problem currently not very -well addressed by current research. Its novelty lies in the design of kernel functions that exhibit time-decaying effect. The experimental results are compelling. However, some points in the model/ training algorithm are not very clear while the experiments should be extended to further demonstrate the model. In particular,

(1) the model uses a soft-plus transformation to guarantee the positivity of the intensity function while maintaining the inhibitory effects imposed by the term in Eq 4. My impression is that this transformation renders the integral in the objective function of Eq 9 not computable in closed form. How do the authors address this (monte carlo etc?)

(2) it is not very clear to me why the authors do not consider a posterior for the type embdeddings u in the final objective L after Eq (10)

(3) I haven't seen partition on event-based datasets with respect to the types of events when performing variational inference? are there convergence guarantees? i think it would help a lot, if convergence, was at least demonstrated with learning curves for some parameters of the model.

(4) It would be interesting if an ablation study was provided which shows predictive performance for different dimensionality of type embeddings R, given that it exhibits cubic time complexity?

(5) I believe that the synthetic experiments should be extended so that synthetic event sequences are generated by a Hawkes process, neural point process etc and show how well the proposed model can recover existing point process models.

(6) the real world dataset experiments would become stronger if the authors also compare with:

[1]Shchur, O., Biloš, M. and Günnemann, S., 2019. Intensity-free learning of temporal point processes. arXiv preprint arXiv:1909.12127.

[2] Mehrasa, N., Jyothi, A.A., Durand, T., He, J., Sigal, L. and Mori, G., 2019. A variational auto-encoder model for stochastic point processes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 3165-3174).
limitations_and_societal_impact: yes
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes a multi-variate self-adaptable point process with nonparametric time decays. First, to flexibly represent arbitrary types of influences between events, they introduce an embedding to represent each event type, and model the influence between any two events as an unknown function of the event type embeddings and time span. Second, they also propose a general construction that covers all possible time decaying functions by assigning Gaussian process (GP) priors over the free latent functions and using Gauss Legendra quadrature to derive the integral in the construction. This gives their model more representative power while being computationally efficient. For efficient inference, they also use weight-space augmentation of GPs to prevent computing huge covariances matrices, and neural network feature mapping to enhance the learning capacity. Lastly, a scalable stochastic variational learning algorithm is developed by taking advantage of the re-reparameterization trick.
main_review: - The paper is overall well-written. The problem motivation is clearly described and an efficient approach is proposed accordingly.
- Results on both synthetic and real world data are promising. The interpretable results are interesting.
- The proposed learning algorithm is efficient to carry out, which is important for point process models.
limitations_and_societal_impact: - The novelty of the method itself is limited and introducing Gaussian prior to the hidden embedding seems incremental. 
- The representative power of this method is still limited comparing to other neural network based methods, where only excitation or inhibition effects are considered. However, the influences between events can be very complicated in some real applications. For example, an event can be inhibited in some time period, whereas it can also be excited in another time period.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper proposes a new algorithm to learn point process models. The proposed algorithm employs non-parametric time decay functions, which are more flexible than exciting approaches. Efficient estimation procedures are drafted. Empirical results backed up the authors claims.
main_review: Originality:
The paper proposes a new solution to an important question.


Quality:
My main concern is that the proposed method is very complicated. The authors claim that existing methods make over-simplified assumptions, and propose a very complicated formulation of the time decay function (Eq. 7). The authors further place a GP prior on the formulation and propose to optimize the derived ELBO. I am not sure whether it’s worth the effort than the relatively simple approaches (e.g., RMTPP), given that the performance gain is not quite significant in some cases (Figure 2, prediction accuracy of Retweet and SO).

Clarity:
Presentation of the paper needs to improve.

1. I suggest to move line 146-147 directly after Eq. 4. It took me a while to realize that $\tilde{\lambda}$ is not the intensity function, which is confusing.

2. I suggest to move line 204-211 right after Eq. 7. Section 4.2 has nothing to do with integral estimation.

3. Some notations are not defined, e.g., $\Delta$ in Eq. 6.

4. In Lemma 3.1, any particular reasons that you choose $f(\cdot)=\mathrm{tanh}(\cdot)$ and $z(\cdot)$ to be the integral function? Are there other possible choices? The current choices seem arbitrary to me, so please provide more intuition. 

Significance:
The paper studies an important problem and provides insights that could benefit future works.
limitations_and_societal_impact: The authors addressed the limitations and potential negative societal impact of their work.
review_title: null

Meta-review: The paper proposes a new marked temporal point process model for continuous-time event sequences. While the standard neural stochastic point process models (e.g., Neural Hawkes Processes) parameterized the time-varying intensity functions using recurrent neural networks,  these models are often black boxes and don't allow to estimate the influence (inhibition or excitation) that event types have on future ones. In contrast, the proposed work uses interpretable Gaussian processes and achieves better performance while maintaining interpretability and scalability. In this sense, the approach achieves to both flexibly and transparently estimate various decaying influences between every pair of events. 

With few exceptions on the model/ training algorithm that could be presented more clearly, the paper is overall well-written and mathematically sound. Despite its sophisticated inference scheme, it provides a valuable contribution to machine learning and statistics. 

Please run a grammar check over the paper for the final version since many small mistakes are present. 
Decision: Accept (Poster)