Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 8: Accept
review_content: The paper proposes to directly model the (conditional) inter-event intervals in a temporal point process, and demonstrates two different ways of parametrizing this distribution, one via normalizing flows and another via a log-normal mixture model. To increase the expressiveness of the resulting model, the parameters of these conditional distributions are made to be dependent on histories and additional input features through a RNN network, updating at discrete event occurrences.

The paper is very well written and easy to follow. I also like the fact that it is probably among the first of those trying to integrate neural networks into TPPs to look at directly modeling inter-event intervals, which offers a different perspective and potentially also opens doors for many new methods to come.

I have just three comments/questions.

1. The log-normal mixture model has a hyper-parameter K. Similarly, DSFlow also has K, and SOSFlow has K and R. How are these hyper-parameters selected? I don't seem to find any explanation in the paper (not even in appendix F.1)?

2. To better demonstrate that a more complicated (e.g. multi-modal) inter-event interval distribution is necessary and can really help with data modeling, I'd be interested to see e.g. those different interval distributions (learnt from different models) being plotted against each other (sth. similar to Figure 8, but with actual learnt distributions), and preferably with some meaningful explanations as to e.g. how the additional modes capture or reflect what we know about the data.

3. Even though the current paper mainly focuses on inter-event interval prediction, I think it's still helpful to also report the  model's prediction accuracy on marks in a MTPP. The "Total NLL" in Table 5 is one step towards that, but a separate performance metric on mark prediction alone would have been even clearer.
review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: This paper describes a simple yet effective technique for learning temporal point processes using a mixture of log-normal densities whose parameters are estimated with neural networks that also adds conditional information. The method is shown to perform better than more recent techniques for density estimation such as different versions of normalising flows. Experiments were reported on 6 datasets, comparing the approach against flow models and assessing the benefits of adding extra conditional information, performance with missing data, and benefits of sequence embeddings.

The paper makes an important point which is also my own experience when working with relatively low dimensional problems; simpler neural density estimation approaches such as MDNs usually perform similarly or even better than models using normalising flows. The task here is on learning temporal point processes which have important applications in social networks, criminality studies, disease modelling, etc, but are relatively unpopular within the machine learning community. The paper gives some motivation but I think the authors could elaborate further on the huge number of applications and potential for significant impact from these models. Apart from this the paper is well written and structured, and easy to follow. 

There are not many theoretical innovations as the main contribution is a combination of several well known techniques such as MDNs and RNNs, applied to the specific temporal point process formulation. The main lesson learnt though is that these simpler techniques can perform surprisingly well. With that said, the paper would benefit from discussing the following points:

1) Mixture models are effective in low dimensional problems but require the manual specification of the number of components. How was this done in the experiments and how sensitive the performance is to this parameter?

2) The paper discusses several problems with normalising flows, but in particular the computational cost involved in generating samples or evaluating the density. This is true for some variations of NFs but not for all. For example RealNVP and the recent Neural Spline Flows are efficient in both, sample generation and density evaluation. With this in mind, the paper would benefit from further comparisons to these approaches. Another interesting comparison would be with autoregressive flows, such as inverse autoregressive flow or masked autoregressive flow. They can both  capture sequences and should be able to model inter-event times, instead of an RNN. 

  
review_title: Official Blind Review #3

final_score: rating: 8: Accept
review_content: The authors propose a new paradigm for learning models for point processes which circumvents the need to explicitly model the conditional intensity. This utilizes recent work on Normalizing flows and upends a long-standing paradigm. The paper is a true tour-de-force and the authors make a very convincing case for why instead of modelling conditional intensity, one could (and should) model the distribution of times directly.

The authors do an extensive literature review and place their work in the context very well. The writing is very lucid and the paper was a pleasure to read. The illustrations are also helpful and the extensive experiments complement the discussions very well. The appendix is also very easy to read and has been judiciously separated.

The two primary axes they justify their model is by first comparing it against conditional intensity-based methods in general and show that:
 1. Their approach has universal approximation property (thanks to a result from DasGupta (2008)).
 2. The likelihood function can be evaluated efficiently (thanks to it being a mixture of log-normals).
 3. The expected next action time can be evaluated efficiently.
 4. Sampling from their model is also easy. 

Then they show that all related work fails either one or more of these conditions. Finally, they also address the common concerns of why point processes were historically always taught using intensity functions. A bit of historical context here is that in absence of modern computers and when Cox-Hazard models still needed to be worked out by hand, the conditional intensity functions indeed were much simpler to handle and afforded desirable properties. However, with modern approaches towards modelling probability distributions, those reasons are no longer valid. Further, some of the recent research which has built on the conditional intensity functions has lost much of the advantages anyway because of the use of deep neural networks. Hence, at this point, one might as well move directly to modelling the probability (as the authors have done) instead of taking a detour via the intensity functions. The experimental results for prediction seem to justify taking this route as modelling intensity function does not seem to out-perform the intensity-free models (more on the metrics later). Finally, the authors also show that their model, thanks to its generative nature, can also be used for several ancillary tasks: Sequence embedding, missing data imputation and learning with conditional information. The provided code is easy to read through and execute.

Hence, I do not have any hesitation in recommending the paper for publication.

As a side-note, it is notable that in (Tabibian, 2016), a mixture model is proposed, but I believe it is not for the final probability but instead for the intensity itself to make the learning problem tractable.


Some ways of improving the paper:

 - Theorem 1 can be made more rigorous by making the role of parameters like 'K' more explicit. e.g. does the theorem imply that there 'exist' K, \mu_k, \s_k, or does it say that for any K, \mu_k, and \s_k, such 'w_k' can be found, etc.?
 - Using NLL for comparing models in the experiment results is a bit unusual. Could a different metric (e.g. MAE) be employed?
 - A key discussion missing in the paper is that of the complexity of training the model. RMTPP, for example, can be trained very efficiently while Neural-Hawkes is difficult to train due to a Monte-Carlo sampling embedded in the calculation of the likelihood. Empirical results will also add to better placing the Intensity free method against other methods.

Citations:

 Tabibian, Behzad, et al. "Distilling information reliability and source trustworthiness from digital traces." Proceedings of the 26th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee, 2017.
review_title: Official Blind Review #2

Meta-review: This submission proposes a new paradigm for modelling temporal point processes by using deep learning to learn to mix log-normal distributions in order to directly model the conditional distribution of event time intervals themselves.

Strengths of the paper:
-Introduces a new modelling paradigm that can lead to further research in this direction, for an important problem.
-Extensive experimentation validates the approach quantitatively.
-Easy to read.

Weaknesses:
-Several reviewers wanted more details on how the mixing parameter K was tuned. This was adequately addressed during the discussion period.

The reviewer consensus was to accept this submission.

Decision: Accept (Spotlight)