Intensity-Free Learning of Temporal Point Processes

Abstract: Temporal point processes are the dominant paradigm for modeling sequences of events happening at irregular intervals. The standard way of learning in such models is by estimating the conditional intensity function.  However, parameterizing the intensity function usually incurs several trade-offs. We show how to overcome the limitations of intensity-based approaches by directly modeling the conditional distribution of inter-event times.  We draw on the literature on normalizing flows to design models that are flexible and efficient. We additionally propose a simple mixture model that matches the flexibility of flow-based models, but also permits sampling and computing moments in closed form.  The proposed models achieve state-of-the-art performance in standard prediction tasks and are suitable for novel applications, such as learning sequence embeddings and imputing missing data.

Reviews and author responses:

As you correctly pointed out, we focused on event time prediction in this paper. We agree with you that mark prediction is an interesting research problem in its own right, and that the conditional independence assumption might be too restrictive in some cases. 

In theory, our model could be altered in several ways to improve marks prediction. First, our model is compatible with the “Neural Hawkes” approach --- we could directly model the distribution of K inter-event times (for each mark) separately and use a shared RNN to process the history. However, it’s unclear whether this will boost the performance (Figure 8 of the original Neural Hawkes paper reports performance comparable to RMTPP in event time and marks prediction).

It could also be possible to improve the mark prediction performance of our model by removing the conditional independence assumption (e.g., by again using a mixture model: $p(\tau, m | \mathcal{H}) = \sum_z p(\tau | z, \mathcal{H}) p(m | z, \mathcal{H}) p(z)$).

Regarding loss balancing: Time NLL can be shifted arbitrarily by scaling all the inter-event times by a positive factor (as can be seen from the change of variables formula), so it’s unclear whether time NLL indeed dominates marks NLL during training --- this issue deserves further investigation.

Thank you for the meticulous response and update.

From the new "mark accuracy" column in Table 6, it seems as though all the models considered in this paper are relatively poor at jointly modeling marks and times in a marked TPP, compared to say Neural Hawkes Process, which models an intensity function for each type (I tend to think this unique design helps with mark prediction, more below) and is missing from the comparison.

I understand your argument that the paper is primarily focused on time prediction and therefore marks are only used as additional inputs to help with that in the experiments. But it just came to me that maybe mark prediction in marked TPPs isn't (and probably shouldn't be) as simple as just stacking a softmax module for marks on top of a normal TPP and then "jointly" optimizing the two conditionally independent likelihoods. The reason is that the likelihood on "time" is continuous while that on "mark" is discrete, and therefore when you take log-likelihood and use simple summation to combine them in the objective, one ("time") can easily "outweigh" the other ("mark") simply because it enjoys a much higher "range". This is also clearly exemplified from the small difference between "Time NLL" and "Total NLL" in Table 6, which signals that "Mark NLL" is much smaller than "Time NLL" and is therefore causing mark prediction to be under-trained. This may be a universal problem for all those marked TPP models where marks and times are assumed conditionally independent given the history.



Thank you for your positive review and constructive comments. 


1. We have made the statement of Theorem 1 more precise.


2. First, using MSE / MAE as a metric may discard important information about the learned distribution. For a model to achieve the best possible MSE, it is sufficient for the learned conditional distribution p_model(tau | history) to have the same mean as the true distribution p_data(tau | history). Similarly, MAE is minimized by matching the median. Unlike NLL (i.e. KL divergence), MSE & MAE do not reflect how well p_model captures other important properties of p_data, such as multimodality or tail behavior.

Second, the expected inter-event time cannot be computed in closed form for some of the models that we considered (namely FullyNN, DSFlow and SOSFlow). Approximating the expectation via Monte Carlo is also non-trivial for these models, since they do not permit closed-form sampling. This makes computing MSE / MAE more challenging for these models.

Last, the NLL scores make it easy to compare our models to the _true_ generative models for the synthetic datasets (Table 5). Since DSFlow and LogNormMix achieve nearly the same NLL loss on the test set as the true generative model, we can conclude that DSFlow and LogNormMix have successfully recovered the true data distribution.


3. We have measured the time it takes to perform a single forward pass for different models on the MOOC dataset (averaged over 200 trials).

Model 		 | RNN time (ms) | Decoder time (ms) |
RMTPP 	         | 3.91 | 0.66 |
LogNormMix | 3.70 | 0.96 |
FullyNN 	         | 3.68 | 2.06 |
Exponential 	 | 3.68 | 0.47 |
SOSFlow 	 | 3.79 | 3.18 |
DSFlow 	         | 3.56 | 3.32 |

We make 2 main observations. First, the RNN takes a significant portion of the runtime. Using a different architecture (e.g. LSTM) would increase this even further. Second, the overhead for using LogNormMix (with 64 mixture components) over RMTPP is minor, while the performance gains are significant.


Also, we have added a reference to (Tabibian et al., 2016) to our paper.

Thank you for your positive review and constructive comments. 


1) Overall, we found the results to be very robust to the choice of K. We have included the discussion in Appendix F.1 and an additional Table 3 showing the stability of the results for the mixture model when varying K.

In the main experiments, we used K = 64 in order to make the number of parameters comparable between the mixture and single-layer DSFlow and FullyNN models. When manually inspecting some of the learned conditional distributions, we found that in this case most of the mixture components were assigned a weight w_k close to zero. This means that it's safe to use a large K, and the model will automatically only use the necessary number of components.


2a) Please note that RealNVP and Neural Spline Flow, while successfully used in multivariate density estimation, cannot be applied to 1D data, such as inter-event times.

RealNVP and Neural Spline Flow define an invertible transformation by splitting the dimensions of the variable into two groups, where the first group remains unchanged and the second group is passed through an invertible transformation. In 1D, we cannot split the dimensions. Therefore, these architectures cannot be used in our scenario. We have clarified this point in the revised version of the paper.


2b) All the models considered in our work are autoregressive; SOSFlow model is a strict generalization of MAF / IAF.

The MAF / IAF models specify the joint distribution $p(\tau_1, …, \tau_N)$ by employing an autoregressive factorization $p(\tau_1, …, \tau_N) = \prod_{i=1}^N p(\tau_i | \tau_1, …, \tau_{i-1})$. 

The two main components of such a model are
 - An autoregressive neural network that takes $\{\tau_1, …, \tau_{i-1}\}$ as input and produces the transformation parameters $\theta_i$.
 - A parametric transformation $f( . ; \theta)$  that transforms $\tau_i = f(z_i; \theta_i)$ for $z_i \sim N(0, 1)$, and thus defines the conditional distribution $p(\tau_i | \tau_1, …, \tau_{i-1})$.

In the original IAF paper, the authors use an LSTM as the autoregressive NN that produces parameters of an affine transformation $f(z; a, b) = az + b$.

The SOSFlow transformation used in our paper is a strict generalization of the affine transformation (as pointed out by the authors of the original paper [1]), therefore our model can already be seen as a generalization of the MAF / IAF architecture. We use RNN as an autoregressive NN instead of a more sophisticated architecture (e.g., LSTM, Wavenet, Transformer) since (1) this is the de-facto standard used in other neural temporal point process works [2, 3] and (2) to highlight that the improved performance of our model comes from a better model for the distribution of inter-event times, not because we are using a different conditioner network.


[1] Jaini, P., et al., "Sum-of-Squares Polynomial Flow", ICML 2019
[2] Du, N., et al., "Recurrent marked temporal point processes: Embedding event history to vector", KDD 2016
[3] Omi, T., et al.. "Fully Neural Network based Model for General Temporal Point Processes", NeurIPS 2019


Thank you for your positive review and constructive comments. 

1. We have chosen the values of K such that the mixture model has approximately the same number of parameters as a 1-layer DSFlow or a 1-layer FullyNN model (models with M>1 layers have approximately M times more parameters, respectively). More specifically, we set K=64 for LogNormMix, DSFlow and FullyNN. We found all these models to be rather robust to the choice of K, and values in range [8, 64] produced similar results. For SOSFlow we used K = 4 and limited R to 3 (the resulting polynomial was of degree 7 x #layers), since higher values led to unstable training, even when using Batch Normalization.

We have included these details to Appendix F.1.


2. Based on your suggestion, we have added a visualization and a discussion of the learned conditional density p(tau | history) in Figure 9, Appendix F.1.

As an example, we consider the Yelp dataset. The events represent check-ins into a specific restaurant. Since check-ins mostly happen during the opening hours, the inter-event time is likely to be ~0h (on the same day), ~24h (next day), etc. Figure 9 shows that LogNormMix can fully recover this behavior from data while others either cannot learn multimodal distributions (e.g. RMTPP) or struggle to capture it (e.g. FullyNN).

Having a flexible distribution is beneficial even if the true distribution is not multimodal. This statement is supported by our experiments on synthetic data (Table 5): flexible mixture and flow-based models can almost perfectly approximate different generative processes.


3. We have included a Table 6 that compares the mark prediction accuracy in Appendix F.2. The goal of the marked TPP experiment is to assess whether time prediction quality can be improved with the knowledge of marks. We predict marks in the same way as done by RMTPP, i.e. output distributions of mark types and event time are conditionally independent. Because of this, mark prediction performance is similar across all the models.

The authors propose a new paradigm for learning models for point processes which circumvents the need to explicitly model the conditional intensity. This utilizes recent work on Normalizing flows and upends a long-standing paradigm. The paper is a true tour-de-force and the authors make a very convincing case for why instead of modelling conditional intensity, one could (and should) model the distribution of times directly.

The authors do an extensive literature review and place their work in the context very well. The writing is very lucid and the paper was a pleasure to read. The illustrations are also helpful and the extensive experiments complement the discussions very well. The appendix is also very easy to read and has been judiciously separated.

The two primary axes they justify their model is by first comparing it against conditional intensity-based methods in general and show that:
 1. Their approach has universal approximation property (thanks to a result from DasGupta (2008)).
 2. The likelihood function can be evaluated efficiently (thanks to it being a mixture of log-normals).
 3. The expected next action time can be evaluated efficiently.
 4. Sampling from their model is also easy. 

Then they show that all related work fails either one or more of these conditions. Finally, they also address the common concerns of why point processes were historically always taught using intensity functions. A bit of historical context here is that in absence of modern computers and when Cox-Hazard models still needed to be worked out by hand, the conditional intensity functions indeed were much simpler to handle and afforded desirable properties. However, with modern approaches towards modelling probability distributions, those reasons are no longer valid. Further, some of the recent research which has built on the conditional intensity functions has lost much of the advantages anyway because of the use of deep neural networks. Hence, at this point, one might as well move directly to modelling the probability (as the authors have done) instead of taking a detour via the intensity functions. The experimental results for prediction seem to justify taking this route as modelling intensity function does not seem to out-perform the intensity-free models (more on the metrics later). Finally, the authors also show that their model, thanks to its generative nature, can also be used for several ancillary tasks: Sequence embedding, missing data imputation and learning with conditional information. The provided code is easy to read through and execute.

Hence, I do not have any hesitation in recommending the paper for publication.

As a side-note, it is notable that in (Tabibian, 2016), a mixture model is proposed, but I believe it is not for the final probability but instead for the intensity itself to make the learning problem tractable.


Some ways of improving the paper:

 - Theorem 1 can be made more rigorous by making the role of parameters like 'K' more explicit. e.g. does the theorem imply that there 'exist' K, \mu_k, \s_k, or does it say that for any K, \mu_k, and \s_k, such 'w_k' can be found, etc.?
 - Using NLL for comparing models in the experiment results is a bit unusual. Could a different metric (e.g. MAE) be employed?
 - A key discussion missing in the paper is that of the complexity of training the model. RMTPP, for example, can be trained very efficiently while Neural-Hawkes is difficult to train due to a Monte-Carlo sampling embedded in the calculation of the likelihood. Empirical results will also add to better placing the Intensity free method against other methods.

Citations:

 Tabibian, Behzad, et al. "Distilling information reliability and source trustworthiness from digital traces." Proceedings of the 26th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee, 2017.

This paper describes a simple yet effective technique for learning temporal point processes using a mixture of log-normal densities whose parameters are estimated with neural networks that also adds conditional information. The method is shown to perform better than more recent techniques for density estimation such as different versions of normalising flows. Experiments were reported on 6 datasets, comparing the approach against flow models and assessing the benefits of adding extra conditional information, performance with missing data, and benefits of sequence embeddings.

The paper makes an important point which is also my own experience when working with relatively low dimensional problems; simpler neural density estimation approaches such as MDNs usually perform similarly or even better than models using normalising flows. The task here is on learning temporal point processes which have important applications in social networks, criminality studies, disease modelling, etc, but are relatively unpopular within the machine learning community. The paper gives some motivation but I think the authors could elaborate further on the huge number of applications and potential for significant impact from these models. Apart from this the paper is well written and structured, and easy to follow. 

There are not many theoretical innovations as the main contribution is a combination of several well known techniques such as MDNs and RNNs, applied to the specific temporal point process formulation. The main lesson learnt though is that these simpler techniques can perform surprisingly well. With that said, the paper would benefit from discussing the following points:

1) Mixture models are effective in low dimensional problems but require the manual specification of the number of components. How was this done in the experiments and how sensitive the performance is to this parameter?

2) The paper discusses several problems with normalising flows, but in particular the computational cost involved in generating samples or evaluating the density. This is true for some variations of NFs but not for all. For example RealNVP and the recent Neural Spline Flows are efficient in both, sample generation and density evaluation. With this in mind, the paper would benefit from further comparisons to these approaches. Another interesting comparison would be with autoregressive flows, such as inverse autoregressive flow or masked autoregressive flow. They can both  capture sequences and should be able to model inter-event times, instead of an RNN. 

  

The paper proposes to directly model the (conditional) inter-event intervals in a temporal point process, and demonstrates two different ways of parametrizing this distribution, one via normalizing flows and another via a log-normal mixture model. To increase the expressiveness of the resulting model, the parameters of these conditional distributions are made to be dependent on histories and additional input features through a RNN network, updating at discrete event occurrences.

The paper is very well written and easy to follow. I also like the fact that it is probably among the first of those trying to integrate neural networks into TPPs to look at directly modeling inter-event intervals, which offers a different perspective and potentially also opens doors for many new methods to come.

I have just three comments/questions.

1. The log-normal mixture model has a hyper-parameter K. Similarly, DSFlow also has K, and SOSFlow has K and R. How are these hyper-parameters selected? I don't seem to find any explanation in the paper (not even in appendix F.1)?

2. To better demonstrate that a more complicated (e.g. multi-modal) inter-event interval distribution is necessary and can really help with data modeling, I'd be interested to see e.g. those different interval distributions (learnt from different models) being plotted against each other (sth. similar to Figure 8, but with actual learnt distributions), and preferably with some meaningful explanations as to e.g. how the additional modes capture or reflect what we know about the data.

3. Even though the current paper mainly focuses on inter-event interval prediction, I think it's still helpful to also report the  model's prediction accuracy on marks in a MTPP. The "Total NLL" in Table 5 is one step towards that, but a separate performance metric on mark prediction alone would have been even clearer.

Meta-review: This submission proposes a new paradigm for modelling temporal point processes by using deep learning to learn to mix log-normal distributions in order to directly model the conditional distribution of event time intervals themselves.

Strengths of the paper:
-Introduces a new modelling paradigm that can lead to further research in this direction, for an important problem.
-Extensive experimentation validates the approach quantitatively.
-Easy to read.

Weaknesses:
-Several reviewers wanted more details on how the mixing parameter K was tuned. This was adequately addressed during the discussion period.

The reviewer consensus was to accept this submission.


Decision: accept-spotlight