Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: Paper presents a transformer-based architecture to handle multi-event forecasting in spatio-temporal domains. The proposed architecture allows for multiple events forecast by formulating problem as a self-supervised problem. 
strength_and_weaknesses: Strengths
* Architecture proposed is quite novel in sense of being applied to multiple event prediction task
* All different parts of the model are well motivated and solve a purpose instead of a giant network being embedded for sake of doing deep learning. 
* Problem being attacked is important and has been something people in filed have mostly sidelined and been happy with doing sequential short predcitions. 
* well written and easy to follow paper. 

Weakness
* The baselines in experiments consider only Hawkes alternatives. I would encourage authors to at least present a comparison using other methods in SI. 
 
clarity,_quality,_novelty_and_reproducibility: I would rate this work very high on clarity, quality and novelty. For reproducibility, I found the code to be of good quality and fairly easy to follow. I hope authors will document the code a bit more once paper is accepted.  
summary_of_the_review: In general, the model and problem proposed in the paper are quite novel and of great importance. Authors have done a great job of explaining each individual parts of the model and its specific utility in solving the drawbacks from the previous work. Hence, I think this work deserves to be published.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper studies the problem of simultaneously predicting times and location of *multiple* (fixed number of) future events with a neural spatio-temporal point process model.
The proposed model is based on an encoder-decoder transformer architecture, and the distribution of the future events is modeled by 
strength_and_weaknesses: **Strengths:**
- The paper introduces an interesting variation of the prediction problem for spatio-temporal event data, where the goal is to predict multiple events simultaneously. A thorough analysis of the limitations of existing approaches that could be used in this task is provided. This is an important task and it would be important for practical applications of spatio-temporal point process models.
- The visualizations in section 4 are well done and provide an insight into the model's performance.

**Weaknesses:**

1. My main concern is the clarity regarding the proposed model architecture & the training procedure. Here are some questions that need to be clarified in the updated version:
    - During training the decoder takes the observed $\\{(t_l, x_l)\\}_\{l=n+1\}^\{n+L\}$ as input, and tries to predict the exact same values  $\\{(t_l, x_l)\\}_\{l=n+1\}^\{n+L\}$  as output. Intuitively, it seems that the decoder should learn to perfectly predict these values / the log-likelihood should tend to infinity as the model trains. Why is this not the case?
    - It is mentioned that at test time the inputs to the decoder are replaced with all zeros.
        - In this case, how can the decoder distinguish between different inputs? Is some positional encoding used?
        - If $t_l = 0$ is fed into the model, this means that the distribution $p(x_l | t_l)$ will effectively become independent of $t_l$. What is the purpose of modeling the conditional distribution at training time, if at test time the time and location are always independent?
    -  The softsign function $f(z) = \frac{z}{|z| + 1}$ has domain/codomain $f: (-\infty, \infty) \to (-1, 1)$. Restricting to positive values only (as $z_{t_l}$ comes from the exponential distribution), we get a function $f: (0, \infty) \to (0, 1)$. However, we need both the domain and the codomain here to be $(0, \infty)$ since we're modeling the inter-event times or the arrival times. In other words, the used function is not a bijector so it does not define a valid normalizing flow transformation. Is there a mistake in my reasoning?
        - As a side note, the softsign function has no learnable parameters, so it's unclear how exactly it increases the flexibility of the distribution $p(t_l)$.

2. Baselines: Currently, the experiments in section 4 only include very simple baseline models (homogenous Poisson, self-exciting/self-correcting processes for time & gaussian mixture for locations). There exist other approaches that definitely could be compared against:

    - Neural spatio-temporal point process model from Chen et al. (https://arxiv.org/abs/2011.04583)
    - Other neural TPP models for predicting the event time (many of which are cited in the paper)

    Figure 1 also suggests that the proposed model takes into account the magnitudes $M_l$ of the events, but it seems that none of the baselines have access to this information. If the magnitudes are indeed taken into account, it's important to also compare to the [ETAS model](https://www.ism.ac.jp/editsec/aism/pdf/050_2_0379.pdf) from seismology (using the power-law kernel and, very importantly, taking into account the contribution of the magnitudes on productivity)

3. Lack of ablations: The proposed model consists of multiple components, but no experiments are performed to confirm that each of these components is actually beneficial. To demonstrate that the transformer architecture leads to a meaningful improvement, it would be necessary to compare with:
    - $p(x_l)$ and $p(t)$ completely independent of the history, but parametrized exactly like in the paper (i.e., exponential + softsign for $t_l$, gaussian + real NVP for $x_l$)
    - Encoder only (without feeding $t_l$, $x_l$, $M_l$ to the decoder, i.e. just mapping $h_{t_i}$ and $h_{x_i}$ to the parameters of $p(t_l)$ and $p(x_l)$). Intuitively, it seems that this option will work just as well as the model with the decoder, since in the full model the decoder anyway doesn't receive any input at prediction time anyway.


**Minor comments**
- The assumption that the future events are conditionally independent given the history (as defined in Problem 1) seems a bit restrictive: for example, if the next earthquake has a large magnitude, then the follow-up events will happen in quick succession. Lifting the conditional independence assumption could be an interesting avenue to explore.
- Potentially missing references
    - Neural spatio-temporal point processes: https://arxiv.org/abs/1906.05467,
    - Modeling TPPs with normalizing flows: https://arxiv.org/abs/1910.08281 and https://arxiv.org/abs/1909.12127
clarity,_quality,_novelty_and_reproducibility: See main review.
summary_of_the_review: Several important aspects of the proposed approach need to be described more clearly. A more thorough comparison to the baselines and an ablation study would be really beneficial to justify the utility of the proposed model.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: 

This paper proposes a neural architecture, combining the transformer model and normalizing flows, for multi-event forecasting of the time and location of discrete events. A variety of real spatio-temporal datasets are used to validate the state-of-the-art performance of the proposed method.

strength_and_weaknesses: Strength:
- A new architecture consists of transformer and normalizing flows, to capture the spatio-temporal distribution of multiple stochastic discrete events jointly, conditioning on historical events.
- extensive numerical experiments to demonstrate the performance of the proposed method.

Weaknesses:
- the contribution is incremental, both the transformer and normalizing flows have been used for point processes in the literature, e.g, [1] 
- the numerical experiments are not very clearly explained, and more clarification is needed

[1] Mehrasa, N., Deng, R., Ahmed, M. O., Chang, B., He, J., Durand, T., ... & Mori, G. (2019). Point process flows. arXiv preprint arXiv:1910.08281.
clarity,_quality,_novelty_and_reproducibility: Overall this paper is well-written and sticks to the major theme of tackling the four challenges of existing point process modeling/prediction methods.

I have some questions:
(1) the intensity function \lambda in eq(1) does not include the marker M. Does this mean that the prediction is only for time t and location x of the next events, while not their markers M? However, in the proposed architecture shown in Fig1, the marker M is used in the self-supervised training phase.
(2) in the numerical results Table 1, the author compared the negative log-likelihood for the predicted events, I am wondering how are these likelihoods computed for real-data examples (since we do not know the true intensities, and the true data-generating model is also different from Hawkes most likely). 

summary_of_the_review: Overall this paper is well-written. This work proposes to combine transformer and normalizing flows to predict future multi-events under the spatio-temporal Hawkes models. Numerical results are provided to demonstrate the good performance in real data examples. However, considering that both transformer and normalizing flows are not new and they have been already been used in point processes, the contribution of this work seems incremental. Moreover, more clarification on numerical results is also needed.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper leverages the sequence-to-sequence (seq2seq) model in NLP for multi-event prediction in the Hawkes process. The authors call the task 
the batched prediction. The authors employ
the encoder-decoder architecture that incorporates spatial-temporal aspects. They leverage the transformer architecture, where two embedding vectors capture the temporal event history and spatial dependency. They seem to have used an interesting normalization flow model to handle the temporal development of probability distribution.
strength_and_weaknesses: **Strengths**

- Proposes a new multi-event prediction framework. 
- Seems to have address the probabilistic nature (i.e., properties that require Monte Carlo or its variant) of multi-event prediction.
- Incorporated a new normalized flow approach.
 

**Weakness**

- Some of the claims in the introduction seem misleading. The classical Hawkes process has a decay function, which introduces a significant distinction over the event instances. I'm not sure if "(1) imposes similar behavior across all events over time meaning that every preceding event has the same form of impact on future events' occurrence" is a scientifically correct statement.

-  The proposed probabilistic models are not clearly defined. Most of the descriptions are very high-level and qualitative. As a result, it is hard to verify methodological novelty and reproduce/extend their results. 
clarity,_quality,_novelty_and_reproducibility: The text is overall well-written. The authors seem to have made much effort for clearer descriptions. However, in contrast, there are very limited formal descriptions on the generative process, which should be the key in verifying their technical contributions. As a result, it is virtually not possible to reproduce their model beyond the specific implementation.
summary_of_the_review: This paper is unique in its sharp contrast between readable narratives and hard-to-follow descriptions of the probabilistic model. Because of the very definition of the intensity function (the probability density of the next event), multi-event prediction generally requires Monte Carlo integration. The situation is similar to that of multi-ahead prediction in forecasting. See, e.g., https://ieeexplore.ieee.org/document/8594894 

The proposed framework seems to have solved that issue by introducing the normalized flow approach combined with the seq2seq transformer model. I wish I could give a much better rating, but it is hard for me to endorse the approach due to the massive lack of formal descriptions.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper presents a spatiotemporal point process model based on a transformer, which allows for simultaneously predicting times and location of multiple (fixed number of) future events. The reviewers did recognize that the paper is well written and the contribution is sound. A majority of reviewers were not in support of accepting the paper due to a number of issues, including limited significance and insufficient experimental evaluation. During the discussion period, the authors did not manage to persuade the reviewers to change their original assessment. However, I encourage the authors to revise the paper in light of the reviewers' comments and resubmit to another top tier venue.
justification_for_why_not_higher_score: Limited significance and insufficient experimental evaluation.
Decision: Reject