Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper proposes a meta-learning framework for temporal point processes (TPPs). The paper points out a limitation of the existing TPP models - treating all event sequences in the dataset as realization of the same process. The proposed meta-learning approach addresses this limitation and leads to improved performance compared to other neural TPP models, as shown by the experiments. More specifically, the proposed model uses a cross-attention encoder (based on [attentive neural process](https://arxiv.org/abs/1901.05761)) to extract the necessary information for predicting the tiem of the next event in the sequence.
strength_and_weaknesses: - Strong performance: The proposed approach shows a consistent improvement over the baselines in terms of event prediction capabilities.
- Extensive empirical evaluation: The experiments cover a large selection of datasets and tasks. These include both standard results on predictive performance (event time prediction, log-likelihood), as well as experiments designed to shed light on the properties of different methods (effect of missing events, distribution shifts on performance; effect of model size)


Weaknesses:
- Clarity: Some aspects of the proposed approach are hard to understand or are not described in full detail. The paper would benefit greatly if these are clarified in the revised version.
    - It would be helpful to explain the mapping between the variables $\mathcal{X}, \mathcal{Y}$ in the meta-learning problem definition and $\tau_i$ in the TPP problem definiton early in the paper, and then stick to the latter notation. Currently, section 3.1 is somewhat hard to follow with the switches between two notations.
    - How are the context features $r$ defined for TPP models?
    - Most notation in figure 1 ($r$, $G$) is only introduced a few pages later in the paper, making it hard to interpret the figure the first time it's encountered.
    - What is the interpretation of the global feature $G$ and the latent variable $z$ in the context of TPP modeling (section 3.2)?
    - Experiments in table 1 include accuracy scores (probably, for mark prediction?), but the rest of the paper only discusses unmarked TPPs.
    - $G$ denotes the context set in the beginning of page 4 instead of the $\mathcal{C}$ notation.
    - How is the NLL computed for the proposed Attentive TPP model? Equation 5 seems to imply that the NLL is intractlible, and only ELBO is available.
    - How exactly are the histograms in figure 3 obtained? As far as I understand, the models produce a one-step-ahead prediction of the inter-event times. How are these converted to the histograms? Do we predict the next inter-event time $\tau_{i+1}$ and then feed in the actually observed value of $\tau_{i+1}$ to then generate the prediction for $\tau_{i+2}$, etc.?

----
Post-rebuttal update: The authors have addressed all of my concerns and updated the paper accordingly. I have raised my score to reflect this.
clarity,_quality,_novelty_and_reproducibility: Clarity & reproducibility: While the experimental setup is described very precisely (except the parts mentioned above), some aspects of the model architecture are not clear.

Novelty: The connection between TPPs and meta-learning has not been explored in earlier works, as far as I am aware.
summary_of_the_review: The proposed approach is well-motivated and achieves strong empirical results. However, some important aspects of the model are not described very clearly.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The authors suggest viewing the problem of predicting the next event for temporal point process models as a meta-learning problem. In particular, they advocate a neural process framework, whereby windows of previous event times act as context and target input sets. A cross-attention architecture is suggested to increase modeling capacity that does not scale quadratically in the number of events. The method seems to outperform supervised point process models empirically. 


----
Update following authors' responses:
The presentation and clarity has been improved in the updated manuscript. The authors have also included additional experimental work that improved the experimental validation of the proposed method. I have increased my score to a weak accept.

---
strength_and_weaknesses: Strengths:
- The meta-learning framework is new for point process models, as far as I am aware. 
- The method appears to perform better than point process models trained in a supervised manner.

Weaknesses:
- I feel that the presentation could be improved at some points (see below). 

Actionable feedback:
- The equations (5) to (7) are a bit unclear to me. Why are we using a variational lower bound based on an expectation with respect to $p_{\theta}(z|C_{L})$? Is this still a valid lower bound if we take (as I understand you do, while dropping the KL term?) using samples from $p_{\theta}(z|C_{l})$?
- How are the context features $r_1, r_2,$ … computed/encoded?
- How exactly does z enter the decoder. What are the priors for z given different context sets? 

Comments:
- Can this be extended to multi-dimensional/marked point process setting where there is not just a single task to predict?
- I was wondering if the presence of periodic patterns in the data somewhat makes the required permutation invariance over a context set a constraint that might decrease the forecasting performance?
- Can this method be applied to meta-learning short sequences, e.g., setups like in Xie et al. Meta Learning with Relational Information for Short Sequences, Neuirps 2019?



clarity,_quality,_novelty_and_reproducibility: The paper is largely well written. Some terms/notations could be defined more precisely (see Comments above).
The presentation could be more self-contained at some points, e.g., by not just referring to the transformer point process models. 

summary_of_the_review: Casting predictions with point process as a neural process is new, as far as I am aware. I feel there is scope to improve the presentation and clarity and I would consider increasing my score if this were addressed.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: In this work, the authors propose to use a meta learning way to train neural processes aiming at one-step prediction of event time of point processes. The proposed TPPs employ a latent variable layer and attention mechanism to improve the prediction performance. The method was demonstrated with benchmark datasets.
strength_and_weaknesses: Strength:
Use meta learning training to overcome the generalization issue of supervised learning.
The latent variable layer incorporates uncertainty.
The attention takes into account the similarity between inputs.

Weaknesses:
The meta objective (4) turns out a special case of supervised learning (3). The distinction is blurry.
The writing needs to be improved.
clarity,_quality,_novelty_and_reproducibility: Several definitions and concepts need clarification
- As mentioned, the meta learning learns the distribution of label $y$, $p_\theta(y | x, C)$. It is unclear how $p_\theta$ is specified in this study, especially in (4). What do the authors mean by "learn" the distribution in contrast to that it appears also in the supervised learning objectives. What's the difference?
- What's the output of the decoder? According to the preliminaries, it's a point estimate/prediction of the label s.t. $f_{\theta}(\cdot, C): x \rightarrow y$. However, it's drew like a distribution in Fig 1.
- The text says, "In inference, as we do not have access to $C_L$ at $i$-th event when $l < L$, we use $z$ from $p_θ(z | C_l)$." Following this, would the KL term in (7) become $0$?
- The model involves latent variable. Sampling relies on its posterior. The how is the testing done for a trained model on new sequences as you don't have the posterior for them.
- Information about how RMSE and NLL calculated is needed, like what likelihood is used, how point prediction is obtained if the output is a distribution.

Quality:
- It's better to show the performance of a null model along with the other methods, e.g., the ones using median interevent interval to predict future event time. It's possible, even not expectedly, that no methods do well for certain dataset.
- It's better to describe how the error bar is calculated in the text or caption.
 


summary_of_the_review: This work uses a meta learning to train variant TPP aiming at one-step prediction of event time of point processes. The demonstration with a variety of benchmark datasets shows competitive performance against other SOTA methods. There are a few key concepts and details need to be clarified to make the work a better understandable and solid one.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This work propose to train TPP in a meta learning framework and treat each sequence as a different task via the neural process. The work defines the context dataset, target input and output for TPP model. Also, the work introduces the cross-attention module to introduce local history matching to learn more informative features. 

strength_and_weaknesses: Strength: Learning TPP in a meta learning framework is interesing and the submission provide the design of the model and training procedure. In general, this paper is easy to follow and I did not find any obvious technical errors despite of some notation problems. 

Weakness: The submission provide a detailed description of the encoder part and the training procedure, but almost no description of the decoder. For example, what is the structure of the decoder? How to implement the decoder? If the output of the decoder is the distribution of the next event time or inter-event interval, what distribution did you use? and how did you parameterize the distribution with your decoder? All questions above are not answered in the submission. Furthermore, the decoder takes a random latent variable Z as input, does that mean the output distribution of the decoder is also random? If so, this kind of design contradicts with the TPP model that when the historical information is fixed, the distribution of the next event time or inter-event interval should be determinnistic. 

The design of experiments is also problematic. As stated in the introduction, the goal of meta learning is to improve generalization. To make the experiments convincing, you should demonstrate the meta-learning TPP is better than existing methods on generalization, e.g., training log-likelihood v.s. test log-likelihood. The curret experiment section even did not describe how to split traning and test dataset; and looks more like the submission proposed a more expressive model with lower RMSE or NLL in Table 1,2,3. In conclusion, the experiments did not convince the statement made in the introduction. 
clarity,_quality,_novelty_and_reproducibility: Clarity: The submission provide a detailed description of the encoder part and the training procedure, but almost no description of the decoder (structure, implementation).

Novelty and Quality: The design of encoder and decoder significantly overlaps with previous works, e.g., [Zuo et al.,, 2020], [Lin et al., 2022] and [Shchur et al., 2020]. The only thing that looks new is the meta-learning framework, but this motivation is not consistent with the experimental design.

Reproducibility: I did not check the code.
summary_of_the_review: See strength and weakness.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper proposes a novel idea of formulating the TPP problem in a meta learning framework. Inspired by the NP literature, the authors present a conditional meta TPP formulation, a latent path extension, and an Attentive TPP model. The proposed methods achieve new SOTA on four TPP standard benchmarks. The reviewer agrees that the paper is a solid contribution of the paper. While Reviewer dU5L did not actively participate the discussion, I think the authors' rebuttal did a great job addressing the reviewer's concern. I therefore recommend acceptance of this paper to ICLR.
justification_for_why_not_higher_score: n/a
justification_for_why_not_lower_score: n/a
Decision: Accept: poster