Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: This paper addresses an anomaly detection problem for event sequences as temporal point process (TPP), where they formalize the problem as out-of-distribution (OoD) detection in a batch scenario.
The authors proposed using goodness-of-fit (GoF) statistics to determine whether a sequence is OoD or not, which provides vast options for this purpose to us than an ordinary OoD procedure just using log-likelihood.
They make it possible to use GoF statistics designed for known standard Poisson process by using Neural TPP fitted to training sequences. We can convert a random event sequence distributed according to an arbitrary TPP into a realization of the standard Poisson process with a compensator derived from the fitted Neural TPP.
They also propose a sum-of-squared-spacings (3S) statistic that can detect anomalies w.r.t. the event count N, which is difficult to detect by Kolmogorov–Smirnov (KS) statistic, as well as anomalies w.r.t. the distribution of the arrivals or inter-event times.
Experimental results demonstrate that the proposed statistic performs better than either other statistics or log-likelihood-based OoD.
main_review: ### Originality:
* This work proposes a novel combination of GoF statistic and OoD for event sequences with learned neural TPP for the unknown data-generating distribution.


### Quality:
* The authors introduced the motivation of the proposed method and provided the necessary background, which would be informative for many readers.

* The requirements of the proposed method would limit its practical usability.
    - $T$ number of observations are required for anomaly detection with the proposed method. Also, $T$ and $V$ are assumed to be large enough in their analysis. In that case, it is challenging to perform timely anomaly detection, which is crucial for real applications, such as detecting intrusions in computer networks and detecting fraud or shifts in the market structure on financial systems. It would be better to add analysis w.r.t different configurations for $V$.

* The motivation and justification of the use of 3S statistics are unclear.
    - Is it possible to use other statistics instead of 3S statistics?
    - In l.562 in Appendix, the authors replaced $N$ with its expectation $E[N |V ]$. I cannot find the justification for that. Is it just heuristics for considering $N$ as a random variable? Are there any other variants?

* The compared methods seem naive.
    - They analyze when KS statistic is modified by removing $\sqrt{N}$, but they can also replace $1/N$ in the second equation with $1/V$ similar to the proposed method. It would work better.
    - It would be better to add a comparison with the original 3S statistic without modification.


### Clarity:
* The discussion in l.195 is confusing since it would be mainly for KS statistic, not for 3S statistic, which does not require CDF. If so, the paragraph from l.195 would not be a help to understand the proposed method.

* There is no description of how to choose hyperparameters of models, such as how they split validation set from the training set and test set.

* In section 6.3, there is no description of the model they used.

* Figure 1 is not mentioned in the main text. Also, in general, it is better to locate figures on top.


### Significance:
* The practical usability of the proposed method can be limited because of its requirements, as stated above.
* Since the compared methods seem naive in the experiments, it is not easy to find the significance of the proposed method.


================ Update: The motivation and novelty of the proposed method have been clarified in the rebuttal. Also, contributions on empirical evaluation should be noted, where the proposed approach worked well even on short intervals.
limitations_and_societal_impact: They addressed the limitations and potential negative social impact of their work.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper discusses anomaly detection in continuous-time event sequences as out-of-distribution (OoD) detection for temporal point processes, and statistical tests for the detection. The paper proposes a new statistical test for the detection and demonstrates the usefulness of the proposed test in simulated and real-world data sets.

main_review: This is a well-written paper. The connection between anomaly detection in continuous-time event sequences and out of distribution detection has been discussed, and such a connection is useful for anomaly detection research. The proposed 3S test is well motivated and sound. 3S test has been shown effective for anomaly detection in continuous-time event sequences in experiments.   

Discussions in the paper are focused, clear and easy to follow. The authors evaluate the proposed test in different circumstances, and the strengths and weaknesses of the proposed test have been presented. The connection between anomaly detection and out of distribution detection, and existing and proposed statistical tests are useful for anomaly detection research.

limitations_and_societal_impact: See above.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper focuses on the anomaly detection of continuous-time event sequences with temporal point processes and proposes to leverage the idea of goodness-of-fit testing to check whether sequences follow the in-distribution. Consider the limitation that the current statistics are not sensitive to the event number N, this paper proposes a new statistic called sum-of-squared-spacings (3S) to check the fitness of the sequence. The experimental results show the effectiveness of the proposed statistic for anomaly detection.
main_review: Detecting anomalous continuous-time event sequences is an important task yet under-exploited. This paper proposes a new statistic to identify the anomalous event sequences, called 3S. Evaluating on two synthetic and one real datasets shows the effectiveness of using 3S for anomaly detection. The overall paper is well-written with sufficient references and experiments. 
limitations_and_societal_impact: Two minor comments here. First, I personally would expect more explanation about the advantages of the proposed 3S statistic over the log-likelihood approach. As mentioned in the paper, many existing anomaly detection approaches are actually based on log-likelihood, because the log-likelihood approach is more intuitive. The experiments on the real-world datasets in this paper also show that log-likelihood can achieve good performance. I believe give more comparisons about these two approaches would be appreciated. Another comment may be outside the scope of this paper, but I am curious about how to detect the marked event sequences with the marked TPP model. Say, using log-likelihood, it would be easy to combine the signals about marked sequences as well as the time sequences for anomaly detection because both can be modeled via log-likelihood, but the proposed 3S statistic can only detect anomalous sequences from the time perspective.
review_title: null

Meta-review: The authors formulate anomaly detection in continuous-time event sequences as an out-of-distribution detection problem for temporal point processes.  This allows them to apply a number of common goodness-of-fit measures to detect the out-of-distribution data (e.g., KS tests on arrival or inter-event times), and propose other statistics (specifically, 3S or "sum of squared spacings") and test for typicality of the statistic values under the training dataset.  Overall reviewers found the work clear and novel and recommend acceptance.  Please see the reviews for more detailed suggestions to improve the presentation.

Decision: Accept (Poster)