Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: summary: This paper proposes a mixture model of Multi-level Marked Point Process (MM-MPP) for repeatedly observed multi-category event sequences and develops a efficient semi-parametric Exponential-Solution (ES) algorithm for event sequence clustering. Experiments through simulation studies and a real-world data analysis show the effectiveness of the work.
main_review: Pros:
This paper considers the event sequence clustering problem, which exists widely in practical scenarios. 
Sufficient details of the learning algorithm are given.

Cons:
I’m confused what exactly the ‘multi-level’ refers to. The authors say their model is ‘multi-level’ as the event sequences they target at are ‘repeatedly’ observed. It seems the ’level’ refers to ‘day’ considering the Twitter dataset and section 4. However, according to line 42-44 and section 3.1, the ‘level’ is only related to event types. Then if temporal dependencies are ignored in this work, why is the ‘repeatedly observed’ or ‘multi-level’ important? 
Moreover, the framework assumes there is no temporal dependencies between events. I doubt the rationality of this assumption, especially considering the Twitter data, in which former events would obviously influence the happening of latter events. 
The baselines [15] and [27] were published in 2013 and 2017, respectively. There should be more advanced frameworks proposed for event sequences clustering since 2017. I suggest the authors make broader literature review and add more competitive baselines. Also more real data sets should be tested for evaluating the effectiveness of the proposed framework.
The computational complexity is discussed, but I would suggest the authors add the convergence rate analysis.

--------------------
Update:

Thank the authors for their feedback. However, my concern is only partially served. I keep my score.
limitations_and_societal_impact: There is no negative societal impact of this work.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The paper proposes a clustering method for repeatedly observed marked point process data. The method is based on a mixture model of multi-level log-Gaussian Cox processes. The paper develops an expectation-solution algorithm for implementation, where the solution-step has a closed-form solution. The study is motivated by a Twitter dataset, where the postings (tweet, retweet, or reply) are treated as point processes.
main_review: The paper is on an interesting topic: model-based clustering of complex point patterns. Existing methods build on mixture models of Hawkes processes and rely on EM algorithms for estimation. The proposed method is complementary to previous work and arguably more flexible and computationally efficient. The paper is generally well written.

Specific comments:
1) The abstract should be rewritten to improve clarity. “Expectation” is misspelled as “Exponential.” Description of the point process data is not clear, and no motivating example is mentioned.
2) From the abstract, functional PCA seems a key component in the proposed algorithm, but is not mentioned in the introduction.
3) Is there a convergence guarantee for the proposed algorithm?
4) When should one choose to model the event sequences by the Hawkes process or the log-Gaussian Cox process? Some discussion would be helpful.

Update: The authors have addressed my concerns very well. It is a good paper, and I will keep my rating.
limitations_and_societal_impact: The paper notes the potential risk of misinterpreting the clustering results.
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: This paper presented a method of multi-level marked point processes for clustering using a mixture model. The idea is to model each component as a Gaussian Process w.r.t time t, where the multi-level model is based on the single-level model as well as a semi-perimetric method in [26]. ES method is chosen with modification incorporated from the estimating equations. In E-step, FPCA and MC methods are used to reduce the computational costs. Synthetic experiments validate the correctness of the model and the real-world Tweet dataset is designed to explore the clustering results in an explanatory way. The main contributions in this paper are a new statistical mixture model for repeatedly observed multi-category event sequences and a corresponding semiparametric ES algorithm instead of the classical EM algorithm.


main_review: This paper is generally well-written with strong motivation, a well-defined approach. The main concern for me is regarding the experimental design.

The authors claim that "design an algorithm that can take advantage of array programming and GPU acceleration to further speed up computation", however, not enough discussion and numerical results showed that GPU or array programming can facilitate the computation.
Another claim is "existing EM algorithms for event sequence clustering have a common issue that they typically require iterative numerical optimizations within each M-step, which would drastically overburden the computation." I am wondering if EM can be substantially slower than ES due to the reason described above. Having an iterative scheme in the maximization step might not lead to slower convergence comparing with ES. It might need further empirical verification or theoretical convergence rate analysis.
What is G defined as on page 7? When R is large, how does the sampling complexity is decreased to O(nCQR^2) from O(nCQR^3)? I have no idea what the exact meaning of "R is small to moderate" is. Could the authors show the complexity of R when it is defined as small to moderate?
In real-world experiments, the clustering stability called clustering consistency is evaluated. I am not 100% sure if it is the best way to verify the model's efficacy. The statistical model is built to solve a very specific application problem, so the clustering validation is of importance when one wants to use the model on other datasets. If no proper clustering validation measure is designed, one dataset with K-trial cross-validation seems not enough to convince me at least. I am wondering if the model can work when it is used on some clinical longitudinal datasets such as drug addiction. (https://catalog.data.gov/dataset?tags=addiction)
Figure 3 shows the mean of each cluster, but I am also curious about the in-cluster variance, i.e., the homogeneity in each cluster so that the cluster results are seriously discussed.
Figure 3 in the paper is not clear. Please use a finer resolution and consistent font sizes.

------
Update: While I appreciate the authors' comments, and some of their points are correct, my main concern - which is the experimental design of the paper has not been resolved by their comments. The simple fact that a variety of similar datasets are not used in the paper is highly problematic. Some of the clarifications on the faster convergence speed over EM and the higher GPU efficiency of the method regarding the experimental design sadly do not either answer the questions posed in the review.

Overall the author's comments have not convinced me to change my assessment of the paper. If a future version of the paper would provide a more comprehensive comparison and firmly justify the benefits of their proposed approach, I would be more than happy to accept that version.
limitations_and_societal_impact: Yes. The authors addressed the limitations and potential negative societal impact except for the lack of experimental design.
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary: The paper proposes a method of clustering event sequences, where each sequence represents a series of actions done by an entity, e.g., a tweeter account. The paper models the sequence of each account at each day as a sample of a inhomogeneous Poisson process. The log intensity is modeled by a summation of the account-level, day-level and (account, day)-level latent functions of the time. To cluster the data in the account level, the account-level latent function is assigned a mixture GP prior while the other latent functions GP priors. The paper proposes to use Expectation-Solution (ES) for the model estimation (including estimating the cluster memberships). The paper proposes estimating equations, and performs ES steps which are analytical, and do not require numerical optimizations. The experiments show the effectiveness of the method.  
main_review: I like the topic of clustering event sequences; I am also interested in the ES framework introduced by the paper, which seems solid and has some advantages. However, there are several significant issues in the presentation and experiments that need to be addressed.  

(1) The paper does not well explain why they use inhomogeneous PP. The paper emphasizes that Hawkes processes are "are not suitable to model the data in our case that have multi-level variations". I do not believe so. If for each account you  merge the sequences in different days into one sequence , and model the whole sequence as one Hawks process --- I do not see any problem. You can model the base rate with the same structure as in (1), but also account for the excitation between different events. In fact, it is even more expressive. 

(2) While the ES framework seems interesting, the paper never explains  how it comes out the estimating equation (7). How does it connect to the model likelihood (3). How do you justify it? What is the error of the ES estimate? etc.  From the description of the paper, (7) is proposed more like for easing the S step (avoiding the numerical optimization).  

(3) The paper has messy, undefined/unexplained notations. For example, what is \Gamma_{x,c}^r in (5), \Gamma_{x,c}^{r,r'} in (6)? Do the two Gammas refer to the same object? Why do you introduce a kernel K_h? Why do you use Campbell's theorem?  

(4). There is only one weak baseline.  Why don't you compare with [27], the most recent method of clustering event sequences with Hawkes processes? 
limitations_and_societal_impact: N/A
review_title: null

Meta-review: As pointed out by the reviewers, the paper is missing comparisons against more recent baselines. There are plenty of these baselines that the authors simply did not even mention. The paper also focuses on a single real data example. Including more real-data examples is necessary to assess the generality of the method.  I believe that in its current version the paper is not ready yet for publication.
Decision: Accept (Poster)