Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper proposes a method for modeling exchangeable sets of data, or point processes. Specifically, the paper is interested in applying normalizing flow methods to these point processes. The paper proposes a method using continuous normalizing flows, and compares the performance of their proposed method, Confet, to baselines on simulated and real data. The paper finds that Confet outperforms the other methods in terms of test loss.

The specific method proposed by the paper relies on continuous normalizing flows. These methods take a sample from a simple base distribution, and transform these samples to a more complicated distribution whose density can be calculated by solving an ODE. This method requires specifying a transformation $f$ whose trace Jacobian can be computed efficiently. In the case of point processes, $f$ needs to be _equivariant_, meaning that permuted inputs to $f$ must have the same output up to the same permutation. The authors use neural networks with equivariant layers, and propose three methods for computing the trace: a MC estimate using Hutchinson's estimator, fixing $f$ so the trace is 0, and exact calculation. In experiments, they find that the exact and stochastic computations perform well.

Point process models are an important area, and this paper does a nice job combining two fields: set modeling and normalizing flows. The paper is clear throughout, and the proposed method is simple and well-motivated. It's also nice that the paper provides three different trace estimators and compares their qualities.

My main criticism is that this paper describes an incremental improvement. It is not a new idea to use normalizing flows for modeling exchangeable data -- for example, the authors cite work by Bender et al, which also uses equivariant normalizing flows for exchangeable data. There is not much novelty with the proposed inference method; Hutchinson's estimator and exact trace computation are two common ways to train continuous normalizing flows.

It is not inherently a problem to me that the proposed method only makes incremental changes from existing methods; if experiments are convincing of the proposed method's performance, it would be a reason to accept. However I am not convinced by the current set of experiments. They compare to three baselines -- autoregressive, IHP, and IWAE -- which all consist of some kind of normalizing flow. Since the experiments only compare to baselines that use a normalizing flow for point process models -- rather than the most common point process models -- I'm not sure which hypothesis these experiments are trying to test. Is it that continuous normalizing flows are more appropriate than non-continuous normalizing flows in this context? This doesn't seem as useful to me as an experiment that would show that continuous normalizing flows outperform common and non-flow based models like DeepSets or Set Transformer. 

Additionally, the experiments would be more convincing if there was a more extensive set of real-world datasets. The experiments are on two check-in datasets and one crimes dataset -- all location data. There exist a variety of point process/set data sets, such as point clouds, brain imaging data, and image anomaly detection. Because the real-world experiments are only on one kind of dataset, it's not convincing that the proposed model is useful for general sets and point processes. 

In conclusion, I think the proposed idea is interesting and could be useful. However, it is incremental, and the current experiments do not sufficiently show that the model is important.

Pros:
 - straightforward idea
 - paper does a good job combining two fields: point processes and normalizing flows
 - clearly written and model is reproducible

Cons:
 - proposed method is incremental
 - insufficient experiments
review_title: Interesting application of CNFs to point processes, but perhaps incremental

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper introduces a tractable likelihood model for point processes named Confet. The paper combines continuous time normalizing flows with graph-network equivariant transformations. They introduce some variations (stochastic/fixed/exact) with different approaches to compute the trace of the Jacobian.


Strengths:
The paper is well-written and introduces the subject really well. The background material is covered extensively and makes the paper pleasant to read. The illustrations are used nicely to illustrate concepts. The variations of the methods (stochastic/fixed/exact) is a nice addition in the context of point process modelling.


Weaknesses:
My main concerns stem from similarities between the proposed methods and existing methods in literature. The paper is correct that Eq. (8) is more expressive than the transformation in Kohler et al, but the reason for this is that Kohler et al. enforced an additional equivariance constraint on physical geometry. It is somewhat straightforward that if this constraint is alleviated, the functions g and h can be a more expressive neural networks. Further, there seem to be connections with graph normalizing flows that are not explored, as point clouds can be interpreted as graphs with edges between all points. Examples of works in this area are (Liu, et al. "Graph normalizing flows." 2019) and (Shi, et al. "GraphAF: a flow-based autoregressive model for molecular graph generation." 2020). Further, in the experimental section it would be good to compare against the method proposed by Kohler et al due to the similarities. In addition, I would like to request the authors to clarify and disentangle their work from Graph Flow literature and from the paper by Kohler et al.. 


Final comments:
I would like to again thank the authors for their time and responses. My concerns regarding novelty remain as described below, but the authors did clarify some aspects. For these reasons, I have raised my score from 5 to 6. If the work is not accepted at this venue, I would like to encourage the authors to continue with their work and submit to a later venue.
review_title: review for Equivariant Normalizing Flows for Point Processes and Sets

final_score: rating: 5: Marginally below acceptance threshold
review_content: Summary:
This paper proposes a method based on continuous normalizing flows that can model random sets of exchangeable points. The advantages of the method are as follows: (1) it can handle a complex density function of sets and (2) it is designed so that model learning can be performed based on tractable likelihood while considering dependencies between samples. The effectiveness of the proposed method is shown in the experiments using synthetic and real-world datasets. 

Pros:
1. This paper is well-organized and its presentation is good. Related works are sufficiently cited.
2. It is a first attempt to unify point process models with normalizing flows, which might lead to a new potential research direction. 
3. The authors present a new model, CONFET, which is well-designed so that the model is flexible and its learning is efficient. 

Cons:
1. This work is a good combination of several recently-developed techniques and it does not include a great technical contribution.
2. The goal of this work is misleading to me. The proposed model, CONFET,  is for the probability density estimation of sets and may not be for point process intensity estimation. See detailed comments. 

Reasons for score: 
An attempt to unify point process models with normalizing flows is interesting. Also, the proposed method, CONFET, is well-designed. Overall, the paper is well-written. But, several points are misleading to me; this is important in the determination of the placement of this work in the research field. Accordingly, for now, my opinion is that this paper is marginal. 

Detailed comments:
This work is based on the result in (Yuan et al., 2020, Theorem 1), and the CONFET aims to model the probability density function $p(\bf{x}) = \lambda(x) / \Lambda$ directly, instead of handling Poisson process intensity $\lambda(\bf{x})$. I agree that this option is advantageous to avoid the integral in the likelihood (Eq. (2)) and to utilize rich density estimation methods. However, the aim of Poisson process modeling is to discover the distribution of the number of points and the distribution of their locations, as the authors themselves mentioned. In the CONFET, the distribution of the number, $n$, of points are assumed to be a Poisson distribution with mean $\Lambda$, that is, $n \sim {\rm Pois(\Lambda)}$ (Is that correct?); it is a simplistic assumption. In order to learn the complex Poisson process models, I think that it would be essential either to model the flexible mean $\Lambda$ or to handle the standard likelihood (Eq. (2)) as in Cox processes. 

Also, the evaluation in Section 6 is for the probability density estimation methods. If the authors state that this work is the proposal of the new Poisson process models, the model comparison should be conducted with consideration of the estimation performance of $n$. In that case, it would be better to add the comparison with the recently-published Cox process models (e.g., [R1]). Another option is to place this work as the method of determining point locations given the number of samples $n$.

Could you please address and clarify the above concern?

[R1] C. Lloyd, T. Gunter, M. A. Osborne, and S. J. Roberts. Variational inference for Gaussian process modulated poisson processes. ICML 2015, 2015.  
review_title: Official Review

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: *Summary
This paper provides a novel method of learning density models for sets of points by modelling the sets as samples from a point process, approximated with normalizing flows.  A point process gives a probability to a set of points, not assuming that the points are independent of one another. The authors describe the CONFET method, which uses continuous normalizing flows to map from a uniform point process with a learned transformation. The authors describe a method to tractably compute the exact trace of their transformation, allowing it to scale to high dimensions and numbers of points. Experiments show state-of-the-art performance on benchmarks.


*Positives
The paper is generally well-written and motivated well.
The exact computation of the trace in this setting is a key aspect to the tractability of this work, and is a good contribution.
The experimental results seem convincing that the overall method is an advance in the state-of-the-art of modelling point datasets.
The appendices are discursive and comprehensive

*Questions
How does this method perform on tasks such as modelling point clouds?

*Recommendation
Overall I recommend to strongly accept this paper.  The trace-computation method is a nice fundamental contribution, which is used to present a compelling set of experimental results.


*Minor points
Typo in paragraph 1 of section 6.2: 'studies are in D' should be 'appendix D'
Typo in paragraph 2 of section 2, 'Symmetry requirement comes from' should be 'the symmetry requirement...'
Figure 5 might be better off in a different colour scheme than green/red for those with colourblindness
review_title: Good contribution, well written; accept

Meta-review: The authors model point processes with equivariant normalizing flows. Reviewers agreed that the paper is well written and addresses a problem of interest to the ICLR community, some reviewers considered the contribution to be incremental.  Perhaps the biggest contribution is a closed form expression for the trace that needs to be computed as part of the normalizing flow, which is valuable but not particularly emphasized. The authors combine this trace formulation with an equivariant normalizing flow to model the conditional density of point locations given cardinality. (As an aside, it was unclear to me if and how those conditional distributions share parameters; in some contexts, the conditional density could look very different depending on the number of points in the set.) Overall, the paper is interesting but needs a little more to lift it over the bar.
Decision: Reject