Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 9: Top 15% of accepted NeurIPS papers, strong accept
review_content: summary: This outstanding paper presents an extremely novel approach (for the machine learning community) to the Gaussian process formulation. This formulation is interesting in its own right. But, the paper goes beyond the pure GP interpretation aspect and introduces this formulation for a very specific and precise purpose: to handle the challenging Poisson process likelihood function. This likelihood function is non-i.i.d. as it involves and integral term (equation 2 of the submission). The integral formulation leads to sneaky analytical tricks on the Poisson likelihood, and provides a new way to handle the interesting "doubly stochastic" inference problem.

The authors import some highly technical and novel concepts from physics, to attack the latent GP intensity function of the Poisson process. This leads to a highly efficient and principled algorithm. Moreover, the high degree of novelty is such that the technique may be applied to other problems with latent GP intensity. For example, their method would also allow to handle the likelihood function of survival analysis with latent GP + link function for the hazard function, which also includes an integral term.

The writing is crisp and compelling. This was a pleasure to behold. The experiments are more than sufficient. It is impressive that the authors made this work in practice, and I hope they see fit to release their code, but if they don't it is okay and the paper more than stands on its own.

main_review: My very positive appraisal is in the overview section. Here I give some very minor constructive feedback. 

My main comment regards the discussion of computational complexity. The abstract is slightly misleading, since the stated big O pertains to one iteration, but the number of iterations required clearly depends on N. It would be great to see a table of computational complexities per iteration, for the various alternatives algorithms. This could be broken down by "predictive mean, predictive variance, marginal likelihood", and in some cases with an alternative for primal and dual formulations. Also, the O(N^2) on line 256 is a little unclear as the methods under question can be implemented in primal or dual form (scaling in # basis functions of # data), and in the primal form do obtain a similar scaling to the proposed method. 

These are minor constructive criticisms and would be trivial to handle in the camera ready version. 
limitations_and_societal_impact: Yes.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper gives a path integral formulation of a Gaussian Cox process (GCP) with an accompanying inference framework for efficiently resolving the integrals. The proposed method is empirically validated on a series of synthetic and real-world data examples where the results seem promising.

main_review: I found this article to be an enjoyable read. The authors do a very good job of motivating the topic before presenting the proposed model in an intuitive step-by-step framework. There is a lengthy related work discussion that effectively places the proposed model within the GCP literature. Finally, the experimental results are compelling with the proposed path-integral GCP obtaining comparable performance to existing methods, but with substantial computational accelerations. It is of note that some of the concepts in this paper were new to me, however, the accompanying appendix does a good job of providing background information for readers like myself. I have some minor points that I will outline below.

Main comments:

1. Are you able to provide error bars in Figure 2? It would be interesting to see how stable each of the methods is on real-world data.
2. What is the memory footprint of your method in comparison to STVB and VBPP methods? Given the method's computational runtime performance, this result would be of interest. 
3. Could you clarify what is meant in lines 94-96 of the Appendix? You mention that the kernel's hyperparameters are learned through optimisation of the marginal likelihood, but then write that the same hyperparameter is learned through grid search which confuses me.
4. I am interested to see how this approach scales to large data sets. I can see that you have considered an example with up to 110705 points in Section 8.3 of the appendix but you only report timings. Do you have any metrics to show the quality of your model here, such as test log-likelihood, as I would be interested to see if the model's capacity to recover the underlying process is maintained with such large datasets?


Minor comments:

1. Can you explain the behaviour of PIFq in Figure 2? The model seems unstable 
2. Typo in the caption of Figure 2 - "over10 hours".
3. Given the similarity of all five models' behaviour in Figure 1, I found the plots quite hard to review. My opinion is that these results may be better suited in a table.
4. I agree with the authors that providing a measure-theoretic formulation of the path integral would be a useful next step for future work in this area.
limitations_and_societal_impact: The authors do highlight two limitations of this work.
review_title: null

final_score: rating: 5: Marginally below the acceptance threshold
review_content: summary:    The manuscript tackles the problem of scalable and tractable approximate inference for Gaussian Cox models using a path integral approach. Experiments on synthetic 1D data and on 2D neural data illustrate certain properties of the approach.

   Strengths
   + Gaussian Cox processes are a relevant class of Bayesian models that are computationally intractable and hence a meaningful research objective.


   Weaknesses
   - The paper does not come with code; hence it will be hard to reproduce the results.
   - The data sets considered in the main paper are rather small compared to [24]. A direct comparison to [24] in the main paper would have been instructive.
   - The empirical evaluation does not support a solid claim on the runtime and memory footprint of the method.
   - An empirical comparison to previous methods e.g. [24] is only partly available.
main_review:    1) Clarity

   The technical content of the paper on page 3 in particular which is concluded by the strange sentence in lines 117-118 is hard to grasp as the authors chose to use an "intuitive" way of deriving the results. A concise summary of the key expressions and the relevant computations e.g. in an algorithm box would have been beneficial.

   2) Originality

   The use of the path integral methodology seems not explored beforehand.

   3) Significance

   The method is specific to the Gaussian Cox process and the benefits w.r.t. to previous methods in terms of theory and empirical performance are not fully established. Hence, the significance of the work is limited.

   4) Reproducibility

   As the code to run the experiments is not included in the submission, it should be rather difficult to reproduce the results.

   5) Empirical analysis

   The manuscript contains experiments on synthetic and real-world data. But the evidence backing up the "fast" from the title is limited. It is hard to assess how accurate the approximation is and how it compares to other competing methods e.g. [24]. An experiment backing up the claim in the abstract on the benefits in the "multidimensional input setting" where "the number of inducing inputs tends to be large" should be added.

   6) Minor Issues and Typos

   line 3: In this paper, we propose
   line 49: inferencing, lines 80 and 92: of GP prior, line 133: is that it can
   Figure 2: over10
   Capitalisation in References: Integralgleichungen, Anwendungen
limitations_and_societal_impact: Yes.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The paper addresses the inference of Gaussian Cox processes (GCP), and provides the path integral formulation to derive a MAP estimate and a Laplace approximation for Bayesian posterior inference. The strongest contribution is, that the formulation holds for general GP prior kernels, and for a variety of link functions. Comparison of the new methods is done on simulated and real datasets.
main_review: I found the paper nicely written, and the derivations elegant, and consider them as a good contribution to the active research problem of GCP inference. The formulation allows usage of various link functions and kernels. However, when it comes to the algorithm, I feel that the authors are not discussing all practical challenges.

+ A major point is the optimization of hyperparameters. I know, that is always an ugly task, but it is important for practical applications, and not discussed at all in the manuscript (at least I didn’t find it). I have the feeling, that it is difficult with the proposed task, because you would need to recompute the eigenfunctions of the kernel, every time hyperparameters are updated. That is potentially the reason, why the authors resorted to grid search. However, other approaches allow for optimization with gradient methods (e.g. the VBPP). Also, it is not clear to me, whether the grid search is included in the CPU times, that are reported.
+ I like the complexity analysis, but I miss the discussion of the complexity of finding the eigenfunctions of the kernel operator. Though, this has to be done only the beginning (as long as kernel is fixed), I think, it should be discussed here. Also, that for finding the basis, D one dimensional integrals have to solved numerically, should be part of the story. In general, I’d appreciate, if the authors could include a bit more details about finding the eigenfunctions in the main text.

## Update

I thank the authors for their response, and I raised my score given the promised changes of the manuscript. I think the final manuscript will be a strong publication.
I believe the paper is interesting, and I don’t consider any of the points above as a deathblow to the paper, but I would like to see them discussed.

Some minors:
+ Would be interesting to see comparison also with [24] and [13] in the comparisons, since the first has same complexity as the presented method and the latter is a very related method, as the authors state themselves. If code is available, that is.
+ I didn’t like that t is used for the space variable and transpose operation. Maybe use \dagger or \top for the latter.
+ L.125: an action integral

limitations_and_societal_impact: For limitations see above. Societal impact is briefly discussed at the end of the manuscript.
review_title: null

Meta-review: This paper presents a path integral formulation for inference in Gaussian Cox process models, which is general for any prior kernel and a variety of link functions. Three out of four reviewers recommend acceptance, with one of the reviewers assessing the paper as a top 15% of accepted papers at NeurIPS. The main criticism of the paper (raised by Reviewer SxYf and shared by other reviewers and myself)  is that the lack of commitment from the authors to make the code publicly available will impede reproducibility. I agree with this point but I understand there are cases where this is not possible. I believe the technical contribution of the paper is strong enough to be beneficial to the NeurIPS community but encourage the authors to improve the reproducibility of their approach. For example, they can include an algorithm (in pseudocode) in the main paper detailing the exact computations required in their method. Besides addressing the reviewers’ concerns, I also encourage the authors to discuss the limitations of their approach a bit further, for example by comparing hyper-parameter estimation in their approach versus other methods such as those based on variational inference.
Decision: Accept (Spotlight)