Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.
review_content: summary: This paper proposes an approach for making Maximum A Posteriori (MAP) inference on the intensity function of a Poisson process, regarded as a function of covariates. The representer theorem, which the authors show applies when a Gaussian Process prior is placed on the square root of the intensity function, is used to achieve a complexity linear in the sample size.
strengths_and_weaknesses: 
**It is unclear why this work is needed. Do we really need Augmented Permanental Processes?**

There is nothing in the theory of Poisson processes that requires the input space to be time, space or space-time. 

If the authors want to infer the intensity function as a function of covariates, they could simply define the space of points to be the space of covariates, namely $\mathcal{Y}$. 

The Poisson likelihood would simply read $p(\mathcal{D} \vert  x(\textbf{y}))= \sum_{i=1}^n \log \lambda(\textbf{y}_i) - \int_\mathcal{Y} \lambda(\textbf{y}) d\textbf{y}, ~~~ \lambda(\textbf{y}) := \kappa(f(\textbf{y})),$ and the prior would read $f \sim \mathcal{GP}(0, k).$

For all practical purposes, this would work even if the mapping function $g: \textbf{t} \to \textbf{y}$ is not injective.

Which begs the question: what contribution does this work make to the literature on Permanental Processes [14, 24, 46]?

questions: 
Why do we need Augmented Permanental Processes? Why not use Permanental Processes in the covariates space?
limitations: N/A
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposes an efficient Beyian algorithm to estimate the intensity function as a function of Covariates in a Reproducing Hilbert Kernel Space (RHKS). The purposed algorithm is based on an augmented permanent process and enjoys significant computational advantages. The effectiveness of the proposed algorithm is demonstrated through numerical experiments.
strengths_and_weaknesses: Strengths: the paper is well written, the presentation is clear,  the motivations are valid and the proposed approach is novel and effective.
Weaknesses: there might still be computational bottlenecks when the number of covariates is large; the synthetic data experiment can be improved.
questions: The proposed approach is essentially an extension of Flaxman et al (2017), which focuses on estimating intensity function as a function of time/spatial locations. But I think the extension is meaningful and solves an important problem. My only concern is about the computation of the Equivalent Kernel in Section 2.4. Because of the change of support from t to y, the integral equation (9) indeed can be computationally intensive to solve when the dimension of covariates is high. This somehow offsets some benefits of kernel ridge regression in RHKS, which can be used to fit high-dimensional functions using kernels such as the multiplicative Gaussian kernel. The algorithm in 2.4 essentially proposes to approximate the integral equation (9) through J-point numerical integration. However, when the dimension of y is large, the required J would grow exponentially, and the proposed approach cannot be applied to the multiplicative Gaussian kernel. One possible solution is perhaps to use a stochastic approximation of (9) in a similar fashion to those in  [1]. It will be helpful to give some discussion on this issue.

In addition, an important class of nonparametric models in Statistics is the additive model. So if the reproducing kernel K(.,.) is additive, would the Equivalent Kernel also be additive? If this is the case, the computation of the equivalent kernel can be greatly reduced for the additive model, which would be a great contribution as well.

The simulation study only covers 1-D functions. It would be better if at least 2-D functions can be studied, just to be consistent with the real data analysis.

Finally, the title seems to emphasize "inference" using the proposed method. But I do not see any inferential results in numerical studies. What kind of inferences can be carried? Can you give some examples?

[1] Xu, G., Waagepetersen, R. and Guan, Y., 2019. Stochastic quasi-likelihood for case-control point pattern data. Journal of the American Statistical Association, 114(526), pp.631-644.
limitations: none
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The paper proposes a new inference method for a variant of a Gaussian Cox process, which the authors call augmented permanental process (APP). The model is described as a point process that uses a Gaussian process on the covariate space with an intensity function based on a square link function. The main focus of the paper is to develop a tractable Bayesian inference to estimate the posterior distribution of the latent intensity function that is based on a novel application of the represent theorem. In the next step the predictive distribution is approximated using a Laplace approximation of the posterior. The method is evaluated on a synthetic and three real-world datasets.

Update after rebuttal: increased score to 5.
strengths_and_weaknesses: *Strengths:*
The paper is well written and the math seems to be sound.

The paper develops interesting theory deriving a representer theorem for the latent intensity function. Using this connection the authors obtain a MAP estimator that scales linearly in the number of datapoint. The theoretical insights could be used beyond the scope of the paper.

*Weaknesses:*
The empirical evaluation of the method is not overwhelming. 

First, on all four considered datasets the benefits of using the new method seem to be rather marginal. The performance gaps to the reference methods are rather small. For instance, when comparing KIE_ra_SI and APP_NYS on the real-world datasets we get the log-likelihood values  5.56 vs 5.56, 2.92 vs. 2.87, and 2.39 vs. 2.36. It is not clear to me if these improvement are of any practical value. If the authors belief so, could you find another metric or experiment demonstrating the benefit. If the benefit is speed (at least in comparison to INLA), it would be helpful demonstrating the benefit clearer. For instance, you could show a plot with dataset size on the x-axis and inference speed/prediction performance on the y-axis.

Second, the datasets seem rather small. It would be interesting to see how those algorithms scale with increased dataset size (see above). Is this an area where you method can shine? For instance, due to the linear scaling can we use APP on huge datasets where competing methods fail?

As stated above the paper makes interesting theoretical contributions which is focused on the proposed model. If not demonstrating clear empirical benefits, another way where the paper could have impact would be to extend the theory and to show how similar ideas could be applied beyond the scope of the specific model.

The presentation is quite dense and might be improved by some visual guiding, e.g., a model diagram.



questions: Could you provide a discussion of the accuracy of your approximation methods? How much does the Laplace approximation affect the results? What is the effect of approximating the kernel?

For me the discussion of the "degenerate approach" came a bit late. In the section before it was always given that we have degenerate kernel. It would be helpful to emphasize in the beginning that we can obtain it by different approximation techniques.

Here you could also discuss the connection to inducing point GPs (which are related to the Nystrom approximation)?
limitations: The authors discussed the limitations.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper studies a fast Bayesian inference of point process intensity as a function of covariate.
- Using the path integral formulation of Gaussian processes, the maximum a posteriori (MAP) estimate can be obtained by minimizing the action integral, which results in an optimization problem with O(N^2) time complexity for each evaluation. 
- In particular, as the quadratic link function is used in this context, the Nystroem method, and degenerate kernel method are employed to solve integral equations.
strengths_and_weaknesses: Originality:
-The novelty of this paper lies in the nonlinear dependence of the latent intensity on covariates, \lambda( K( y(t) ) ), compared to the previous work [Hideaki Kim 2021], in which the latent intensity is given by \lambda( K( t ) ).

- In particular, this paper employs quadratic link function, and thus enables to obtain a linear integral equation, which is first solved using Nystroem method, and degenerate kernel by this paper.

- I consider that most results in deriving MAP estimate, marginal likelihood and predictive distribution in Sec 2.2 and 2.3 are a bit incremental, given the results by [Hideaki Kim 2021]. 

Quality:
The work appears to be sound.

Clarity:
Overall, the methodology is well documented, and the main derivations are easy to follow.

Significance:
The work comes up a faster Bayesian inference method for covariates-dependent point processes, compared to previous MCMC and VB methods using domain discretizations. Thus, I would think it will make some impacts on the Bayesian inference community for point processes although the novelty is not that strong.
questions: Can you provide the running time comparisons of your method against the conventional Bayesian method, such as INLA, for the final experiments? 
limitations: The authors claim that the proposed method may harm people's privacy by predicting their fine temporal patterns.
I expect the authors to present more in-depth analysis about the limitations of the method.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper proposed an augmented permanental process whose intensity is modeled by a Gaussian process on covariate space, and derive a efficient Bayesian inference method without the need of discretizing the domain. The proposed algorithm is based on path integration and Laplace approximation, which facilitates the computation of predictive distribution (uncertainty) and marginal distribution (hyper-parameter). Also the paper designed different methods, including naive methods, Nyström approximation and random features, to construct the equivalent kernel. 
strengths_and_weaknesses: strengths:
The paper is writting very well and easy to follow. The methodology is solid and seems correct to me but I did not check all the derivations carefully. 

weaknesses:
However, I have two concerns about the current version, which makes me hasitate on the rating of this submission. 
The first is the experimental comparision with baseline models is rather weak. The proposed methods (NAI/NYS/RFM) are only compared with very old works. For example, KIE and INLA are published in 2012. I did not find any SOTA baselines in this work, which cannot convince me of the merits of this work. 
The second is the contribution of this work is rather incremental. The key components of the proposed inference method all come from previous works, e.g., the MAP estimate using path integral is from [1], let alone the Laplace approximation of posterior and Nyström or random features approximation of kernel. 

[1] Fast Bayesian inference for Gaussian Cox processes via path integral formulation. Neurips 2021.
questions: I suggest to provide more experimental comparison results with SOTA baseline models, e.g., sigmoidal Cox process [2], permanental process [3]. Provide more analysis expeirmentally or theoretically on the ' Nodal Lines' problem which is a common issue in permanental processes. 


[2] Efficient Bayesian Inference of Sigmoidal Gaussian Cox Processes. JMLR 2018. 
[3] Large-Scale Cox Process Inference using Variational Fourier Features. ICML 2018. 
limitations: The work does not have potential negative societal impact. 
The authors did not mention the limitations of their work if I did not miss any important points.
review_title: null

Meta-review: metareview: The paper looks a the Augmented Permanental point process as a model of (spatial) point phenomena. 

One reviewer was unconvinced that the method was needed at all, which the authors refuted. There was a long exchange but I'm on the side of the authors here - the method is clearly distinct from a point process defined on the covariate space. I'm happy that the reviewer was able to make their point and that the discussion was enabled, and I applaud the authors for their patient responses. I'm disregarding that reviewer's score. 

The reviewers suggest that an important and interesting contribution is the representer theorem for squared processes. I also appreciated the authors discussion on the approximation error on the integral operator. These should be highlighted in the manuscript. 

There was the occasional confusion from the reviewers on notations: for example, the reviewers failed to spot the the performance of the method was presented in the paper using the $\tau$ column in the tables. Please, double check all the reviewer feedback for clarifications. 

Overall, I think that there are a couple of interesting ideas in the paper that people working on Point Process data will be impacted by, and am recommending that this is just above the acceptance threshold. 
confidence: Less certain
Decision: Accept