Nonstationary Sparse Spectral Permanental Process
Authors: Zicheng Sun
Yixuan Zhang
Zenan Ling
Xuhui Fan
Feng Zhou
Venue: NeurIPS 2024 poster (2024)
Keywords: permanental process
nonstationary kernel
Bayesian inference
Primary area: probabilistic_methods

Abstract: Existing permanental processes often impose constraints on kernel types or stationarity, limiting the model's expressiveness. To overcome these limitations, we propose a novel approach utilizing the sparse spectral representation of nonstationary kernels. 
This technique relaxes the constraints on kernel types and stationarity, allowing for more flexible modeling while reducing computational complexity to the linear level. 
Additionally, we introduce a deep kernel variant by hierarchically stacking multiple spectral feature mappings, further enhancing the model's expressiveness to capture complex patterns in data. Experimental results on both synthetic and real-world datasets demonstrate the effectiveness of our approach, particularly in scenarios with pronounced data nonstationarity. Additionally, ablation studies are conducted to provide insights into the impact of various hyperparameters on model performance.

Nonstationary Sparse Spectral Permanental Process
Zicheng Sun's Yixuan Zhang Zenan Ling", Xongg"Xune Xuhui Fan" Feng Zhou 1,5
CCenter for Applied Statistics and School of Statistics, Renmin University of China School of Statistics and Data Science, Southeast University
Schooo of EIC. Huazhong University of Science and Technology
Schooo of Computing. Macquarie University Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing (sunzicheng2020, feng. feng.zhou) @ruc. edu. cn, zh zhixuanChotman com
Abstract
Existing permanental processes often impose constraints on kernel types or sta- tionarity, limiting the model's expressiveness. To overcome these limitations, we propose a novel approach utilizing the sparse spectral representation of non- stationary kernels. This technique relaxes the constraints kernel types and stationarity, allowing for more flexible modeling while reducing computational complexity to the linear level. Additionally, we introduce deep kernel variant by hierarchically stacking multiple spectral feature mappings, further enhancing the model's expressiveness results on both synthetic and real-world datasets demonstrate the effectiveness our approach, particularly in scenarios with pronounced data nonstationarity. Additionally, ablation studies are conducted to provide insights into the impact of various hyperparameters on model performance. Code publicly available https://gthub.com////C//////
capture complex patterns in data. Experimental
Introduction
Many application domains involve point process data, which records the times or locations of events occurring within a region. Point process models are used analyze these event data, aiming to uncover patterns of event occurrences. The Poisson process [1], as an important model in the field of point processes, plays significant role neuroscience
finance [1], criminology [3I,, epidemiology [5]. and seismology [ Traditional Poisson processes assume parameterized intensity functions, which severely restricts the flexibility the model. To address this issue, a viable solution is to employ Bayesian nonparametric methods, by imposing nonparametric Gaussian process (GP) prior on the intensity function, resulting in the Gaussian Cox process R This greatly enhances the flexibility of the model, while also endowing it with uncertainty quantification capabilities.
In Gaussian Cox processes, posterior inference of the intensity challenging because the GP prior is not conjugate to the Poisson process likelihood that includes an intensity integral. Furthermore, to ensure the non-negativity of the intensity, we need to use a link function transform the GP prior.
Commonly used link functions include exponential 220 12 sigmoid [ 9 6) square 18
ReLU 16 and softplus 24 Among these, using square link function, i... the permanental process allows for the analytical computation of the intensity integral [18 33 29), thus receiving widespread attention in recent years. For this reason, this work focuses on permanental process.
Currently, the permanental process faces three issues: (1) The permanental process inherits the notorious cubic computational complexity GPs, making impractical for use with large amount
"Corresponding author.
$ 'Equal contributions.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

of data [4]. (2) Existing works on permanental processes either require certain standard types of kernels, such as the squared exponential kernel, etc., to ensure that the intensity integral has an
analytical solution 18
7 3 or they require the kernels to be stationary [14 29 These constraints
limit the expressive power of the model. (3) Furthermore, existing works have predominantly utilized
simple shallow kernels, which restricts the flexibility of the model. Although some shallow kernels 34, [2) are flexible and theoretically capable of approximating any bounded kernel, they are limited in representing complex kernels due to computational constraints in practical usage.
In this study, we utilize the sparse spectral representation of nonstationary kernels to address these limitations in modeling the permanental process. The sparse spectral representation provides a low-rank approximation of the kernel, effectively reducing the computational complexity from cubic to linear level. The nonstationary sparse spectral kernel overcomes the limitation of stationary assumption. By treating the frequencies as kernel parameters for optimization, we can directly learn the nonstationary kernel from the data in a flexible manner without restricting the kernel's form. We further extend the shallow nonstationary sparse spectral kernel by hierarchically stacking multiple spectral feature mappings to construct deep kernel, which exhibits significantly enhanced expressive power compared to shallow ones, We term the constructed model as Nonstationary Sparse Spectral Permanental Process (NSSPP) and the corresponding deep kernel variant DNSSPP.
We conduct experiments on both synthetic and real-world datasets. The results indicate that when the data is (approximately) stationary. (D)NSSPP achieves similar performance to stationary baselines. However, when the nonstationarity in the data pronounced, (D)NSSPP can outperform baseline models, demonstrating the superiority of (D)NSSPP. Additionally, we perform ablation studies to assess the impact of various model hyperparameters.
2 Related Work
In this section, we present some related works on how to reduce the computational complexity of the stationary permanental process, as well as some research on nonstationary kernels.
Efficient Permanental Process Due to the adoption of the GP prior in the permanental process, posterior inference involves computing the inverse of the kernel matrix. This results in a computational complexity of O(N3), where N is the number of data points. To reduce computational complexity, several works have introduced low-rank approximation methods from the GP domain into the permanental process, such as inducing points [18), Nyström approximation 77 B3), and spectral representation [14 29]. These methods essentially involve low-rank approximations of the kernel matrix, thereby reducing the computational complexity from O(N3) ) to O R?), where R < N is the number of rank, such as the number of inducing points or frequencies. In this work, we utilize the spectral representation method, reducing the computational complexity from cubic to linear level. Another reason for adopting the spectral representation is that facilitates the construction of the subsequent deep nonstationary kernel.
Nonstationary Kernel Stationary kernels assume that the similarity between different locations depends only on their relative distance. However, previous studies have indicated that the stationary assumption is not suitable for dynamic complex tasks, as the similarity not consistent across the
input space [23]. To address this issue, some works proposed nonstationary kernels by transforming the input space 30 or using deput-dependent parameters [0]. Additionally, other works leveraged the generalized Fourier transform of kernels [37] to propose nonstationary spectral kernels [26 332
Although the aforementioned shallow nonstationary kernels are flexible and theoretically capable of approximating any bounded kernels, they are limited in representing complex kernels in practical usage. In recent years, many deep architectures have been introduced into kernels, significantly enhancing their expressive power [35]6]]. Despite the development of (deep) nonstationary kernels in GP, interestingly, to the best of our knowledge, there has been no prior work applying them in Gaussian Cox processes. This gap is what this work seeks to address.
3 Preliminaries
In this section, we provide some fundamental knowledge about the permanental process and sparse spectral kernel.

---

3.1 Permanental Process
For Poisson process, x E & RR,, ? the intensity function X(x) lim - E[ x+ 6x))1/xx can be used to quantify the rate of events occurring at location x. A Cox process can be viewed as Poisson process whose intensity function itself is a random process. The Gaussian Cox process employs the GP to model the intensity function: X(x) = 1o f(x) where / is a GP function and 1: R >> IR + is link function ensure the non-negativity of the intensity function.
The nonparametric nature GP enables the Gaussian Cox process to be flexible modeling events in space time. However, the posterior inference of Gaussian Cox process is challenging. According to the likelihood function of a Poisson process:
((xx)]= [A(x) - [(x)) - II (Xx exp (-/x M(x)dx)
N
i=1
and the Bayesian framework, the posterior of latent function f can be written
Il exp(- fxx ((x)dx)5P((10
p(I [x
(2)
M(x)) fxx X Xx))d))(P(110)
In Eq. (2), there integrals. The first one fg X(x)dx cannot be solved due the stochastic nature J. Additionally, the integral over f in the denominator a functional integral in the function space, which is also infeasible. This issue is the well-known doubly-intractable problem.
The link function
+ R has many choices, such as exponential, sigmoid, square, ReLU, and softplus. This work focuses the utilization of the square link function, also known as the permanental process. Furthermore, if we set the kernel in the GP prior to be stationary: x2) = k(x1 -x2), then the model is referred to stationary permanental process.
X2).
3.2 Sparse Spectral Kernel
The sparse spectral representation a common method for the low-rank approximation of kernels. This approach is based on Bochner's theorem.
Theorem 3.1. 12] A stationary kernel function k(x1, = k(xi - -x2) IR - is bounded, continuous, and positive definite if and only if it can be represented
expliw (x1 (xpliwT(xx: f
k(x1 x2)
-xx))dd((w))
where (((w)) is bounded non-negative measure associated to the spectral density p(w)
Defining dxx) exp(iw Tx), x), we have k(x; x2) o² Ed )0 [] where denotes the complex conjugate and o² = ((RPP) If we utilize the Monte Carlo to sample [w independently from p(w), then the kernel can be approximated by
k(x1 x2) R R
Σ to
( (x1) (oxs)TT(())))) (x2)""
where (x) = e x). + (x)
vR |chn (x),
dx)] which corresponds to the random
Fourier features (RFF) proposed by (25
can be proved that Eq. (3) is equivalent to 2R-sized
trigonometric representation (proof is provided in Appendix A
+ (1) (x)
[cos(w) x), ... cx), sin(w] x)...- sin.w:s))
R
When we specify the functional form of the kernel (spectral measure p), the RFF method can learn the parameters of the kernel (spectral measure μ) from the data. However, when the functional form of the kernel (spectral measure μ4) unknown, RFF can not flexibly learn the kernel (spectral measure μ)) from the data. Another approach is to treat the frequencies J,R as kernel parameters. By optimizing these frequencies, we can directly learn the kernel from the data in a flexible manner, which corresponds to the sparse spectrum kernel method proposed by [17].

---

4
Our Model
The permanental process inherits the cubic computational complexity of GPs, rendering impractical for large-scale datasets. To efficiently conduct posterior inference, many works employed various low-rank approximation methods 7| These methods require certain standard types of kernels, such the squared exponential kernel, polynomial kernel, etc., to ensure that the intensity integral has analytical solutions. This limitation severely affects the flexibility of the model. To address this issue, sparse spectral permanental processes proposed in recent years [4 [4). The advantage of this method lies in its ability to analytically compute the intensity integral, while not requiring restrictions the kernel's functional form. However, the drawback is that it is only applicable to stationary kernels. Therefore, a natural question arises: can we further generalize the sparse spectral permanental processes not only remove restrictions the kernel's functional form but also make them applicable to nonstationary kernels?
4.1 Nonstationary Sparse Spectral Permanental Process
Existing sparse spectral permanental processes rely on the prerequisite of Bochner's theorem, which limits the extension of the sparse spectral method to nonstationary kernels. Fortunately, for more general nonstationary kernels, the spectral representation similar to Bochner's theorem also exists.
Theorem 4.1. (37) A nonstationary kernel function k(x,, X2) : RDD RDD - R bounded, continuous, and positive definite if and only if it can be represented as:
x2x = [ JgpxRD exp (i(w] x - w/ x2)) w2),
where p(w), w2) is a Lebesgue-Stieljjer measure associated to some bounded positive semi-definite spectral density p(w),
p(w))w)))
use the Monte Carlo method to approximate it. However, it is worth noting that, to ensure that the spectral density p(wi, W2) is positive semi-definite, we must first symmetrize it, i.c., p(w1, w2) = p(w2, W1); and secondly introduce diagonal components p(w1, w/) and p(w2, w2y. We can take a general density function g on the product space and then parameterize p(wr, W2) in the
Similarly,
following form to ensure its positive semi-definite property (26 32]:
p(w1, (و2) = (o(w)) (22 + g(w2, w1) + g(w,, w1) + g(w), w2)).
(5)
4
Then, obtain the sparse spectral feature representation of any nonstationary kerne::
0² X2) Eggwe w2) [exp (i(wixx x1 w/ wxxx) x2)) + exp (i(w)) x1 xx)x
exp (i(w] x1 - W/ x2)) + exp (ilw]x, xI - x2)]
2
exp (i(wT,x: x1 w1,x2]) + + exp (i w1,xx))]
(6)
4R
Φ (1t) (x1) T 22 (1)) (x²) - -b// (2) (x1) T (1)) (x,) + क (8 (x1) T TT (11) (x2) + Φ (x1) (R) (x2)
(1)
(8)( (x1) T (R) (x2),
where 0² = MMRD,, RR,), (w)r, War R are independent samples from أ(wو, W2), and
$ (R) (x)
0
[oo((w)
cos(w] x) ( [i), sin(w[Rx))
2VR
8 (10 (x)
o
[cos(w]) x), co)(w)Rx)) sin x), "in([]]x)]))))
2/R
6 (8) (x)
o (R) (x)
+ क (R) (x).
If the GP's kernel in the permanental process (Eq. adopts the above sparse spectral nonstationary kernel, then we refer to this model as NSSPP.

---

4.2
Deep Nonstationary Sparse Spectral Permanental Process
We can further enhance the expressive power of the nonstationary kernel by stacking multiple spectral feature mappings hierarchically to construct a deep kernel. The spectral feature mapping sp (R)) (x) has another equivalent form:
(0(1) (x)
coo((w] x b11) cos(w>) x b21)................))).
bin) + coss(w2 b2n)
bbn))
V2R
(8) where {bi b2r b uniformly sampled from [0, 2m]. The derivation of Eq. is provided in Appendix
The spectral feature mapping in Eq. can be viewed more complex single-layer neural network: employs two sets of weights (w) and biases (b) to linearly transform the input, followed by the application of cosine activation functions, and then adds the results to obtain single output dimension. By stacking multiple layers on top of each other, we naturally construct deep nonstationary kernel:
(1 (x) = L o
(x), 1x(1, = (( (x1) (T(( (x2).
where ()(R) (x) is the spectral feature mapping with L layers and o denotes the 1-th laye This actually recovers the deep spectral kernel proposed by (36]. If the GP's kernel in the permanental process (Eq. (2)) adopts the above deep sparse spectral nonstationary kernel, then refer this model as DNSSPP.
DNSSPP exhibits enhanced expressiveness compared to NSSPP due to its deep architecture. In subsequent sections, we consistently use (R( as the spectral feature mapping. When it has single layer, the model corresponds NSSPP; when multiple layers, the model corresponds DNSSPP.
5 Inference
For the posterior inference of (D)NSSPP, this work employs fast Laplace approximation exploit- ing the sparse spectral representation of nonstationary kemnlls. Other inference methods, such as variational inference, are also feasible. Specifically, we extend the method proposed in [29 from stationary kernels to nonstationary kernels.
5.1 Joint Distribution
When we approximate a kernel using Eq.
66 or Eq.
9.. the GP prior can also be approximated as:
f(x)
3
R (x),
G ()..
(10)
Additionally, we set the intensity function as A(x) (f(x) + a) Following 14 we introduce an offset a to mitigate the nodal line problem caused by the non-injectiveness of f This issue results in the intensity between positive and negative values of being forced to zero, despite the underlying intensity being positive.
Combing the likelihood in Eq.
| and the equivalent GP prior in Eq. 10p. we obtain the joint density:
log (xx) 3)) = log p( (x, N
+ 11)
13,0)
p(3)
1B3 (R) (x,) + a)) /
1 - B+C
(1I)
where C' is a constant and θ denotes the model hyperparameters. The intensity integral term can be further expressed as:
/ A(x)dx ,3 M,3 2a m + +
(12)
For simplicity, here assume all layers have the same width H, but it also vary.
5

---

where [2) is the window size, M is a R x R matrix, and m is a R-sized vector with each entry:
M - f
vp (8) (x) (x)(xx
m. J v (8) (x)dx, iJE,,..... R.
(13)
It is worth noting that the intensity integral is potentially analytically solvable. For single-layer spectral feature mapping (NSSPP), analytical solutions for and m exist (see Appendix(C), However, for multiple-layer mapping (DNSSPP), analytical solutions are not possible. In the following, analytically solve M and m for NSSPP, while using numerical integration for DNSSPP.
5.2 Laplace Approximation
j We use the Laplace's method approximate the posterior p( : 6). The Laplace's method
approximates the posterior with Gaussian distribution by performing a second-order Taylor expan- sion around the maximum the log posterior. Following the common practice:
log ((g (x:), x log.N Q) = -3 - (3- log Q - log log2r, 2n, (14)
D
where @ is the mode of the true posterior and Q is the negative inverse Hessian of the true posterior evaluated the mode:
Volog (xx 310)| 0, Q
log (x
(15)
The mode 3 is typically not directly solvable based on Eq. 15 so we employ optimization methods find it. Then, the precision matrix Q-1 can be computed analytically (see Appendix D
5.3 Hyperparameter and Complexity
The model hyperparameter consists of the kernel parameters a, w. b (if there are multiple layers, these hyperparameters include parameters from all layers) and the bias of the intensity function We can learn these hyperparameters from the data by maximizing the marginal likelihood:
log | ())
=l ((x; [x:
( f I(3) {x:) Q) x logp( (x;) ,10) - log N W/333, Q). (16)
Substituting Eqs. [IT and (14) into Eq. (I6).
obtain marginal likelihood approximation.
Optimizing this yields the optimal hyperparameter e.
Similar to common low-rank approximation methods for kernels, the computational complexity of the proposed inference method reduced from (NN3) O(NR2) where & N, i.e., linearly with N. A complete algorithm pseudocode is provided in Appendix
5.4 Predictive Distribution
After obtaining the posterior B {x, I θ ( Q), compute the posterior of the intensity function. For any x* E X, the predictive distribution of f(x') + is
f(x") + al (x,
[x ~ "((x(""),((xx")
where
m(x") = (x))
(xx))xx))=)) + +a d²(x") (R) (x") (x)))
Then, the intensity X(x") (f(x*) + +a)² can be proven have the following mean and variance:
[[A(x")) = =²((x)))))))))))))))'>)) + +z Var[X(x")] = (x"(xx) + 4x²(x"))x (x²))")xx)
(17)
6 Experiments
In this section, we mainly analyze the superiority in performance of (D)NSSPP over baseline models on both synthetic and real-world datasets, as well as the impact of various hyperparameters. We perform all experiments using the server with two GPUs (NVIDIA TITAN V with 12GB memory), two CPUs (each with 8 cores, Intel(R) Xcon(R) CPU ES-2620 v4 @ 10GOHzz), and 251GB memory.

---

326
316
L723
2236
2754
1762
5/05
172.7
115
1753
372
313
dgg
220
learteg
0.001 3105 5.0
-1,00
LPLS
0766
375.7
156.
112
1122 196.
2,00
Loke
274.7
1750
-395
500.
3634
1974
100
20
30
200
(a) Stationary Data
(b) Nonstationary Data
(c) Width depth
(d) Epoch v.s. learning rate
Figure 1: (a) The fitting results of the intensity functions for all models on the stationary synthetic data; (b) those the nonstationary synthetic data. The impact of (c) network width and depth, (d)
the number of epochs and learning rate on the Lis of DNSSPP on the nonstationary data.
6.1 Baselines
We employ several previous works as baselines for comparison. These baselines include VBPP which utilizes the inducing points method [8], LBPP employing the Nyström approximation [3] as well as SSPP and GSSPP utilizing the Fourier representation (2 We use GSSPP based on Gaussian kernel, Matérn kernel with parameter 1/2 and 5/2 respectively, and we denote them as GSSPP, GSSPP-MIT and GSSPP-M52. Besides, we offer a baseline that employs stacked mappings of stationary kernels which we name it Deep Sparse Spectral Permanental Process (DSSPP). All of these methods utilize stationary kernels. As far as we know, there currently no work employing nonstationary kernels in permanental processes. To compare with nonstationary baselines, we implement nonstationary permanental process using the nonstationary spectral mixture kernel 26 referred to as NSMPP.
6.2 Metrics
We employ two metrics to evaluate the performance of various baselines. One is the expected log- logliiklihhod on the test data, denoted as Cesa, with details provided Appendix [ Additionally, when the ground-truth intensity is available, e.g., for the synthetic data, the root mean square error (RMSE) between the expected posterior intensity and the ground truth serves as another measure.
6.3 Synthetic Data
Datasets To better compare the performance of (D)NSSPP with baseline models on point process data in both stationary and nonstationary scenarios, simulated stationary and nonstationary permanental process on the interval [0, 10), respectively. Specifically, we assume two kernels:
xx x2 + 1) exp( - - -x21²x)
An (x1, x2) = exp(
||x - -1n1-x113), x2) X2) = 100
where 1 is stationary Gaussian kernel and l nonstationary kernel obtained by multiplying Gaussian kernel with polynomial kernel. We use these two kernels to construct a stationary GP and a nonstationary GP, respectively, and randomly sample two corresponding latent functions. Based on X(x) (f(x) + 2)², we construct two corresponding intensity functions. Finally, we use the thinning algorithm [22 to simulate two sets of synthetic data. For each intensity, we simulate ten datasets and use each dataset alternately as the training set and the remaining ones the test sets.
Setup For NSSPP, the number of frequencies (network width) is set to 50. For DNSSPP, we experimented with three different configurations: DNSSPP-150,30, DNSSPP-[100)) [30,50,30]. Each number represents the width of the corresponding network layer. Therefore, DNSSPP-(50,30) implies model with two layers, where the first layer has a width of 50 and the second layer has a width of 30, and so on. For baseline models, we need to set the number of rank for kemel low-rank approximation. We set the number of inducing points to 50 for VBPP, the number of cigenvalues to 50 for LBPP, and the number of frequencies to 50 for SSPP, GSSPP, and NSMPP For DSSPP, to facilitate comparison with DNSSPP, adopt the same layer configurations.
and DNSSPP-
The fitting results of intensity functions for all models are depicted in Fig. | To quanti- tatively compare the fitting performance of different methods in both stationary and nonstationary
Results
7

---

Table 1:
The performance of Lust RMSE and runtime for DNSSPP, DSSPP, NSSPP and other baselines on two synthetic datasets. For Liss, the higher the better; for RMSE and runtime, the lower the better. The upper half corresponds to nonstationary models, while the lower half stationary models. On the stationary dataset, DSSPP performs comparably to DNSSPP. On the nonstationary dataset, DSSPP does not outperform DNSSPP due the severe nonstationarity the data.
Stationary Synthetic Data
Nonstationary Synthetic Data
Lust
Runtime(s)
Last
Runtime(s)
(1005P-)100,50)
+ 37.52)
0.061(+ 0.020)
9.27
175.70(+
26.89)
0.076 0.015)
9.40
DNSSPP-150.30|
156.55(4 37.44)
0.017)
9.00
173.68( 26.70)
0.094:= 0.012)
9.11
DNSSPP-(30,50,30)
156.55(+ 37.66)
0.068( 0.017)
10.32
174.47(+ 26.57)
0.013().013)
10.07
NSSPP
153.56(+ 36.82)
0,070(+ 0.009)
8.39
172.87(+ 26.86)
0.10C+ 0.012)
8.96
NSMPP
153.05(4: 36.96)
0.065(
3.19
172.17(# 26.42)
0.079(1 0.019)
4.63
(10,5-(100,50]
155.62(++ 37.31)
0.060 0.015)
7.64
173.91(+ 10.27)
0.090( 0.013)
8.27
[SSPP-[50,50]
155.51(4 36.98)
0.065(20.012))
7.56
172.54( 26.74)
0.103(000155
7.54
153.75(4 37.34)
0.033(=00016)
8,95
174.35( 26.70)
0.08((
9.74
SSPP
36.70)
0,71((+
1.95
165.94(+ 30.00)
0.140 0.019)
GSSPP
154.12(#:7.009
0.073(
7.72
170.13( 26.43)
0.001(0.023)
14.19
GSSPP-M12
150.65(1336884
0.071( 0.018)
9.30
168 26.67)
0.08K(
15,33
GSSPP-M52
154,46 37.24)
0.07 0.022)
9.63
169.3364 26.61)
0.107
14.49
LBPP
150 36.13)
0 0.008)
0.31
168.97(+ 26.70)
0.126 0.006)
0.37
VBPP
(5, 36.52)
0.072 0.021)
L.68
172.95 30.00)
0.087 0.010 0.016)
1.83
LBPP for Redwoods DNSSPP for Redwoods
LBPP for Taxi
DNSSPP for Taxi
Figure 2: The fitting results of the intensity functions from LBPP and DNSSPP the Redwoods and Taxi datasets. Additional results for various baselines on three datasets are provided in Appendix
scenarios, we employ Listt and RMSE as metrics. The comparison results are presented in Table [ DNSSPP with three different configurations exhibits significant advantages on both stationary and nonstationary datasets. NSSPP and NSMPP show better performance in the nonstationary scenario compared to most stationary baselines. This is reasonable, as stationary baselines face challenges in accurately modeling nonstationary data. Because stationary kernels are subset of nonstationary ker- nels, nonstationary models typically perform well on stationary data as well. Compared relatively shallow models, DNSSPP achieves better performance but also incurs a longer running time.
6.4 Real Data
three sets of common real-world datasets evaluate the perfor- mance of our method and the baselines. Coal Mining Disaster [[3]: This a I-dimensional dataset containing 191 incidents that occurred between March 15, 1875, and March 22, 1962. Each event represents a mining accident that killed more than 10 people. Redwoods [27] This 2-dimensional dataset that describes the distribution of redwoods in area, consisting of 195 events. Porto Taxi [11] This is large 2-dimensional dataset containing the tracks of 7,000 taxis in Porto during 2013/2014. We focus the area with coordinates between (41.147, -858) and (41.18, -8.65), and extract 3,000 pick-up points as our dataset. For each dataset, we randomly partition the data into training set and set of approximately equal size.
Datasets In this section,
Setup For real datasets, because we do not know the ground-truie intensity, we use Lus as the evaluation metric. Since NSSPP generally performs worse than DNSSPP, we report only the results of DNSSPP in this section. For DNSSPP, we consistently use the same three configurations as in the synthetic data. For baseline models, we choose different numbers of rank for kernel low-rank approximation for 1-dimensional and 2-dimensional datasets. Specifically, for the 1-dimensional dataset (Coal), we set the number of rank, i.c., the number of inducing points for VBPP, the number

---

Table 2: The performance of Les and runtime for DNSSPP and other baselines on three real datasets. The upper half corresponds to nonstationary models, while the lower half to stationary models.
Coal
Redwoods
Taxi
Lest
Runtime(s)
Lus
Runtime(s)
Lesst
Runtime(s)
DNSSPP-[100.50)
225.73(+ 2.96)
10.54
79.56(+ 0.014)
11.09
7171(+ 73)
73.3
DNSSPP-[50,30]
225.85( 2.61)
11.38
79.50(+ 0.020)
10.45
6682(+ 37)
70.
DNSSPP-(30.50.30)
225.79(:: 4.49)
11.76
79.55(= 0.025)
12.72
7246(1: 37)
81.0
NSMPP
223.28( 3.60)
2.88
77.64(# 6.21)
4.43
6492(# 62)
94.6
SSPP
221.42(# 1.87)
1.72
78.57(# 2.83)
2.45
6245(1: 45)
58.4
GSSPP
221.08( 6.32)
5.05
76.97( 5.80)
5.94
6445(+ 97)
110.64
GSSPP-MI2
223.11(# 5.02)
5.61
72,96( 11.89)
5.90
6599( 76)
104.75
GSSPP-M52
221.89(1: 3.06)
77.98(1 6.05)
5.23
6526( 113)
112.54
I.PPP
218.30 4.12)
0.33
80.40( 0.72)
0.34
6096(+ 25)
3.16
VBPP
219.15(# 4.54)
1.69
77.06(= 0.88)
2.14
6156(1 34)
9.10
of eigenvalues for LBPP, and the number of frequencies for SSPP, GSSPP, and NSMPP, to 10. For the 2-dimensional datasets (Redwoods and Taxi), set the number of rank to 50. We discuss the reasonableness of the above setup in Appendix H
shows the fitting results of the intensity functions from LBPP and DNSSPP on Results Fig. 2
the Redwoods and Taxi datasets. The quantitative comparison results are shown in Table 2 Since real-world data more or less nonstationary to varying degrees, nonstationary models generally perform better datasets and achieves second place on the Redwoods dataset. For simpler datasets, such as Coal and Redwoods, which have relatively simple data patterns, the performance of DNSSPP is comparable to that of the simpler baselines. However, for the more complex Taxi dataset, DNSSPP demonstrates significant advantages the simpler baselines. This indicates that DNSSPP has clear advantage in handling data with more complex patterns, such as large-scale or nigh-dimeessioal datasets.
data. DNSSPP secures both first and second places the Coal and Taxi
6.5 Ablation Studies
In this section, we analyze the impact of four configuration hyperparameters on the performance of DNSSPP: the network width and depth, as well as the number of epochs and learning rate during the update of hyperparameter e.
6.5.1 Width and Depth of Network
We investigate DNSSPP's performance with varying network sizes by adjusting width and depth on nonstationary synthetic data. In experiments, maintain consistent width across layers. Results are illustrated in Fig. Tcl As the number of layers and the width of the network increase, DNSSPP's performance initially improves and then declines, reflecting the phenomenon of overfitting as the network size increases. Regarding the number of layers, DNSSPP generally exhibits underfitting when there is only single layer, while overfitting tends occur with four layers. In terms of network width, due to the relatively small size of the synthetic data, overfitting becomes apparent when the width large. However, for larger datasets, a more complex network structure should be chosen.
6.5.2 Epoch and Learning Rate of +
During the inference process, we need to perform numerical optimization on the hyperparameter O, implying the need to determine the epoch and learning rate for optimization. We conduct experiments on the nonstationary synthetic data investigate the coordination of epoch and learning rate. Results are illustrated in Fig. [d When the learning rate is too low or the number of epochs is insufficient, the update of θ sluggish, leading to inferior performance. However, when the learning rate too high or the number of epochs is excessive. model performance generally declines well. This is bi-level optimization: the inner level is the Laplace approximation, and the outer level is to update the hyperparameter θ An excessively high learning rate too many epochs can cause è converge too quickly, making the overall model training more prone to getting stuck in local optima, thus resulting in suboptimal performance.
because our model training

---

7 Limitations
NSSPP not only removes restrictions the kernel's functional form but also makes it applicable nonstationary kernels. DNSSPP further enhances its expressive power through a deep architecture. However, the deep architecture of DNSSPP prevents analytical computation of the intensity integral, necessitating numerical integration. This reliance on numerical integration imposes a limitation on the model's computational speed.
8 Conclusions
In this study, we introduced NSSPP and its deep kernel variant, DNSSPP, to address limitations in modeling the permanental process. This approach overcomes the stationary assumption, allowing for flexible learning of nonstationary kernels directly from data without restricting their form. By leveraging the sparse spectral representation of nonstationary kernels, we achieved low-rank approximation, effectively reducing computational complexity. Our experiments synthetic and real-world datasets demonstrated that (D)NSSPP achieved competitive performance on stationary data while excelling on nonstationary datasets. This study underscores the importance of considering nonstationarity in the permanental process and demonstrates the efficacy of (D)NSSPP this context.
Acknowledgments and Disclosure of Funding
This work was supported by NSFC Projects (Nos. 62106121, 62406 19), the MOE Project of Key Research Institute of Humanities and Social Sciences (22JID110001) the fundamental research funds for the central universities, and the research funds of Renmin University of China (24XNKJ13), the Natural Science Foundation of Hubei Province (2024AFB074), and the Guangdong Provincial Key Laboratory of Mathematical Foundations for Artificial Intelligence (2023B121201000)
References
[1] Adams, R. P., Murray, I., and MacKay, D. Tractable nonparametric Bayesian inference in Poisson processes with Gaussian process intensities. In Proceedings of the 26th Annual International Conference on Machine Learning, pp. 9-16. ACM, 2009.
[2] Bochner, S. Vorlesungen uber Fouriersche Integrale. Akademische Verlagsgesellschaft, 1932.
[3] Cox, D. R. Some statistical methods connected with series of events. Journal of the Royal Statistical Society. Series B (Methodological), pp. 129-164, 1955.
[4] Dezfouli, A., Nock, R., Arabzadeh, E., and Dayan, P. Neural network Poisson models for behavioural and neural spike train data. bioRxiv, 2020.
[5] Diggle, P., Rowlingson, B., and Su, T.-l. Point process methodology for on-line spatio-temporal disease surveillance. Environmetrics: The official journal of the International Environmetrics Society, 16(5):423-44 434, 2005.
[6] Donner, C. and Opper, M. Efficient Bayesian inference of sigmoidal Gaussian Cox processes. Journal of Machine Learning Research, 19(1)2210-27733 2018.
[7] Flaxman, S., Teh, Y. W., Sejdinovic, D., et al. Poisson intensity estimation with reproducing kernels. Electronic Journal of Statistics, 11()):081-5504 2017.
[8] Geng, J., Shi, W., and Hu, G. Bayesian nonparametric nonhomogeneous Poisson process with applications to USGS earthquake data. Sputial Statistics, 41:100495, 2021.
[9] Gunter, T., Lloyd, C., Osborne, M. A., and Roberts, S. J. Efficient Bayesian nonparametric modelling of structured point processes. Conference on Uncertainty in Artificial Intelligence, pp. 319-319, 2014.
[10] Heinonen, M., Mannerström, H., Rousu, J., Kaski, S., and Lähdesmäki, H. Non-stationary Gaussian process regression with Hamiltonian Monte Carlo. In Artificial Intelligence and Statistics, pp. 732-740, PMLR, 2016.
10

---

[11] Ilala,, D. A Poisson process with random intensity for modeling financial stability. The Spanish Review of Financial Economics, 14(2):43-50, 2016.
[12] Illian, J. B., Sørbye, H., and Rue, H. toolbox for fitting complex spatial point process models using integrated nested Laplace approximation (INLA). 2012.
[13] Jarrett, R. G. A note on the intervals between coal-mining disasters. Biometrika, 66(1):191-193, 1979.
[14] John, S. and Hensman, Large-scale Cox process inference using variational Fourier features. In International Conference on Machine Learning, pp. 2362-2370 PMLR, 2018.
[15] Kingman, J. F. C. Poisson processes, volume 3. Clarendon Press, 1992.
[16] Ko, Y.-J. and Seeger, M. W. Expectation propagation for rectified linear Poisson regression. In Asian Conference on Machine Learning, pp. 253-268. PMLR, 2016.
[17] Lázaro-Gredilla, M., Quinonero-Candela, J., Rasmussen, C. E., and Figueiras- Vidal, A. R. Sparse spectrum Gaussian process regression. The Journal of Machine Learning Research, 11: 1865-1881, 2010.
[18] Lloyd, C., Gunter, T., Osborne, M., and Roberts, S. Variational inference for Gaussian process modulated Poisson processes. In International Conference on Machine Learning, pp. 1814- 1822, 2015.
[19] McCullagh, P. and Møller, J. The permanental process. Advances in applied probability, 38(4): 873-888, 2006.
[20] Møller, J., Syversveen, A. R., and Waagepetersen, R. Log Gaussian Cox processes. Scandi- navian journal of statistics, 25(3):451 482, 1998.
[21] Moreira-Matias I.., Gama, J., Ferreira, M., Mendes-Moreira, J., and Damas, L.. Predicting taxi-passenger demand using streaming data. IEEE Transactions on Intelligent Transportation Systems, 14(3):1393-144 2013.
[22] Ogata, Y. Space-time point-procest models for earthquake occurrences. Annals of the Institute of Statistical Muthematics, 50(2):379-402. 1998.
[23] Paciorek, C. J. and Schervish, covariance functions. Envirommetrics: The official journal of the International Environmetrics Society, 17(5):483-500 2006.
Spatial modelling using a new class of nonstationary
[24] Park, M., Weller, J. P., Horwitz, G. D., and Pillow, J. W. Bayesian active learning of neural firing rate maps with transformed Gaussian process priors. Neural computation, 26(8):1519-1545 1541, 2014.
(25] Rahimi, A. and Recht, B. Random features for large-scale kernel machines. Advances in neural information processing systems, 20, 2007.
[26] Remes, S., Heinonen, M., and Kaski, S. Non-stationary spectral kernels. Advances in neural information processing systems, 30, 2017.
[27] Ripley, B. D. Modelling spatial patterns. Journal of the Royal Statistical Society: Series B (Methodological), 39(2):17--192. 1977.
[28] Samo, Y.-L. K. and Roberts, Generalized spectral kernels. arXiv preprint arXiv:1506.02238 2015.
[29] Sellier, J. and Dellaportas, P. Sparse spectral Bayesian permanental process with generalized kernel. In International Conference on Artificial Intelligence and Statistics, pp. 2769-2791. PMLR, 2023.
[30] Snoek, J., Swersky, K., Zemel, R., and Adams, R. Input warping for Bayesian optimization of non-stationary functions. International conference on machine learning, pp. 1674-1682 PMLR, 2014.
11

---

[31] Taddy. M. A. Autoregressive mixture models for dynamic spatial Poisson processes: application tracking intensity of violent crime. Journal of the American Statistical Association, 105(492): 1403-1417, 2010.
[32] Ton, J.-F., Flaxman, S., Sejdinovic, D., and Bhatt, S. Spatial mapping with Gaussian processes and nonstationary Fourier features. Spatial statistics, 28:59-78, 78, 2018.
[33] Walder, J. and Bishop, A. N. Fast Bayesian intensity estimation for the permanental process. In International Conference Machine Learning, pp. 3579-3588. PMLR, 2017.
[34] Wilson, and Adams, R. Gaussian process kernels for pattern discovery and extrapolation. In International conference on machine learning, pp. 1067-1075 PMLR, 2013.
[35] Wilson, A. G., Hu, Z., Salakhutdinov, R., and Xing, E. P. Deep kernel learning. In Artificial intelligence and statistics, pp. 370-378. PMLR, 2016.
[36] Xue, H., Wu, Z.-F., and Sun, W.-X. Deep spectral kernel learning. In International Joint Conference Artificial Intelligence, pp. 4019-4025, 2019.
[37] Yaglom, A. M. Correlation Theory of Stationary and Related Random Functions, Volume I: Basic Results, volume 131. Springer, 1987.
12

---

A Proof of Equation (4)
The derivation of Eq.
is provided below:
R
k(x1
X2)
~
Φ (xx)))
R
==1
0²
exp(iw x1) exp(-ius) x2)
R
d²
(cos(w)] x1) + issn(w] x1 (c)S((w x2) -isin(,,"xx) x2))
R
a²
R
(cos(w) x1) cos(w, x2) + sin(w? x1) sin(w? x2))
R
$ (8) (x1) (x(()
(18)
In the derivation, due the symmetry of the spectrum distribution, can be assumed that 2R points are symmetrically sampled, which allows the elimination of the imaginary part in the equation without altering the expectation.
B Proof of Equation (8)
The derivation of Eq.
is provided below:
x2)
exx (i(wi,xx w2xx2)) + .... exp (ilw)xx1 x))x
4R
0²
Σ
[oos(w]]xx wxxx22 +... + cos(weg, w3,x2)]
(19)
4R
R
[oos(w),>i - w]xx))];
4R
where the second line is because the kernel is real-valued, so we can safely eliminate the imaginary part. Additionally, considering the law of total expectation, have
Ew, w [co xı + w. x2 + 26)] Ew,, [cos((,, x; - w x2 + 2b) (, will,
(20)
where b ~ Uniform(0) (0, 2T). And consider the periodicity of cosine function, we have
E, [o. x1 + w/ x2 + 2b) wر] O.
(21)
so Eq. 20 finally equal to Therefore, we can further obtain another equivalent form of the spectral feature mapping, through the following procedure:
x(x1,X2)
[oos(w, - _,,2 + cos(w,,,x, + J,x2 + bir + byr)
4R
1=1
(6.))=(1,2)
0²
2 + bir) cos(ww),x + byr)
(22)
4R
(i.) (1.2)
( (x1)) (x2))
where (bir, b2r R uniformly sampled from [0, 2m..
13

---

C Analytical Computation of Intensity Integral
The intensity function of the permanental process has the following expression A(x) = (B3 y (B)
(x)
)²². For ease of derivation, we adopt the form of Eq. (7) for the spectral feature mapping u (2) (x) in
subsequent steps, i.e., a 2R-sized vector. When (x) takes the form of Eq. (8). i.e., an R-sized vector, the calculation result of the intensity integral remains the same. The intensity integral on X is:
Jx X(x)dx= J (g) (x) + d
= A3 / (x)((B) dxB + 2a / (x)) + / a²
= 3 33/ + 2aB m + az
where
M.
/ w (x) (x)xxx
m,
(x) i.j E ,,...,2,,,
(23)
worth noting that M and m are analytically solvable. The analytical solution for the station- ary case is provided in Appendix C.2 in (29). and we provide the analytical expression for the nonstationary case here.
For M, the product of two spectral feature mappings is:
Σ
cos(w (x))
if 1..... R,
(1.2)=
Σ
sin(w sin W x):
if R+ 1,...,2R,
o²
v (x)
(x) 4R
(a.b)= (1,2)²
Σ
cos sin (w)
if 1, ,,..., R, j=R -R+ 11.,..... .....: 2R,
(a.b) (1,2)
Σ
sin(w cos (w]
if = R 1, ,,..................... 2R, j 1, R.
(1.2)
(24)
We can further transform the product term of trigonometric functions:
cos(w, cos (w/]w)]) = cos ((wa whj) x) + cos ((was + wbj) {x0))) Tx)]:
2
1
sin (awlTxx Tx) sin (w]]x))
= coos ((wai Wb;) xx - cos ((was + Wbj) ) Tx)]:
(25)
2
cos ( x sin sin(w
2 isin ((wai - why)) T x) sin ((was + Wby) ) xx]].
Without loss of generality, the integral of the above expression analytically. Since we only consider the D = and D point process data in this paper, we specifically demonstrate the analytical integral expressions in the one-dimensional and two-dimensional cases here.
consider the integral domain X' as [-d, d. Then
compute
One-dimension:
1.aa cos (nz) do sin(nd)
if 1 +
(26)
2d
if 0,
f |-ac sin (1z² dx = O.
(27)
Two-dimension:
[ |-dd0 cos (17 T x)xx
(cos((n - (2 cos((m + +2))d))
if m 122 # 0
mama 4d
(28)
if 1) 7/ = 0,
[La0p sin (n) (nTxx) x)xxx = O.
(29)
14

---

where denotes the first ordinate of 77. and 7/2 denotes the second ordinate of 7). It should be noted that theoretically. the integral in Eq. (28) only has two possible results explained in the right side. The analytical solution of the integral of Eq. (25) can be obtained by replacing r with Wai Whj or Was + Whj- Note that the case a and = corresponds to the second case Eq. (26) and Eq. (28). which means the computation of the diagonal entries of M is different from that elsewhere.
Next, we can compute m:
[[-d ( cos (os((w]))) cos (os(w dx, [ (si((w][x)) + sin(w}{x)) dx,
if = 1,..., R.
m,
(30)
if = R 1,....,,,. 2R.
is easy to see that if D = 1, m can be analytically computed using Eq. (26) and Eq. (27). while if D 2, can be analytically computed using Eq. (28 and Eq. (29) is worth noting that the analytical expression of the intensity integral above applies only to NSSPP (single-layer network). For DNSSPP (multi-layer network), the nested structure of trigonometric functions leads to M and m lacking analytical solutions, and we need resort to numerical integration.
D Derivation of Laplace Approximation
The gradient and the Hessian matrix of the posterior of B are:
N vrl (x:)
Va log 310) -2M 1) 2am +
(31)
3 (r) +
N ( (x;)
va log po ((0) = (2M - - ( + a²)²
(32)
In theory, setting Eq. 31 to 0 can solve for the mode of the posterior, denoted as B. However, in practice, we cannot analytically solve it, so numerical optimization to obtain B. Then according to Eq. 32 we can obtain the precision matrix Q V log ((x)) 3|0) 9_و
E Pseudocode
The pseudocode our inference algorithm is provided in Algorithm
Algorithm 1: Inference for (D)NSSPP
Define the network depth L and width R, and initialize the hyperparameter θ; for Iteration do
Calculate M and m by Eq. 13
Update the mode / by maximizing Eq. 14
Update the covariance matrix Q by Eq. 15
Update the hyperparameters θ by maximizing Eq. 16
end
For any x", output the predicted mean and variance of A(xx) by Eq. 17
F Computation of Expected Test Log-likelihood
The computation of expected test log-likelihood is provided in Appendix E.2 (2)] For the completeness of the article, we restate the specific derivation process here. Considering the posterior of the intensity function, we have the approximation of the expected test log-likelihood:
Ea[logp((xx)
[x
N
= =-Es/
/ (BT (T + aa) +
Σ
Eg log(B (x;) + a.
(33)
15

---

derived from Eq. I
First, we compute the first term in Eq. (33)
Eal (3 (x)+ = / Eg|3 (o (x) + af + / Varls (o(R) (x)]xx
x
1 (v)(H) (x) 33 T ( (x))dx 2a (BT wil (x)) + a² 1 / (V)10 (x)) (x))
= MB + tr( (M) + 2033 T m + x21
(34)
where M and
defined as Eq.
Then, we consider the second term in Eq. (33 can obtain the following expression:
Applying the computational method used by [18] we
Egllog(3 @
(
(xx) + a
(35)
T y + + log( ( T (Qu(((()))) - Const,
G
(36)
2
(x)(R)
2
where Const 0.57721566 is the Malsh-MMacheronii constant. G is defined as
j1zd
G(=) 22
(2)((122))
where (.), denotes the rising Pochhammer series:
(a)o 1, (a)s aaa 1) + 2)... ++1.
In practical computation, we first prepare a lookup table for the G function and then obtain numerical approximations through linear interpolation.
G Additional Experimental Results
In this section, we provide additional experimental results for the real data. Specifically, we present the fitting results the intensity functions from all models on the Coal dataset Fig. 3 the fitting results of the intensity functions from VBPP, SSPP, GSSPP, and NSMPP on the Redwoods dataset in Fig. 4 and the fitting results of the intensity functions from VBPP, SSPP, GSSPP, and NSMPP on the Taxi dataset in Fig.
VBPP
LBPP
DPPS
SSPP
Events
3.000
Expected
2.004
3.007
3.002
1200
3000
1002
1920
2942
3963
Figure 3: The fitting results of the intensity functions from all models on the Coal dataset.
H Some Discussion about Experimental Setup for Real Data
For the real data, we chose different numbers of ranks for the 1-dimensional dataset (10 for Coal) and the 2-dimensional datasets (50 for Redwoods and Taxi) for all baseline methods, except for DNSSPP: The reason is that the Coal dataset is very small (only 191 events) and has a simple data pattern. A
16

---

GSSPP for Redwoods Figure 4: The fitting results of the intensity functions from VBPP, SSPP, GSSPP and NSMPP the Redwoods dataset.
VBPP for Redwoods
SSPP for Redwoods
NSMPP for Redwoods
VBPP for Taxi Figure 5: The fitting results of the intensity functions from VBPP, SSPP, GSSPP and NSMPP on the Taxi dataset.
SSPP for Taxi
GSSPP for Taxi
NSMPP for Taxi
higher number ranks would not further improve performance but would significantly increase the algorithm's 's running time. Therefore, set the number of ranks to 10 for Coal, but 50 for Redwoods and Taxi because the latter two datasets are larger and more complex, requiring higher number of ranks improve performance.
To demonstrate this, we further increased the number of ranks for all baseline models to 50 on the Coal dataset and compared the result to rank=10. The result is shown in Table As can be seen from results, when we increased the number of ranks from to 50, the performance all baseline models did not improve significantly, but the algorithm's 's running time increased substantially.
In summary, for the same dataset, the number of ranks for all baseline methods kept consistent, except for DNSSPP because it is not a single-layer architecture. The choice of the number of ranks for the baseline methods was made considering the trade-off between performance and running time. Further increasing the number of ranks would not significantly improve model performance but would result a substantial increase in computation time.
Table 3: The performance of Liest and runtime for all baselines on the Coal dataset, with the number of rank 10 and 50. When increase the number of ranks from 10 50, the performance of all baseline models does not improve significantly, but the algorithm's running time increases substantially.
Coal (rank
Coal (rank 50)
Liea
Runtime(s)
Len
Runtime(s)
NSMPP
223.28(1: 3.60)
2.88
222.92(1 3.63)
4,74
SSPP
221.42( 1.87)
1.72
221.77( 2.92)
3.16
GSSPP
221.08(1 6.32)
5.05
221.67(1: 4.28)
19.08
GSSPP-MI2
223. 5.022
5.61
219.33(+ 4.25)
18.88
GSSPP-M52
221.89(1: 3.06)
5.17
217.80(1 5.25)
16.94
LBPP
218.30 4.12)
0.33
219.88(+ 2.68)
2.68
VBPP
219.15(1 4.54)
1.69
219.25(1 4.51)
6.26
17

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope. See the Abstract and Introduction sections.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The limitation of the work is discussed in the Limitations section.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
18

---

For all theoretical results, the paper provides the corresponding proofs in the main text or appendix.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The paper fully discloses all the information needed to reproduce the main experimental results in the paper. See the Experiments section.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
19

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We provide the data and code in the supplemental material to reproduce the main experimental results.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code permitted.
6. Experimental Setting/eetails
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: specify all the training and test details necessary to understand the results. See the experimental setup in the Experiments section.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We report the statistical significance of the experiments. In the tables presenting the experimental results, provide the standard deviations across multiple runs, while some figures, we also include confidence intervals.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
20

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provide sufficient details the type of GPU, memory, time of execution needed to reproduce the experiments. See the experimental setup in the Experiments section.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics http:://hheeaaee.ccccppppb///aaa
Answer: [Yes]
Justification:
The research conducted in the paper conform with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws regulations in their jurisdiction)
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [No]
Justification: This paper presents work whose goal is to advance the field of machine learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Guidelines:
21

---

The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymentt of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All code, models, and datasets mentioned in the text are appropriately cited with their original papers.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
22

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperssi thood cww datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: New assets introduced in the paper, such as code, are well documented. The documentation is provided alongside the assets the supplementary material.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question:
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
23

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
24


Reviews:
[{"review_id": "rmmQIMkjOy", "reviewer": "Reviewer_WJmT", "summary": "The paper introduces an approach to modeling permanental processes by utilizing a sparse spectral representation of nonstationary kernels, termed as Nonstationary Sparse Spectral Permanental Process (NSSPP) and its deep kernel variant (DNSSPP). This method addresses the limitations of traditional permanental processes which often require specific kernel types and assume stationarity, thus restricting the model's flexibility and computational efficiency. The deep kernel variant (DNSSPP) uses hierarchically stacked spectral feature mappings, significantly enhancing the model's ability to capture complex patterns in data. The paper presents experimental results on both synthetic and real-world datasets, showing that NSSPP and DNSSPP perform competitively on stationary data and show improvements on nonstationary datasets.", "questions": "- Performance comparison with a baseline that employs stacked mappings of stationary kernels", "limitations": "Yes.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper introduces a novel approach by integrating sparse spectral representation with nonstationary kernels in the context of permanental processes. The introduction of a deep kernel variant, DNSSPP, which stacks multiple spectral feature mappings to enhance the expressiveness of the model significantly, is another novel idea.\n- The paper is well-structured and articulates the motivations, methodology, and findings clearly.\n- Experimental results show the effectiveness of the approach when compared to baselines and other recent approaches.\n- The significance of this work is considerable, given its potential impact on various fields that utilize point process models.", "weaknesses": "- The experimental results (Table 1) indicate that performance enhancements are observed mainly with the deep kernel variant (DNSSPP) when compared to stationary kernels. It would be beneficial for the authors to include a comparison with a baseline that employs stacked mappings of stationary kernels to more clearly ascertain the source of these improvements.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> Q: Performance comparison with a baseline that employs stacked mappings of stationary kernels. \n\nA: Thanks for the suggestion. A baseline that employs stacked mappings of stationary kernels is an important baseline. To show the source of the performance improvement, we have re-compared with a deep stacked stationary kernel model. \n\nThe results are shown in Table 1 of the rebuttal PDF. The corresponding deep stacked stationary kernel model is referred to as DSSPP, which has the same deep architecture as DNSSPP. Therefore, DSSPP also has three implementations: DSSPP-[100,50], DSSPP-[50,30], and DSSPP-[30,50,30]. \n\nAs can be seen from the results, the performance of the stationary kernel model significantly improves after introducing the deep architecture. On the stationary synthetic dataset, DSSPP performs comparably to DNSSPP. On the nonstationary synthetic dataset, DSSPP does not outperform DNSSPP due to the severe nonstationarity in the data. However, on any dataset, DSSPP outperforms other shallow stationary kernel models due to the introduction of the deep architecture. \nTherefore, this demonstrates that the source of performance improvement comes from the stronger expressiveness of both the deep architecture and the nonstationary kernel.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uWnlW5eH6E", "reviewer": "Reviewer_QimU", "summary": "Point process models are a commonly used technique for analysis of event-based data. Gaussian Cox processes are an example which use GPs to model the intensity function in a Cox process, which itself is a specific case of a Poisson process where the intensity function is a stochastic process. Generally, these types of processes are used to model rates of events occurring at given input locations. The link function within the Gaussian Cox process is a design choice; if we choose a square link function and a stationary kernel for the GP prior, this yields a stationary permanental process. The authors aim to address three key problems with these existing approaches; the cubic complexity of exact GP inference, the requirement to use kernels which allow for analytical solutions to the intensity integral, and the reliance on shallow kernels which limit the expressiveness and flexibility of the model. Introduced is a new form of permanental process model which uses a sparse spectral representation to improve computational efficiency, allows for nonstationary kernels to be used without restriction on the form of said kernel, and allows deep kernels to be formed by composing spectral feature mappings.", "questions": "-\tAs mentioned earlier, I think the rationale for using of a very small number of inducing points etc. for some of the baselines needs to be clarified in the text; is it the case that if you increase this number to 50+ that the baselines begin to outperform the models proposed by the authors?\n-\tYou mention other variational methods are feasible, was there a certain specific reason that you decided upon a Laplace approximation-based approach?\n-\tSee earlier point about discussing the impact of the deep kernel learning framework in general on overfitting.\n-\tSee typos in weaknesses section, please fix.", "limitations": "The Limitations section of the paper briefly discusses the fact that the deep formulation of the DNSSPP does not allow for analytical computation of the intensity integral, which necessitates computationally inefficient numerical integration. There are also some practical considerations/limitations discussed elsewhere such as in Section 6.5.2.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "-\tIn terms originality and significance, this paper fills a clear gap in the existing literature on the topic, namely probing the usefulness of deep models in the permanental process setting. Whilst conceptually not a particularly complicated extension, there are obviously various nuances to moving to a deep kernel setting which are outlined clearly by the authors throughout the work.\n-\tThe experimental evaluation overall is thorough, and the results are well presented in a tidy and accessible manner, with clear figures. Particularly appreciated is the inclusion of the ablation study in the main text, which is very useful information to a reader/practitioner.", "weaknesses": "-\tThis is just a reoccurring minor typo but I’m fairly certain it’s the Porto taxi dataset (as in the city of Porto, Portugal), not Proto, so just edit all mentions of this.\n-\tI wasn’t entirely sure of the rationale behind some of the baseline selections, for example, for some of the real-world experiments, you use as low as 10 inducing points/frequencies for the baselines, but use more than this for the DNSSPP (‘same configurations as for the synthetic data’). It would be good to have some discussion of this, and/or how the performance of the proposed methods compares to that of the baselines as we increase the number of parameters. Obviously what constitutes a “fair” comparison between models is highly subjective and depends on various factors such as computational complexity as well as parameter count, but I just wanted a little more context on this.\n-\tI’m aware that there has been some work by  on understanding and exploring overfitting in the context of deep kernel learning (Promises and Pitfalls of DKL, Ober et al, 2021), it would be useful to touch on some work from this area during your discussion on overfitting just to clarify to the readers that this phenomenon is not arising solely due to the size of the datasets you are modelling.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> Q: As mentioned earlier, I think the rationale for using of a very small number of inducing points etc. for some of the baselines needs to be clarified in the text; is it the case that if you increase this number to 50+ that the baselines begin to outperform the models proposed by the authors? \n\nA: Thanks for your suggestion. For the synthetic data, we used the same number of ranks (50 inducing points, eigenvalues, frequencies, etc.) for all baseline methods, except for DNSSPP because it has many layers and different layers can have different layer widths (frequencies). For the real data, we chose different numbers of ranks for the 1-dimensional dataset (10 for Coal) and the 2-dimensional datasets (50 for Redwoods and Taxi) for all baseline methods, except for DNSSPP. The reason is that the Coal dataset is very small (only 191 events) and has a simple data pattern. A higher number of ranks would not further improve performance but would significantly increase the algorithm's running time. Therefore, we set the number of ranks to 10 for Coal, but 50 for Redwoods and Taxi because the latter two datasets are larger and more complex, requiring a higher number of ranks to improve performance. \n\nTo demonstrate this, we further increased the number of ranks for all baseline models to 50 on the Coal dataset and compared the result to rank=10. The result is shown in Table 2 of the rebuttal PDF. As can be seen from the results, when we increased the number of ranks from 10 to 50, the performance of all baseline models did not improve significantly, but the algorithm’s running time increased substantially. \n\nIn summary, for the same dataset, the number of ranks for all baseline methods is kept consistent, except for DNSSPP because it is not a single-layer architecture. The choice of the number of ranks for the baseline methods was made considering the trade-off between performance and running time. Further increasing the number of ranks would not significantly improve model performance but would result in a substantial increase in computation time. \n\n> Q: You mention other variational methods are feasible, was there a certain specific reason that you decided upon a Laplace approximation-based approach? \n\nA: Variational inference methods, to the best of our knowledge, require certain standard types of kernels, such as the squared exponential kernel, to ensure that the intensity integral in the likelihood has an analytical solution [18]. The advantage of the Laplace approximation is that it can analytically compute the intensity integral without restricting the types of kernels. Therefore, for NSSPP, we derived a fully analytical Laplace approximation inference algorithm. \n\nWhen we further extended NSSPP to the deep variant DNSSPP, the introduction of the deep architecture meant that neither variational methods nor the Laplace approximation had analytical solutions. For convenience, we continued to use the Laplace approximation method employed in NSSPP. \n\n> Q: I'm aware that there has been some work by on understanding and exploring overfitting in the context of deep kernel learning (Promises and Pitfalls of DKL, Ober et al, 2021), it would be useful to touch on some work from this area during your discussion on overfitting just to clarify to the readers that this phenomenon is not arising solely due to the size of the datasets you are modelling. \n\nA: Thanks for your suggestion. We agree with your advice. In fact, in the ablation study, we also observed some overfitting phenomena in deep kernel learning. We will provide more discussion on this issue, touching on related works from this area, in the camera ready. \n\n> Q: This is just a reoccurring minor typo but I'm fairly certain it's the Porto taxi dataset (as in the city of Porto, Portugal), not Proto, so just edit all mentions of this. \n\nA: Thanks. We will fix all these typos in the camera ready.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "c4vtt5M69Y", "reviewer": "Reviewer_vqtT", "summary": "Point process are generative models for datasets of points in (typically 1D, 2D, 3D Euclidean) space, e.g. datasets of rain drops, taxi pickup locations, and a Poisson process consists of an intensity function $ \\lambda: \\mathcal{X} \\to \\mathbb{R}^+ $ that roughly shows how likely a point is to appear at $x$. The number of points $N_S$ in a region $S\\subset \\mathcal {X}$ is Poisson distributed $N_S\\sim \\text{Poi}[\\lambda_S]$ with mean parameter $\\lambda_S = \\int_S \\lambda (x) dx$.\n\nA Poisson Process with a (squared for positive output) Gaussian Process prior over $\\lambda(x)$ is a permanental process, from point clouds in 1D, 2D or 3D space, one can learn smooth underlying intensity functions. There has been much study on fitting GPs to learn intensity functions using well known GP tools such as inducing point methods and this work builds closely upon the work of [27] which uses the popular method of Random Fourier features for stationary kernels.\n\nThis paper makes two novel contributions, firstly using Random Fourier features for non-stationary kernels and  deriving the corresponding approximate marginal likelihood. Then secondly using deep kernel by nesting Fourier features that increases model capacity at the cost of requiring using numerical integrations for the approximate marginal likelihood. Experiments are performed and positive a range of results and model hyperparameters showing results both with and without improvements over baselines.\n\n[27] [Sellier 2023](https://proceedings.mlr.press/v206/sellier23a/sellier23a.pdf)", "questions": "- how is the marginal likelihood for DNSSPP computed? (The paper just says \"numerically\" and I couldn't see details in the appendix)\n- do the error bars represent a one/two standard errors or standard deviation over test set likelihoods?", "limitations": "The authors have described the method in full.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "- the idea seems a nice intuitive extension of prior work\n- I felt the writing was very clear and concise and introduced all the necessary background and new ideas very clearly.\n- the method and experiments appear to be well designed\n- the new method DNSSPP introduces more parameters and the authors provide parameter sweeps and show results with and without improvement over baselines.", "weaknesses": "My only concern is the lack of substantial novel contribution, particularly when compared to the work of [27] that introduced GSSPP.\n\n- Replacing the approximation based on Bochner's Theorem with the approximation based on Yanglon 1987 [37] significantly increases model capacity and flexibility but, unless I am mistaken, appears to be a rather small change from GSSPP, we sample over pairs of frequencies instead.\n- consequently, deriving the Laplace approximate marginal Likelihood for NSSPP also appears to be not too difficult given the result for GSSPP, use trig identities to combine frequencies and I believe the results follow from the stationary case.\n- using deep features understandably helps model fitting however the analytic result of NSSPP is lost.\n\nOverall, this felt like GSSPP with different kernels, which introduced some difficulties that seemingly had fairly simple solutions.\n\n\nI feel that it is an elegant although not-too-difficult extension of GSSPP and I do not feel the contribution is significant enough for publication at NeurIPS. I enjoyed the paper and I feel the contribution is certainly very respectable and well designed and demonstrates the efficacy of the method.\n\n\n[27] [Sellier 2023](https://proceedings.mlr.press/v206/sellier23a/sellier23a.pdf)\n\n[33] [Fast Bayesian Intensity Estimation for the Permanental Process, Walder and Bishop, ICML 2017](https://proceedings.mlr.press/v70/walder17a.html)\n\n[37] [Correlation Theory of Stationary and Related Random Functions, Akiv aYaglom](https://link.springer.com/book/9781461290865)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> Q: My only concern is the lack of substantial novel contribution......\nI feel that it is an elegant although not-too-difficult extension of GSSPP and I do not feel the contribution is significant enough for publication at NeurIPS. I enjoyed the paper and I feel the contribution is certainly very respectable and well designed and demonstrates the efficacy of the method. \n\nA: The reviewers' main concern is that the method proposed in this paper seems \"not-too-difficult\" and therefore not very \"novel\". However, we politely disagree that a sufficiently complex method is the only significant contribution for publication at NeurIPS.\n\nOur method may appear straightforward because we have presented it in a concise manner. From a high-level perspective, our main contributions are: (1) We used a nonstationary sparse spectral kernel to replace the stationary sparse spectral kernel, thereby constructing NSSPP, (2) we then derived the corresponding Laplace approximation inference algorithm, and (3) subsequently, we built a corresponding deep variant (DNSSPP) by stacking spectral feature mappings to further enhance expressiveness.\n\nTo the best of our knowledge, there has been no prior work applying (deep) nonstationary kernels in Gaussian Cox processes. This gap is what this work seeks to address. Each of these three steps above involves greater challenges when extending from the stationary case to the nonstationary case, both in theoretical derivation and algorithm implementation. Especially the third step, the deep variant (DNSSPP), has never appeared in previous Gaussian Cox processes work. This idea cleverly utilizes the relationship between spectral feature mapping and deep models, resulting in its design. This contribution has also been praised by the other two reviewers. For example, Reviewer WJmT stated: \"The introduction of a deep kernel variant, DNSSPP, which stacks multiple spectral feature mappings to enhance the expressiveness of the model significantly, is another novel idea.\" Reviewer QimU commented: \"In terms of originality and significance, this paper fills a clear gap in the existing literature on the topic, namely probing the usefulness of deep models in the permanental process setting. Whilst conceptually not a particularly complicated extension, there are obviously various nuances to moving to a deep kernel setting which are outlined clearly by the authors throughout the work.\"\n\nWe understand that different people may have different definitions of \"novelty\", which may be difficult to reconcile. We greatly respect your review and hope that the above response can lead you to reconsider the cleverly designed ideas in our work. We sincerely hope you can increase the rating. \n\n> Q: How is the marginal likelihood for DNSSPP computed? (The paper just says \"numerically\" and I couldn't see details in the appendix). \n\nA: For DNSSPP, the nested structure of trigonometric functions leads to the intensity integral ($\\mathbf{M}$ and $\\mathbf{m}$) lacking an analytical solution, and we need to resort to numerical integration. Numerical integration has many choices, such as Monte Carlo integration or quadrature methods. In this work, we used Gaussian quadrature. \n\n> Q: Do the error bars represent a one/two standard errors or standard deviation over test set likelihoods?\n\nA: It is one standard deviation.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We sincerely thank all reviewers for their efforts in providing insightful comments and constructive feedback. We are encouraged that reviewers recognize that our paper proposes an interesting extension on nonstationary permanental processes [R1,R2,R3], introduces an deep kernel variant to enhance expressiveness [R1,R2,R3], includes solid numerical experiments [R1,R2,R3], and is written very clearly and concisely [R1,R2,R3]. In the following, we address reviewers' comments point by point.

Meta-review: This paper proposes an approach to using sparse spectral representations of non-stationary kernels to improve the expressiveness of permanental processes, providing more flexible modeling via less constrained kernels. The majority of the reviewers initially assessed this paper as having clear motivation, strong writing, strong novelty, effective experiment designs, promising results and considerable significance. Following the author response and discussion, all of the reviewers were in favor of accepting the paper and all concerns were addressed. The scores support an accept decision. The authors are encouraged to incorporate clarifications from the discussion and reviewer suggestions into the final manuscript.
Decision: Accept (poster)