Identifiability for Gaussian Processes with Holomorphic Kernels
Authors: Ameer Qaqish
Didong Li
Venue: ICLR 2025 Poster (2025)
Keywords: Equivalence of Gaussian random measure; kernel parameters; periodicity; identifiability; interpretability
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
TL;DR: We characterize the identifiability of kernel parameters for Gaussian process kernels that are holomorphic around 0.

Abstract: Gaussian processes (GPs) are widely recognized for their robustness and flexibility across various domains, including machine learning, time series, spatial statistics, and biomedicine. In addition to their common usage in regression tasks, GP kernel parameters are frequently interpreted in various applications. For example, in spatial transcriptomics, estimated kernel parameters are used to identify spatial variable genes, which exhibit significant expression patterns across different tissue locations. However, before these parameters can be meaningfully interpreted, it is essential to establish their identifiability. Existing studies of GP parameter identifiability have focused primarily on Mat\'ern-type kernels, as their spectral densities allow for more established mathematical tools. In many real-world applications, particuarly in time series analysis, other kernels such as the squared exponential, periodic, and rational quadratic kernels, as well as their combinations, are also widely used. These kernels share the property of being holomorphic around zero, and their parameter identifiability remains underexplored.
In this paper, we bridge this gap by developing a novel theoretical framework for determining kernel parameter identifiability for kernels holomorphic near zero. Our findings enable practitioners to determine which parameters are identifiable in both existing and newly constructed kernels, supporting application-specific interpretation of the identifiable parameters, and highlighting non-identifiable parameters that require careful interpretation.

Published as a conference paper at ICLR 2025
IDENTIFIABILITY FOR GAUSSIAN PROCESSES WITH HOLOMORPHIC KERNELS
Ameer Qaqish, Didong Li
Department of Biostatistics, University of North Carolina at Chapel Hill (ameer, ,didongli)
(annnggiinnnnnnnnnnnn @unc.edu
ABSTRACT
Gaussian processes (GPs) are widely recognized for their robustness and flexibility across various domains, including machine learning, analysis of time series, spatial statistics, and biomedicine. In addition to their common usage in regression tasks, GP kernel parameters are frequently interpreted in various applications. For example, in spatial transcriptomics, estimated kernel parameters are used to identify spatial variable genes, which exhibit significant expression patterns across different tissue locations. However, before these parameters can be mcaningfully interpreted, is essential to establish their identifiability. Existing studies of GP parameter identifi- ability have focused primarily on Matérn-type kernels, as their spectral densities allow for more established mathematical tools. In many real-world applications, particuarly in time series analysis, other kernels such as the squared exponential, periodic, and rational quadratic kernels, as well as their combinations, are also widely used. These kernels share the property of being holomorphic around zero, and their parameter identifiability re- mains underexplored. In this paper, we bridge this gap by developing a novel theoretical framework for determining kernel parameter identifiability for kernels holomorphic near zero. Our findings enable practitioners to determine which parameters are identifiable in both cxisting and newly constructed kernels, supporting application-specffe interpretation of the identifiable parameters, and highlighting non-identifiabll parameters that require careful interpretation.
INTRODUCTION
Gaussian Processes (GPs) are powerful and flexible tools extensively used across multiple fields, such as machine learning (ML), geospatial and spatiotemporal analysis, biomedicine, finance, and environmental modeling (Rasmussen and Williams, 2006; Banerjee et al., 2014; Cressie and Wikle, 2015). They serve various purposes: as regression or classification methods through GP regression GP classification; as priors over functions in Bayesian inference (Ghosal and van der Vaart, 2017); for modeling latent distributions via Gaussian Process Latent Variable Models (GPLVM, Lawrence (2003)); and in demonstrating equiv- alencies to deep neural networks with infinite width (Lee et al., 2018). The flexibility of GPs as universal approximators, their inherent interpretability especially regarding kernel parameters and their capability to quantify uncertainty, are among their key advantages.
The kernel function, also known as the covariance function or covariogram, which defines the covariance structure within GP. is pivotal to application and effectiveness. Over recent decades, there has been a proliferation of research into developing specialized kernels tailored for specific data types including time-scries, spatial, imaging, and spatiotemporal datasets. Popular choices such as the squared exponential (SE, also known as RBF or Gaussian), rational quadratic (RQ), periodic (Per), and Matérn kernels are frequently employed, often in innovative combinations that enhance model performance (Wang et al., 2018). These combinations involve operations such summation, multiplication, and spectral mixtures (Duvenaud et al., 2011; 2013; Kronberger and Kommenda, 2013; Wilson and Adams, 2013; Samo and Roberts, 2015; Remes et al., 2017; Cheng et al., 2019; Verma.

---

Published as a conference paper at ICLR 2025
and Engelhardt, 2020), which enable the leveraging of individual kernel strengths to better capture complex data patterns.
Despite the extensive literature on GP theory and its application regression or prediction tasks, less attention has been paid to the parameter inference, particularly the identifiability and interpretability of kernel parameters. Parameter inference is critical in applications that use estimated parameters in downstream tasks such as model comparison and problem-specific parameter interpretation.
One such application is in the study of spatial transcriptomics, which measures gene ex- pression across different tissue locations understand cellular and lissue-level biological processes (Marx, 2021). One important task within this field is identifying spatially vari- able genes (SVGs), which are genes that show significant changes in expression patterns across spatial locations, among tens of thousands of genes. Svensson et al. (2018) mod- cls gene expressions as a GP across spatial coordinates using a SE kernel with nuggets: K(x,z') 03 exp (- |====""") (o The kernel parameter of was then interpreted as the magnitude of the spatial effects to identify SVGs, estimated by the Maximum Likelihood Estimator (ML.E). Other applications of GP parameter inference to spatial transcriptomics include Weber et al. (2023) and Sun et al. (2020).
Another example where kernel parameter estimates are interpreted is the decomposition of the Mauna Loa CO2 time series data (Tans and Keeling, 2023) into four kernel components in the impactful book Rasmussen and Williams (2006):
K (zx, x') = Ki (I, + K2((I x') + K3 x')
+ (2()z²²)
= 8 exp (- (x (x'xx") x²)
(- (a-20 (x-2²)²
2 2sin²(T x'))
+ 88 exp
202
263
03
1+ (I x²)²
-0s
(x 2²)2
+ 0g
+ 02 exp
+ 0f 1 ((x=0))
(1)
20
20
where K1 (I, ) is a SE kernel that captures the long-term smooth rising trend, K2 x'), called the damped periodic kernel, is a multiplication of a SE kernel and a periodic kernel that accounts for seasonal variations, K3(2, x²) ) is a rational quadratic kernel that models medium-term irregularities, and (1(z,²') a sum of a SE and white noise kernel that measure correlated and independent noise respectively. This kernel is also used as example in the tutorial of the widely used Python package "sklearn ggausrnngauusian process", with detailed interpretation of all 11 parameters proposed by the authors in Section 3. Although the interpretation seems reasonable, a theoretical understanding with a rigorous proof is missing.
Although parameter identifiability in a GP model might seem straightforward at first glance, a challenging and nuanced problem. In fact, not all parameters in widely used GP kernels are identifiable: if parameter is not identifiable, consistent estimation and subsequent interpretation are impossible. For example, for the Matérn kernel in dimension p S 3 with spatial variance o², lengthscale f, and known smoothness parameter v, Zhang (2004) proved that neither o² nor is identifiable or consistently estimable, no matter how sophisticated the estimator is. In fact, the only identifiable parameter in the Matérn kernel, termed the microergodic parameter, is o²L-2" Follow-up studies for a single Matérn kernel include Anderes (2010); Kaufman and Shaby (2013); Li (2022); Li et al. (2023), and Chen et al. (2024) for a linear combination of Matérns with different smoothness. Such negative results raise a natural question: are all parameters used in practice, including those in Equation (1), identifiable so that. their interpretat ions are justified?
As far as we are aware, identifiability of the parameters Equation (1) has not been proven before. More importantly, there is still a gap in the literature for more complicated kernel combinations like those popularized in ML, especially when the combinations involve periodic kernels. The lack of theoretical examination is partly due to the failure of traditional methods used to study GP parameter identifiability, such as the integral test (Stein, 1999), which requires conditions on the spectral density not met by common kernels like SE, Per, and RQ, and even more so when these kernels are combined. This necessitates the development of new analytic tools to better understand kernel parameter identifiability and interpretability.
2

---

Published as a conference paper at ICLR 2025
Motivated by these observations and challenges, this paper proves general theorem (Theo- rem 3.4) that determines all the identifiable functions of the parameters in any family of stationary kernels holomorphic around 0. The result applies to complex combinations of ker- nels, particularly those common in the MT. community, such as the one used in Equation (1). We demonstrate that all parameters in this kernel identifiable under mild constraints, supporting the interpretation of kernel parameters in Rasmussen and Williams (2006) and the gaaearnngauusian process" Python package tutorial. Additionally, we establish general result that is used to determine the identifiable functions of the parameters for kernel that sum of products of other kernels.
The paper is organized as follows. Section 2 provides a comprehensive background, introduces the necessary notation and concepts, and reviews the relevant literature. Section 3 presents our main theoretical contributions. Section 4 contains simulation studies to support our theories, followed by Section with a discussion of limitations and future work. brief discussion of the connection between parameter identifiability and prediction is given in Appendix C.
BACKGROUND
This section defines key concepts and notations and summarizes existing literature on GP kernel identifiability and interpretability. We begin with the definition of GPs.
2.1
GAUSSIAN PROCESS
Definition 1 (GP). A stochastic process Z is said to follow GP in domain Q with mean function μ Q - R and a positive definite covariance /ernel/ haaee//ovrriogrmm function K Ω x Q + R if for all æ1, (Z(x1,), ... (x,)) ~ N(v, E), v == [u(x1), ... (ME,),, Egj = K (x),))))
E S2
For our study, as well as presentation simplicity, we assume μ = 0, without loss general- ity (Stein, 1999). In this situation, since the distribution of / is completely determined by K, we sometimes call K GP, which refers to a GP with covariance kernel K.
Throughout this paper, focus the infill domain (also known as fixed domain or interpolation). iee, the domain Ω = [0, [ does not grow with sample size, a situation commonly considered in the literature (Stein, 1999). Next, we introduce the commonly accepted stationarity assumption:
Definition 2 (Stationarity). K is called stationary if K ((xxx)) K(x hh, +1))) Vh E RP.
For a stationary kernel, we can reformulate the kernel a function on R instead of R2 RR by Ko(x) = K (xx,0) 0). Stationarity is a common assumption in GP literature duc to its salislactory practical performance and simplicity in both implernentation and theoretical analysis. Throughout this paper, focus only stationary kernels, and still denote the simplified kernel K without causing any confusion.
2.2 KERNELS
We first note that all kernel functions considered in this paper are continuous functions unless noted otherwise. Then we introduce the following commonly used kernels in Table
In this table, o² is called the spatial variance, partial sill, which measures point-wise variance; is called the length scale that measures the spatial dependency; 7 > the period parameter; is called the scale mixture parameter; v is called the smoothness parameter. Among them, Per well-defined only when = 1, i.e. = [0, T] is a closed interval, while others well-defined on RRP for any p.
Each individual kernel in the above table captures some unique behavior in the process Z. However, when the process has complicated structure, common approach is to combine some of these kernels to create a new one. Such a combination can be simply a sum of products of these kernels, which is guaranteed to be positive definite function.
3

---

Published as a conference paper at ICLR 2025
Table 1: Example kernels, parameters, and domain dimension
Name
(
Paramcters
Dimension
SE
o² exp
|
o² 0
p21 1
212
2sin
Per
o² exp
exp
, 0
p=1
P
RQ
o² 1
||=||²
-a
d2,,a=>0 p21
2at?
Matérn
2- T(p)
22v
||
Kr
2v x
d2,,,v>0
p21
The following example extends Equation (1) used in Rasmussen and Williams (2006) to study the Mauna Loa. CO2 time series data:
x²
x²
sin2
Ko(x) = 09 exp
0g
exp
20
26
03
x
0g
x²²
+ 0%
+ 6g exp
(11-(==0)
(2)
20g0?
20%
where (01, (0,, 011:7 7) be the vector of all parameters in the above kernel.
Note that this kernel is more flexible than the one in Equation (1), which assumes the period = 1. We adopt this more challenging modification since the period is sometimes unknown in practice so practitioners have to estimate from the data.
2.3 IDENTIFIABILITY
The study of identifiability of GP kernel parameters relics on the notation of equivalence of measures defined below:
Definition 3 (Equivalence of measures). Two measures P1 and P2 are said to equivalent if they are absolutely continuous with respect to each other, denoted by Pi P2. That is, (A( = 0 + (2(A) = O. Two measures are said to be orthogonal, denoted by Pi 1 P2 if there crists a measurable
such that Pi(A) =0 but P2(A)= =.
Two GP laws Pi and P2 are either equivalent, or are orthogonal (Feldman, 1958), which means they assign probability disjoint sets: P1 and P2(A) = We define the identifiability of GP parameters as follows:
Definition 4 (Microcrgodicity). Let (Ko)eeo be a family of covariance kernels of a GP. Then function h = h(0) of 0 is said to be microergodic if Ke, = Ko, > h(01)= h((2))
II h and h are both microergodie, then h(0i) h(02)) = ((00)) h(02), are related by a bijection. Thus the migroergodic function is unique up to a bijective transformation, and makes sense to speak of 'the' microergodic function h.
and h
Definition 5 (Identifiability. Let (Ko)øee be family of covariance kernels of a GP. function g g(0) of is said to be identifiable if Ks, K8, = g(01) = g(02), or equivalently, g is function of the microergodic function h. We say that the family (Ko)oeo identifiable if 0 identifiable.
Note that a consistent estimator of g(0) can exist only when g(0) is identifiable - when g()) not identifiable, say Ko, = Ko, with g(O,) # g(22), it is not possible to find consistent estimator of ((), since there is no way to distinguish between data generated from Ko, and those from Ke, almost surely (see Stein (1999); Zhang (2004) for more detailed discussion). Thus anything that can be consistently estimated is identifiable. The microergodic function h(0) the maximal identifiable function, so knowing the microcrgodic function completely solves the identifiability problem for the family of kernels. However, in some cases, it is difficult to fully determine the microergodic function h(0), whereas easier to determine that some specific function (0) is identifiable.
4

---

Published as a conference paper at ICLR 2025
To study the identifiability of GP kernel parameters, it suffices determine when two GPs in the same parametric family are equivalent. However, to determine whether two GPs are equivalent is not easy task, and the methods for doing highly depend on the form of the kernels. There is a rich literature focusing on identifiability for Matérn kernels, where has been shown that
{ (ofe72m 2 .1 (036220 v2) (of,,,1)) = (02, U2) p p25. 5.
1 15p53,
Koj Kof-i1-1 = Kagder
That when the domain dimension is greater than equal 5, then h(o² () (a² L,v), so all three parameters are identifiable (Anderes, 2010; Bolin and Kirchner, 2023); when (o26-2 .V: is
the domain dimension is less than or equal 3, then h(o2, identifiable (Loh et al., 2021), but not o² or (Zhang, 2004). As a result, there no consistent estimator of o² or L, but instead, consistent estimator of 0²p-2v, called the microergodic parameter, does exist (Kaufman and Shaby, 2013; Loh al., 2021), namely the MLE. The microergodic function for p 4 is an open problem.
Although the identifiability of Matérn has been understood, the study of other kernels including Per and RQ is much sparser. The key reason is that the tool to study equivalence between Matérn kernels, known as the integral test (Stein, 1999), requires strong conditions on the spectral densitics of the kernel, which are not often satisfied by other kernels. The spectral density is defined below:
Definition (Spectral measure). For a stationary kernel K, its spectral measure, denoted by F, is defined Uhrough
K(x)
= K(x)=
{())=/c* c0===""" (F((w)
Bochner's theorem guarantees the eristence and uniqueness of F. The density of F w.r.t. the Lebesgue mensure dw, denoted by J, if exisls, called the speciral densily.
The condition to use the integral test is that [(w)| x
for some > 0. That is, the spectral density is required to behave like |||||- for some positive a >> 0. The
spectral density of Matérn is f() 0² 2= p/2r(u+
(2v)" ()2))) T'meas (Rasmussen and Williams, 2006, p. 84) (note that we use a different. Fourier transform convention than (Rasmussen and Williams, 2006).) However, this condition is not met by RBF, Per, or RQ, as their spectral densities decay very rapidly due to the infinite differentiability of the kernels.
( + ||]l²)
[|]l)))(+++)) (v+
||w|--207
Due to the popularity of these kernels in ML, we aim to address these challenges and study the equivalence of GPs with these kernels and their combinations. The next section provides theoretical support for the success of these kernels in terms of identifiability and interpretability.
3 THEORY
In this section, we present our main theory regarding equivalence of GPs, as outlined in the previous sections. We first determine the identifiable parameters of the individual kernels used in Equation (2), i.e., SE, Per, and RO, with some extensions.
Theorem 3.1. The microergodic functions of 5 individual kernels in Table 1, including all four components K1, K2, K3, KA in Equation (2) and an additional kernel, Cosine, are summarized in Table 2.
Theorem 3.1 supports the identifiability and interpretability of each kernel parameter in SE, Per and RQ, as discussed in Section 2.2. In addition, include the cosine kernel, which will be revisited later in this section.
Then consider the combination of SE, PER and RQ in Equation (2), extension of the kernel used by the impactful book Rasmussen and Williams (2006) and the tutorial the widely used Python package "sklearn.gaussian _prccess__
5

---

Published as a conference paper at ICLR 2025
Table 2: Microergodic functions of five kernels
Name
(())
Parameters
Microcrgodicity p
SE
o² exp
||
o2
(a2.4)
21
2/2
sin (
Per
o² exp
09,47>0
(o2,4n)
12
Damped Per
o² exp
2sin
2
24,,0
(o 00,4,621 (2,))
289
RQ
2² 1
||x113
0
2 1
2a12
Cosine
o² cos(s" x)
0 > 8,
21
Theorem 3.2. All parameters in Equation (2) are identifiable provided 010, the length-scale of the SE component model the corvelated noise, is less than 82, the length-scale of the SE component lo model the long-term Ivend.
Such a constraint is necessary, and not surprising, since otherwise, say, if 02 I1o then we can merge the two SE components into single SE: (0) + 0%) exp z² 203
making of + 0g identifiable instead of of and 0g. This distinction of Lwo SE components is also discussed in Section 5.4.3 Rasmussen and Williams (2006). Excluding this trivial case, all parameters interpretable, discussed in the
are identifiable As consequence, these parameters same section in Rasmussen and Williams (2006). For example, 0 mcasures the amplitude and 02 mcasures the characteristic length-scale of the long-term smooth rising trend; within the seasonal trend, 03 gives the magnitude, 04 gives the decay time for the periodic component, Y gives the period while O5 is the smoothness of the periodic component; for the (small) medium term irregularities, Og is the magnilude, 07 is the typical length-scale and Og is the shape parameter determining diffuseness of the length-scales 0g the magnitude of the correlated noise component, 010 is its lengthscale and 011 is the magnitude of the independent noise component.
Now we would like to answer the following more challenging question with broader implication: Given a new kernel, how do we determine the microcrgodic function? Spocifically, if we combine a finite number of kernels, such as those in Table 2, by finite multiplication and addition like Equation (2), what is the microergodic function of the resulting kernel? To answer these questions, need to introduce the following notions first.
Lemma 3.3 (Kernel decomposition). For any stationary kernel K, K can be uniquely decomposed as K = K + K", where KS is kernel with continuous spectral measure and Kd is another kernel with discrete spectral measure.
A direct consequence is that if K admits spectral density, then K = Ks; while if K periodic, then K Kd. We call K the contimuous component. and Kd the discrete component. Note that this notion is different from continuous and discrete functions, and we do assume all kernels are continuous functions themselves. Here the contimuous and discrete notion is at the spectrum level. For example, the Per kernel, is continuous has a purcly discrete spectrum. Moreover, we denote the spectral measure of K as F and the spectral measure of K'd as Fd, where F Fe Fd is the spectral mcasure of K.
function, but
Such a decomposition offers deeper insights to understand different types kernels. Morcover, to undersland the equivalence of GPs, it suffices to undersland the equivalence of its continuous component. and discrete component separately, given by the following key theorem, which the main result of the paper:
Theorem 3.4. Given two kernels K₁ and K2 with K₁ holomorphic on some ball around 0 in CP, the p-dimensional complex space, then Ki K2 if and only if the following two conditions hold:
1. Kf(x)= Kg(x) for every RP.

---

Published as a conference paper at ICLR 2025
CFF((W)) for all w R and 2. There are C >>>0 0 such that cFf((w)) s (f()))
Σ (1 (1- (1()))) w:Ff((w))>0
Pf(#w)) <0..
Note that K Rp R is said to be holomorphic holomorphic extension K to some ball B C C* around 0, such that K = K on B n IRP. While being holomorphic on a ball around is a stronger condition than being infinitely differentiable, most infinitely differentiable kernels used in practice, including all those in Table 2, are holomorphic on a ball around 0. Condition means the contimuous components of Fi and F2 are the same, while Condition 2 mcans the discrcte components of F1 and F2 have the same support, and their relative difference, although allowed to be nonzero, should decay fast enough.
ball around in CP if it has
Notably, Theorem 3.4 provides a general pipeline to study the identifiability of kernel parameters, summarized in the following theorem:
Theorem 3.5. Let (Ko)ace a family of stationary kernels on R', each of which is holomorphic some ball around O in C², We have the following assertions regarding the microergodic, function:
1. If h(0) is microergodic for the continuous component (KG))ge and g(0) is microer- godic for the discrete component (Kg)aee, then (h(0). g(0) is microergodic for (Ko)
2. Moreover,
(a) h(0) is microergodic for (Ko)oee if and only if K2, (z)) K2,(x), (x), Va R
h(()) h(02).
(b) g(0) is microeryodic for (Kg) if and only if Ee, C > O, s.t. cFf((w)) ((w)) S (CF((WW)) Va E ER
(1
F2, (143)
g(0,) 9(02)
Σ
<0
Fo, (wW))
w:Fo ([w))>0
That is, in order to find the microergodic function of a parametric family of kernels Kg, it suffices find the microergodic function h(0) of the continuous component, and g(0) of the discrete component separately. Moreover, to find h(0), it suffices to understand when two continuous components are equal everywhere; find g(0), we need to investigate the conditions 2b about the discrete measure pd.
Having established the foundational aspocts of kernel identifiability, we now apply our results to determine the microcrgodic function of various combinations of kernels. Our general strategy to use Fourier transform identities compute the spectral mcasure of the combined kernel (see, for example, Theorem B.6) and then apply Theorem 3.4. These combinations not only illustrate the practical applications of our theoretical findings in Theorem 3.4, but also provide insights into designing new kernels with desired properties. We start with the squared exponential kernel with automatic relevance determination (ARD).
Theorem 3.6. For the family
(z) - o2 exp (- [x
where o² > and M is a positive-definite matriz, the microergodic function is (o²,M)
Next, we study the sum of cosine kernels:
Theorem 3.7. For the family
Kor...om om. (x) = of cos()137) + of cos(823) +... of cos(sm where of...., and s Sm E R, under the natural constraint 0 5 $1 < 82 < .... Sm, the microergodic function is (81,-.., 8m)-

---

Published as a conference paper at ICLR 2025
Theorem 3.7 shows that when cosine kernels are combined linearly, their individual frequencies (or periods) remain identifiable, provided they are distinct. This scenario often arises in signal processing where different periodic components need to be isolated and identified. Notably, the last kernel in Table 1 is a special case of the kernel in Theorem 3.7 with m = 1.
Next, we study the product of Cosine kernels:
Theorem 3.8. For the family
Ka (2) = o² cos( (0s((11)) cos()22) ... cos(smm2)
where o² > 0 and 81...., 8m E R, under the natural constraint 0 < 81 S 82 S S Sm, the microergodic function is {#ts) t..... == {a1s1 + 0282 + am3m : 01, .......... Om {-1,1}} If m 1, then the mircoeryodic function simplifies to ($1,...,
In Theorem 3.8, when m 2 4, we do not have identifiability of (S1,..... (9....,8m)) 8m). For example, for 4, when (51, 82, 84) = (1, 2, 2) and (अ1, 82, 83. 84) (1,1,333) the values of the microergodic function coincide, that value being {0, =2, #4. =8}. Theorem 3.8 shows that for a product of discrete spectrum kernels that are all a function of the same variable z, the parameters of each individual kernel may not be identifiable.
Finally, we explore the sum of periodic kernels as previously discussod.
Theorem 3.9. Let Ke denote the periodic kernel with variance parameter 1, length-scale E, and period n.. For the family
Ko (2) = of Ke (x) (x) 0 (a
where o7.03, 11,21 > O, n 12 0, the microergodic function (of i,, is, all parameters are identifiable.
n, 02, l2, 12), that
This result is crucial for scenarios where multiple periodic processes operate at different scales or periods, as often encountered in geospatial, financial, and environmental data analysis.
4 SIMULATION
In this section, provide empirical support lo our theoretical results on kernel parameter identifiability, presented in Section 3, by investigating the behavior of the maximum likelihood estimators (MLEs) the sample size increases.
Before moving to the simulation details, we would like to clarify the broader picture of parameter inference for GPs, which involves three steps: first, determining which parameters are identifiable; second, finding a consistent estimator of identifiable parameters, such as the MLEs or others estimators; and third, developing numerical methods to compute these estimators. While the second and the third steps are crucial, they fall beyond the scope of this paper, which focuses solely the first step-a a theoretical framework to find all the identifiable parameters. In fact, even for simple kernels like the SE and Matérn kernels, whether the MLE consistent remains open (Loh and Sun, 2023).
Despite these complexities, we use standard optimization packages commonly applied in the GP literature find the MLEs. Our simulations are not intended to solve the open problem of MLE consistency or introduce new numerical techniques; rather, they serve to illustrate the theoretical results on identifiability through practical examples.
We start from individual kernels, followed by the combination in Equation (2).
4.1 INDIVIDUAL KERNELS
We consider the individual kernels: SE, Damped Per (DPer), Per, RQ. and Cosine. For the cosine kernel, we parameterize in terms of the period 7 so that 2r Input samples are
7
1. 1 random shift to n evenly spaced points in [2] 4 generated by adding a unif(- An where {500, 1000, 2000, 50003. After generating the outcomes by sampling GP with the given kernel at the inputs, we added independent Gaussian noise from N (0, =), E = 0.01, to model measurement errors (see Section D of the appendix for the experiments repeated with
8

---

Published as a conference paper at ICLR 2025
0.1). All kernel parameters were estimated by MLEs, with 100 replicates for each kernel configuration to assess the convergence of the MLEs. The results are summarizzed in Figure 1. These boxplots demonstrate that the MLEs of all parameters except o² in the cosine kernel appear consistent, as conjectured by their identifiability, proved Theorem 3.1.
Some of the MLE standard deviations appear platcau for large n. One explanation for this is numerical limitations - for our squared exponential simulation, where o² = and t= - 1/500, the condition numbers of the covariance matrix of the observations are 6.82 10", 1.80 109, 1.89 10th, and 3.37 1024 for sample sizes 500, 1000, 2000, and 5000, respectively.
The failure of the MLE of o² in the cosine kernel to converge in agreement with the microergodicityy of 7. In fact, if we treat y known and let the noise variance decrease to then since the covariance matrix has rank for all 2 2, can be shown that the MLE (e) converges to (0)) o² 2x
17
0.0552
0.0104
11
4.0
0000
1.0
0.0000
3.0
0.
01046
Samre
Sameie 5000
Samrae 600
Samaate Size
Samrle 5:20
(a) Squared Exponential (SE)
(b) Damped Periodic (DPer)
e
0,0838
0.00315
13
33
0.0006 12
C.00945
4.0
3.0
6,00340
C.00330
38
0.8
0.0000
Nar
Sampp Aie
Seee Nam
www. 10
Na
(c) Periodic (Per)
(d) Rational Quadratic (RQ)
0.2500
0:2495
2
0.2890
500
2000
3000
500
1000
2000
5000
(e) Cosine
Figure 1: Simulation results for various kernel types. Each subfigure shows the boxplots of MLEs for the corresponding kernel, with ground truth in horizontal dashed line.
4.2 THE COMBINED KERNEL IN EQUATION (2)
Then, we study the combined kernel, one motivating kernel of this paper, defined in Equa- tion (2). Since the kernel was proposed for forecasting COzlevel on the Mona Loa dataset, we set the time interval to be 0.45), presenting the time span of 45 years. Input samples are generated by adding unif( 45 45 random shift to evenly spaced points in 45
4n 45 45 where n E (50, 100, 200, 500). All kernel parameters were estimated by MLEs, with Ax.
replicates to assess the convergence of the ML.Es. Moreover, to further mimic this dataset,

---

Published as a conference paper at ICLR 2025
the ground truth parameters and noise variance fi are set to be the MLEs learned from running the "Gaussian process regression" package from the scikit-learn Python package. All truth parameters to be estimated are given by Table 3.
0,
8
700
1000
230
000
2000
132
20
293
333
2300
1
160
260
Sample Size
Somple
Sampll
0
y
0
6,
10-
500
10-
59
100
2003
900
228
200
S02
100
2000
500
190
903
Sample le Size
Sample Size
Sample
Sample Size
0g
6
100
300
223
226
520
50
150
200
Sample Size
Somple Size
Sample Size
Figure 2: MLEs of parameters in Equation (2), with ground truth in horizontal dashed line.
In Figure 2, again observe that the MLFs generally are unbiased, but for some parameters, their variance does not strictly decrease with sample size. This is likely due to the relatively large number of parameters (10) compared to the small sample size of 500.
5 DISCUSSION
This paper has introduced a novel analytical framework that advances the theory of iden- tifiability of kernel parameters in GPs for large class kernels, those holomorphic around O. We have demonstrated that all the parameters certain combinations of kernels, such as the example employed on the Mauna Loa CO2 time series data, are indeed identifiable. This establishes a robust thcoretical foundation for selecting or constructing GP kernels and determining the identifiable functions of the parameters practical applications.
Looking ahead, several avenues of future research present themselves as particularly promising and interesting. First,, while establishing the identifiability of kernel parameters is a critical step, it does not necessarily guarantee the consistency of the MLE. The analysis of MLEs complicated due the complex nature of the likelihood function involved, which is often mulli--modal and difficult to handle. Second, extending our theoretical framework to encompass non-stationary kernels could enhance the flexibility of GPs in modeling data with evolving trends and dynamics. This area is notably challenging due to the current limitations in mathematical tools available, presenting a largely open problem in the field. Third, another intriguing direction for research involves exlending our findings to infinitely differentiable kernels that are not holomorphic near 0, though most infinitely differentiable holomorphic near 0.
kernels used in applications
Reproducibility Statement: All code used to produce the results of this paper are provided Appendix A. Complete proofs of all lemmas and thcorems stated in the paper are provided in Appendix B.
Our paper does not deal with sensitive experiments, data, any methods that can be expected to cause harm. We have no conflicts interest and have no data privacy concerns.
Ethics Statement:
10

---

Published as a conference paper at ICLR 2025
Acknowledgment: AQ was supported by NIH grant R37 A1029168; DL supported by NIH grants P30 ES010126, R01 HL149683, R01 HL173044, Ro1 LM014407, R56 LM013784, UMI TR004406.
REFERENCES
Abramowitz, M. and I. A. Stegun (1964). Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tobles, Volume 55 Applied Mathematics Series. Washington, D.C.: National Bureau of Standards. Reprinted by Dover Publications, 1972.
Anderes, E. (2010). On the consistent separation of scale and variance for Gaussian random fields. The Annals of Statistics.
Banerjee, S., B. P. Carlin, and A. E. Gelfand (2014). Hierarchical modeling and analysis for spatial data. CRC press.
Bolin, D. and K. Kirchner (2023). Equivalence of measures and asymptotically optimal lincar prediction for Gaussian random fields with fractional-order covariance operators. Bernoulli 29 (2), 1476-1504.
Chen, J., W. Mu, Y. Li, and D. Li (2024). On the identifiability and interpretability of Gaussian process modols. Advances in Newral Information Processing Systems 36.
Cheng, L., S. Ramchandran, T. Vatanen, N. Lietzén, R. Lahesmaa, A. Vehtari, and H. Lähdesmäki (2019). An additive Gaussian process regression model for interpretable non-paramctric analysis of longitudinal data. Nature communications 10(1), 1798.
Cressie, N. and C. K. Wikle (2015). Statistics for spatio-temporal data. John Wiley & Sons.
Duvenand, D., J. Lloyd, R. Grosse, J. Tenenbaum, and G. Zoubin (2013, 17-19 Jun). Structure discovery in nonparametric regression through compositional kernel search. In S. Dasgupta and D. McAllester (Eds.), Proceedings of the 30th International Conference on Machine Learning, Volume 28 of Proceedings of Machine Learning Research, Atlanta, Georgia, USA, pp. 1166-1174. PMLR.
Duvenand, D. K., H. Nickisch, and C. Rasmussen (2011). Additive Gaussian processes. Advances in neural information processing systems 24.
Feldman, J. (1958). Equivalence and perpendicularity of Gaussian processes. Pacific
Math 8 (4), 708-
Ghosal, S. and A. W. der Vaart (2017). Fundamentals of nonparametric Boyesian inference, Volume 44. Cambridge University Press.
Ibragimov, I. and Y. Rozanov (1978). Conditions for regularity of stationary random processos. In Gaussian Random Processes, pp. 108-143. Springer.
Kaufman, C. and B. A. Shaby (2013). The role of the range parameter for estimation and prediction in geostatistics. Biometrika 100(2), 473-484.
Krouberger, G. and M. Kommenda (2013). Evolution of covariance functions for Gaussian process regression using genetic programming.
Lawrence, N. (2003). Gaussian process latent variable models for visualisation of high dimensional data. S. Thrun, L. Saul, and B. Schölkopf (Eds.), Advances in Neural Information Processing Systems, Volume 16. MIT Press.
Lee, J., Y. Bahri, R. Novak, S. S. Schoenholz, Pennington, and Sohl-Dickstein (2018). Deep neural nerworks Gaussian processes. In International Conference on Learning Representations.
Li, C. (2022). Bayesian fixed-domain asymptotics for covariance parameters a Gaussian process model. The Annals of Statistics 50(6),
11

---

Published as a conference paper at ICLR 2025
Li, D., W. Tang, and S. Banerjee (2023). Inference for Gaussian processes with Matérn covari- ogram on compact Riemannian manifolds. Journal of Machine Learning Research 24 (101), 1-26.
Loh, W.-L. and S. Sun (2023). Estimating the parameters of some common Gaussian random fields with nugget under fixed-domain asymptolics. Bernoulli (3)3,, 2519 2543.
Loh, W.-L., S. Sun, and J. Wen (2021). On fixed-domain asymptotics, parameter estimation and isotropic Gaussian random fields with Matérn covariance functions. The Annals of Statistics (6), 3127 3152.
Lukacs, E., O. Szász, et al. (1952). On analytic characteristic functions. Pacific J. Math 244,, 613-625.
Marx, V. (2021). Method of the year: spatially resolved transcriptomics. Nature meth- ods (1),, 9 14.
Rasmussen, E. and K. I. Williams (2006). Gaussion processes for machine learving. Adaptive computation and machine learning. MIT Press.
Remes, S., M. Heinonen, and S. Kaski (2017). Non-stationary spectral kernels. Advances in neural information processing systems 30.
Samo, Y.-.. K. and S. Roberts (2015). Generalized spectral kernels.
Stein, M. L. (1999). Interpolation of spatial data: some theory for kriging. Springer Science & Business Media.
Sun, S., J. Zhu, and X. Zhou (2020). Statistical analysis of spatial expression patterns for spatially resolved transcriptomic studies. Nature methods 17(2), 193-20 100-
Svensson, V., S. A. Teichmann, and O. Stegle (2018). Spatialde: identification of spatially variable genes. Nature melhods 15(5), 343 346.
Tans, P. and R. Keeling (2023). Trends in atmospheric carbon dioxide. https: htop::///mll noaa.
gov/ccgg/trends/ttte wttteerdddaaaaa.. html. Accessod: 2023-08-01.
Verma, A. and B. F. Engelhardt (2020). robust nonlinear low-dimensional manifold for single cell RNA-seq data. BMC bioinformatics 21, 1-15.
Wang, J., W. K. Yam, K. L. Fong, S. A. Cheong, and K. M. Wong (2018). Gaussian process kernels for noisy time series: Application to housing price prediction. In Newral Information Processing: 25th International Conference, ICONIP 2018, Siem Reap, Cambodia, December 13 16, 2018, Proceedings, Part VI 25, pp. 78 89. Springer.
Weber, L. M., A. Saha, A. Datta, K. D. Hansen, and S. C. Hicks (2023). nnSVG for the scalable identification spatially variable genes using nearest-neighbor Gaussian processes. Nature communications 14 (1), 4059.
Wilson, A. and R. Adams (2013). Gaussian process kernels for pattern discovery and extrapolation. In International conference on machine learning, pp. 1067-1075. PMLR.
Zhang, H. (2004). Inconsistent estimation and asymptotically equal interpolations in model- based geostatistics. Journal the American Statistical Association 99(465), 250 261.
12

---

Published as a conference paper at ICLR 2025
APPENDICES
A CODE AVAILABILITY
All codes can be found and downloaded at https://github. com http:///thuubcccoomeeeeeeeeeee ic1r2025- simulation.
eng/
B PROOFS
We begin with the proof of Lemma 3.3, Theorem 3.4, and Theorem 3.5, which are the general tools we developed to study equivalence of GPs on which the other theorems depend.
B.1 PROOFS OF LEMMA 3.3, THEOREM 3.4, AND THEOREM 3.5
We first define the measure associated to GP Z.
Definition 7. For set Q2 covariance kernel Ω2 x + R and mean function m Q -> R, define the measure Gm K) to be the distribulion of Gaussian process (Z(x)) zen) with mean m and covoriance K. Thus Ga(m, K) is measure on (IR" where An is the a-algebra generated by the coordinate maps (xx) for x E Q
Now we state Bochner's Theorem, which is the key tool for working with stationary kernels. Lemma B.1 (Bochner's Theorem (Stein, 1999)). A continuous complex-valued function K R + C is the covariance function of complex valued slationary process on IR? if und only if there a posilive fonite measure F on TRP such that K con be expressed as
f {(x)=)<>"' e F(dw).
= K(x)=
K(z)
K. is real valued if and only if the distributions of w and -w are the same under F.
Now prove Lemma 3.3.
Proof of Lemma 3.3. Let F be the spectral measure of K. Let Ω = {w e R' F(fw)) > 0}. We claim that Q is most countable. For contradiction, suppose Q is uncountable. Then {w F((W)) > 1/n} must be uncountable for some n. But then F(R²) 2 Hl(w: F((w)) 1/n)] = 20, contradicting finiteness of F. Let
Fd F(WW)))
WEN
F-=F-FU Fd.
Then let Kd be the inverse Fourier transform of Fd, ie. kd(a) = f e pdd and let K be the inverse Fourier transform of F", By Bochner's theorem, both K and Kd are complex-valued stalionary covariance kernels. Since F is symmetric, so is pd, Hence pe pd is also symmetric. Thus K and Kd are real valued.
Now prove condition under which we recover process (Z(t))ERR from it's values on a box T.P Lemma B.2. Suppose K is a stationary covariance kernel on RR that is holomorphic on some ball containing O. For Ω C R², let
An =t
Then for any T >
Agr C Ajo.Ty
where Ajor denotes the completion of ATo.T by sets of probability with respect to P Gzr (0, K). In other words, the entire process (Z(t))eER can be recovered with probability 1 from (7()))eoo.T
13

---

Published as a conference paper at ICLR 2025
Proof. For ease of notation, assume p == 1. The proof for p is the same. Say K is holomorphic on D 26), > 0. By Theorem Lukacs et al. (1952), J F((w) < co for |4] 26. For & N, since ga e te
[2R)]
f W 2k F(dd)) v
(2k) 82h
/ [c00 +e (w) < co.
Thus the process is infinitely mean-square differentiable. For 8, RR,
(Z))) = (Z)()(s), ((1)) K(s - 1) "e"","""))
Hence the map Z(t) + ei extends by linearity and continuity to unitary isomorphism &
from (R) to Lg (F), where (R) the closure of span (Z) :t t IR} in L2PP), and (x(F) is the closure of spanfelwt :t tt R in (F).
Let t E [-2.0] We claim that
Z z(m) (0)
Z(t)
Σ
n!
where the converges in (P). Using the unitary isomorphism above and diffcrentiating with respect to yields
2 jut
@Z'(t) two
: $ (1) = (iw)) pist
Thus to show that Enzo (0m) tm = Z(t) in LP(P), it is equivalent to show that En (iw)" and nt fm erul in (F.... We know Σ (dw) p ciul pointwise as N oo, mn!
n!
( n
N
W
t"
1t1"
n!!
n!!
n=
n=0
S Wi t
S se- fu + +ew
Since, as mentioned earlier, e e eew E (F),), the dominated convergence theorem yields edw
IN (tw)" nf t - eint in (F) N co.
Thus A[-g.r] C. Ajo. Repeating the same argument with the process Y(t) Z (t 9 which has the same covariance Z, we obtain Ap-8T C A.or. and by induction, A1-4.T] M)..r for all & E N. Similarly, A(o,T+kg) C Ajory for all le E N. Hence Ar C A/o.T-
C
We immediately conclude the following, which reduces the problem of determining equivalence of GPs on a bounded interval to determining their equivalence on all of RP, which is a much simpler task.
Lemma B.3. Suppose Ko and K₁ are continuous stationary covariance kernels on RP and that Ko is holomorphic on some ball containing O. Let T > 0. Then.
Gorr (0, Ko) (o,,ry K1) < GR (0, Ko) Gp (0, K1).
14

---

Published as a conference paper at ICLR 2025
Proof. Let P3 Gze K3),,, 0, Note (0, PylAamm Suppose (0, Ko) G- K1), that is, Po and P1 equivalent Ap.. Then Po and P1 are equivalent on (oTTee the completion of AIo.TP by sets of probability 0 with respect to Po (or equivalently, with respect to Pt, since Po = Pi A1oTTo) By Lemina B.2, Agr C AfoTTo" Consequently, Po and P1 are equivalent on Age.
Parts of the following lemma proved in Tbragimov and Rozanov (1978), but for our application, we need precise statement of the result. In what follows, for real valued functions and b defined on S, we use a(s) x b(s) for s S mean that there exist C. > 0 such that ca(s) S b(s) I Ca(s) for all E S. Lemma B.4. Suppose continuous stationary kernels Ko and K2 on R have spectral measures Fo and F1. Write
F3 = Ff+ Fy, 0,
6) is the discrete part of Fj, and F7. = F, - Fy is the where F = EWn continuous part of Fj. Let Pj Gg (0, Kj), j 0, For j = 0, 1, let HR())) be the closure of span| =1 RP} in (2))). The sollowing are equivalent:
((W))S) Fy
(i)
Po P1.
(ii)
|| 11-1122(P3) x x11 (11a())))) on span(Z(t) :t RR),
II - K1l (S( 00.
(181)
Fe Pf,
Fo((W)) x (fW()))
(1 F(((W)) Fd((w)))
Σ
<..
(:
Proof. To clarify (ii), HS(Ho(R)) is the space of Hilbert-Schhiie Schmidt operators on Ho (RP). The continuous bilinear map K1 HR()) HR(PP) C is identified with the bounded linear map K₁ : (R*) + (R?) by
(KUU, V) Ho(r)) K((,)) U,V Ho())).
Under this identification, Ko
By Theorem 7 on page 129 of Stein (1999), (i) and (ii) are equivalent.
Suppose (ii) holds. Then Ho(R)) H1 (RR") and || (12(()))) || [[(()))) on HR()).. Hence by the unitary isomorphism 4 Hy (RP) + Lær (F₃), 4Z(t) = = defined in the proof of Lemma B.2, Ler (Fo) Tur (Fi) and || [1( = 11 [I) on (Fo).
We claim that LR (Fo) (FF)). Let (HP). Let R be rectangle containing the support of f. By the Fourier inversion theorem, for every E Rº, f(x) = Sn f({)e dg. The integral is a pointwise limit of Riemann sums. Each Riemann sum has absolute value bounded by [ Vol(R) E (FF),, so by dominated convergence, the Riemann sums converge f in 2 (Fo). This shows that f (Fo). Since C (R²) is dense in (Fo), we oblain L (Fo) (FF...)
Then the equivalence || (1)((F))) || (I))))) on Tax (Fo) (.2(Fo) implics I odF = dddFi Fi(dw) and odFo ʃ OdFi for for all o > 0. Thus Fo Fi, so there is density f(w) Foldw)
15

---

Published as a conference paper at ICLR 2025
all @ 2 0 implies f(w) x 1 for Fo a.c. WER.. IU,V E Ho(R)) and ΦU, 2V, then
(K1U,V) KI(U,V)
((W)
/ f(W)
(fu, (fu)))
Hence via the isomorphism Φ, K2 is identified with M,, the operator of multiplication by f. Thus
|| - M, (ASCL200) = || Ki (IllAS(H((PP) < oo.
Let.
Ω = {w IP Fo((w]) > 0}.
Since Fo
F6 + Fá and Fū
Føl, we have the orthogonal dccomposition L²²
L (R \ F8) @ (2, Po.. Thus
|| M, MIllAs(L((F)))
= -| My Myllas(cc(eee....) + ||T Myll -yllis(
Note 7 M, = M1-1- We claim that 1 f(w) = for Fo-a.e. w E RP f. For contradiction,
suppose (0)
0. Then either (1- > 0)RR (2) 0 or F8((1-f < > 0, or both. Say [ n (2) > 0. Then there is s > 0 such that Q. Define m(t) (Anooo)))
0 (
(2) > Let A (1
+ 0, m continuous. Since m(-20) = 0 and m(x)) = (A), there is with m(t) = (A).. Thus A can be divided into two sets of positive Fe measure. Inductively, A can be divided into sets of positive F measure for every k. Thus dim( F8)) 2 for all k, so (Ja(1 N1 15 (dww) 2 elleeiiAcE)
Then since m((+ Pr (An (t)+)))))))))))
1)P - +
Ff)) co. Since ((1 (1- (ep,
for all & E (A, F8), we obtain
|| My MIliSSLAAA
11 MIllHS((((AF5)))
2 2 (A,
=00
This is a contradiction. Thus 1- f(w) for Fj a.e. w E R Since (T F(w)F) we obtain Fi = Fj. Thus || - My (a = 0. To compute | M (1(1)
note that (1w)) 1/1 Ω) is orthonormal basis of L2(
|| Mylaa((2(00EE))
(c), Fo(( 1/2 (m)) (
WEN
2
(1 (())))()) Fd((ww)
[w)
WEn
Σ f(w))²
1 F(((W) 2
Σ
WEB
This proves (iii)
16

---

Published as a conference paper at ICLR 2025
Now suppose (iit) holds. Then || [1)2 x || (1)())) so || 11l ll (I(a(s)) span span(Z(t): 1 E R²). Then since F6 = Ff, [(w) = for aa..e w E TR and
||r K (
|[ M, (iS(LP(FF))
= | My ((SILA + 1| - Myllis(1a(0FF)))
= | My (AS(LP(),F))
F(((W))
Σ wEn
(o((w)))
< 00.
This proves (ii).
We are now in a position to prove Theorem 3.4.
Proof of Theorem 3.4. This is a direct consequence of Lemma B.3 and condition (iii) of Lemma B.4 for the equivalence of measures on RP.
To obtain Theorem 3.5, we want to apply Theorem 3.4 to the families (oee and (K*) but since Theorem 3.4 required the kernels to be holomorphie near 0, need to show that K and Kd are holomorphic near whenever K is.
Lemma B.5. Suppose K is a stationary covariance kernel on R that is holomorphic on the polydisc D = {2 CP :2yl < d,,vyjj Vj}, where 81.....8p O. Then both K and Kd are holomorphic on D.
Proof. Note that K is the characteristic function of F. By straightforward generalization of Theorem of Lukacs et al. (1952) from R to Rº, we have for arbitrary contimous stationary kernel K that
K is holomorphic on D = f feluiluultmtthhele (rrlllnttttttllllllll twollll F(dw) < when 1را| 85 for all j.
Thus f F(dw), it follows that I (/wallell++-+twl (dww) < oc and I elus livil [uollell (dw) < 00 for luyl < 8/. Hence K and Kdd are holomorphic on D.
Fdd) < 0o for l8s1 < 8j. Since (dw) S F(dw) and F (dw)) S
Now we are ready to prove Theorem 3.5.
Proof of Theorem 3.5. 1. Suppose ((0) is microergodic for (Ki)Eee and g(0) is microergodic
for (K&) Let 02 E O. By using Theorem 3.4 to write down the conditions for K6, = Ko,, KG, = Kÿ and Kdd = Kg we obtain
Ko, = Ko, = Kë, = Kū and Kg = K
Then by definition of microergodicity,
K = K2 and K = Kyg - 1(01) = h(2)) and g(0)) = g(02) (h(01), = (h(0)).g()))
Thus Kø, = Ko, + (h(01), = ((2)), so (h(0).g(8)) is microergodic for (Ko)oee)
Let 02 E e. By Theorem B.5, both Kÿ and K2, are holomorphic on some ball around 0. By Theorem 3.4.
K Ke (x) Kz)(x) for all R..
17

---

Published as a conference paper at ICLR 2025
Similarly, by Theorem B.5, both Ki and K2 are holomorphic some ball around 0, and by Theorem 3.4,
Fg ((w})
K2 K - Fd (1W)) x Fo, ([w)) and
Σ
< 00.
Fgj ([w})
wiFe, (Ww))20
B.2 PROOF THEOREM 3.1
We study the kernels in Table in the order of RBF, Per, Damped Per, RQ and Cosine.
Proof for SE. Let (x) denote the SE kernel with parameters 02,6> 0. Since the SE kernel has spectral density, by Theorem 3.4, it suffices to show that Ko,,f, (x) Kaz fa (2)
for all æ implies of = of and & = l2. To show this, it is equivalent to show that and can be written functions of Ka,g. We have
|
(x)e(x) 0² exp
262
||
log Ko,(
1og(
26
log Ko,e
1
||
21
+0
||>||²
as x - 0o,
1
log (x))
lim
2/
|
||0||
a² (o,)(0).
Thus and are functions of Kg.t-
Proof for Per. To apply Theorem 3.4, we first need compute the spectral measures Fale. The period of K is 7. Since K1,22 is 2n periodic, it has Fourier series coefficients R1,025 (j) satisfying
(z) E
(1)
(1))22(0) =
K K1 (z)e 1jz dx.
2π
Thus
2TT (x) r2 0211,
R (i)e
=
==-00
10
σ²
R1.t22 (1) (w)eter dw..
Thus
Fott,, o² Σ
R1,,2 (i)0,z2
8=-00
18

---

Published as a conference paper at ICLR 2025
By 9.6.19 page 128 of Abramowitz and Stegun (1964),
1
R1)
cxp
dx
2T
e2
1
/
-2sin(x//23
exp
cosljz)de
π
22
2
cOs co
π
exp
@2
82
By 9.6.10 on page 375 of Abramowitz and Stegun (1964), k1e22=(0 > 0 for all j E Z. By 9.6.3 on page 127 and 9.3.1 on page 117 of Abramowitz and Stegun (1964),
1/
i J
2
وー 1
(1+ O(= as x
j
2j
e
(1 (1+0
2
2
Thus
R1,022 (f) = exp
=exp
fT
82
12
exp
(-12)
j
(1+0
v2mj
2
1
e
(1+0
j2m
2j
exp
p2
Suppose (ou, fu 7o) and (01, (1) are two parameter sets with
Fd oo.forra (o ([w)) = Fdl Fa (W))),
pd
2 aubi (ww))
Σ
1
70 ([w)) <0.
Fd
wi fo (1w))>
Due to the formula Fo,e, = 0² -00 R (1, and the fact that R1 for all j, must have 70 71 and
,2 of
of of
Σ
<0.
of
==-00
We have
à
1 27 (-2)
21 (1+00-1)
of K1 (j)
52ng
2j
exp
0, K1 2x
o3
1/2
(1+
exp
V2nj
2j
of exp
1
(1
(1+0 '))
of exp
19

---

Published as a conference paper at ICLR 2025
If to # 4, then of of + 0 or 0o as + 00, so we must have lo = E1 Then
of 11.1,.0.2000
2f
and the assumption Dj- (1- of of 2 %
o. %
of 2
00
implies 00 -I
Proof for Damped Per. Let &2,r denote the damped periodic kernel with parameters a, f1, y > 0. Note that the damped periodic kernel is the product of the SE kernel and periodic kernel, so it's spectral measure is the convolution of the SE spectral mcasure and the periodic kernel spectral measure. If denotes the spectral density of the SE kernel, and E3 aydw, denoles the spectral measure of the periodic kernel, then their convolution has spectral density g(w) = E, ay f(w - wj). Thus by Theorem 3.4, we need to show that a, l1, 7 can all be written functions of Ko We have
x²
2sin
x-
K (ad1d25(1) exx
exp
26
Note that Ka has a unique holomorphic extension to C
as
2²
2sin
Ka.ls 0² exp
exp
E
Putting by yields
y22
sinh
(ash = o2exp exp
7
28
sinhh
2
y²
log K (6, = log(o²)
+
28
o² Since sinh(x)
_0
2
exp
log Ko Ko (o,
2
log log K (,,41,22nnn
πy
logoo2 + o(1),
log log Kart (iy) + 000
7
y
Thus is function of K Ko Then
exp
26
log K (o,ts.22.n11))
so lg is function of Ka,8, 82.7 Note
sinhh 2
I2
y²
log K (ott,
log(o?) (
207
2sinh? =z
log
+ +ly
2
y²
d²
K
Thus & and o² are functions of Ka,ty
20

---

Published as a conference paper at ICLR 2025
Proof for RQ. Let Ko, denote the RQ kernel with parameters o² a, l > 0. Since the RQ kernel has a spectral density, as before, we need to show that o² a all are functions of K Ka,at We have o² Ka Kaa,e(0), so o² is a function of Ko Let
K(t)=
x² (tee)
12
+
2al2
Then as t 00,
22a
(tt)
(2al2)-a
2
fa
log ((t) = a (p 2a log(t) o(1)
log K(t) -2 +0
log(t)
log(t)
Thus a is a function Ka,a, Since
fa (2 K(t)
q
l is function of K Ka
Proof for Cosine. Let Ka,s denote the cosine kernel with parameters 0² > 0, s RR.. The spectral mcasure is
Fa.s 0² (6_, 182 +b).
This is a discrete measure with support {-s,s}. By Theorem 3.4, Kay = Kozies if and only if
(1
F Foz.e2 ([w)) 2 For.s ([w7t) < 00.
For,o (fw)) x ((w.s(((w)) and
Σ
oofor (1w))>0
Since the supports of For and Fo..9, are finite sets, the condition For ([W}) F Fo (fw))
equivalont to the condition that the supports are equal, that is, { --81, 81 = f - 82, 82)
The second condition always holds because the sum is over finite set. Thus Kay,s1 K Kag.22 and only if [-81,833 82) Thus {-8,8] is microergodic.
B.3 PROOF OF THEOREM 3.2
Proof of Theorem 3.2. First of all, Theorem on page 123 of Stein (1999) tells us that for continuous kernels K1, K2, the kernels K (x) + (==0) and K2(x)) + (x=0) equivalent if and only 7 = and K₁ = K2. Thus it suffices to prove identifiability of the parameters under the assumption that 011 = 0.
Write
(x) Ki (x)x) K2(x) K2(x) Ka(x),
21

---

Published as a conference paper at ICLR 2025
where
x²²
K1 (z) 89 exp
20g
x22
sin
2
K2(x) = 0g exp
20
05
x²
0g
K3(x)= 0% 11
20
03
x
K4(x)= = 0g exp
20
Note that K1, K2, K3, K4 analytic on the disks of radius co, oc, (20g 1/2, oo respectively. Thus Ka,y is holomorphic the disk of radius (20g02)2/2 Since K1, K2, K3, K4 each have spectral density, K has spectral density, and consequently, K Ko,; K 0. That is, focus only on the continuous component.
By Theorem 3.4, need to show that if (0,7) and (0, 7) are such that (x,((x) = K (x) for all R, then (0, (0, Let
θ {(0, 7) {Ko, (0, E O).
01u,7 > 0, 010 602.
Let S be the collection of functions h h(0, 7) with domain θ such that there is function
P : C - @ with h(0,7) = <(Ko,)) for all (0, 7) E O. We need to show that (0, 7) E S. We have
(o, ~ (2) 20,
log( (o, log (2) %b) 20g log(z) o(1),
log (x) -20 o(log(x)-
log(z)
Thus
0g E S.
and from this deduce
030200 S.
Note that the largest r >> 0 for which Ke,, extends to a holomorphic function on D r) is (20,63)1/2, Thus (20,03)1/2 E S, so
07 E
And from 05020 S obtain
0s S.
Set
x²
x
2 sin 2
x²
f(x) = 0 exp
8 exp
6 exp
203
203
0g
203
Since 0s, 07, 0g S, f S. Let f denote the unique holomorphic extension of f to C. For x E R,
2
sinh2
2
m..
x²
x²
x²
f(-ix) = 02 exp
+ 0 exp
+
+ 03 exp
203
20
0
20
22

---

Published as a conference paper at ICLR 2025
So -> 00,
2sinh 2
x²
f(- ((-ix) ~ 0g exp
20
8
x
2 sinh
log (iix)) = log((3)
+
+
+ o11)
20
0
sinh?
2
0
2
π.
exp
g
2
2TT
26
exp
?
2TT
log log f(-ix) (2gg)
o())
log log f( ((-ix)
+ o(x )
2
Thus
7 S.
Then from
f(-ix) sinh? n0
2 + o(1)
we get
02
0s E S.
We have
sinh 2
x²
log f(-ir)
log(Ob)
+ o())
0
20}
2sinh (
log (-ix)
85
+ o(x o(x
x²
20}
04 E S,
2 sinhh
x2
x
log f-iiz)
log( (8%) o(1),
03
20
0g E S.
Let
2 sinh 2
x²2
g(x) f(-ix)
0g exp
26
0
x²
x²
0f exp
0g exp
203
203
Since 0g < 010,
x
g(x)
0f exp
20%
.12
log(g(x)) log(0?)
28
02 E S,
E
Then by applying the same argument to g(x) - 07 exp
2
we get exp
x²
E
S.
207
207
23

---

Published as a conference paper at ICLR 2025
B.4 PROOF OF THEOREMS 3.6 TO 3.9
Proof of Theorem 3.6. Note that KA,M o² =2K((M2/2s) where K(x) exp (- [1=113) is the standard squared exponential kernel. Let R. Let A M2/2 Since
K(Ax)= = / (ww)elw Axe dw
/ [(A-F
1 det A dy,
we obtain
Ko.M (w) 0²
(det M) (MM
1
Thus Ko,M has a spectral density. Suppose Koo.Mo Ko, By Theorem 3.4, of exp (-2x² Mox) = of exp (- (-3x² M1x) for all x E IR. Putting x yields of = of and then taking logs yields x² Mox = x² Mi for x E R². Since Mo and Mi are symmetric, the polarization identity yields Mo Mi. Thus (o², M) is identifialle, claimed.
Proof of Theorem 3.7. The spectral measure is
m
F Pm....mms
Σ
3 (ō-s; + 5zz).
The support of Fo......... Mt..s is {+s J l..... m). As in the proof of for the cosine kernel, we conclude that { j =-= I,..., m) is microergodie. Since we assumed 0 < < < 8m; the largest m elements of the set {+s} :j I,...,m) m. are 8m, 8m- , Hence 81.
(81) is microergodic.
Proof of Theorem 3.8. If K₃ and K₂ stationary kernels with K2 K2 - 1, then the spectral measure of K2K2 is the convolution F1 F2, which is defined as the distribution of X1 + X2 when Xj- Fj and X1 and X2 are independent. Thus the support of Fo Fo.e As in the proof for the cosine kernel, we conclude that (+81 + 82 +
is {+881 + 82 ..... +... +m ... sm) is microergodic.
Now suppose m E (1,,,3), say m = 3. Then the 3 largest clements of the set {+ 8 + ss) are 81 + 82 + 83, -31 + 82 + 83, and 81 82 + 83. From these we can algebraically solve for 81, 82, 83- This shows that (81, 82, 83) is a function of {ts) + 82 + sa). Conversely, (#1) 82 + 833 is also function of (81, 83). Thus (81, 82, 88) is microergodic.
Proof of Theorem 3.9. This proof mimics the proof for the sum of cosine kernels, but is bit more elaborate due to the infinite support of the spectral measures. Let Ke(j), j E Z denote the Fourier series coefficients of the periodic kernel with o² = 1, length-scale f, and y 2T. In the proof for the periodic kernel, we showed that
Fa.t. = Σ RR.
-=-00
Re()) > for all jEZ,
Re() 1
1 (1+000)))
exp
(3)
V
2j
of
of exp
(-1)
(
(1+0(-')).
(4)
of exp
-
24

---

Published as a conference paper at ICLR 2025
Suppose Ko,.4.m K o 62.
Write
81 2m
2m
82 22
2π
$1 71
2n
72
Note that 0 <<a 81 < 82 and 0 < &1 < 82. Let and F denote the spectral measures of
(iniimos and K on, We have
F= Σ
Kf U,8j. +
Σ
of og (j)6
j=-m0
-=-00
F 2 U)8 +
Σ 22 12 (j)0
By Theorem 3.4, the supports of F and F are the same, that is,
(jst j (jsz j E 2) = {Jz : e 2)
Equating the smallest positive element of both sides yields
81 81-
Now split the rest of the proof into two cases. Case 1: Suppose and 82 is not integer multiple of 81, there is j E Z such that 82 j8g. This implies that $2 cannot be an integer multiple of 81, and since 32 E supp(F") there must exist k E Z such that 52 ks2 jkšz. Thus jk = 1, and therefore j = k = Thus $2 52. Now note that
81 & Q. Since 82 E supp(F)
supp(F) n (0, 00) = {isı : j E N) U {js2 1 N)
and since #2 $ Q the sets {js) : E NJ and (jo2 j E N} are disjoint. By Theorem 3.4 and the symmetry Re(t) Re(-)))
F(w)
Σ
>>
1
F(w)
w:F(w)30
of R2 (1)
2
2 (i) o R2 (i)
= =C+2 Σ
+2
oiKn (1)
5ml
j=l
87 82K2,(1) (1)
23 ( o2 (2)
In particular, ofKe, (1)
+ and
+ 1 as j + 00. From (4), we conclude & = Pi,
a = ē1, f2 = 82, and 02 = ō2.
Case 2: 22 E Q Since 32 is an integer multiple of 31 or 82, Ee E Q. By writing 82 = 991,
8g = q81 with 7.7 E Q it is clear that m = lem(s1, 82, 82) = (81(81, q81) exists. Note that
F(jm) = oiRa,
(j
j
m
+
o
Rez
jm
m
$1
$2
25

---

Published as a conference paper at ICLR 2025
Equation (3) can be written as Re(j)= = ---10)--000000 where C(t) > 0. Thus when >> 1 an integer,
(aj) a Ri)
j j-1/2j-
= =a (a-12au-oMC((,11
Ke (ai) (aj) (1 -a) -a)jlogj + j log C(a, 81, (2) (2)++)+000 +01)
log
kiz
+ -00 as + 00,
80
(oj( Ri,(i)
- O.
Therefore, since
F(jm) = of
j=
m
+ 02
Res
{ m
j=
82
02 Re2
m
82
For the same reason,
F(jm) R2
m
|= $2
02 R2 2/
Since 0o E}= 1
F()m)
we have Flim) F(ymj
1, 50
Again by (5), we
F(im)
0 K,, mm)
conclude that 82 = 82, and by (4), we obtain &g = lg and 03 = oî. Recall m = lcm((1, 52) 2
82 31. Thus 81
1
$1
1
2
51
$1.... are all not integer multiples of s2. Thus
F
1 j
81 = of Rrs
+ j
81
81
F
[1
81
R
m + $1
81
F( F((1+) 81
aR (1+j
3R (v) lim ofKe (b)
Since
1, have 1 = limj-v00
From (4),
F F((1+) ) )1)
Rig
[+5 m
we deduce [ l1 and o. = of.
B.5 AN ADDITIONAL THEOREM
In this subsection, we introduce a theorem that useful in conjunction with Theorem 3.4 for determining the identifiable functions in new kernels constructed from existing ones.
Let C be the set continuous stationary kernels K on RD for which = K and D be the set of continuous stationary kernels for which K = Kd. Theorem 3.4 tells us K1 = K₂ if and only Kf- = Kद and KJ = Kÿ. Thus it can help to know how to decompose K into K and K', When K is a sum of products of other fully discrete fully continuous spectrum kernels, the decompasition is easily found by inspection:
Theorem B.6. Suppose Ko and K₁ are continuous stationory kernels on E with spectral measures Ko = Fo and Ki = Fi. The following hold
(i) Ko + K1 = Fo + F1
(ii KoKi Fo Fi.
26

---

Published as a conference paper at ICLR 2025
Consequently. (ii) Ko E
EC,
(tv) Ko C = KoKi EC,
(v) KO Ko Ki ED, KoKi ED.
Proof. (1) and (ii) standard results for Fourier transforms. (iii) holds since the of two continuous measures is continuous. (iv) holds since the convolution of continuous measure with any other measure is continuous. (v) holds since the sum of two discrete measures is discrete and the convolution of two discrete measures is discrete.
This theorem, together with Theorem 3.4, allows us to study the identifiability of parameters in new kernels constructed the multiplication and summation of existing kernels.
C RELATION BETWEEN PRAMETER INFERENCE AND PREDICTION
While the primary focus of this paper is on parameter identifiability in GPs models, it is worth briefly discussing its connection to prediction performance, as prodiction one of the most common applications of GPs.
In the literature, there well-ostablished distinction between parameter inference and prediction accuracy. For example, shown in Theorem 8 of Chapter in Stein (1999), if two measures P1 and Po are equivalent, then assuming Po is the true measure and using Pi to obtain the best linear predictor e₁ at new observation location zo, the ratio of the MSE of ej to the MSE of the best linear predictor eo under Po converges to as the sample size +
For the Matérn family with known smoothness parameter v. Theorem 12 of Chapter 4 in Stein (1999) further shows that the asymptotic ratio of the MSEs under two Matérn kernels parameterized by (of (1) and (až.Ez E2) converges to 1, regardless of the values the ofe2" parameters (Equation 49, कें
As a consequence, we gel asymplotically optimal prediction performance by having P: be in the correct parametric family, even if P1 1 Po- This underscores an important point: prediction is, in an informal sense, "simpler" than parameter inference, as incorrect parameter specification parameter estimates may still yield asymplotically optimal predictions.
Beyond Matérn kernels, for the holomorphic kernels studied in this paper, Stein's Theorem 8 still holds, However, this is beyond the scope of the current paper, which primarily focuses parameter inference.
D ADDITIONAL FIGURES
In Figure 3, we present the simulations repeated for = 0.1 side by side with e = 0.01. It appears that the variances of the MLEs are smaller when ε = 0.01.
27

---

Published as a conference paper at ICLR 2025
0.0022
1,0
0.9
0.001
0.8
500
1000
2000
5000
500
1000
2000
5000
Sample
Sample
Sample
(a) SE (5 0.1)
(b) SE (5 0.01)
MASSON
11
14
08040
0898
120 300
Sumple
100
Sumph Sec
Sanol SkY
Samble Sa2
(c) DPer (s = 0.1)
(d) DPer (E = 0.01)
0.00355
0.00345
4.0
0.00330
322
1020 A000
2000-400
Samppe
Samoll
Sample
Sample
(e) Per 0.1)
(f) Per (5 0.01)
010016
0.0034
0.8
500
1000
2200
1000 2002 9C00
500 1000
2000 5000
Sampl
Sanoo
SATOR
(g) RQ (E = 0.1)
(h) RQ (5 0.01)
0 2.350
0.2505
0.2480
0.2485
500
5000
500
5000
sample
Sirea
(i) Cosine (== 0.1)
(j) Cosine (5 0.01)
Figure 3: Simulation results for various kernel types with 0.1 and 0.01. Each subfigure shows the boxplots of MLEs for the corresponding kernel, with ground truth in horizonial dashed line.
28

---

Published as a conference paper at ICLR 2025
E
ADDITIONAL EXPERIMENTAL DETAILS
The ground truth parameters used Section 4.2 are given by the following table:
81
Bg
83
04
0,
?
8g
87
0g
Bg
810
44.8
51.6
2.64
91.5
1.48
0.536
2.89
8.97
0.188
0.122
Table Ground truth parameters for the combined kernel.
All the experiments rum Linux-basod virtual computer with 6500 conventional compute cores 472 delivering 13,000 threads. We used 24 CPUs.
Table 4: Runtime and memory usage for each experiment
Experiment
Time
Memory
Squared Exponential 0.1
8 hours, 53 minutes
32 GB
Damped Periodic & = 0.1
6 hours, 24 minutes
32 GB
Periodic 0.1
10 hours, 45 minutes
32 GB
Rational Quadratic E = 0.1
10 hours, 16 minutes
32 GB
Cosine 0.1
hours, 49 minutes
32 GB
Squared Exponential = 0.01
7 hours, 30 minutes
32 GB
Damped Periodic 0.01
6 hours, 45 minutes
32 GB
Periodic 0.01
10 hours, 12 minutes
32 GB
Rational Quadratic & 0.01
hours, 42 minutes
32 GB
Cosine 0.01
5 hours, 44 minutes
32 GB
Combined Kernel
hour, minutes
32 GB
Acknowledgment Assets Used:
Code Libraries: We utilized the following Python libraries in program:
NumPy: BSD License
SciPy: BSD 3-Clause "New" or "Revised" License
Matplotlib: PSF License Agreement for Python
Scikit-learn: BSD 3-Clause "New" or "Revised" License
29


Reviews:
[{"review_id": "HNz3Jwe2GK", "reviewer": "Reviewer_wkyQ", "summary": "This paper provides a new theoretical framework that can be used to determine identifiably of parameters in a stationary kernel under the infill asymptotic. Using this framework, the authors are able to determine the parameter identifiably of several popular kernels in the literature, extending the theoretical framework for the Matern class in the literature. Some numerical experiments are used to validate the theoretical findings.", "questions": "I have to admit that I did not go through the proof line by line, but the arguments make intuitive sense.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "This is a very well-written paper addressing an important research question in the literature. The theory is solid, especially Theorem 3.4, can be very useful for researchers in related areas.", "weaknesses": "No apparent weakness. (Note: I have reviewed this paper for another conference before, and all my previous questions have been addressed. Personally, I think this is a great work based on my research experience and should have been accepted.)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MbYD3LkqzR", "reviewer": "Reviewer_ygPM", "summary": "This paper investigates the identifiability of parameters in a Gaussian process (GP) model with a kernel that is stationary and holomorphic around zero. Identifiability in GPs is a critical issue for both parameter estimation and interpretability. However, according to the authors, identifiability remains insufficiently studied for a wide range of kernels, as most existing methods apply only to a limited set of kernels.\n\nTo address this, the authors introduce a new method for establishing the identifiability of GP parameters. Using this approach, they demonstrate that parameters for several well-known kernels are identifiable. They also derive conditions under which parameters of sums of products of kernels remain identifiable, extending the applicability of their method. Finally, experiments on datasets support their theoretical findings, confirming the identifiability of parameters across various kernels.", "questions": "It is true that the decreasing variance with an increasing number of samples is reasonably convincing evidence that the parameters identified as \"identifiable\" are indeed identifiable. However, in cases where this pattern does not hold, do you have any additional arguments to support that the experiments still validate the theory's conclusions about identifiability? Conversely, what kind of behavior would you expect to see if a parameter were not identifiable? More broadly, is there a way to develop a meaningful statistical test to empirically confirm that the parameters theoretically classified as identifiable or non-identifiable actually exhibit these properties?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The motivation for studying identifiability conditions for kernels of GP is strong, as it enables clearer interpretation of model parameters. Additionally, the focus on widely used kernels is a practical and relevant choice.\n\n- The authors introduce a novel reasoning approach to derive the identifiability condition, which could be of interest to researchers beyond the immediate scope of this paper.\n\n- Overall, the paper is well-written, and the authors have provided code to facilitate reproducibility of the results.", "weaknesses": "- While the results are presented for various kernels, the scope is still limited to stationary and holomorphic kernels around zero, which restricts the broader applicability of the identifiability condition.\n\n- The experimental results are somewhat inconclusive. It would be helpful to clarify what should happen when parameters are identifiable. For instance, if the parameters are indeed non-identifiable, we might expect the estimated values to fluctuate across a range of possibilities. Can the experiment demonstrate that this is not occurring? Perhaps there is a statistical test that could be developed to verify whether the model is, in fact, identifiable. The observed reduction in variance is promising evidence, yet it leaves unanswered questions about cases where variance does not decrease. It would also be useful to test the behavior of parameters that are known to be non-identifiable—particularly those that would not be easily identifiable without the specific reasoning introduced in this paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "h2olTWyFxS", "reviewer": "Reviewer_Ec9P", "summary": "The paper studies identifiability of the parameters of certain kernels (squared exponential, periodic, rational quadratic, and cosine) and some of their sums and products, in Gaussian process regression (fixed domain asymptotic scenario). In certain cases, it finds microergodic parameters: the functions of parameters that _are_ identifiable, and thus can be consistently estimated. It illustrates the results with an empirical study of convergence of maximum likelihood estimators.", "questions": "A key technique used with RBF kernels (and not only with them) is automatic relevance determination (ARD), when each input coordinate corresponds to its own length scale parameter, each optimizable; or even when an input vector is multiplied by an optimizable matrix. The separate length scales are often used for interpretation, as a measure of relevance of each individual coordinate, which makes them a natural target for your study. Can your results be applied in this setting? Adding this would make the paper stronger.\n\n(Minor) suggestions:\n- In the abstract, perhaps you want to change “time series\" into “time series forecasting” or “analysis of time series”, or something like that. Just “time series” doesn’t read like a “domain”.\n- Lines 093-096, you mention non-identifiability of Matérn covariances, but you forget to mention that it only holds for dim <= 3.\n- Line 132. “if for any” -> “if for all”.\n- Line 198. “or they are supported on disjoint sets” - I strongly object to this intuition. Consider a non-degenerate Gaussian supported on $\\mathbb{R}^2$ and a degenerate Gaussian supported on the line $\\\\{0\\\\} \\times \\mathbb{R} \\subset \\mathbb{R}^2$. These measures are orthogonal, but the support of the latter is a subset of the support of the former, the supports are not at all disjoint. Yes, you could exclude $\\\\{0\\\\} \\times \\mathbb{R}$ from the support of the former measure if you treat things up to probability 0 events, but this is not very natural and thus doesn't give a good intuition.\n- Line 248. Please give a reference for the spectral density of Matérn kernels for the specific notion of the Fourier transform that you are using.\n- Line 377. The notation $\\\\{ \\pm s_1 \\pm s_2 \\pm \\dots \\pm s_m \\\\}$ is unclear, please expand on what you mean exactly by this.\n- Lines 404-406. “In fact, even for simple kernels like the SE and Matérn kernels, whether the MLE is consistent remains open, and we still do not know whether the likelihood is unimodal or not.” - please support this claim by a reference.\n- Please polish your reference list. As an example, in line 545, the word “gaussian” is missing a capital “G”.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- The paper presents new theoretical results on identifiability, which are, in principle, intersting.\n- The results look plausible (I didn’t check the 13 pages of proofs in the appendix though).\n- The code is provided in the supplementary materials.", "weaknesses": "- The paper promises a theoretical framework for determining the identifiability of parameters in arbitrary combinations of squared exponential, periodic, rational quadratic, and cosine kernels. However, the results don’t appear as such: they consider a few special cases, like sums and products of cosine kernels only. The general sum/product combination of the mentioned kernels is not explicitly handled.\n- The proofs are hidden in appendices. This is normal for a theoretical paper submitted to a conference. However, in such cases I expect the main ideas/methods to be covered in the main text, so that a reader who doesn’t have time to read the actual proofs could at least get some idea of how they go and how plausible they are.\nI believe the paper falls short of this. In particular, the paper studies kernels on $\\mathbb{R}^n$, but uses the term “holomorphic” from the world of complex numbers, and assumes the properties of kernels with traditionally real inputs on complex domains. This connection between the world of real-input kernels and the world of complex-analytic methods seems crucial, but remains unclear to me from the main text.\n- There are certain problems with writing/presentation, like tables going over the right margin (Table 2) and figures with overly small fonts (Figure 1).\n- Empirical study doesn’t illustrate the theoretical findings well. It doesn’t identify them as wrong, but in many of the studied cases does not really show anything.\n- I noticed that in the second paragraph of Introduction, all the citations are copied from the 4th paragraph  of introduction of https://proceedings.neurips.cc/paper_files/paper/2023/file/dea2b4f9012686bcc1f59a62bcd28158-Paper-Conference.pdf. The general flow is also similar. The mentioned paper seems to tackle a different problem, so I don't suspect this paper plagiarizing it, but borrowings like this don’t look good. I hope the authors will make sure not to copy-paste things (almost) directly from other papers.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "B4JSPI29t9", "reviewer": "Reviewer_iSg8", "summary": "This paper addresses the identifiability issue of the hyperparameters of Gaussian Processes when holomorphic kernels are used. A novel theory is proposed that may help identify when hyperparameters are identifiable and when not, allowing meaningful interpretations in the future. From my own experience, the identifiability of hyperparameters for GPs is an important issue to address.  \n\nThe writing style and obvious practical implications made this paper an easy and interesting read.", "questions": "- What is the impact of identifiability on the function approximation and uncertainty quantification of a real dataset? \n- How does non-identifiability might affect prediction quality? \n- What does the log marginal likelihood function look like in such a case?\n- How do MCMC and variational inference deal with non-identifyability?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- well written.\n- a great contribution to answering the long-standing and practical-important question of identifiability for GPs.\n- digestible theory.\n- immediate practical impact.", "weaknesses": "This is a strong paper, I only have one major and a few minor issues with some of the presentation: \nMajor: \n(1) The simulation results, while certainly confirming the theory, are insufficient to convince a critic. Identifiability of hyperparameters is such a rich and practical topic that the results seem somewhat bleak and unimpressive. My suggestions are to apply the methodology to a real dataset --- something of high impact, such as climate, or the mentioned spatial transcriptomics --- show the effects of misidentification by plotting and evaluating results (CRPS, RMSE) and possibly showing relevant snapshots of the log marginal likelihood function. In addition, where the data comes from right now is not well explained. The inputs are defined, but it only says \"After generating the outputs\". Please be more specific about where the data comes from. \n\nMinor:\n(1) There are some errors in the text \"summarize(s) existing literature\", or table \"above\" when it is below. \n(2) The paragraph after Def. 3 is a little convoluted, possibly due to a grammar mistake, but it's hard to tell. Please be a little more specific and explicit in your logic there. \n(3) It would be valuable to the reader to discuss how other training methods (MCMC, Variation Inference) deal with the identifyability challenge. I would guess MCMC as a sampler might deal really well. This would be a great addition.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)