Understanding Smoothness of Vector Gaussian Processes on Product Spaces
Authors: Emilio Porcu
Ana Paula Peron
Eugenio Massa
Xavier Emery
Venue: Accepted by TMLR ()

Abstract: Vector Gaussian processes are becoming increasingly important in machine learning and statistics, with applications to many branches of applied sciences. Recent efforts have al- lowed to understand smoothness in scalar Gaussian processes defined over manifolds as well as over product spaces involving manifolds.
Under assumptions of Gaussianity and mean-square continuity, the smoothness of a zero- mean scalar process is in one-to-one correspondence with the smoothness of the covariance kernel. Unfortunately, such a result is not available for vector-valued random fields, as the way each component in the covariance kernel contributes to the smoothness of the vector field is unclear.
This paper challenges the problem of quantifying smoothness of matrix-valued continuous kernels that are associated with mean-square continuous vector Gaussian processes defined over non-Euclidean product manifolds. After noting that a constructive RKHS approach is unsuitable for this specific task, we proceed through the analysis of spectral properties. Specifically, we find a spectral representation to quantify smoothness through Sobolev spaces that are adapted to certain measure spaces of product measures obtained through the ten- sor product of Haar measures with multivariate Gaussian measures. Our results allow to measure smoothness in a simple way, and open for the study of foundational properties of certain machine learning techniques over product spaces.

Published in Transactions on Machine Learning Research (MM/YYYY)
Understanding Smoothness of Vector Gaussian Processes on Product Spaces
Emilio Porcu
omilio-poreukkk.cco
Department of Mathematics, College of Computing and Mathemationl Sciences Khalifa University, Abu Dhabi & ADIA Lab, Abu Dhabi
Ana Paula Peron
apperon@@ceme..up.bb
Departamento de Matemático, Instituto de Ciências Matemáticas de Computação, Universidade de São Paulo, São Carlos SP, Brazil
Eugenio Massa
eug.masoo@gaiilcco
Deportamento de Matemática, Instituto de Ciências Matemáticus de Computação, Universidade de São Paulo, São Carlos SP, Bruzil.
Xavier Emery
Semery@ing.uchte uchile.
Department of Mining Engineering, Universidad de Chile & Advanced Mining Technology Center, Universidad de Chile
Reviewed OpenReview: https: // openreview. net/ permmm XXXX
Abstract
Vector Gaussian processes are becoming increasingly important in machine learning and statistics, with applications to many branches of applied sciences. Recent efforts have al- lowed to understand smoothness in scalar Gaussian processes defined over manifolds well as over product spaces involving manifolds.
Under assumptions of Gaussianity and mean-square continuity, the smoothness of zero- mcan scalar process in one-to-one correspondence with the smoothness of the covariance kernel. Unfortunately, such a result is not available for vector-valued random fields, as the way each component the covariance kernel contributes the smoothness of the vector field unclear.
This paper challenges the problem of quantifying smoothness of matrix-valued continuous kernels that are associated with mean-square continuous vector Gaussian processes defined over non-Euclin Con-Euclidean product manifolds. After noting that a constructive RKHS approach is unsuitable for this specific (ask, we proceed through the analysis of spectral properties. Specifically, we find a spectral representation to quantify smoothness through Sobolev spaces that are adapted certain mcasure spaces of product measures obtained through the ten- sor product of Haar measures with multivariate Gaussian measures. Our results allow to measure smoothness in simple way, and open for the study of foundational properties of certain machine learning techniques over product spaces.
1 Introduction
1.1
Context
The paper deals with the smoothness of continuous matrix-valued kernels associated with mcan-square continuous vector-valued Gaussian processes defined on the product of two spaces, with one of them being non-Faclidean, namely hypersphere of d dimensions embedded in (d + 1)ddimensional Euclidean space.

---

Published in Transactions on Machine Learning Research (MM/YYYY)
Gaussian processes (Seeger, 2004) are ubiquitous in machine learning, statistics and numerical analysis. Vec- for (i.e., mult ivariate) Gaussian processes have recently received attention after the constructive approaches proposed by Hutchinson et al. (2021). The impact of such processes on the machine learning community ranges from regression (Chen et al., 2023), Bayesian optimization and active learning (see the discussion in Hutchinson et al., 2021, and references therein), to relevance vector machines (Quinonero-Candllla 2004), sensor networks (Osborne et al., 2008), text categorization (Kazawa et al., 2005), informance vector machines (Lawrence et al., 2002), gradual learning (Yuan et al., 2022), and multitask learning (Bonilla al., 2007; Xing et al., 2021).
Vector Gaussian processes arisc within the framework of multiple output learning of a vector-valued function I = (f,,..,, fp) that is observed over finite set f collected over the training set X = ;xy Specifically, we suppose that æ is defined over a product space ((d.k) = s x TR. with g being the unit sphere of dimension d and R& being the k-dimensional Euclidean space. The output space Y, where f is defined, has dimension p.
(f, (()) from training data a
The problem can be tackled either assuming that f belongs to reproducing kernel Hilbert space (RKHS) of vector-valued functions assuming that f is drawn from vector Gaussian process.
We start by illustrating the RKHS perspective for vector-valued functions that are reproduced through matrix-valued kernels, denoted K throughout, being matrix-valued functions from Y(d,8) x ((dA) into Rexx,
A vector RKHS is a Hilbert space, HK composed vector-valued functions f such that, for all c E IR? and all (dd.), the linear combination f(x) (x) is obtained through
f (f(), ((,
where b,cc is the inner product on Hg. K
K The kernel is positive semidefinite: for any arbitrary system fculi= of p-dimensional real vectors and any finite collection of points {ox 1N in the input space, we have
N
c (( It )c 2 O.
For thorough review about RKHS for both scalar and vector-valued scttings, well for material about regularization in RKHS, the reader is referred to Hofmann el al. (2008) and Alvarez et al. (2012).
Although RKHS are a powerful instrument to quantify smoothness of scalar Gaussian processes, the same does not hold for the case of vector Gaussian processes, where the role of the matrix-valued kernel remains unclear. Hence, we opt here for the alternative of matrix-valued kernels through vector-valued Gaussian processes.
1.2 Why Study Smoothness? Why Product Spaces?
Smoothness plays fundamental role in numerous applications machine learning and statistics. We men- tion here the most recent development in Gaussian process regression. The recent work by Rosa al. (2023) dcals with Bayesian contraction rates under the framework of Gaussian process regression with random de- sign. Posterior construction rates provide nice way to illustrate how given class of posteriors concentrates around the true data generating process. Rosa et al. (2023) prove that the contraction rates depend on the smoothness of the underlying Gaussian process, the prior of which is defined through a Matérn kernel (Porcu et al., 2023).
Well-known results from probability theory connect the smoothness properties of a scalar Gaussian process with those of the associated kernel (Yadrenko, 1983; Adler and Taylor, 2007). Unfortunately, these results are not available for the rector-valued case, and the same definition of RKHS in Equation (1) clearly shows that the contribution of cach component K to the smoothness of Z unclcar. This explains the lack of literature this direction and the fact that the available contributions are centered to the smoothness of the kernel K rather than the smoot anoss hness of Z.
2

---

Published in Transactions on Machine Learning Research (MM/YYY)
An intuitive way to look at the geometric smoothness properties of the kernel is by working under the framework of Sobolev spaces.
Another relevant motivation for studying smoothness of Gaussian processes on manifolds is related to the use of computational tools of kernel cubature and kernel discrepancy beyond the usual Euclidean manifold. Barp et al. (2022) illustrate the importance of Sobolev spaces when quantifying kernel cubatures. These topics have been popular in statistics, machine learning, and numerical analysis. Kernel cubature has been applied in several contexts, and the reader is referred to Hubbert et al. (2023), with the references therein.
Studying smoothness on non-Fuclidean manifolds has been important to several disciplines. For the special case of the manifold being a d-dimensional sphere, applications include kernel cubature (Marques et al., 2013; 2022), Stein's 's method to numerically calculate posterior expectations in directional statistics (Barp et al., 2022), and approximation of solutions of some classes of PDEs (see e.g. Fasshauer, 2007). Not to mention that certain classes of kernels on spheres ensure that the solution of the PDE belongs to the RKHS and, through the use of an appropriate kernel method, can be consistently approximated (see Fuselier and Wright, 2009; 2012; Hubbert et al., 2015).
The product space ((d.) has received increasing attention in the statistics and machine learning communitics (Atluri et al., 2018; Wang et al., 2022; Porcu et al., 2016; 2021). Applications from several branches of science justify this context, such as
atmospheric, environmental and oceanographic sciences: variables such as air pressure, air humidity, wind speed, surface temperature, solar radiation, acrosol optical depth, daily ozone concentration, ground level concentration of particulate matter, ocean current velocily, sea surface height anomaly, sea surface salinity, sea surface chlorophyll-a concentration, or sea water density are indexed by the position (longitude and latitude) on planet Earth (S²) and by time (I²) (Castruccio and Stein, 2013; Oleson et al., 2013; Faghmous and Kumar, 2014; Jeong et al., 2017; Wu et al., 2022);
geophysics: scismic and volcanic events can be represented by point processes indexed by a position on planet Earth (82) and by time (RR2) (Illian et al., 2008; Connor et al., 2009);
structural geology and geotechnics: variables such as the linear discontinuity frequeney, the rock quality designation, the uniaxial compressive strength, used to measure the geotechnical quality of a rock mass, are indexed by the position (casting, northing, clevation, i.c., point of R3) of the rock core sample in the rock mass and by the orientation (azimuth and dip, i.e., point of S?) of this sample (Sánchez et al., 2019; 2021);
social science: crime and security data can be indexed by their position in the geographic space (IR2) and by time at the scale of day (S") (Tompson et al., 2015; Shirota and Gelfand, 2017);
neuroscience: functional magnetic resonance imaging (fMRI), electroencephalograpyy (EEG) and magnetooncophaloggaply (MEG) signals are indexed by the position on the human head (S²) and by time (R+) (Wingeier et al., 2001; Atluri et al., 2016).
1.3 Challenges and Contribution
The smoothness of scalar Gaussian processes was studied in Lang and Schwab (2013) for the sphere, and in Clarke De la Cerda et al. (2018) for the product of sphere with R.
While scalar Gaussian processes are well understood, the literature on smoothness of matrix-valued kernels in machine learning is scarce, with the notable exception of Cleanthous (2023), who provides an ingenious construction for a Gaussian process defined over ball embedded in &
Our paper contributes in this direction. Specifically, a) Section 2 considers mean-square contimuous zero-mean vector Gaussian processes defined over the space (dde)) being previously defined;
b) we take a spectral path smoothness of their covariance kernels in Section 3, through a proper spectral representation for vector Gaussian process on ((d.k) and consequently for the related matrix-valued
3

---

Published in Transactions on Machine Learning Research (MM/YYY)
kernel;
c) we construct, in Section 3.3, suitable Sobolev space for such a matrix-valued kernel;
d) Section 3.4 provides a spectral characterization of smoothness of the kernel.
1.4 Outline and Notation
Section 2 provides a suceinct mathematical background. Section 3 illustrates the way to construct proper Sobolev spaces through spectral representations over the space T(d,k), Proofs are technical and deferred to Appendix A. Section 5 concludes the paper with discussion.
Hereinafter, Z is the set of integer numbers, Z, {N Z: 2 0), stands for the complex imaginary unit, d and k for positive integers, and I| for the Euclidean norm on RR. Bold letters denote vectors or matrices of size x p. A refers to the conjugate of a complex matrix A, and A to its transpose. In order to work in multidimensional spaces, we consider the multi-index notation: for a = (a1...., ax) E zf. and h (h1, ..., h4) E R&,, wC sct
lac
Σ i=1
a! II ag! ==1
8 f² 0h 0h ... ...0%t f(h)
as
and for a B E z
k a
2 3 - a2 2 8r ii,
with the usual understanding that 0 1.
2 Vector Gaussian Processes
Let
((d = Sd x R = (x = (x, t) E Rd+1, x IR |x|| = 11, e R*).
A y-variate (vector) Gaussian process, [Z(z)
((d))) is an uncountable collection of random vectors
such that, for any finite collection of points Ty.... LNv E T(d.)), the vector (Z(æ,) ..... Z<Ex) having dimension (p x N) 1, is a Gaussian random vector. In what follows, without loss of generality, we suppose that such a Gaussian process has a zero mean at any point x in T(d.).
A p-variate covariance kernel on (dd.)) is a matrix-valued function
K (ddA)) ((d.)) R
defined as
K [x. x') [Kij (I, x E')R, म E T(d.))
where Kij (x, = K3 (x' Φ) for all i,j E p} and æ. I' r(dk), and where K is positive semidef- inite, that is, the pN PN block matrix [K (z,, T 1 is symmetric and nonnegative definite for any set of points Ty. EN m(dd))
Hereinafter, we focus on the case where the mapping K is continuous, isotropic on Sd and stationary in Rk meaning that
K( K ((x, x'), t'), z (x, (x',t')
with ( denoting the dot product in Rd+1, and K [- R Rpxp being continuous mapping. Throughout, K will be called kernel for simplicity, albeit this should be called the isotropic profile of the kernel K. This will not give rise to confusion as, from now, only the mapping K will be used.
4

---

Published in Transactions on Machine Learning Research (MM/YYYY)
On the one hand, Alegria et al. (2019, Theorem 6.2) proved that continuous kernel K of the type (2) be uniquely decomposed as follows:
k(s h) Σ dim(H() (h) 9 (s),
s sE|-1,11 h EER,
for sequence ())m)))))))) of matrix-valued stationary covariance kernels such that (dim (2) is
summable. For d > 1, In (ld-1)/2 is defined in terms of the Gegenbauer (ultraspherical) polynomial G G/-11//2 normalizing as -/- g/2-1)/2 = G 1/2-1)/2 / (1)) while for d 1, 9h Ta the nth Chebyshev polynomial of the first kind.
On the other hand, the generalized addition theorem for spherical harmonics (Erdélyi, 1953, Equation 11.4.2) states that
Σ QEAA..
(n. (a(') = dim(H()) ((x,,)) a. E Es", n E Z+,
(4)
where An,d is set of finite cardinality dim()()) associated with the spherical harmonics Un.g.d: which form an orthonormal basis for all Lebesgue-squarriinteegn measurable functions on the d-dimensional sphere gd.
Owing Equations (3) and (4), the Karhunen theorem on the generalized spectral representation of ran- dom processes (Yaglom, 1987b, Equation (2.31')) allows decomposing mean-square continuous p-variate Gaussian process Z having mean and K as its covariance kernel in the following fashion:
( ② = Σ Σ
= (x, t) E ((2k),
(5)
n=0 qEA..
where (-),, is a sequence of mean-square continuous zero-mean vector Gaussian processes in R such that (cov refers the covariance operator)
cov (1) of 8n,n' 5 C (((t-t)) t'), t,t' e ERR, n, 24, q Q AEAdd E Anid
(6)
The reciprocal, that vector Gaussian process of the form (5) has covariance kernel of the form (3), has been established in the scalar case (p 1) by Clarke De la Cerda et al. (2018, Proposition 3.1). For the reader's convenience, we sketch the proof for the vector case. For any x (x, t) and x' = (x',t' in r(d.)) the bilincarity of the covariance implics
K( x. x²) = cov Z(zz),((z))
2 Σ Σ
20
cov
(t)
Σ A (t)
Σ
Σ
n=0 qEA.E
Σ
( alxx'())))))))))) "f cov (t), A (1))
==0 gEAnn n2' 0'=00''E E'A.'
Σ
==
(I Ct - t1) 9/d-11/2(((,
with the last two equalities derived from Equations (6) and (4).
Equation (5) can be coupled with the spectral representation of the mcan-square continuous stationary
random process Ad (.) (see Yaglom (1987a, Equation (4.70) Chilès and Delfiner (2012, Equation (2.16)))
5

---

Published in Transactions on Machine Learning Research (MM/YY) (M/YYYY)
to attain
(
elit.w) (holdw))nggdde RR
g (x, t) r(dd).
(7)
n=0 ned
where (dn.(d-))mgg is sequence of vector-valued measures with orthogonal increments, that is, E (B)) 8n=n FAANB), for all q, n, n' and all Borel sets A and B in R*, where F, is matrix-valued measure of bounded varialion such that (a(dw) is positive semidefinite matrix for all w E RK.
Under the additional condition Σn, dim()() JR+ En.((dw) < co, Equation
(3) bocomes
Σ dim (JC() (/c ((hww) ) ((4-1)/2(ss,,
K(s, h)
8 SE 1-111,11 h E
(8)
Clearly, Cd is real matrix-valued if, and only if, (A) Fa(-A), for all Borel sets A in RR.. A stronger condition for this to happen is that (-(-) = (A) Throughout, we shall always work under the assumption of real matrix-valued covariance kernels.
3 Understanding Regularities of Matrix-Valued Kernels
To study the properties of the matrix-valued kernel K, an intuitive approach to provide Karhmeen-Love expansion of vector Gaussian process in RR, with input space T(d.)) Since the product space (ddkk) is not compact, extension of the arguments provided for the scalar case by Clarke De la Cerda et al. (2018) suggests that sensible strategy is needed to provide a legitimate Karhunen-Loeve expansion of vector- valued functions. There are indeed two possibilities:
a) to compectify the space (4.)) by considering the space (d.kk) == S [0. with T positive constant. Under such a construction, Karhunen-Loove expansion can be namely obtained. Yet, this approach has cost in that it does not allow for traditional spectral expansions as much as in Equations (7) and (8), respectively:
b) to consider the measure space
(r(d,)), B.Hrra.)))
(9)
where B is the Borel sigma-algebra over P(dik,, and where prrtas)) is a product measure defined through
ur (dz) = (x() x v(dt), 2 E T(d))
where ae is the Haar measure, i.e. the Lebesgue measure for the sphere, and v is the Gaussian measure in & with zero-vector mean and identity covariance matrix, i.c.,
v(dt) 2/2 1119//2 122
(10)
Under this choice, the Karhunen-Looève expansion for the vector Gaussian process can be attained at the expense of defining suitable orthonormal basis that is legitimate for this measure space. Our paper takes this path. Hence, we start by defining a proper orthonormal basis for the case considered here.
We illustrate routine through the following scheme.

---

Published in Transactions on Machine Learning Research (MM/YVV (YYY)
The Route to Smoothness
Consider the measure space in Equation (9).
Provide an orthonormal basis.
3. Provide suitable Karhunen-Loêve expansion.
Define a proper Sobolev space.
5. Quantify smoothness of the kernel K.
The following sections detail each of the steps in this routine.
3.1 A Constructive Approach to Orthonormal Bases
Consider the normalized Hermite polynomials Hx on the real line defined by (Olver et al., 2010, Table 18.3.1)
(-1)² 2² dt + (E) (k!) de e
EER, 0 1,,,,2,....
The family {H₂} nEz, forms a complete orthonormal system for L2 1/), with the standard Gaussian mea- sure dv = (2T) e < de, i.e.
Hx(E) + df 8,
(11)
2m
Moreover, the 1-th derivative of the Hermite polynomials satisfies
d' del
k!!
Hx(E)
. (n 1n-
H (())
(12)
On R & 222, 2, we define the normalized multiple Hermite functions Pa, with E Z through the identity
(h) II Ho. (h+), h E 12*
(13)
It can be verified via Fubini's theorem, by Equuatin (11) and the definition of v in Equation (10), that these functions form an orthonormal basis 2 (R* v).
Hence, we have completed Step 2 in our Route to Smoothness.
3.2 Expansion for the Matrix- Kernel
The sequence ( in the series expansion (3) is summable at zero. Further, from well-known
properties of matrix-valued positive semidefinite functions (Chilès and Delfiner, 2012), we have that, for every
0, the matrix-valued function Cn having elements Cij.ns i,j 1,...,P P, satisfics Cig.n()² (h)² < Ca,, (O)C
for i,j 1,...,. This in turn implies that, for every finite measure X, the mapping C, is in [RR A) for all n. This is obviously true when À is the Gaussian measure v.
From Equation (3) in concert with the fact that
Gldd 1)/2
(gdd)1)/3())
$1, (-1,1
(dd-1)/2
we conclude that the convergence of the series (3) is uniform.

---

Published in Transactions on Machine Learning Research (MM/YYYY)
At this point, since the multivariate Hermite polynomials that have been defined in Equation (13) form complete orthonormal basis in (R* v), we have that, for every = 0, the positive semidefinite functions Cn I + RPxP can be uniquely expanded in terms of Hermite polynomials, that is,
(h) = Σ
nn (h), h E
ac21
where the series converges in (22 v), and where { arE2 C CCP is summable sequence of matrices such that
an = / (h) n e Z,, a E ž
(14)
Consequently, the kernel K in Equation (3) can be uniquely expanded as
K(s, h)
E dim (H
Σ
Yn Pa (h)9g (1)2) (8, E [-1,1| x
(15)
a
We call the indexed set (1h (n.a)) where the indices take value in the set
C CPPp the Gegenbouer- Hermite spectrum of the p-variate kernel K,
S4 = (n, 2,, a }
(16)
3.3 Defining the Sobolev Spaces
For given S.m E Z4, let ec.m ((- 1), PP*P) be the space of functions R defined (-1, x R², with values in Rexp, such that a² 03 R exist and are continuous for 0, I...., and 0 S |8| s m, where d represents the differentiation in (-1,1) and a the partial derivative in IR of multi-index B.
Define
2 of
R 2
Σ
3 mm Rk
(1 (1- 2/2 ds v(dh),
(17)
1=0
where I the Fröebenius norm RPx,, induced by the Fröebenius inner product (A, (rp:=I((AAB]] so that || tr(AAT), ) with tr denoting the trace operator.
Finally, define the Sobolev space WC. ((-1,1),R*; RPXP) the completion of the space (5.m((11,1)) Rt; RPXP) with respect to the norm (17) (with the usual identification of a.e. equal functions). Remark 3.1. The choice of this particular norm is due to the actual meaning of the variables in our setting: in fact the differentiaaion with respect to connected to a differentiation on the sphere with respect the geodesic distance, defined as arccos((:,.)) between any pair of points on the spherical shell. The measure (1 (1- (2= 1/2-111 ds corresponds to the surface (Haar) measure aa on gd.
3.4 Quantifying Smoothness
In the following we intend to obtain estimates from below and from above for the Sobolev norm (17). We will use the equivalence relation y to relate functions S,g, meaning that eg < ʃ S Cg with constants c, C > 0 that can only depend on (d, k, (.m. Note that this is the case if the constants depend also on j or on B, since it will always be intended that j < Ç and | S m, so they only take values in a finite set depending on k. k.
Our search from smoothness starts by defining a proper spectral inversion of K under the Fröebenius norm || 11.. To do so, for B E Z and E Z4 such that |&| < and j S C, define
1 = / /')
di K(s,
8
h)
2
(1 - 2) d/2-1t5 vdhh).
(18)
Jpt -1
F
8

---

Published in Transactions on Machine Learning Research (MM/YYYY)
We now define sequence {8H with generic element $j being identically equal to
8, 8, Σ
Inall (n + 120-
(19)
n=)a28
We are going to prove that the quantities (18) and (19) are actually related, and that they are both crucial to quantily smoothness.
We start with a technical result that clearly illustrates the relation between these two quantities.
Proposition 3.1. Let Ç. m E Z4. Given the continuous kernel K (-1,1) R Rexx that is isotropic
Sd and stalionary on IR in Equation (15), we have that
K
Σ
ΣER
1
Σ sj.B.
W.C
d.k
1815m
Hence, ||||
wa. < 3o if and only if 83 < co, for all j SC and B E 24 such that | S m.
Proposition 3.1 derives from Lemma A.2 given Appendix A. Clearly, does not provide a friendly way to check when given function K belongs to the Sobolev space W for given quadruple (d, k, m) of suitable integers. The next result (see the proof in Appendix A.2) provides an estimate that helps shedding some light in this direction.
Proposition 3.2. In the conditions of Proposition 3.1,
K W 80,0 + 80,0 +
Σ -m
80,3* +
Σ
8c,8".
|
A further step ahead can be done by introducing the space of square summable selti-sequences, with respect to measure j in the set Se defined in Equation (16):
((1)= = {{ { (maa)maaen C CPXP
1m,, ,na < }00). )
n=0a20
We are ready to state the main result (sce the proof in Appendix A.2), which completes our quest for smoothness over product spaces, giving a condition on the Gegenbauer- Hermite spectrum {rd, (n.a)en,
of the p-variate kernel K, which holds true if and only if K W. ((-1, R²; RPxP), thus quantifying its smoothness in terms of the two parameters m, E+.
Theorem 3.3. Let S.m E 24 and the measure fis. be defined as
pi.m = (n+ [1 (n 1) Xmze]
1+ Σ 18
9' Xazs
(n.a) Ωx.
(20)
with Xnzc and Xaza being equal to 2 Ç and B¹, respeclively, and to otherwise. Then, for given continuous kernel K (-1, 1) R RPxP that is isotropic on gd and stationary R$ in Equation (15), we have that K belongs to the space W.. if and only { (((7.. d.k
Hence, we have proved that, under the spectral construction proposed this paper for a Gaussian measure space, quantifying smoothness oquivalent to prove summability conditions for the matrices > One can certainly argue that these conditions are analytically tricky to check. Yet, Theorem 3.3 provides the building block to deduce the simpler condition below (see the proof in Appendix A.2).

---

Published in Transactions on Machine Learning Research (MM/YYY)
Corollary 3.4. Consider pi one of the following measures in 12
C.m. = (n + 1)0-1 [1 + (n + 1)
1 Σ a
, (n, E Su.
(21)
=m
Ph (n+ [1+ (n 1)2{ [1 |a ["]"
(n, E St.
(22)
If {1,,aa E ((bim), then K belongs to the space W.... d,k
Remark 3.2. Our work is general, and the following special cases are covered:
1. The case & = 0 and p has been considered by Lang and Schwab (2013).
The case h and p has been considered by Clarke De lo Cerda et al. (2018).
3. The case d = 0, general case, but the domain restricted to Ba C R with By denoting the k- dimensional ball, has been considered by Cleanthous (2023).
The work of Cleanthous (2023) is the first, to our knowledge, where multivariate smoothness is considered in the literature.
4 Example
For d> 1, a 20, and v) E (0, 1), consider the following univariate nonseparable kernel (Emory et al., 2021):
(1 - y) exp(-b|lh1l3
((s. h,s, = (1- - 2ns exp( -a - (2 + 172 expl -appl-2aloo 1(d--))
8 8 [-1,11 h E R*
To calculate its Gogenbauer-Hermite spoctrum, we start with the Gegenbauer expansion (Emcry et al., 2021)
K(s, ((s,h;a,b,,) = (1 - -1) -1)=)=- Σ n" exp( (am b)
==
dim ( a, b, ())2 sE(-1,1], hER,
n=0
with
(a (d 1)(1 1) 1-1 n7 exp(-(an+ 1)11)
2n +
The Gegenbauer-Hermiee spectrum given by Equation (14). Accounting for the properties of Hermite polynomials (Magnus al., 1966, Section 5.6.2), one finds
n In,a
Cd a, b, m) (h)dh R*
one or more components of a is odd (d=1)(1-md=-44) 2- sha!
(1+2(an+0) - 1) /x//2 otherwise.
2n+d-1 (a/2)) V (1/2+an+bj*
Using the duplication formula for the gamma function (Olver et al., 2010, formula 5.5.5), seen that 2-1a//2
x'a! (a/2)1 V +oj belongs (0, (2n)) )/2)) Since, furthermore, (142(0n4)) - 1) and 7 belong to (-1, and (0, 1), respectively, it follows that (xd.a) (ā5.i)) for any Ç, m E 24. Accordingly, owing to Theorem 3.3,
K belongs to WS.m for any (, Z4.
10

---

Published in Transactions on Machine Learning Research (MM/YYX)
5 Conclusions
Our work provides the foundations to smoothness quantification of Gaussian processes defined over some specific product space involving a d-dimensional sphere. Some comments are in order. The results presented in Section 3 can be extended to product spaces involving other manifolds. For instance, classic harmonic analysis arguments prove that the d-dimensional sphere might be replaced by a compact two-point homoge- ncous space at the expense of replacing the normalized Gegenbauer polynomials Equation (3) with their counterpart such spaces, known as Jacobi polynomials (Cleanthous et al., 2020). We not. aware of whether results would hold for other general networks such graphs with Euclidean edges (Porcu al., 2023). For such cases, even spectral representations become questionable, so that more mathematical effort is needed in this direction.
Future works may involve the verification of the results presented in this paper for specific classes of scalar and matrix-valued kernels, such as the ones proposed by Porcu (2016; 2018), Alegría et al. (2019) and Emery al. (2021).
Also, extensions our work to kernels that are not isotropic the sphere could be based spectral characlerizations such as the one proposed by Jones (1963) for axially symmetric processes on S i.e., processes that are stationary over longitudes, but not over latitudes, of the 2-sphere. Having some insight in this direction would help to overcome the restrictive assumption of isotropy and allow for wider classes of kernels in vector Gaussian process regression.
Acknowledgments
This work was partially funded by the São Paulo Research Foundation (FAPESP, Brazil), through grant 2021/04269-0 (APP) and grant 2022/16622 164416 (EM), and by the National Agency for Research and Devel- opment of Chile, through grants ANID PIA AFB230001 and ANID Fondocyt 1210050 (XE). The authors are very grateful the Action Editors and the Reviewers for their thorough reviews that have allowed for considerably improved version of the manuscript.
11

---

Published in Transactions on Machine Learning Research (MM/YYN)
A Appendix
A.1 Technical Lemmas
Lemma A.1. Let R. 24. If Jate? 2 8, then one has:
(a+E)!
a..
(23)
(a+s 3)
Proof. The claim holds because ei (3,, E,), S (a;+e:- 9:] S c2(5,, e; i= == 1,2,.... (Olver et al., 2010, formula 5.11.12). In particular, the constants depend and e.
(a.+e.)!
For the next lemma we will need to state some formulas. From Olver et al. (2010, formulas 18.9.19, 18.9.21 and 18.7.4), we have
d
(x 22 (A), (s) (x)) Vn 2
AY 0,
(24)
dsl
d (s) =
d (s(s) =
vn 2 1,
(25)
ds
ds
where (A), = T(X)
r(++1) is the Pochhammer symbol (Olver et al., 2010, formula 5.2.5). Using Olver et al. (2010,
Table 18.3.1 and formula 18.14.4), we get
[ ()(s))
2) ds =22-2A((n + 2A) nl(n + (A) 8nxe, Vn, 0, YA >>>0 0,
(26)
[ 9h (s) (1- 5²) 1/2 ds 8n,w' Vn,n'20 0.
(27)
Finally, from Muller (1966, equation 11),
dim()())
2n + d d-1
(28)
G Gld-1)/2
dim dim(H() 2,
Vn 1
Lemma A.2. Let f.m E E4. For E z and j E E. such that |3] S m and j S S, define I and s5 as per Equations (18) and (19). Then the following estimates hold:
1
ΣΣ n=j a=)a28
Yn 2 (n 1)d-1+2)
14 a!!
(29)
(a (a-3)
and
I ~ 8j
(30)
Proof. By Equation (15),
1
Σ dim dF Σ a a ah)
di
9(2-1)/2
(1 (1- 222d/2114 ds v(dh)
dst
|a|20
(nn In a.) Jnnn
n,,'=0 |al, a 20
12

---

Published in Transactions on Machine Learning Research (MM/YYYY)
where
Jn = dim )im.))) (x%) ["
dej 9h 1)2
d
dsi dss 92 (s) (1 82) 22) d8
d
and
Ja = ( 0 (1)0 ae (h) v(hh).
Note that, Jn. = for n j. For n 2 j 0, we distinguish two cases, depending on whether d is greater than 1 or not. First, let us examine the case when and 2 j 2 0. In this case, we have, by Equation (24),
dim(HH) dim (Ha)
J² de dad Gh (1 &i
Jnz
der Ght ( (1 - 2))00//221 ds
G (dd-1)/((1)
dim()C))) dim (3C))
cdd-11/2+3 (s) (s) (1 - 32) 22)0/2-1+3 ds.
Gh
((d- 1)/2 (1) G G/d-1)/2 (1)
By Equations (26) and (28), we obtain
J Jn
2n + d
2n' d + /" G (s)c (s) (1 22)
d d-1
d-1
-
2n + d d-1)²
-22-d-2j11 2 +
8n,n'
d-1
(n-j)\(n + d-1 |
d-1 j)²
2
Since
2n + d 1 d-1
11
and (Olver al., 2010, formula 5.11.12)
22 222-0-21T(6 + ++j+d - 1)
(n - j)!(n + 2 1 + + (n 1n+1)d-3+42)
the previous result. simplifies into
Ja
(n + 1) 8m
(31)
Let us now address the case when
1. For n 2 = 0, we have
Jan = dim (3C)) / 3h (s)s (s) 22-- ds
~ 8n,n':
based on Equation (27). For n 2 > O, we have, by Equations (23), (24), (25) and (26):
Jnw = dim (IC, (3C)) /"
def (s) (s) d Sh (s) (1 - 22)5-1/2 ds
d'
dim((() dim (3C) nmn 40 [(j 1)11 [ GØ, (s),()) (1 1-12 ds
dimm(C) (dim (3c)) nn 1 ( + j 1)!
2(n - i)! 8nmn'
(n +
1)288 an
Hence, Equation (31) remains valid when d = 1.
13

---

Published in Transactions on Machine Learning Research (MM/YYN)
On the other hand,
Ja
J
0
II
Haj (h₂) 08
II Hai (h1) v(dh)
2k
j=1
J=1
II
S Hay (hg)
II
80 Ha, (hi) v(dh)
3k
==1
ar!
a''l
Hos (h2)
II
Hap (hz) v(dh)
(og
Bz)1
(ag-Ba) B,)!
3
f=1
/ (a - 3)1 V (a'- - 3)! /
a!
a'!
b
1I
Ha, -, (h2) II
Hag (h₂) v(ch)
j=1
j=1
ad!
o'! a'!
/ Pa-s (h) a (h) v(dh)
V (a 18--pp B)
(a'- 3)
and then, since the multivariate Hermite polynomials are an orthonormal basis of T,2(RR v),
ad!
Ja (a 311 8a a B.
(32)
Thus, from Equations (31) and (32) we obtain Equation (29) and then Equation (30) using Equation (23). Lemma A.3. Let a, a, B, zt. I 2 ge z
g 2. 0, then a S a
Proof. In the scalar case, a, b, b' E4 with a 2 b 2 implies a² s a.. By applying this to each component we obtain the claim.
Fix B E 2 with |8| S m, lot
Ig = 1B' E Z+: 88223, 18" = m]
and
A-: (a E Z, a a 3).
Lemma A.4. The set Ag can be written as
Ag Ag U Ag
(33)
B'Elp
where Ag {a 24 :a] < m, 2 .
Proof. If E Ag, then either |al| < m or there exists B' Ig such that 8 < B' < az. One can construct
such Be by increasing those components Br of g that satisfy Br < os until reaching 1
A.2 Proofs of the main statements
Proof of Proposition 3.2. Given j S and |B] S m, in the definition (19) of 8,,8, the sum runs over every
2 j and a E Ag.
We have that
n 2 Ç then (n + 1)20 < (n + 1)
if n then (n 1) (( 1)25.
14

---

Published in Transactions on Machine Learning Research (MM/YYY)
Moreover, by Equation (33), if a Ag then either Ag Ag for some B' lp:
if a Ag then a I mm
a E Ag with 3' 1æ: then by Lemma A.3, it holds a 2 a
Then we can estimate from above all the terms I l (n 1)a-+++++2 a in the definition (19) of js with the corresponding term
in sc, with a 2 8¹ 2 B lol 2 2m,
in (C + s0, with a 2 S 2
in sç, if |a| < m, n2c
in (C + 1) 50,0 if aa| < m, n c.
As consequence
$3 S 1) t2 Su,o + m sg0 m,,sso + (5 ( 1)
Σ
Su,gr +
Σ $g,9r,
=m|=m
|
and then summing up all the terms (the number of such terms only depends on S. k, m), we get
K
Wm d.k
Σ
SI 80,0 + $0.0 +
Σ
80,8' +
Σ
sc,8'.
==0 |8
-m
Bm
since the estimate from below is trivial.
Proof of Theorem 3.3. By Proposition 3.2, all we have to do is to prove that
80,0 + 8ç.0
Σ -
Su,8' +
Σ SG
Iaal? Ps.m
(34)
-m|
n=0a20
where ph.m is given in Equation (20). Indeed, from Equation (19), one has:
Σ Σ
Imall (n+ aa
80.0
n=0a20
$c,8/ = Σ
ba I (n + 1+26
| 2 (n + 1)d-1+24 1+25 Xnos á Xa2s
n=C
22
n= a2
oc
8
1 2
F. (n + 11d--11+
| ma.aa (n+ + 1)d-1+2
Xnze a.
80,0
n= a20
0=0>0
00
8
Σ
a (n (n+ 1)
d v (n + a Xazø
80,9'
n=0 a20
where a 1. This all adds up into
7/.m = 1 1 + (n + +))5xx26
Xnzs
Σ 3
Xazgr + (n + 1)26
Xa2s Σ a Xaz#
| =m
=m
= (n + 1)--1 [1 (n + xn2c
1+ Σ a g Xazo
15

---

Published in Transactions on Machine Learning Research (MM/YYY)
Proof of Corollary 8.4. Obviously,
Th S (m+ (n+ [1 (n + 1)2{
1+ Σ a
18
Moreover, a 2 |ax"", so that Em a8 < D(m, k)|a(""; l") where D(m, k) > indices in 2 of module m (an integer depending only m and k). Accordingly,
the number of multi-
Ta 5 (n + () [1 (n + 1)²$ [1 + (m,, km
< (m,, k) 1) [1 (n + 1) [1 [ax|")
Thus, considering the measure in Equation (22) or in Equation (21), if { Vm E ( then {, E ] e (fpti)) and the result follows by Theorem 3.3.
References
Adler, R. J. and J. E. Taylor (2007). Random Fields and Geometry. New York: Springer.
Alegría, A., E. Porcu, R. Furrer, and J. Mateu (2019). Covariance functions for multivariate Gaussian fields
evolving temporally over planet earth. Stochastic Environmental Research and Risk Assessment (8-99) 1593-1608.
Alvarcz, M. A., L. Rosasco, and N. D. Lawrence (2012). Kernels for voetor-valued functions: A review. Foundations and Trendso in Machine Learning 4 (3), 195 266.
Atluri, G., A. Karpatne, and V. Kumar (2018). Spatio-temporal data mining: A survey of problems and methods. ACM Computing Surveys 51 (4), 1-41.
Atluri, G., A. MacDonald III K. Tim, and V. Kumar (2016). The brain-network paradigm: Using functional imaging data to study how the brain works. Computer (10), 65-71.
Barp, A., C. J. Oates, E. Poreu, and M. Girolami (2022). Riemann-Ster kernel method. Bernoulli 28(4). 2181-2208.
Bonilla, E. V., K. Chai, and C. Williams (2007). Multi-task Gaussian process prediction. Advances in Neural Information Processing Systems 20.
Castruccio, S. and M. L. Stein (2013). Global time models for climate ensembles. The Annals of Applied Statistics 7(3), 1593-1611.
Chen, Z., J. Fan, and K. Wang (2023). Multivariate Gaussian processes: definitions, examples and applica- tions. METRON 81 (2), 181-191.
Chilès, J.-P. and P. Delfiner (2012). Geostatistics: Modeling Spatial Uncertainty, Second Edition. New York: John Wiley & Sons.
Clarke De la Cerda, J., A. Alegría, and Porcu (2018). Regularity properties and simulations of Gaussian random fields on the sphere cross time. Electronic Journal of Statistics 12(1), 399 426.
Cleanthous, G. (2023). On the properties of multivariate isotropic random fields on the ball. Technical Report. DOI: 10.21203/ r....r-27000022 vl.
Cleanthous, G., A. G. Georgiadis, A. Lang, and E. Poreu (2020). Regularity, continuity and approximation of isotropic Gaussian random fields on compact two-point homogencous spaces. Stochastic Processes and their Applications 130(8), 4873 4891.
16

---

Published in Transactions on Machine Learning Research (MM/YYYY)
Connor, C., N. Chapman, and L. Connor (2009). Volcanic and Tectonic Hazard Assessment for Nuclear Facilities. Cambridge University Press.
Emery, X., A. Alegría, and D. Arroyo (2021). Covariance models and simulation algorithm for stationary vec- tor random fields on spheres crossed with Euclidean spaces. SIAM Journal on Scientific Computing 43)). A3114 A3134.
Erdélyi, A. (1953). Higher Transcendental Functions, Volume II. New York: McGraw-Hill
Faghmous, J. H. and V. Kumar (2014). Spatio-tempores data mining for climate data: Advances, challenges, and opportunities. In W. Chu (Ed.), Dala Mining and Knowledge Discovery for Big Dala, pp. 83 116. Springer.
Fasshauer, G. F. (2007). Meshfree Approzimation Melhods with Matlab, Volume 6. World Scientific.
Fuselier, E. and G. B. Wright, (2012). Scattered data interpolation on embedded submanifolds with restricted positive definite kernels: Sobolev error estimates. SIAM Journal Numerical Analysis 50(3), 1753-1776,
Fuselier, F. J. and G. B. Wright (2009). Stability and error estimates for vector field interpolation and decomposition on the sphere with RBFs. SIAM Journal on Numerical Analysis 47(5), 3213 3239.
Hofmann, T., B. Schölkopf, and A. J. Smola (2008). Kernel methods in machine learning. The Annals of Statistics 36(3), 1171 1220.
Hubbert, S., Q. T. Le Gia, and T. M. Morton (2015). Spherical Radial Basis Functions, Theory and Applications. Springer.
Hubbert, S., E. Porcu, C. J. Oates, and M. Girolami (2023). Sobolev spaces, kernels and discrepancies over hyperspheres. Transactions in Machine Learning Research. Accepted.
Hutchinson, M., A. Terenin, V. Borovitskiy, S. Takao, Y. Teh, and M. Deisenroth (2021). Vector- Gaussian processes on Riemannian manifolds via gauge independent projected kernels. Advonces Neural Information Processing Systems 34, 17160 17169.
Mlia,, J., A. Penttinen, H. Sloyan, and D. Stoyan (2008). Statistical Analysis and Modelling of Spatial Point Patterns. John Wiley & Sons.
Jeong, J., M. Jun, and M. G. Genton (2017). Spherical process models for global spatial statistics. Statistical Science (4), 501-514 513.
Jones, R. H. (1963). Stochastic processes on sphere. The Arnals of Mathematical Statistics (1), 213-218.
Kazawa, H., T. Izumitani, Н. Taira, and E. Maeda (2005). Maximal margin labeling for multi-topic text categorization. In Advances in Neural Information Processing Systems 17. MIT Press.
Lang, A. and C. Schwab (2013). Isotropic Random Fields on the Sphere: Regularity, Fast Simulation and Stochastic Partial Differential Equations. The Annals of Applied Probabilty 25, 3047-3094.
Lawrence, N., M. Seeger, and R. Herbrich (2002). Fast sparse Gaussian process methods: The informative vector machine. Advances in Newral Information Processing Systems 15.
Magnus, W., F. Oberhettinger, and R. Soni (1966). Formulas and Theorems for the Special Functions of Mathematical Physics. Berlin: Germany: Springer-Verlag.
Marques, R., C. Bouville, and K. Bouatouch (2022). Gaussian process for radiance functions on the sphere. In Computer Graphics Forum, Volume 41, pp. 67-81. Wiley Online Library.
Marques, R., C. Bouville, M. Ribardière, L. P. Santos, and K. Bouatouch (2013). A spherical Gaussian framework for Bayesian Monte Carlo rendering of glossy surfaces. IEEE Transactions on Visualization and Computer Graphcs 19(1)), 1619-1632.
17

---

Published in Transactions on Machine Learning Research (MM/YYYY)
Muller, C. (1966). Spherical Harmonics. Lecture Notes in Mathematics. Springer Verlag.
Oleson, J. J., N. Kumar, and B. J. Smith (2013). Spatiotemporal modeling of irregularly spaced acrosol optical depth data. Eneironmental and Ecologioul Stalisties (0) 297-33 334.
Olver, F. W., D. M. Lozier, R. F. Boisvert, and C. W. Clark (2010). NIST Hondbook of Mathemalical Functions. Cambridge: Cambridge University Press, Cambridge.
Osborne, M. A., S. J. Roberts, A. Rogers, S. D. Ramchurn, and N. R. Jennings (2008). Towards real- time information processing of sensor network data using computationally efficient multi-outuut Gaussian processes. In 2008 International Conference Information Processing in Sensor Networks (IPSN 2008), pp. 109 100. Institute of Electrical and Electronics Engineers.
Porcu, E., Alegria, and R. Furrer (2018). Modeling temporally evolving and spatially globally dependent data. International Stotistical Review 82).), 344 377.
Porcu, E., M. Bevilacqua, and M. G. Genton (2016). tentooal covariance and cross-covariance
functions of the great circle distance on a sphere. Journal of the American Statistical Association 111 (514), 888 898.
Porcu, E., M. Bevilacqua, R. Schaback, and C. J. & Oates (2023). The Matérn model: A journey through statistics, numerical analysis and machine learning. arXiv preprinl or XxXw:2303.02259.
Porcu, E., R. Furrer, and D. Nychka (2021). 30 years space time covariance functions. WIREs Compu- tational Statistics 13),), e1512.
Porcu, E., P. A. White, and M. G. Genton (2023). Stationary nonseparable space-time covariance functions on networks. Journal of the Royal Statistical Society, To Appeor.
Quinonero-Candela J. (2004). Learning with uncertaindy: Gaussian processes and relevance vector machines. Ph. D. thesis, Technical University of Denmark Lyngby, Denmark.
Rosa, P., V. Borovitskiy, A. Terenin, and J. Rousseau (2023), Posterior contraction rates for Matérn Gaussian processes on Riemannian manifolds. arXie preprint or urXiv:2309.1090 10918.
Sánchez, L. K., X. Emery, and S. A. Séguret (2019). 5D geostatistics for directional variables: Application in geotechnics to the simulation of the linear discontinuity frequency. Computers & Geosciences 133, 104325.
Sánchez, L. K., X. Emery, and S. A. Séguret (2021). Geostatistical modeling of rock quality designation (RQD) and geotechnical zoning accounting for directional dependence and scale effect. Engineering Geol- ogy 293, 106338.
Seeger, M. (2004). Gaussian processes for machine learning. International Journal of Neural Systems 14 (02), 69-106.
Shirota, S. and A. E. Gelfand (2017). Space and circular time log Gaussian Cox processes with application to crime event data. The Annals of Applied Statistics 11 (2), 481-503.
Tompson, L., S. Johnson, M. Ashby, C. Perkins, and P. Edwards (2015). Uk open source crime data: Accuracy and possibilities for research. Cartography and geographic information science 42(2), 97-111.
Wang, S., J. Cao, and P. Yu (2022). Deep learning for spatiotemporal data mining: A survey. IEEE Transactions on Knowledge and Data Engineering 34 (8), 3681 17000.
Wingeier, B. M., P. L. Nunez, and R. B. Silberstein (2001). Spherical harmonic decomposition applied to spatial-temporal analysis of human high-density eleetroencephalogran. Physical Review E 64 (5), 051916.
Wu, S., X. Li, W. Dong, S. Wang, X. Zhang, and Z. Xu (2022). Multi-source and heterogencous marine hydrometeorology spatio-temporal data analysis with machine learning: a survey. World Wide Web 26(3). 1115-1156.
18

---

Published in Transactions on Machine Learning Research (MM/YYYY)
Xing, W., F. Yu, P. Leung, X. Li, P. Wang, and A. Shah (2021). A new multi-task learning framework for fuel cell model outputs in high-dimensional spaces. Journal of Power Sources 482, 228930.
Yadrenko, M. I. (1983). Spectral Theory of Random Fields. New York: Springer- Verlag.
Yaglom, A. M. (1987a). Correlation Theory of Stationary and Related Random Functions. Volume I: Basic Results. New York: Springen-Verlag.
Yaglom, A. M. (1987b). Correlation Theory of Stationory and Related Random Functions. Volume II: Supplementary Notes and References. New York: Springer-Verlag.
Yuan, Y., Q. Wang, and X. Yang (2022). Traffic flow modeling with gradual physics regularized learning. IEEE Transactions on Intelligent Transportation Systems 23(9), 14649 14660.
19


Reviews:
[{"review_id": "o9PkOk9rJV", "reviewer": "Reviewer_TRSJ", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\nNovel construction to define smoothness of vector-valued Gaussian process.\n\nWeaknesses:\nThe construction depends on some recent references, e.g. Porcu et al. 2021. And therefore it is not introduced in a self-contained way.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper characterizes the smoothness properties of vector Gaussian processes. The authors first use Hermite polynomials to construct a spectral decomposition of a matrix-valued kernel, then define a corresponding Sobolev space, then quantify the smoothness of a given kernel by investigating the $s_{j,\\beta}$ functions in (14). The main result is in proposition 3.3, stating that smoothness is equivalent to summability conditions of the matrix coefficients of the spectral decomposition.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Somewhere around sec 2, the authors should explain how smoothness is quantified for scalar Gaussian processes, so the readers can see the developments are not trivial. Overall, this paper introduces a construction but does not compare it with alternative constructions.\n\nHow significant is the studied product space? Can either the sphere or the Euclidean space be zero dimension (d=0 or k=0), so that the product space includes the sphere/Euclidean space as special cases?\n\n1.4 introduce \\mathbb{Z}\n\nP4 \"cov\" is not introduced\n\n3.1 \"It can be namely verified that these functions ...\" Add more explanations and intuition.\n\n3.3. Explain why $(-1,1) \\times R^k$ is investigated. Why the interval (-1,1) is not closed?\n\nsec 3. In the end, how smoothness is \"quantified\"? Is it a binary property (whether or not K belongs to the space $W_{d,k}^{\\zeta,m}$)?", "broader_impact_concerns": "No ethical implications were identified.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "84jXWrRMo2", "reviewer": "Reviewer_wYMA", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\nThe spectral analysis is well-founded and the paper is carefully written.\n\nWeaknesses:\n\nThe choice of notation, such as using $\\lVert\\cdot\\rVert_k$ for the Euclidean norm and $\\lVert\\cdot\\rVert_*$ for the Frobenius norm, might confuse readers familiar with the conventional usage of these notations in other contexts. Also, the reliance on external references, such as Erdélyi (1953) and Cramér's theorem, without fully integrating their details into the discussion, detracts from the paper's self-contained nature and accessibility.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper addresses the issue of quantifying the smoothness of vector Gaussian processes defined over the product space $\\mathbb S^d \\times \\mathbb R^k$. The paper considers a class of kernels over this space which are isotropic and stationary, i.e., given inputs $(x, t)$, $(x’, t’)$, the output of the kernel only depends on $\\langle x,x’\\rangle$ and $t-t’$. For such kernels, by leveraging the expansion in the orthonormal basis of corresponding $L^2$ space (induced by the product measure of the spherical Haar measure and the Gaussian measure), the paper relates the decay of the Fourier coefficients in this basis with appropriate Sobolev-type norms. The flavor of the result is expected and the main contribution seems to be to write everything down explicitly.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "The notation $\\lVert\\cdot\\rVert_k$ is usually used for the $\\ell_k$ norm, and $\\lVert\\cdot\\rVert_*$ is usually used for the nuclear norm; please change these notations (it would be fine to use $\\lVert\\cdot\\rVert$ for both, or $\\lVert\\cdot\\rVert_{\\rm F}$ or $\\lVert\\cdot\\rVert_{\\rm HS}$ for the Frobenius norm).\n\nPlease make Section 2 more self-contained by formally stating and explaining the application of formula 11.4.2 from Erdélyi (1953) and Cramér's theorem within the context of your analysis.\n\nTypos:\n- Section 1.2: change the title from \"Why Studying\" to \"Why Study\".\n- In eq. (5), the last comma should be a period.\n- After eq. (10): \"indexes\" -> \"indices\".", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Wiksb55YOz", "reviewer": "Reviewer_v83p", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: the paper is primarily theoretical and the theoretical results are, to the best of my knowledge, novel and interesting.\n\nWeaknesses: \n1. The motivation on the study of smoothness properties, and the specific choice of product is a bit vague and unclear; e.g., more connection to applications may be helpful.\n2. The paper is rather technical and the presentation can be improved.\nSee my detailed comments below.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "In this paper, the authors provide characterization of the smooth properties of vector Gaussian processes defined on the product space of as unit sphere manifold and standard Euclidean space.\nThrough a spectral representation approach (e.g., via the normalized Hermite polynomials), the authors propose to assess the smoothness by studying the corresponding matrix-valued kernel.\n\nFrom my personal viewpoint, the major contribution of this paper is:\n1. propose to study the smoothness properties of vector Gaussian processes on product space via a spectral approach; and\n2. derive precise smoothness characterizations in Section 3.4.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. why the setting of product space between unit sphere and standard Euclidean space? The authors mentioned on possible applications at the end of Section 1.2, but the connections are a bit vague and unclear. In fact, it could be to good idea to showcase the application of the proposed characterization on a concrete ML example.\n2. it would be helpful to point to, when talking about the contribution of this work at the top of P3, the corresponding precise results in the paper. For example, the route is given in P5, the spectral characterization of smoothness that relies on the properties of the matrix-valued kernel is given in xxx, etc.\n3. Please refer to the precise section of proof in the Appendix.\n\nA few minor issues:\n1. Corollary 3.4: \"one of the following measureS\"?\n2. what does $\\ell^2$ mean in Proposition 3.3?", "broader_impact_concerns": "This paper is primarily theoretical and I do not see any ethical concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "oiXgiJ0Ir2", "reviewer": "Reviewer_3DrE", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strenghts:\n- The presentation is relatively clear, with the \"route to smoothness\" helping the readers understand the proof process, despite the technicalities.\n- While I haven't been able to check every minute detail of the proof, the logic is presented clearly and the arguments are mathematically sound.\n\nWeakness:\n- The space that is considered in this paper is oddly specific. The authors simply list a bunch of works where the particular product space is used but making this more explicit (e.g. giving an example in one of these works cited) would help us understand better the value of the paper.\n- The work feels incomplete. If I understand correctly, the authors are interested in characterising the smoothness of the Gaussian sample paths but the proof stops at proving the smoothness of the kernel. Some comments or extra arguments on how to go from the kernel regularity to the sample path regularity is necessary in my opinion.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The contribution of this paper is to quantify precisely, the smoothness of vector Matern GPs defined over a product space, with one part consisting a $d$-sphere and the other consisting a $k$-dimensional Euclidean space. The proof follows by constructing a Karhunen-Loeve expansion with respect to a basis on a particular measure space, one part consisting the Haar measure of the sphere and the other part consisting a Gaussian measure on $\\mathbb{R}^k$, then bounding the coefficients in such a way that the kernel belongs to a particular Sobolev-space of matrix-valued functions defined over the product space. The authors provide an explicit bound on said coefficients.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I think the work doesn't fully resolve what it sets out to do, which is to characterise the smoothness of Gaussian processes (in my understanding, this means sample paths). Some extra arguments to go from the kernel smoothness to sample path smoothness is necessary.", "broader_impact_concerns": "This paper is mathematical in nature and there are no concerns regarding ethical implications.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper is primarily theoretical and the theoretical results seem to be novel and interesting.

Strengths:
- Novel construction to define smoothness of vector-valued Gaussian process.
- The spectral analysis is well-founded and the paper is carefully written.
- Derivation of precise smoothness characterizations in Section 3.4.
- The presentation is relatively clear, with the "route to smoothness" helping the readers understand the proof process, despite the technicalities.
- The logic is presented clearly, and the arguments are mathematically sound.

Virtually all weaknesses have been addressed after a revision.
Decision: Accept (poster)