Integration-free Kernels for Equivariant Gaussian Process Modelling
Authors: Tim Steinert
David Ginsbourger
August Lykke-Møller
Ove Christiansen
Henry Moss
Venue: ICML 2025 poster (2025)
Keywords: Gaussian processes
equivariance
kernels
molecular data
Primary area: probabilistic_methods->gaussian_processes
TL;DR: Introducing a novel class of integration-free equivariant Gaussian process kernels, enabling efficient modelling of vector-valued random fields with applications to velocity fields and molecular dipole moment estimation with high accuracy.

Abstract: We study the incorporation of equivariances into vector-valued GPs and more general classes of random field models. 
While kernels guaranteeing equivariances have been investigated previously, their evaluation is often computationally prohibitive due to required integrations over the involved groups. In this work, we provide a kernel characterization of stochastic equivariance for centred second-order vector-valued random fields and we construct integration-free equivariant kernels based on the notion of fundamental regions of group actions. We establish data-efficient and computationally lightweight GP models for velocity fields and molecular electric dipole moments and demonstrate that proposed integration-free kernels may also be leveraged to extract equivariant components from data.

Integration-free Kernels for Equivariant Gaussian Process Modelling
Tim Steinert David Ginsbourger'
August Lykke-Moller
Ove Christiansen'
Henry Moss 34
Abstract
We study the incorporation of equivariances into vector-vare vector-valued GPs and more general classes of random field models. While kernels guarantee- ing equivariances have been investigated previ- ously, their evaluation is often computationally prohibitive due to required integrations over the involved groups. In this work, we provide ker- nel characterization of stochastic equivariance for centred second-ord -order vector-valued random fields and we construct integration- free equivariant ker- nels based on the notion of fundamental regions of group actions. We establish data-efficient and computationally lightweight GP models for veloc- ity fields and molecular electric dipole moments and demonstrate that proposed integration-fre kernels may also be leveraged to extract equivari- ant components from data.
Figure I. Rotational equivariance of electric dipole moments (red arrow) of acetylacetone molecules.
ensuring positive definiteness in matrix-valued covariance kernels. However, attracted by GP's explicit posterior distri- butions that facilitate uncertainty quantification and active learning, especially in scientific applications Moss al. (2020); Griffiths al. (2022); Ranković et al. (2024), there has been significant recent effort to encode invariances and equivariances through tailored kernels Ginsbourger et al. (2012); Scheuerer & Schlather (2012); Ginsbourger et al. (2016); van der Wilk et al. (); Holderrieth et al. (2021); Henderson (2023).
1. Introduction
Unfortunately, building equivariant kernels comes with sig- nificant computational effort, and choices are typically made that alleviate costs at the price of reducing expressiveness. Such computational challenges have been addressed in spe- cific contexts, c.g., for accurate modelling of interatomic force fields Glielmo et al. (2017), by transforming a scalar argument-wise invariant kernel into an equivariant matrix- valued kernel with a single group integration.
The incorporation of structural knowledge such as physical laws into machine learning models has gained significant attention for improving predictive accuracy and realism. For example, equivariances are ubiquitous across molecular chemistry, where applying simultancous rigid motions on underlying atoms typically results in equivalent motions to their vectorial properties (as demonstrated in Figure 1). The incorporation of such equivariances well-established in deep learning Cohen & Welling (2016), allowing neural networks to exploit knowing the responses across entire orbits of group action from a single data point.
Our work proposes novel class of equivariant kernels that simultaneously overcome the previously high computational cost and limited flexibility of existing equivariant kernels. We exploit the group-theoretic notion of projecting onto fun- damental regions of group actions, substantially extending the approach of Ginshourger al. (2012) as proposed for scalar-valued invariance. Our main contributions are:
In contrast, progress incorporating such knowledge into Gaussian process (GP) models has been more moderate, potentially due to the intricate mathematical constraint of
IMSV, University of Bern, Switzerland DDpartment of Chem- istry. University of Aarhus. Denmark "School of Mathematical Sciences, Lancaster University, UK DDeparmment of Applied Maths and Theoretical Physics, University Cambridge, UK. Correspon- dence to: <tim.steinertte unibe.ch.>.
a theoretical framework for stochastically equivariant random fields,
a class of integration- -ereattiioo-rreeeeeee equivariant kernels that are computationally efficient, flexible, and stable,
Proceedings of the nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
3. empirical results on equivariant fluid flows and dipole predictions from quantum chemistry.

---

Integration-fre free Kernels for Equivariant Gaussian Process Modelling
2. Background
be relevant Uteva al. (2017). Symmetry-adapted GPs of Grisafi et al. (2018) effectively address equivariance but involve computationally expensive double sums, In the spe- cific setting of -euueiuuireeiane,, Wigner kernels Bigi et al. (2024) offer an efficient approach to modelling covariances, built upon recursive constructions and analytical simplifica- tions derived from properties of Wigner D-matrices. Con- ceptually aligned with our approach is the work of Aslan et al. (2023), which enforces equivariance in determinis- tic machine learning models within the setting of discrete groups by employing similar notions of fundamental regions and projections onto them.
Preliminaries For a detailed summary of the necessary background on multivariate (Gaussian) random fields and group theory, we refer the reader to Appendix A. In what follows, interchangeably denote vector-valued Gaussian random fields as GPs.
Equivariances In this work, we focus on efficiently en- coding equivariances in random fields models. We con- sider -Va-uul second-order, centered random fields Z = (z DE where D C I2d. We denote by * a group ac- tion of a linear group G on D. Equivariance of a mapping f: D + I2 means that for any group element g E G and any x E D, the (vector) value taken by f at the point g * æ is related its value via multiplication with a matrix Pg RPx representing g in IR,, i.c. f(g x x) (z))
3. Kernel characterizations of stochastic equivariance
We begin by characterizing in broad settings the stochastic equivariance of centred second order random fields in terms of their matrix-valued kernel. Theorem 3.1 lays the ground for the subsequent development of our computationally effi- cient kernel class.
In the context of a random field Z we introduce a notion of stochastic equivariance, expressed as:
Vg E G. x D, P(Z Pg Zx) =.
(1)
Theorem 3.1 (Kernel Characterization for stochastically equivariant random fields). Let Z (Z=)=ED, D Rd, be IR -vauued gquare-integrade matrix-values kernel K : D x D + Rexx. Furthermore let G be a linear group acting on D via * and represented in RPP by p : 9 E G - Pg E TRXY,, Then, the following equivalence holds:
Let point out that for G countable, Vg G can be brought inside of P without further assumptions, deliver- ing property of equivariance up to a modification, that is, Vx E D. IP(v E G, G. g.z = pg a) = 1. Adding conditions on D (e.g., D countable) similarly leads to a stronger notion of almost sure equivariance, namely that P(Vy E G, x E D, Zg = Z=) = 1. Sufficient con- ditions for those types of equivariances in more general settings go beyond the scope of this work. In the following. we focus on stochastic equivariance.
centred random with
Ve E D,gEG, P(Z = P,Za) =
Va, x' D, g, K(gxa, = Py
Recall that a centred Gaussian random field Z is character- ized by its valued covariance kernel K: D x D + Rexp, where for ax x' E D and 1 Sisp p.
(2)
Proof. Assuming Z stochastically equivariant, we have that for any x, x E D, g, g,
K(x, x') Dis = Cov
= Z x
(17.z())
E G,
a x') Cov (Z gear Z (inaa)
where the superscript (i) refers to the i-th th vector compo- nent. As we prove in Section 3. ensuring stochastic equiv- ariance (I) for broad classes of random ficlds can be charac- terized in terms of a notion of kernel equivariance such as introduced in Reisert& Burkhardt (2007) for deterministic kernel-based algorithms.
= E Z Z TTraal
(pn I
gu
Pgg E Z ar)
Po K(x x²) )T.
Conversely, assuming (2), then, for any D, g E
Cov(Z Z g+a Zy Z gz Z
Related work The Helmholtz kernel is currently consid- ered a suitable kernel for GPs in the case p = d as it leverages the Helmholtz decomposition for vector fields de- fined over 2 While it ensures equivariance in the posterior mean (Prop. 4.2 Berlinghieri et al. (2023)), the Helmholtz kernel does not satisfy the requirements for stochastic equiv- ariance. In the context of molecular properties, incorpo- rating symmetries in GP models has been demonstrated to
= =K(g x g x) + Pg K (a. x)pg T
- -K(g x, x po Pg K (x, x) = 0,
where from T
gmm gg
Pg 2, tr((0) = 0 and hence
9.20 Pg Zz) 1.
Property (2), i.. for K to be equivariant in the first argument and anti-equivariant in the second, referred to in Reisert
2

---

Preegratioonfree Kernels for Equivariant Gaussian Process Modelling
& Burkhardt (2007) as (kernel) equivariance. In that sense, Theorem 3.1 establishes equivalence for centred second- order random fields between stochastic equivariance and equivariance of the underlying matrix- kernel. It is noticeable that Theorem 3.1 is quite general concerning the linear group G and its action * on D. In particular, note that while we assume D C IRd throughout the paper, the result can be directly generalized to any D.
100
12
10-
K
250
500
250
500
Test
We now consider more specific assumptions. Following settings from Reisert& & Burkhardt (2007), given compact, linear and unimodular group G with continuous represen- tation in case of G being a Lie group, such equivariant matrix- matrix-valued kernels can be constructed by Haar integra- tion. Considering base matrix-valued kernels Ko such that the integrals are well-defined, one obtains in fact as class of equivariant matrix-valus kernels by taking
Figure 2. Log computation times (in seconds) for integration- bbaed vs integration equivariant kernel evaluations.
D, and (g * A) n A = 0, for any g G {e}. For such an A and any x D, there exists at least one g E G such that g* E A. We call section any mapping
gion of * subset A c D such that G A A
D - G, satisfy-
Ky ( x') = f Pg Ko * xt, h * pn dg dh.
ing s(x)) x E A, and denote by II, D - Å the associated projection onto A, characterized by [x) s(x) * x for x E D. The resulting class of kernels and their equivari- ance are presented in Proposition 4.1, followed by a worked example illustrating construction principles and resulting computational benefits.
(3)
In particular, the equivariance of Kj can be checked by using the translation invariance of the Haar measure (de- fined up to a constant). Assuming further that the Haar measure is normalized, choosing any K2 already satisfying equivariance in Eq. leads to - K,. such settings, cquivariant kernels can thus systematically be represented with this construction. In practice, however, the integral nature of Eq. (3) can make GP modelling with such kernels very computationally demanding.
Proposition 4.1 (Integral equivariant kemnels). Let G be linear group acting on D via *, possessing unitary group representation p :: 9 E G - Pg E IRPxP, and let A C D be fundamental region of x. Then, for any matrix-valued kernel K on À x A, section 8 and associated projection II, Kn below defines a nautix-valled kernel equivariant (W.r. k and p) on (G * A) x (G k A):
4. Integration-free equivariant kernels
Knlæ, x') = Po(e) K (II,, (I))))aa))))) (4)
The primary limitation of modelling equivariant random fields with KJ is the need of evaluating the cumbersome double integral in Eq. (3) which is rarely available in closed form and must often instead be approximated, c.g., by quadrature methods. In addition, these approximations need to be accurate to allow the inversion of the Gram matrix required when fitting the GP. For instance, in the rotation-equivaarint GP in Section 5. around 1000 func- tion evaluations are needed, making posterior simulations over few hundred locations computationally challenging.
We refer to Appendix C for proof of Proposition 4.1.
Remark 4.2. In case of a free group action, Kn is equiv- ariant on the whole domain D x D. Otherwise, s(g * x s(x) og gg may not hold for all x E OA
Example 4.3. Assume D = RR, p = 2, and G = SO(2) (An example of a SO(d) eqquivarrn prediction task is pre- sented in Appendix F). A fundamental region is then given by A (xx, 0), > 0}. To each point x E D we assign a section s(x) that rotates æ into A, represented for x # O by:
The computational burden inherent to the group integra- tion formulation of (3) motivate us to introduce a new class of integration-free kernels. Taking inspiration from a previously developed approach for scalar-valued random fields with invariant paths, as proposed by Ginsbourger et al. (2012), we propose projecting inputs onto a fundamental region of the group action, which is subsequently incorpo- rated into base matrix-valued kernel. Equivariance of the resulting kernel is enforced by left and right multiplication of the base kernel with suitable matrices following from the considered group representations.
coos (0(z)) -in (0(x))
Pa(m)
sin (6(z)) cos (0(x))
Here, 0(x) arctan (21))/(1)) is the angle needed to rotate onto A (clockwise). For x 0, we fix (()) = 0. since any rotation is valid choice for s0))
The corresponding projection II, onto À maps each x to II, = (r(x),0), O), where r(z) = |- In Figure 2, we compare the total time to compute the posterior covariance matrices associated with GP KT1) and GP(O, KJ) on
As detailed in Appendix A, we denote by fundamental re-
3

---

Integration-free Kernels for Equivariant Gaussian Process Modelling
training and test locations in 1-,,1],, f. for different train- ing and test set sizes. Here, K is computed by adaptive integration with the adapt Integrate function a maximum of 1000 function evaluations.
5.1. Rotation-equivariant vector fields
Data generation To assess the predictive performance of zero-mern mean rotation-equivarian GP with proposed integration-free kernll, we build dataset of noisy mea- surements D = {(0 v.) with y, given as realisa-
funtiiion in R
The base kernels K,- = X are chosen to be the simple diagonal RBF matrix-valued kernel. All computations in this paper were performed on a cluster equipped with single- core AMD EPYC2 CPUs running at clock time of 2.25 GHz. With straightforward implementation, the difference in computation time for moderate training and test set sizes is considerable. While computing the posterior distribu- tion of GP p) at 500 test locations given 100 training points requires 45 hours, with GP(O, Kx) it takes a total of 55 seconds.
tions of
F(4:) t
E ~ M00 (tsm))
for two synthetic
vector fields
F (x) = (-xx (2) 1 (..)).
E 1_?,
F² (x) 0.5 +
2,
with n = 8, 10 observations and Johs = 0.15, 0.1 for F and F², respectively. See Appendix A.3 for a detailed explanation of the training and evaluation procedure.
Proposition 4.4 (Continuity of Kn). Under the conditions of Proposition 1.1, assume that for a subset B of A, both and II, are continuous on G B, and K is continuous on
B x B. Then, Kn is contimuous on (G+ B) x (G * B). In particular, for B A, Kn is continuous on D D.
We consider four kernels constructed from diagonal squared-exponential base kernel matrix function
Baselines
(a, x²: ofe 11=-2111/226
0 O2e || x 121
Proof. Assume (ar, (ar, x²) E BG x (G B). Then,
0
(I, (x')) E B x B. Since II, is continuous
(G+B) (G+B) and K is continuous on B x B, it follows that K a (-), (.). is continuous on (G+ B)x x (G B). By the continuity of matrix products of continuous matrix- valued functions, KI is continuous on (G * B) x (G k B).
with tunable kernel parameters O (f1, 1, Eg, 02 obs)). We build two (02))-equivarian kernels following the setup of Example 4.3, i.e. setting K, = Кд = KSE, to pro- duce our integral-free -qquivaaqaaaa kernel Kn and the double-integral
kernel Ky.
If B A, then by definition of A, we have G B D, which implies that Kn is continuous D D. 0
We also consider two additional kernels proposed for ocean modelling by Berlinghieri et al. (2023): (1) using KSE di- rectly and (2) the Helmholtz matrix valued kernel KH, as de- rived by modelling the components & and Li of Helmholtz decomposition a vector field F = grad + rot as in- dependent GPs (see Berlinghieri et al. (2023) for a detailed derivation).
Remark 4.5. If the continuity properties are not fulfilled
with B = A but with B = A, this results in continuity on (G A) x (G * A). Appendix B examines the continuity of Kn in specific applications.
The posterior mean predictions and root mean squared error (RMSE) over the ground truth field for the four GPs are shown in Figure 3, where we see that all kernels except KSE provide equivariant pos- terior means. However, we stress that the computational costs of our GP (0, KII) were 500 times faster than those of GP (0, Ky). where the double integral (3) is approx- imated with the adapt Integrate function in R using 1000 evaluations.
Mean predictions
5. Experiments
We now present series of experiments designed to high- light the advantages of incorporating equivariances in GP models, alongside the specific benefits of our integration- free equivariant kernel. First, we model equivariant velocity fields, comparing with the popular Helmholtz kernel, which exhibits equivariance only in the posterior mean. Next, we consider challenging real-world test case involving the prediction of molecular dipole moments, demonstrat- ing the enhanced uncertainty quantification and practical applicability of our proposed kernel. Finally, we explore the efficacy of our GP models in a parameter estimation problem disentangling a real-world ocean velocity dataset from equivariant perturbations.
Probabilistic predictions and sampling Figure 4 presents single realizations (samples) from the posterior distributions of each model, alongside the logarithmic pos- terior density of predictions over the ground truth field (LogS). Table summarises average RMSE and LogS

---

Integration-fre Kernels for Equivariant Gaussian Process Modelling
Sqaared apponetaal EMCCK
Sandvarral SMOC
Doudda Kangadann
RMSD
Vester Bel
Figure 3. Ground truth (left), blue: posterior means of the squared exponential (Ksø), Helmholtz, fundamental (Kn) and double integration () GP. The top row corresponds to Fl, the bottom row to F22 and red vectors indicate (noisy) observations.
scores for each 1000 random draws of n 8, training and The 17 20 test locations.
Table I. Mean performance metries [standard deviation]. Best
scores are in bold, and values within the standard deviation of the best score indicated by (*).
We see that the proposed integration-fiee free kernel enjoys sub- stantial advantages over all the other methods for two key reasons. Firstly, unlike the kernels K and KI which satisfy our formal equivariance condition of Eq. (2), the Helmholtz GP does not provide rotation-equivarian pos- terior samples - see Corollary 5.1 below about posterior stochastic equivariance. Secondly, unlike numerically stable Kn. integral approximation errors when calculating y lead to instability along the the boundary of the domain.
K
Kge
Kg
Km
K
RMSE
0.27 [0]
[D2300
[0.06]*
0.08 (0.04]
LogS
2.06 1.06|137,88]
88.44/256.96)
-6.03|0.511
5.95 [0.46|
RMSE
0.33 (0.06)
0.26[0.06]
0.13
0.22 [0.2]
LogS
149.4 (1086.66)6
(84.355
(0.81
-3.41 [2.06]
of sparse GPs, building upon an equivariant kernel (and equivariant mean) leads sparse GP models retaining the equivariance properties in both their mean and covari- ance ance-thus preserving stochastic equivariance. We give further detail on this in Appendix G. This observation opens the door to extending equivariant GP modeling to large- scale datasets in the future.
Corollary 5.1 (Posterior equivariance). Assume Gaussian random field Z and group G satisfy the assumptions of Theorem 3.1, with the kernel of Z being equivariant (2). Then, given any observed realization 2tr of Ztr (whereby the notation of Appendix A is used). the resulting posterior distribution retains stochastically equivariant, i.e.,
Continuity Kπ and effect of A. Downsides of Kn compared to Kj include potential discontinuities in s and II, potentially resulting in discontinuity of Kn, well as challenges with pathological choices of fundamental regions. To reduce boundary-related issues, one may fa- vor connected fundamental regions. For example (See Fig. 5 for an illustration), partitioning {(x, 0), > 0} into P = U,P, x (0) with P/'s intervals of equal length and defining = U x {0} as the fundamental region for the S0(2) (2))-qquvaari kernel in Example 4.3 leads to reduced performance. In Experiment 5.1, a partition of size 10 results in GP(0, Kr)) achieving an average RMSE (resp. LogS) score of 0.49 (resp. -2566 when predicting Fz. Appendix E provides an illustration this and further- more the impact of a similarly ill ill-specified A on the learning curves Experiment 5.2.
Vx ED DEG, P(Z = - Zo Zie Z(z) 11
Remark 5.2. !fa Gaussian random field Z is stochusticully equivariant on a subset of the domain D, then such posterior distributions retain stochastic equivariance on that subset. In particular, under the assumptions of Proposition 4.1, the resulting posterior distributions for Gaussian process con- structed via the kernel Kμ retain stochastic equivariance on G A.
Remark 5.3. The cubic computational complexity of Gaus- sian process inference restricts its application to datasets of only a few thousand training and test points. Sparse approximations alleviate this limitation and enable GP mod- eling at larger scales. Interestingly for usual constructions
5

---

Integration-free Kernels for Equivariant Gaussian Process Modelling
Squared exponential
Helmholtz LogSn -1.877
Fundamental Logãe 6.005
Double
5.941
-3.404
Squared exponential
Helmholtz LogSe 2.305
Fundamental Log5 -7.804
Double Integration LogS -2.002
-5.647
Figure 4. Single realizations the Gaussian process posterior distributions. Unlike the Helmholtz and squared exponential GP posteriors, the right-most models ensure equivariant realizations, shown by Corollary 5.1
22
manage computational costs for larger molecules.
The electric dipole moment of molecule is here modeled vector funetion
E D R m(z)) R3
where = Vec (a1.- ......... az)) encodes the position vectors in Euclidean space of the atoms (a;, i 1,..,, within the considered molecule. From physical principles, l is known to be translation invvriian and cotation-equvvaree i.e. for all D,
T
u(t +1 x) p(x) 1 +2 x)) (z)(x)
Vt R
(5)
Vi SO(3),
Figure 5. Visualisation of disconnected fundamental region for Experiment 5.1 and the associated and II..
where denotes the action of translations (encoded by elements of RR3) on RR3 *2 denotes the usual action of SO(3) on 123, and Pa E 3233 is the rotation matrix (representation) canonically associated with g E SO(3). The actions are extended D (and later Î) with the conventions:
5.2. Water molecule dipole moments
The electric dipole moment, a vector indicating the imbal- ance molecule's electron distribution, is key to under- standing intermolecular interactions Israelachvili (2011); Stone (2013) and predicting IR spectra intensities Califano (1976). Estimating the dipole moment across molecular surface is computationally intensive, often requiring recal- culations for numerous configurations, which can take days even for small molecules. Therefore, accurate statistical models are crucial, as they reduce the number of required calculations while still effectively describing the dipole sur- face, thereby making feasible predict spectra and
t x x Vec (t +1 a1.....
++ a.):
(6)
x* x Vec (g +2 01 *2 a.)
We now provide an explicit example of how apply our Eq. (4) to a quantum chemistry problem. In particular, consider the case of water molecules, where E D C R29 now represents the position vectors of an oxygen and two hydrogen atoms. the considered case water molecules where = and a2, ag both stand for hydrogen atoms, there also permutation- sermutation-invariane with respect these
6

---

Integration-free Kernels for Equivariant Gaussian Process Modelling
Mean log(RMSE) Scores
Mean Logs Scores
Kh
-1
-8
24
-32
66
20
30
50
110
20
30
40
50
80
110
Training size
Training size
Figure 6. Predictive scores (mean + sd over 10 reps) of GP models versus training set size.
two columns. Specifically, for any Vec (a1, a.),
with equivariant kernel Kn, considering the fundamental region of x2 on D:
(ln x3 x) pa(=)) f E
{{(0
(0.
where *y stands for the action of z/2Z that swaps (for I) the last and penultimate atom position vectors as and dx-1-
A=
0,
012 0.
221
222 0 : o + a < d
ER
To define section for point Ø = Vec (ā,, .2) E δ, we apply a rotation v/ (ã) E SO(3) that maps ã1 to (0, 0) with = max/ri, r2 and 22 to (CI, C22 0) with >
Naive baseline As a baseline kernel, we take diagonal squared exponential matrix-valuee kernel:
0. This rotation can be represented as a product three elementary rotations in 50(3): ( I W,
K)::0): D 3x3
(x, x + 0 1
The corresponding projection map is given by
13+
II ( (0, 0, c,,,0,,).e,00))
where 0 0= (c,o?) denotes the vector containing the tunable kernel lengthscale and variance hyperparameters. For ease of notation we write K1 = K1 (-; 0). We use a single length- scale and variance, as there is no reason to assume different marginal variances for the components of μ., given that they all depend on the intrinsic geometry of the position vectors æ. Analogous experiments with separate lengthscales and variances did not lead significant changes in the results.
We obtain KI on Ó as
Kn .') = (x) K (II, (L'))
Finally, a kernel of that is invariant under x1 and and equivariant under *2, is given for æ, x' E D by:
K (ax x²) Kn(Ø((), 6(x'))
Constructing a tailored matrix-valued kernel We as- sume that p(æ) = f(Ø(z)), where f: D stochastically equivariant GP on Ú 1g The map
An illustration this procedure shown in Figure 7. Note that the choices of and v not unique. Here, K follows the same form as K1 (-: 0), with inputs II, (A,) parameterized in (a subset of) R (by and C2).
3 is
Φ: D + D is defined by (xx))
7(A(x)) *3 A(x)
with A D - D defined by A
Vec (ā,, a2)
a1) and 7: D + 2/22 is defined by 7(ā)= = lff 2 72 (and 0 otherwise), where r; = ||a,|]
Vec (22 a1,
Experimental Results We consider a dataset of dipole moments f obtained from 850 water molecule configura- tions I, which were computed using quantum chemical methods. For details on the dataset generation process, see
( (1,2)). & can be checked to be invariant under *1 and *3 and equivariant under *2. We model of as centered GP
7

---

Integration-Fes free Kernels for Equivariant Gaussian Process Modelling
Preliminary learning curves in Appendix J on newly ob- tained dipole moment dataset of 21, 21,000 N-Methylformamide molecules of 9 atoms highlight the significantly improved predictive performance on test sets of size 500 of the equiv- ariant Gaussian process over its base GP, particularly in data-scarce regimes (n < 1000) where structural priors are crucial. Sparse GP modeling on the full data set is part of ongoing work.
d1
a2
1
r2
a1
y
5.3. Ocean data with equivariant noise
C1
We finally investigate the performance of weighted combi- nations of (rotation- Setttiirnnneiiviaiant and non-equivariant kernels combinations of vector fields with equivariant perturha- tions in the case d p = 2.
C2
Figure 7. Illustration of the rotations for Kn--
Appendix H. To enhance the dataset's diversity and rep- resentativeness, we applied random rotations to the data points. See Figure 15 of the Appendix for a visualisation of this augmented dataset.
We compare the predictive accuracy of GP models for the dipole moments of water molecules using the baseline ker- nel K1 and the proposed kernel Kn- See Appendix I for visualisation of the optimised parameter values, sensitivity to initialisation, and additional information about our train- ing schemes. For additional comparison, we can include invariances separately with the following kernels:
Figure 8. Gulf data with 50( variation
In experiment (see the visualization in Figure 8),
F(x) a Fof (x) + (1 - ()F a E 1]. (8)
F represents ocean drifter velocities on a set of 564 loca- tions in the Gulf of Mexico, as taken from the Gulf Drifters Open dataset Lilly & Pérez-Brumics (2021), after standard- izing x and FC (xx). and
Translation-ivaarinne::
K2(0, ((A()))
Permutation--ivarianne:
P((x))
0.5 + |
Ka(æ, x²) = (II,, (æ), II,, (x))), II, (x) = 17( (x)) *3 x,
In Table we compare model fits to F, using 1000 replicates of 100-observation training sets and a 0.2 train/test ratio, of five different GPs:
Permutation and translation-invarianee:
Ka Ka x') = (1(())).((z)))
1. GP (0, KSE),
GP (0,72Ks + (1 -)
In Figure 6, we see that incorporating structural knowledge into kernels consistently improves predictive accuracy. The proposed integration-free free equivariant GP significantly out- performs the other GPs across all training set sizes, the small order of magnitudes of the RMSE suggest that our proposed model is accurate enough for use in quantum chemistry.
3. GP KH),
4. GP 2 (1- and
5. GP (0, 2 KH + (1 (1-7)²km)
Remark 5.4. The construction of the argument-wise rotation-equivarian and translation-invan invaaiiann kernel K for the dipole moment prediction task in (65) transfers analogously to molecules of larger numbers of atoms s.
Here KH is the kernel matrix function which enforces the Helmholtz GP to be fully potation-equivariin bytaking KHelm as K, in the integration- kernel of Ex- ample 4.3. The mixing coefficient ? E [0, 1] is an additional
obtained
8

---

Integration-fre Kernels for Equivariant Gaussian Process Modelling
Table Mean performance metrics and [standard deviation].
Best scores are in bold, and values within the standard deviation of the best
score given an asterisk (*),
GP
a
0.8
0.6
0.4
0.2
0
RMSE
0.787 [
0.630 [0.008]
0.476[0.022]
0.323 [0.016]
0.168 [0.010]
0.021 [0.007]
1
LogS
0.962 [0.082]
0.035
[0.096]
-1.111 [22]
-2.617
[0.
-5.292 [0.085]
-18.872 [1.153]
RMSE
0.766 [
0.617
[0.02]]
0.468 [
0.317
[0.016]
0,166 [0.009]
0.021 [0.014]
2
LogS
0.857 [
0.003 [0. 16
-1.121 [0.163]
-2.738 [0.162]
-5.420 159]
-20.317 [4.1
RMSE
0.787 [
0.629 [
0.471 [0.022]
0.316[0.015]]
0.162 [0.008]* 1*
0.015 [0.005]*
3
LogS
0.932 [0.087]
0.001 [0.098] *
-1.201 [0.117]
-2.829 [O. 133
-5.537 [0.140]*
-13.536 [0.022]
RMSE
0.818 [0.050]
0.644 [0.038]
0.490 [0.030]
0.340 [0.032]
0.192 [0.026]
0.011 [0.006]
4
LogS
1.231 [0.343]
0.264 [0.289]
-0.798 [0.265]
-2.258 [0.366]
-4.623 [0.545]
-13.763 [0.059]
RMSE
0.761 [0.031]
0.608 [0.023]
0.458 [0.018]
0.310 [0.012]
0.158 [0.006]
0.021 [0.004]
5
LogS
0.773 [0.172)
-0.137 [6.164]
-1.302 [0.171]
-2.898 [0.166]
-5.541 [0.093] -7.308 [0.005]
kernel parameter optimised in the maximum likelihood set- ting, with initial value 0.5.
shown to allow for expressive kernel combinations and ob- tain competitive probabilistic predictions for ocean velocity data with equivariant perturbations.
The results presented in Table allow to compare models both in terms of point predictions (via the RMSE) and of probabilistic predictions (via the logarithmic score). Combi- nations of equivariant and non-equivariant kernels appear to consistently yield better performances than the use of single non-equivariant kernels alone. This stands out in particular for the logarithmic score, which underlines the importance of fine-tuning GPs in terms of distributional properties be- yond the resulting posterior means. In this regard, the fifth combination, which stands out for most values of a, appears as intriguing blend between GPs equivariant in the mean and stochastically equivariant. Future numerical experi- ments will aim at exploring this and further combinations more extensively.
Future work includes scaling up equivariant GP modeling to large molecule datasets like the full N-Methylformmmiie dataset using sparse equivariant GPs. Beyond scaling to larger data, we also aim to extend our approach to other nat- ural and artificial systems, broadening the applicability of equivariant GP modeling in scientific research. Considering proposed kernels within wider machine learning pipelines providing prediction of molecular properties or active leam- ing (e.g. Moss & Griffiths (2020); Griffiths al. (2024)) could be of interest. Also, further exploring mathematical properties of those kernels and investigating potential syn- ergies with recent developments pertaining to kernels on graphs, Lie groups, and other structures (See, e.g., Azan- gulov et al. (2024)) are of interest.
6. Conclusion and perspectives
Data availability and code We provide the dipole moment data from Section 5.2 along with notebook containing the code necessary to reproduce the experiments in this Github repository.
We presented theoretical framework for stochastically equivariant second order random fields and introduced method to construct class of equivariant random field models by leveraging fundamental regions, that avoids cum- bersome group integration. It is worth noting that neither integration- based nor fundamental region approaches can be declared universally superior for stochastically equivari- ant GP modeling. While the former offers an elegant con- struction principle, the latter provides a fast and practical alternative that still satisfies the equivariance requirements and enables tackling challenging prediction tasks. Our ex- periments on rotation- -quiiiin-quivariiant synthetic and real-world data show that the proposed approach enables obtaining lightweight GP models honoring prescribed equivariances, leading to benefits both on the computational side and in terms of probabilistic prediction performance. Our approach was found in particular to allow for efficient predictions of dipole moments of water molecules by incorporating physi- cal principles directly into the GP framework. It was also
A Acknowledgments
The authors would like to thank all people having evalu- ated and provided feedback on this work for comments and suggestions having led to substantial improvements. Special thanks to Sebastian Baader and Jan Draisma for useful feedback pertaining to the terminology of funda- mental regions. GP calculations were performed on UBE- LIX http:://www.id.uibe..hhhccc University of Bern. Tim Steinert and David Ginshourger acknowledge the support of the Digitization Commission (DigiK) of the University of Bern via the project "Percep- tion in Statistics, Econometrics and Probability". Part of this research was performed while Tim Steinert and David Ginsbourger were visiting the Institute for Mathematical and Statistical Innovation (IMSI), which is supported by the
the HPC cluster at the

---

Integration-fres Kernels for Equivariant Gaussian Process Modelling
National Science Foundation (Grant No. DMS- 192934.2.48). David Ginsbourger would like to thank the Isaac Newton Institute for Mathematical Sciences, Cambridge, for support and hospitality during the program Representing, calibrat- ing & leveraging prediction uncertainty from statistics to machine learning, where work on this paper was undertaken that was partially supported by EPSRC grant EP/Z000500/1 and by grant from the Simons Foundation. The work of August Lykke-Möller and Ove Christiansen was sup- ported by the Danish National Research Foundation through the Center of Excellence for Chemistry of Clouds (Grant Agreement No: DNRFI72). Ove Christiansen also acknowl- edges support from the Independent Research Fund Den- mark through Grant No. 1026-00122B. The research work of Henry Moss was supported through Schmidt Sciences, LLC and Lancaster University's Mathematics for Al in Real- world Systems E3 grant.
matter simulations. Journal of Chemical Physics, 152 (18):184107, 2020.
Bartók, A. P., Kondor, R., and Csányi, G. On representing
chemical environments. Phys. Rev. B, 87:184115, 11,,, May 2013.
Berlinghieri, R., Trippe. B. L., Burt, D. R., Giordano, R.,
Srinivasan, K., Özgökmen, T., Xia, J., and Broderick, T. Gaussian processes at the helm(holtz): a more fluid model for ocean currents. In Proceedings of the 40th In- ternational Conference on Machine Learning. JMLR.org, 2023.
Bigi, F., Pozdnyakov, S.N., and Ceriotti, M. Wigner kemnls:: Body-ordered equivariant machine learning without a basis. The Journal of Chemical Physics, 161(4))044116, 2024.
Califano, S. Vibrational States. Wiley, 1976.
Impact Statement
Christiansen, O., Artiukhin, D. G., Bader, F. D., Godtliebsen,
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which feel must be specifically highlighted here.
I. H., Gras, E. M., Györffy, W., Hansen, M. B., Hansen, M. B., Højlund, M. G., Høyer, N. M., Jensen, A. B., Klinting, L., Kongsted, J., König, C., Madsen, D., Madsen, N. K., Monrad, K., Schmitz, G., Seidler, P., Sneskov, K., Sparta, M., Thomsen, B., Toffoli, D., and Zoccante, A. Midascpp, version 2024.04.0, 2024.
References
Aslan, B., Platt, D., and Sheard, D. Group invariant ma- chine learning by fundamental domain projections. In Proceedings of the 1st NeurIPS Workshop on Symmetry and Geometry in Neural Representations, volume 197 of Proceedings of Machine Learning Research, pp. 181-218. PMLR, 2023.
Cohen, T. and Welling, M. Group equivariant convolutional networks. In International conference on machine learn- ing. pp. 2990-29999999 PMLR, 2016.
Dunning, T. H. J. Gaussian basis sets for use in correlated molecular calculations. i. the atoms boron through neon and hydrogen. The Journal of Chemical Physics, 90(2): 1007-1023, 1989.
Azangulov, I., Smolensky, A., Terenin, A., and Borovitskiy, V. Stationary kernels and gaussian processes on lie groups and their homogeneous spaces ii: non-compact symmet- ric spaces. Journal of Machine Learning Research, 25 1281):1-51, 2024.
Ginsbourger, D., Bay, X., Roustant, O., and Carraro, L. Argumentwise invariant kernels for the approximation of invariant functions. Annales de la Faculté des sciences de Toulouse Mathématiques, Ser. 6, 21(()):5115555 2012.
Bak, K. L., Gauss, J., Helgaker, T., Jørgensen, P., and Olsen, J. The accuracy of molecular dipole moments in stan- dard electronic structure calculations. Chemical Physics Letters, 319(5):563-566 2000.
Ginsbourger, D., Roustant, O., and Durrande, N. On de- generacy and invariances of random fields paths with applications in gaussian process modelling. Journal of Statistical Planning and Inference, 170:117-128, 2016.
Balasubramani, S. G., Chen, G. P., Coriani, S., Diedenhofen, M., Frank, M. S., Franzke, Y. J., Furche, F., Grotjahn, R., Harding. M. E.. Hättig, C., Hellweg, A.. Helmich- Paris, B., Holzer, C., Huniar, U. Kaupp, M., Khah, A. M., Khani, S. K., Müller, T., Mack, F., Nguyen, B. D., Parker, S. M., Perlt, F., Rappoport, D.,, Reiter, K., Roy, S., Rückert, M., Schmitz, G., Sierka, M., Tapavicza, E., Tew, D. P., van Wüllen, C., Voora, V. K., Weigend, F., Wodynski, A., and Yu, J. M. Turbomole: Modular pro- gram suite for ab initio quantum- chemiccll and condensed-
Glielmo, A., Sollich, P., and De Vita, A. Accurate inter- atomic force fields via machine learning with covariant kernels. Phys. Rev. B, 95:2 143224 Jun 2017.
GmbH, T. Turbomole v7.8. A development of University Karlsruhe and Forschungszentrum Karlsruhe GmbH, 1989-2007, TURBOMOLE GmbH, since 2007, 2023.
Griffiths, R.-R., Greenfield, J. L., Thawani, A. R., Jamasb, A. R., Moss, H. B., Bourached, A., Jones, P., McCorkin- dale, W., Aldrick, A. A., Fuchter, M. J., et al. Data-driven
10

---

Integration-fre' Kernels for Equivariant Gaussian Process Modelling
discovery of molecular photoswitches with multioutput gaussian processes. Chemical Science, 13(45):13541- 13551, 2022.
Reisert, M. and Burkhardt, H. Learning equivariant func- tions with matrix valued kernels. Journal of Machine Learning Research, 8(15):8854008 2007.
Griffiths, R.-R., Klarner, L., Moss, H., Ravuri, A., Truong, S., Du, Y., Stanton, S., Tom, G., Rankovic, B., Jamasb, A., et al. Gauche: a library for gaussian processes in chemistry. Advances in Neural Information Processing Systems, 36, 2024.
Scheuerer, M. and Schlather, M. Covariance models for divergence-free and curl-free random vector fields. Stochastic Models, 28:433-455 07 2012.
Stone, A. The Theory of Intermolecular Forces. Oxford University Press, 01 2013.
Grisafi, A., Wilkins, D. M., Csányi, G., and Ceriotti, M. Symmetry-adane adaated machine learning for tensorial proper- ties of atomistic systems. Phys. Rev. Lett., 120:036002, Jan 2018.
Toffoli, D., Kongsted, J., and Christiansen, O. Automatic generation of potential energy and property surfaces of polyatomic molecules in normal coordinates. The Journal of Chemical Physics, 127(2))2204 106, 11 2007.
Henderson, PDE constrained kernel regression meth- ods. PhD thesis, INSA Toulouse; Université de Toulouse- Toulouse IIIUPSS 2023.
Uteva, E., Graham, R. S., Wilkinson, R. D., and Wheatley, R. J. Interpolation of intermolecular potentials using Gaussian processes. The Journal of Chemical Physics, 147(16):161706,, 06 2017.
Holderrieth, P., Hutchinson, M. J., and Teh, Y. W. Equiv- ariant learning of stochastic fields: Gaussian processes and steerable conditional neural processes. In Interna- tional Conference on Machine Learning, pp. 4297-4307. PMLR, 2021.
van der Wilk, M., Bauer, M., John, S., and Hensman, J. Learning invariances using the marginal likelihood. Ad- vances in Neural Information Processing Systems, 31, 2018.
Israelachvili, J. N. (ed.). Intermolecular and Surface Forces. Academic Press, San Diego, third edition edition, 2011.
Veit, M., Wilkins, D. M., Yang, Y., DiStasio, Robert A., J., and Ceriotti, M. Predicting molecular dipole moments by combining atomic partial charges and atomic dipoles. The Journal of Chemical Physics, 153(2):024113, 2020.
Kingma, D. and Ba, for stochastic optimization. CoRR, abs/1412.6980, 2014. URL https:/ /tppaaap.pa/// tttaaapi.aaaaaaa sen.ssmmmaiicchhll.... otr//CorpusID:6628100
Adam: method
Weigend, F. and Häser, M. RI-MP2: first derivatives and global consistency. Theoretical Chemistry Accounts, 97 (1):331-340, 1997.
Lenz, R. Group theoretical methods in image processing, volume 413. Springer, 1990.
Lilly, J. M. and Pérez-Brunius, P. GulfDrifters: A con- solidated surface drifter dataset for the Gulf of Mexico (1.1.0), 2021. URL https: orgoorgg///ooo 0./.0....0000 5281/ zenodo. , 4 421585.
Moss, H., Leslie, D., Beck, D., Gonzalez, J., and Rayson, Boss: Bayesian optimization over string spaces. Ad- vances in neural information processing systems, 33: 15476-15486, 2020.
Moss, H. B. and Griffiths, R.-R. Gaussian process molecule property prediction with flowmo. arXiv preprint arXiv:2010.01 118, 2020.
Møller, C. and Plesset, M. S. Note on an approximation treatment for many-electron systems. Physical Review, 46:618-622, 1934.
Ranković, B., Griffiths, R.-R., Moss, H. B., and Schwaller, P. Bayesian optimisation for additive screening and yield improvements beyond one-hot encoding. Digital Discov- ery, 3(4):654-66)) 2024.

---

Integration-fre Kernels for Equivariant Gaussian Process Modelling
Appendix
A. Preliminaries
A.I. Multivariate random field
Let PP be a probability space. We denote by IR IR--velued random variable measurable mapping
V (2, FF - (R", (R",)(R)))
where (TRP) stands for the Borel sigma-algebra IRP. We call a R R--valued random variable V square-integrable whenever E[| [1+) < co, where III stands for the Euclidean norm in 12,, For such square-integrable V, there exist vector m R2 and positive semi-definite matrix K E RPxP such that
= =m, E V = -BIV((01)*
i=1
and
[((), )( 1S1.SP =K,
Cov [V]
Cov
where (+) denotes the i- component of the random vector V.
Now, for D, a - random field is collection of R2- - random vectors defined on the probability space PP), indexed over D, that denote by Z (Z=)mED- We call Z square-integrable, if for all æ E D, Z is square-integrable in the above sense. Then there exist mappings m: D - RR" and K: D D : R2x defining the expected value at any x E D by
m(x = R [a]
and the matrix-valued kernel of Z describing cross-covariances by
K(x, x') = Cov (z Dz))s
where æ, x' The kernel K needs to satisfy K(x', æ.) K(x, x¹) (for any x, E D) and be positive-sems definite, meaning that, for any 1, a1,..... On RP, and $1...., . nn D, holds
Σ a K (2,, 2y)a 2 0.
1StjSn
Throughout this work, we consider the index set D to be a subset Rd, where 1.
A.2. Multivariate Gaussian random field
Z is said to be a multivariate Gaussian random field if for any n 2 1, and 3T Un E D, Vec ......., Zzn) has
multivariate Gaussian distribution. In what follows, we think Z Vec Z Z....... Z Zan) as responses at set training points denoted X (......, Fnl E Rdx Further denoting by 2tr vector standing for observed realization of Z use the notation
D, {xx;, 2z, )
While the x, points are not considered as random here, we use a D,, subscript in forthcoming equations as a shorthand notation to summarize the information needed condition Z based on the considered n training evaluations of Z (i.e. to condition Z on the event "Vec Z....... Zan) =
Prior to any observation, the distribution of Z.r is N (m (Xtr), K (Xt+)), where
m (Xxx) (x)) (xn) m (xx))...............
m(() (x)
(9)
and
KKi (Xor)
Kip (Xur)
K (Xor)
E ppp
(10)
Kp1 (Xer)
Kpp (Xer)
12

---

Integration-fre Kernels for Equivariant Gaussian Process Modelling
For i, S p, the block matrices in (10) are given by
R e Kiy (Xtu) = (KK (11 Ist.mnn
Similarly, the cross-covariance for any two sets of locations
X1 = [.i,...., Inll E Rdxms
X2 = [z'1, x'nol E Rdxn2
given by
K11 (x)
Kip (x)
K (x)
:
R pnj Xpn2
Kpt (X1,X2)
Kpp (1,
where
R Xm2 > Kig X2) (IK(2), x' m))is) igis 15m5ng
Posterior distribution For a centred Gaussian random field Z and test locations Xte. the posterior distribution of Zoo (defined analogously to Zu in terms of Xte instead of Xxx) given Dn is thus characterised by the posterior mean
mD, (Xte) K (Xte, X1t) (Xu)) Ztr
(11)
and the posterior covariance
KD. =K()ee)))))) - K(Xto,
((x,) (XX,,, X10)-
Observation Noise To account for observation noise, we can augment the training covariance matrix by adding the noise covariance matrix E. This results in a modified covariance matrix:
K (Xtr) E.
For i.i.d. normal observation noise with a common variance Ld to the components, the noise covariance becomes a scaled identity matrix, so the training covariance matrix is given by:
K (Xu) + o T
(12)
A.3. Training of the Gaussian Process
For valuix-valued kernel Ko parametrised by 0 E IR9, at which Ko is invertible, we tune the kernel parameters O using the maximum likelihood approach, i.c. minimizing the negative twice log- ligeliiel (n2ll) given in noise-free settings by
1(0) = 2 Ko (Xa))- Zir + log Ke (Xt))l + 2n log 2n.
(13)
In our experiments, training is performed using the gradient- gradient-based Adam optimiser. The gradient of (13) with respect for values at which Ko (Xtz) is differentiable and invertible is given by
(V)(()), 2,, K5 (Xtr) kKe (Xx) Ko1 (Xtr) žtr - tr KKo (X.x) 00,
oKg (Xxz)
00,
In the noisy setting (12), the n2ll and its gradient are computed analogously with the observation noise additional tunable kernel parameter. The parameters O are optimized using maximum likelihood estimation with the bassed Adam optimizer Kingma & Ba (2014). For experiment 5.1, the optimization is run for 1000 iterations with learning rate of 0.01, starting from the initial values Ont (1, 1, 1, 1, 0.1).
To evaluate the predictions, we measure the average magnitude of prediction error on the test set, given by the RMSE:
RMSE
= - m.p,, (X
(14)
The
where Zte E 1RP is the stacked vector of observed test set responses, and mDD (Xtc) is the GP posterior mean at locations Xte, as defined in Eq. (11) of Appendix A.
To measure the probabilistic predictive accuracy, we use the average logarithmic score LogS. For given train/test split, the LogS is defined by the logarithmic posterior density of the test data.
13

---

Integration-free Kernels for Equivariant Gaussian Process Modelling
A.4. Group and representation theory
In this section, we outline the essential background on groups and group representations required for constructing equivariant kernel matrix functions. Our discussion is primarily based the group theory framework in Reisert & Burkhardt (2007)
and fundamental regions in Ginsbourger et al. (2012). For clarity and consistency, we harmonize some notations. Definition A.1. A group (G, o) is set G equipped with binary operation
o:GxG-G. (a. + a
which satisfies
1. Va, (bbbb))
2. Be G Va
3. Ya G. Gst. =aa-oaa==aaaa
Definition A.2. If the set G is furthermore topological space and the group operation o as well as its inverse o are continuous with respect to the topology of G, we call G topological group. In addition, topological group G is compact if it is compact with respect to its topology, i.e. if each open cover of G admits a finite subcover.
Definition A.3. A group representation maps elements from a group to L(V) where V is a finite dimensional Hilbert space and L(V) the space of linear transformations on V. Furthermore p is homomorphism, i.e. for any 91, 92 E G, p(g1 92) p(g))p((2)
Remark A.4. There exist definitions of group representations where V is infinite dimensional. However, these cases go
beyond the scope of this paper. which focuses exclusively on finite-dimension representations.
Definition A.5. A group G is called a linear group if there exists injective homomorphism & G + GL(v, F) to the general linear group GL(p, F)) for some integer and some field E, such that the image @(G) is a closed set in the natural topology on (L, which corresponds to the topology induced by the standard norm on KP.
As consequence, linear group admits invertible matrix-valued representations.
Definition A.6. A linear group representation is called unitary for any g E G it holds Pa-1= =p.
Remark A.7. Since our work focuses on the case IF R. unitary representation equivalent to an orthogonal representation, meaning that for any g E G. it holds that Pg- PP-
Following Reisert & Burkhardt (2007), it can be shown that any representation of finite group or of compact continuous group with continuous representation is equivalent to unitary representation.
Definition A.8. A measure p defined on the a- alalgebra generated by the open sets of a compact group G, called the Borel algebra of G, is called left cuaslation-imvariian if for any open subset S C G and E G it holds p(gS) = ((S), where gS [g s|s E S}.
By Haar's theorem, on any compact group there exists a unique left translation-invariant measure, called the left Haar measure. Analogously, there exists unique right translation-invariant measure, called the right Haar measure.
Definition A.9. A compact group is said to be uninodular, its right and left Haar measure coincide.
It can be shown that representations of compact, linear, unimodular groups have determinant one, which simplifies reparametrisations in Haar integrals Sc fg) dg.
Definition A.10. left group action of group G on a set X is a mapping
b: G X +
(g.x) x
satisfying for any X. G
1. exx
2. g * (h x x) = (g h) x
14

---

Integration-free Kernels for Equivariant Gaussian Process Modelling
Fundamental regions
Definition A.11. The orbit of point E X under the action of G on X is the set
(x))
G}.
Definition A.12. The stabilizer of a set S C X in G is defined by
Stab4 = {g E G Va E =
Definition A.13. Let X be a topological space. We call subset A C X a fundamental region for the action * if the
following conditions hold:
1. G A
2. (g* A))A)A=00 for all g E G {e)-
Remark A.14. A fundumental region A is a subset of X that intersects each orbit under the action of G at most once. That
is, for any two distinct elements y E A, there is no group element E G such that x y. Definition A.15. Given a fundamental region A, we call section any mapping s: X - G, satisfying for all x X,
s(x) * x A.
We further define the associated projection map by
I1; X A,
s(x) x.
Remark A.16. By definition of A, the restrictions to G A of the section and therefore also of the projection map II, uniquely defined. If the group action is free (i.e., 9 x = x implies for all x E X), then and II, are unique.
B. More on the continuity of Kn in specific applications
B.1. Continuity of Kn in Experiment 5.1
In Experiment 5.1, was chosen be the positive x-axis and with the section for non-zero æ
coos (0(z)) -in (0(=)))
Pa(a) sin (0(z)) cos (f(x)),
and (0) = 1z. The corresponding projection map is thus II, = (1|=||2, 0). With 0(x) arctan (z(21/z((3) Ps discontinuous on {0} x E and on {(-2, 0), 0}, there exists no subset B A for which P. is continuous on G B.
However, if we use non-angular parametrization of the representation of i.e.
z
(2))
Ps(w)
| 2
(2)
(1)
SO(2),
(15)
then for any subset B C. A, is continuous on G B, as discontinuity occurs only at 0 E ĐA. Furthermore, II, is continuous on * B and K is continuous on B x B. Thus, by Proposition 4.4, Kn is continuous on (G * A) x (G A).
B.2. Continuity of Kn and K in Experiment 5.2
For T = Vec (ā1, a2) E D, the section s()) a rotation composed of three elementary rotations (M,(#)) C SO we parametrize the respective 2 ddimensionnl rotation components of V. V
angularly in (15), = I__ v is continuous except at z Vec (0, 0), which lies in the boundary of A. By definition of A, holds for any subset B c that Vec (0, 0) & + B, hence Ps is continuous on G B. Since II, (#) = Vec (Pa(z) á1, P=(z)ā2) II, is continuous on G * B for any B C A. As K continuous on A, it follows from Proposition 4.4 that Kn is continuous on (G A) (G A).
Now, & may be discontinuous on the subset of D of Lebesgue measure zero given by Eg fxr D ((x) E II (8A)). Thus, & continuous on (D Eo) (D) \ Eo).
15

---

Integration-free Kernels for Equivariant Gaussian Process Modelling
Mean log(RMSE) Scores
Mean LogS Scores
Time Reynolds/Time
Reynolds Ka Kn
3.6
3.4
-38.5
3.2
Time
-7
-8
-39.5
20
40
110
30
50
110
20
40
80
110
Training size
Training size
Training size
Figure 9. Comparison KB and original Kn by the predictive scores the ratio of computation times (in seconds).
Extending continuity using the Reynolds operator We can use the Reynolds operator with respect to *3, to con-
struct fundamental region kernel which is discontinuous only on subset of Es x Eg. For this, denote by KЛ (. Kn (.), (.)) the *z-equivariant and =ı-invariant kernel obtained by omitting the permutation of ā₁ = 02 01 and 22 ag 01 in the construction of Φb We can ensure invariance under *y by applying the Reynolds operator for Z/22 on K K resulting in the -,,-iinvariant, kz-equivariant kernel K RoA given by
KR (x, x')
1 Σ 4
KA (g x. x'))
(16)
9:9'EG
Without permuting ã and ā2, II, maps onto the fundamental region
==((0
=== (0.500 20 20
A 4-=1
212 O.
#21 222, 0
D. 221, 212 0 o
with the same section 8. The points of potential discontinuities of KÅ thus reduce to E EA, where & (a E A (a) E II, (AA*)} C Eg. As the employed Reynolds operator may not introduce additional discontinuities, it follows that KA and K AoA both continuous (D {EA) (D \ &a).
Comparing the GPs GO, K and GP(O, k ( by their learning curves in Figure 9, we see that continuous K RoA provides improved predictive performance, requiring a moderate multiple of K 's computation time.
C. Proofs
C.1. Proof of Proposition 4.1
Proposition 4.1 Let G be linear group acting on D via *, possessing a unitary group representation p : E G +
Pg TRPx, and let A D be fundamental region of Then, for any matrix-valued kernel K A Á x Á, section and associated projection II,, Kn below defines matrix matuix-valued kernel equivariant (w.rt and p) on (G A) x (G A):
Kn( x.) = PI(aa) KA (II, (2))))))s
16

---

Integration-free Kernels for Equivariant Gaussian Process Modelling
Proof. (a()) Kn (a', T)) holds for any a. x'' E D, and positive semi-definiteness follows directly as for any 1, ..,...; On IRP, and a1.-.... Zn E D,
Σ a (n)(, ISiJEn
Σ ai Pala.)
K (L)))) (((s))))
15ijsn
Σ
K ( (L))
15ijšn
2 0,
where b, P,( 0z and the last inequality follows from positive definiteness of K Now, let (x, (G A) (G A). Since the projector II, is constant on the orbits of æ and æ', it holds for any 9.h G:
KI xx, Palgra)) K (T az), (1 * x') ((thaa))
Pa(gen) Ka (z). (1, (a)))) Ps(heee))
Its straightforward to see that s(g * xx) = s(x) o og and thus Ps(gaæ) = Pa(w)) Hence,
Kn(9 * x, h x²) = PaPs(an) K (x)
Pg K (x, x )hh
C.2. Proof of Corollary 5.1
Corollary 5.1 Assume Gaussian random field Z and group G satisfy the assumptions Theorem 3.1, with the kernel of Z being equivariant (2). Then, given any observed realization 2tr of Zt (whereby the notation Appendix A is used), the resulting posterior distribution retains stochastically equivariant, i.e..
Va E D, 9. g = Zg Zt = Ztr) = 1.
Proof. For simplicity assume the noiseless setting, as the case of present observation noise is analogous. For test point x E D and the training set Xte. we have cross-covariance matrix of the form
(sx, X.r) [K( x1 ((x 2n)) RPXP"
Since K satisfies the equivariance (2), it follows that for E
(2, K(x K( xx,),,
and therefore it holds the equivariance of the posterior mean
mpn (g * x)) = Pg MpD (x:).
Furthermore the equivariance (2) of K transfers to the posterior covariance K Let x E D, and g, h G, then
Kp (g * x, x) PyKK (x,
Hence, analogously to the proof of Theorem 3.1, it follows that
Cov Cov(Z (Z
Py Zy Z 9xx0 Zo Zu: 21,) 0.
17

---

Integration-free Kernels for Equivariant Gaussian Process Modelling
D. Experiment 5.3 continued
extend Experiment 5.3 the case where F ~ GP(O, k*), is realisation of an equivariant Gaussian process, i.e. K I. Under the assumption that the ocean drifter velocities F are realisation of second-order centred Gaussian process, we can study the ability combinations of GPs with non-equivariant and equivariant kernels to recover the mixture parameter parameter estimation problem.
To analyze the parameter estimation of a with a combination of the squared-exponential kernel K and the equivariant squared exponential kernel Kπ (resulting in GP 2 below), we generate realisations of
a + (1 - (O) Km).
(17)
Analogously, we consider the case of the equivariant part coming from the fully equivariant Helmholtz GP with K defined in Experiment 5.3:
FH oF + (1-a) (P(O,))
(18)
Figure 10 presents realisations of the random vector field (18) for different values a. This experiment considers four Gaussian process models evaluate the performance single, non-equivariant kernels against combinations of equivariant and non-equivariant kernels and the ability to recover the mixture parameter a through an additional tunable mixture parameter 7 E [0, 1] of such combinations:
1. GP (0, KSE),
2. GP (O,>>KsE + (1 - 7)2 Km),
3. GP (0, KR), and,
4. GP (0, 2H KH + (1 - x))
0.875
a-6.225
0.5.
0.375
0.255
0.125
Figure 10. Realisations of Fu for different values of a. Red indicates the training set of size 100.
18

---

Integration-fre Kernels for Equivariant Gaussian Process Modelling
Table Mean performance metrics and [standard deviation]. Best scores are in bold, and values within the standard deviation of the best
score given an asterisk (*).
GP
0.8
0.6
0.4
0.2
0
RMSE
0.781 [0.033]
0.634 [0.030]
0.501 [0.048]*
0.397 [0.127]
0.306 [0.197]
[0.255
1
LogS
(..16(0.144)*
-0.123 [0.185]*
-1.037 [0.434]
-2.154 [1.090]*
(2.00(2.184)
-4.273 [2.688]
Bias
RMSE
0.782
[0.033]*
0.631 [0.029]
0.482 [0.031]
0.334 [0.048]
0.177 [0.048]
0.039 [0.096]
2
LogS
0.711 [0.138]
-0.147 [0.
-1.20 [0.331]
-2.691 [0.722]
-5.008 [0.927]
-6.975 [1.718]
Bias
-0,265 100]
-0.124 [0.090]*
-0.060 [0.090]
-0.027 [0. 100]*
-0.018 [0.100]
0.014 (0.020)
RMSE
0.766
[0.030]
1.017 [1.237]
[3.7153
1.809 [3.656]
2.278 [5.015]
3.416 [7.64]]
3
LogS
0.863 [0.135
1.184 [3.798]
2.126 18.3
1.083 189
0.270 [8.439]*
1.029 [10.986]
Bias
RMSE
0.768 [0.031]*
0.770 [0.721]
1.216 [3.32]]
0.931 [2.577]
1.174 [4.278]
1.928 6.00
4
LogS
0.855 [0.145]
0.359 [1.844]
0.166 [5.398]
-1.584 [4.291]
-3.511 [6.079]
-4.407 [8.403]
Bias
-0.139 [0.120]
-0.054 [0.140]
-0.068 [O. 130]*
-0.022 [0.180]
0.002 [0.140]
0.062 [0.210]
Table 3 summarizes the average RMSE, LogS, and bias scores across six values of a, evaluated over 1000 samples of 100
training points and 464 test points. The kernel parameters (l1, TT, l2. 02) of Kn and Kμ were uniformly sampled between O and 2. The optimization of the kernel parameters 0 = (f1, 01, l2, 022 Jats a) was performed using maximum likelihood estimation with the Adam optimizer over 1000 iterations and learning rate of 0.01.
We observe that for both random vector fields F and FH. the corresponding combined GPs 2 and 4 consistently achieve lower logarithmic scores compared to the single non-equivariant kernel GPs. The combined GPs also appear to generalise better than the single non-equivariant GPs and 3 in terms of point predictions, with lowest RMSE scores across all investigated values of a but a I, where the difference in scores insignificant. This equivalence of point predictive accuracy is explained by no additional equivariance being present in the case a
Both combined models (GP 2 and GP 4) thus suggest strong evidence that incorporating equivariance into the model leads to better predictive performance, with the combination of non-equivariant and equivariant kernels appearing to provide a flexible framework that adapts well to varying levels of equivariance in the data.
Figure 11 presents the distribution of the estimated y, showing that both GP 2 and GP correctly identify the fully equivariant case a = 0. However, as a increases, GP 2 tends to estimate the mixture parameter more accurately for S 0.6, while GP 4 performs better for a 0.8 and a 1. Increasing the number of training iterations to 3000 reduces the uncertainty in r. as seen in the lower panel of Figure 11, confirming the benefit of extended training for more accurate parameter recovery.
This experiment demonstrates that incorporating equivariant kernels into Gaussian process models noticeably improves performance in tasks involving physical systems with inherent equivariance for better generalization and predictive accuracy.
E. Pathological choices and their effect of fundamental regions
In Section 5.1, we diseuss a shortfall of the fundamental region approach, which can occur for unfortunate constructions of the fundamental region. The left panel of Figure 12 illustrates the connected construction of A from Example 4.3, for a SO( equivariant KIT taking values from R² 2RR2. The right panel shows a disconnected construction of a fundamental region using an alternating partition of the x axis, which lead to reduced performance. Figure 13 shows the posterior means of two fundamental region GPs learned on samples of observations of the SO(2) (()))))-uavriian random vector fields in Experiment 5.1. The middle column shows the posterior means of the equivariant GP with a disconnected fundamental region of 1000 sub-intervals P, The right column shows the posterior means for the connected fundamental region. In contrast to using the connected fundamental region, the posterior means of the equivariant GP with disconnected fundamental region are discontinuous and deviate significantly from the ground truths.
An analogous investigation provided equivalent conclusions for the dipole moment GPs. We constructed a disconnected fundamental region in the same way as for the S0(2) equivvriaat case, by inducing similar alternating partition in the Z22 component, which only requires occasional left-multiplication of the section by an additional rotation around the y-axis with angle 0 = æ. The resulting discontinuous fundamental region GP decreases in performance, as can be seen by the elevated learning curves of Figure 14.
19

---

Integration-fre Kernels for Equivariant Gaussian Process Modelling
Iterations: 1000
1.0
GP 2
0.8
GP 4
0.6
0.4
0.2
0.0
0
0.2
0.4
0.6
0.8
a-1
Iterations: 3000
1.0
GP 2
0.8
GP 4
0.6
0.4
0.2
0.0
0
0.2
0.4
0.6
0.8
Figure 11. Optimised values of from the mixture GPs 2 and 4 for 1000 vs. 3000 training iteration steps.
F. An example of matrix to vector prediction task featuring SO(d) equivariances
We present an interesting example extending the fundamental region concept of Experiment 4.3 featuring SO(d) equivariance in arbitrary dimension Consider class of matrix to vector mappings constructed as follows. The Input matrices X [x1,.... xa] Raxdd are chosen such that their columns form orthogonal (not necessarily orthonormal) bases of TR... and f(X) Rdd is defined as g[[]x] |))x; where g (0, oo) R.. Such f is automatically equivariant with respect SO(d) (acting matrix multiplication columns of X and similarly fXX) as, for any orthogonal matrix R,
f(RX) = [x]] =
= Rf(X).
Besides, S(X does not depend on the d 1 last columns of X, which is additional invariance property. Taking both cquivariance and invariances properties into account, we arrive at the fundamental region
FR {[aer, e2,.... ed). a > 0}.
where the es's are the canonical basis vectors.
The key to construct a section here is to observe that Vr
X
[xal Xy an orthogonal matrix with
T X
|| |x
[|x]|
which can then be sent to FR by modifying the inactive columns appropriately (e.g., by norming them). Hence f can be modelled via an equivariant GP model defined in terms of the latter sequence of operations and of a kernel on FR, that is, a kernel that can be parametrized on (0, oo) (0, oo) (and corresponds to solely modelling the function g).
20

---

Integration-Fis freeeeeoo--eeeeeeeee Kernels for Equivariant Gaussian Process Modelling
D2
22
((x)
II (())
01
A
A
Figure Visualisations of connected disconnected fundamental regions Experiment 5.1 and the associated II,
Vector field
Disconnected A, RMSE=0.74 Log5=-1.95
Connected RMSE=0.18 LogS=.1.37
Vector field
Disconnected A, RMSE=0,39 LogS=-2.68
Connected A,
RMSE=0.15 Log5=.1.56
Figure 13. Ground truths sampled from the equivariant random fields of 5.1 (left column), posterior means of GP(O, Kn) with discon- nected A (middle column) and connected (right column).
21

---

Integration-fres Kernels for Equivariant Gaussian Process Modelling
Mean log(RMSE) Scores
Mean LogS Scores
-2.5
-8
K
disconnected Kg
-3.625
-4.55
-24
-5.875
-7
-40
20
30
40
50
80
110
20
30
40
50
80
110
Training size
Training size
Figure 14. Learning curves of GP(O, Kn)) (dotted: disconnected fundamental region) for the dipole moment prediction task.
G. Equivariance of sparse GPs
To broaden the applicability of equivariant GP modeling large datasets like our N-Methylffrrmmiid dataset, we introduce (centered) sparse Gaussian Process based << inducing locations X, Rmxd with posterior mean and covariance respectively denoted m and K with, for x. x' E D,
m (x) Kιx (X,) m (Xu), K (xx K(x, x²) ((x,)
Pm (X.))(XX -X(,, x').
Here, mm---m (X,,) and Kyp-m (Xu) are the posterior mean and covariance at the inducing points given the remaining observations D given by
mmna-m (NDn=m(X.)+ = K(X,, (Xx) Ztrs KDa-m(X.l K(X.) - K(X,, X (Xer) K K((t;;
If this sparse GP is built upon an equivariant kernel, it will be equivariant in its mean and covariance and therefore enjoy stochastic equivariance. To see this, consider any g, E G. holds
"mp. m (X ()+ = K(g Xx)K((x) (m 1ampp--aa (Xw)) = Xx)(((X))KXX))) -
(X (x),
and
Kb (5*x,,,x++)
K(g (X) (K({)) KD. (X.)) K(X,, h x')
= (,, x² )pT Py K(x, (x.) - KDD m (X.))) TT
Km (x, x PT.
It is worth noting that additional assumptions for computational efficiency like FITC and PITC will not affect stochastic equivariance the posterior distribution the sparse GP, as these assumptions only replace m-pn-m (X,,) and Km (X,)) with approximations thereof. Similarly, conditioning on a finite number of derivatives or linear forms (c.g., Fourier coefficients) will preserve stochastic equivariance.
22

---

Integration-fre free Kernels for Equivariant Gaussian Process Modelling
H. Data generation
The dipole moments were generated by optimising an initial guess for the geometry of water using the RI-MP2 module in the TURBOMOLE quantum chemistry program. The theory for this module is described in the original MP2 paper by Møller & Plesset (1934) and the RI-MP2 paper by Weigend Häser (1997). Furthermore, cc-pVDZ basis set was used, created by Dunning (1989). Using this optimised structure, numerical hessian found. Diagonalising the mass weighted Hessian then gave set of three Normal coordinates, corresponding to bending, symmetric and asymmetric bond stretch. Using the static grid method Toffoli et al. (2007) in the MIDASCPP program package a grid of twenty points was constructed along each normal coordinate. These points were linearly spaced between the classical turning points of the tenth excited state of the quantum mechanical oscillator approximation which can be constructed from the mass weighted hessian. Furthermore, three twenty by twenty grids were constructed describing displacements along two out of the three coordinates simultaneously. Combined with the optimised structure this gives 1261 geometries. For each of these geometries, the electric dipole moment calculated using the RI-MP2 method and cc-pVDZ basis set in TURBOMOLE. To ensure dataset efficiency, 410 pairs of points that were rotations of each other were identified and removed. The remaining 851 points were then randomly rotated to cover larger region of Cartesian space. This approach samples the geometries of chemical interest around the optimised structure and them rotates them in space resulting in diverse and representative dataset.
Figure The point cloud representing the datasct: (light) blue indicate positions of the hydrogen atoms, red the position of the oxygen atoms and orange the dipolc moments.
23

---

Integration-free Kernels for Equivariant Gaussian Process Modelling
I. Optimized kernel parameter values
train the GPs using Adam optimisation over 1000 iterations, with all kernel parameters initialised to I, maximising the training likelihood. Figure 16 shows the distributions of the optimised values (after initialised by of the parameters ł, for 2000 draws of the training and test split for different training sizes.
The choice of starting kernel parameter values plays an influential role, the left panel of Figure 17 shows the distribution of the optimal parameter values for GO,, K1) and GO,,, KT) on training sets of size 50, when initialised uniformly on [10 We see that GO, K1) tends to increase the lengthscale and decrease the variance, which indicates the kernel to smooth out the function excessively, likely because it lacks the capacity to capture the complexity in the data, which not the case for (O,, Kn). The right panel shows distributions of the corresponding log(RMSE) and LogS scores after optimising the kernel parameters, indicating stable predictive performance for both GPs with respect to initialisation.
2.3
0.50
-+- Kj -+-
Kg - Kg
2.1
0.45
1.9
0.40
17
0.35
1.5
0.30
1.3
0.25
20
40
50
80
110
20
30
40
50
80
110
Training size
Training size
Figure 16. Optimised parameters (mean + sd over 2000 samples) for different training sizes for the dipole moment prediction in 5.2.
J. Dipole Moment Prediction for the N-Methylformamidd Molecule
We consider dataset comprising 21,000 distinct configurations of the N-Methylformmmide (NMF) molecule, which consists of nine atoms. As outlined in Experiment 5.2, constructing a kernel K7T that is rotation-equivariant and follows the same principles as in the water molecule setting.
Figure 18 shows preliminary learning curves comparing the baseline Gaussian process GO using squared exponen- tial kernel with the *2 -zuuqiivar and * -iiinvariant Gaussian process GP(0, KTT). Evaluation was conducted on 500 test points across 1,000 random train-test splits. The kernel parameters water molecule experiments. For additional comparison, we also report performance using fixed kernel hyperparameters (a, E) = (1, 1) (dashed lines), which notably reduced -acimaannce-artiulaalyy for GP Kn) across both metrics and for GP(O, K)) in terms the log score.
optimized via the same Adam routine used in the
The relatively flat learning curves of the baseline GP highlight inability to model the structured nature of the data. In contrast, the equivariant GP demonstrates improved ability to capture the structural of the data. However, achieving high predictive accuracy requires larger training sets, motivating the of sparse GP methods, which is part of ongoing work.
24

---

Integration-Fag free Kernels for Equivariant Gaussian Process Modelling
Sensitivity to hyperparameter Inttialisation,
Sensitivity scores to hyperparametor initialisation, train
100
8
-00
Cist
Opt
K opt
(RMSE))
logs
log(RMSE)
logs
Figure 17. Left panel: Distributions of initial kernel parameters and the corresponding optimised values. Right panel: distribution of log(RMSE) and LogS scores after optimisation.
Mean RMSE Scores
Mean LogS Scores
0.40
3
K1
Kn
0.35
Hyperparameters trained
0
untrained
0.30
0.25
0.20
0.15
-12
0.10
0.05
-15
20
120
220
320
420
520
620
720
820
920
20
120
220
320
420
520
620
720
820
920
Training size
Training size
Figure Predictive performance (mean + over 10 repetitions) GP models versus training set size N-Methyfforrmmiie molecule dataset.
25


Reviews:
[{"review_id": "BGlhaKqcs5", "reviewer": "Reviewer_HxJv", "summary": "This paper introduces integration-free equivariant kernels to avoid computationally expensive integration. The method is claimed to be computationally efficient while preserving equivariance. Applications in velocity fields and molecular dipole moments are used to demonstrate effectiveness. While promising, the approach remains limited to low-dimensional groups, lacks extensive theoretical guarantees on stability, and fails to provide a rigorous scalability analysis.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We warmly thank the referee for taking the time to review our work and the stimulating comments. The question and comments pertaining to scaling to higher-dimensional equivariant problems call for a distinction between group order, input dimensions, and training set size (in particular). The statement “While promising, the approach remains limited to low-dimensional groups” puzzled us at first because our examples include infinite groups (SO(2), SO(3)), so we hypothesized that the mentioned limitation could pertain, e.g., to either the dimensionality of inputs or the data set cardinality. In the submitted paper, our examples featured 2- and 9-dimensional inputs. During the rebuttal, on the molecule side, we have been able to tackle a bigger molecule (9 atoms, i.e. 27-dimensional), and we are happy to report that our first results on the applicability of kernels based on analogous fundamental regions are very promising. Cf. learning curves of GPs for N-Methylformamide molecule dipole moments (20 to 100 training molecules at this stage, 500 test ones) at \n\nhttps://equivariantrf.github.io/Equivariant-Random-Fields/reviewer-discussion-html.html \n\nAlso, we have set up a proof of concept fundamental region example of matrix to vector prediction tasks featuring SO(d) equivariances in arbitrary dimension. It basically consists in mapping matrices with columns forming an orthogonal (but not necessary orthonormal) basis of $R^d$ to a scalar non-linear function of the norm of the first column times the first column. In such a case one may prove that ${[\\alpha e_1,e_2,...,e_n], \\alpha >0}$ forms a fundamental region and provide a section and projection. We may include this example in appendix. \n \nComing to the scalability in terms of training data size (n), while standard GP implementations can be tricky to scale up (notably due to O(n^3) covariance matrix inversion costs), there are options such as sparse GPs (see discussion with referee iBFz) that allow tackling large n applications. The discussion led us to look further into the matter of extending our results and constructions to sparse GPs are we found out that our results do carry over nicely in that context, in the sense that if one uses an argumentwise equivariant kernel as original kernel in sparse GP modelling, the resulting sparse GP will retain the equivariance property in the mean and the covariance. \n\nWe will stress this interesting fact and also stress in the paper’s discussion that a research perspective that expects us beyond this work is to scale it up via sparse GPs and benchmark the resulting sparse equivariant GPs against equivariant neural networks (sending us to the referee’s third question). On the latter point, we are very open to suggestions as to which classes of equivariant neural networks the referee would suggest to us for future comparisons. A particular challenge in our considered settings of probabilistic prediction (the GP models being returning probability density functions, which is in turn  necessary for the log-score we are using in evaluation) is to come up with probabilistic approaches to equivariant neural networks, which is not something we found off-the-shelf and calls in our opinion for work that goes substantially beyond our present contributions.  We are by the way very thankful for the referee to point that “the theoretical characterization is rigorous and a strong point of the paper”. \n\nConcerning the choice of the fundamental region (and section/projection), while we provided an example at the end of section 5.1 highlighting the performance loss in case of a poorly chosen A, we will stress this further in the discussion and add new illustrations in appendix of how things can go wrong in particular cases with disconnected A’s (See the three figures on the topic added to the anonymous GitHub). Besides this, as developed in the response to referee U1Nr, we are not claiming the fundamental region approach to be uniformly better than others, but found it to be practically applicable in cases where integrations appeared to be prohibitively cumbersome.  As exposed in Appendix B2, we found it promising indeed to hybridize fundamental region (on an infinite group) and double sum (on a small group). We consider putting this more to the fore as this subtle combination may be extended to further contexts. Coming finally to remarks pertaining to sensitivity and robustness, we spent considerable energy working on hyperparameter tuning and arrived at reducing variability in implementation results. The choice of starting hyperparameter values plays an influential role, and to illustrate this and address the referee’s comment we conducted new experiments (available figure on anonymized Github) that will be reflected in Appendix.  We truly hope that in light of our efforts, the referee will consider increasing their score. We are thankful for the opportunities given to us to clarify things and improve our paper with a broader perspective.", "claims_and_evidence": "The main claim of achieving computational efficiency while preserving equivariance is partially substantiated. While empirical results show significant speedups, they are restricted to low-dimensional cases, and generalization to larger groups remains unclear. Some ablation studies on region choice and its impact on kernel continuity are missing.", "methods_and_evaluation_criteria": "Theoretical derivations are solid but rely on restrictive assumptions which do not always hold in real-world applications. The evaluation criteria focus on RMSE and LogS. Hyperparameter selection discussion is lacking wrt sensitivity and initialization effects, potentially affecting model reliability.", "theoretical_claims": "The theoretical characterization is rigorous and a strong point of the paper.", "experimental_designs_or_analyses": "The velocity field experiments and molecular provide a useful baseline, albeit at times offers only marginal improvements. The ocean dataset experiment is interesting but limited.", "relation_to_broader_scientific_literature": "The work builds on prior studies of equivariant kernels and methods methods.", "essential_references_not_discussed": "N\\A", "other_strengths_and_weaknesses": "The method is computationally efficient and theoretically justified. However, the lack of scalability analysis, absence of hyperparameter sensitivity studies, and limited exploration of high-dimensional problems weaken the overall contribution.", "other_comments_or_suggestions": "More structured discussion on practical limitations would strengthen the impact of the work.", "questions_for_authors": "How does the method scale to higher-dimensional equivariant problems? \n\nHow sensitive is performance to fundamental region choice? \n\nHow does it compare to equivariant neural networks in terms of sample efficiency?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1U7BObJVNo", "reviewer": "Reviewer_bAhY", "summary": "The paper introduces an integration-free approach to constructing equivariant kernels, leveraging the concept of fundamental domains of the action. The authors demonstrate the effectiveness of their method through one synthetic example and two real-world applications, highlighting its practicality and potential impact.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We warmly thank the referee for taking the time to check our results and for the very encouraging positive evaluation. We will of course fix the citation command and precise the closure notation. We hope that the referee will also appreciate the overall discussion and our efforts to further improve the paper (including new experiments with bigger molecules, stochastic equivariance carrying over to sparse GP models, and further response points that can be found in rebuttals to other referees). Please see https://equivariantrf.github.io/Equivariant-Random-Fields/reviewer-discussion-html.html\nand the other rebuttals for new illustrations and discussions on our results and extensions thereof.", "claims_and_evidence": "Yes, all the claims made are supported by convincing evidence.", "methods_and_evaluation_criteria": "Yes, the datasets used are relevant for the problem tackled, and the synthetic experiment is a nice visualization of the impact of using $K_{\\pi}$ rather than $K_\\int$.", "theoretical_claims": "I checked the proof of Theorem 3.1, Proposition 4.1 and Corollary 5.1; they look correct to me. I didn't check the proof of Theorem 3.2.", "experimental_designs_or_analyses": "N/A.", "relation_to_broader_scientific_literature": "The paper is a timely contribution to the literature, providing a computationally feasible approach to equivariant kernels. This is crucial for images, proteins, and graph-structured data, where symmetry plays a key role. Introducing an integration-free method offers a practical solution to a long-standing challenge in this area.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "The paper is well written and it does a great job of breaking down complex mathematical concepts and using examples to make them easier to understand. Overall, I think it’s a solid and valuable contribution, especially in making equivariant kernels more computationally feasible, an important challenge for the community.", "other_comments_or_suggestions": "- Please use the \\citet comment when citing in text, otherwise looks odd.\n- I guess $\\bar{A}$ is the closure of $A$, but it's not explained in the paper.", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_flag": "Flag this paper for an ethics review."}}, {"review_id": "5hsVmXEwiZ", "reviewer": "Reviewer_u1Nr", "summary": "This paper introduces the group-theoretic notion of fundamental regions and proposes a feasible method to construct kernels for equivariant functions. The proposed method is free of integration operations and much faster than conventional methods. Experiments on synthetic and real-world data confirmed the model's validity.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the referee for taking the time to review our work and the constructive comments that will truly help us improving the paper. We really appreciate the suggestion to open on other physical knowledge that can be incorporated in GP models and kernel methods. We discovered the suggested reference on symplectic GP with great interest. We found in particular that a parallel result to our kernel characterization of stochastic equivariance could be established for the relevant symplectic property, so that the kernel proposed in the symplectic GP paper could drive under suitable assumptions (regularity) not only the symplectic nature of the GP posterior mean but also of the posterior sample functions (almost surely). While this is not falling into the umbrella of equivariance, we will explain how the two could be connected via null spaces of linear operators. This connection has already been made in other contexts, for example in scalar-valued GP settings. We will open perspectives on how, under appropriate assumptions, equivariances, the symplectic property, and other properties of vector fields such as being divergence- or curl-free may be treated in a unified way (via linear constraints) when it comes to their incorporation in GP modelling. Let us remark that the fundamental region approach might not be straightforward to transport broadly beyond group invariances and equivariances.        \n\nComing to clarifications regarding comparisons between integration vs fundamental region approaches, we are happy to have the opportunity to clarify this is in the paper, as it is really not our intention to suggest that the fundamental domain approach is generally superior to integration.  We do think that integration-based equivariant kernels are very nicely theoretically grounded, as our projection result illustrates, and also may deliver higher or lower performances depending on the applications. However, we stumbled across prohibitive costs in case of larger / infinite groups, and we found fundamental region approaches to deliver a fast alternative also passing the argumentwise equivariance requirements, and doing the job on our challenging molecule application(s). It is yet very interesting to note, and we will stress this further, that combining the fundamental region approach for SO(3) with an Reynolds operator approach (with a group of order two in that case) delivered better performances than the pure fundamental region approach (See Appendix B). As mentioned also in the response to reviewer HxJv, we consider putting these results more to the fore as this subtle combination may be extended to further contexts.\n  \nWe also wish to thank the referee for stressing the context of “limited observation” within which our work takes its roots. GPs are known to be especially suitable in such contexts, providing a flexible family of probabilistic predictors able to work with scarce training data. Standard GP models are actually known to possess limitations when it comes to bigger data sets, and sparse GP models have been developed to extend the scalability of GPs.  Reviewer iBFz attracted our attention on that and enquired whether our results and constructs could be transposed to the field of sparse GP modelling. As we develop in the response to reviewer iBFz, we are happy to respond positively to the latter and explain how sparse GPs built upon an argumentwise equivariant kernel (and in particular on the fundamental region kernels) will have their mean and kernel inheriting that property and therefore enjoy stochastic equivariance. As illustrated on  https://equivariantrf.github.io/Equivariant-Random-Fields/reviewer-discussion-html.html and further discussed in other rebuttals, we conducted experiments highlighting how poor fundamental region choices may affect performances, and also obtained promising first results in extending the integration-free approach to bigger (9-atom) molecules.  \n\nWe hope that our efforts to address the various comments from this and the other reviews will be appreciated and considered to clarify what needed to be / improve the paper, and that this could justify score increases enabling our work to be part of ICML 2025.", "claims_and_evidence": "The main claim that the proposed method can efficiently construct kernels with equivariant property is supported by clear and convincing evidence.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria make sense.", "theoretical_claims": "Theoretical claims seem to be correct.", "experimental_designs_or_analyses": "No issues specified.", "relation_to_broader_scientific_literature": "Equivariance, along with invariance, is an important property in physics. The proposed method could provide a more accurate approach to estimating underlying functions related to physical phenomena with limited observation.", "essential_references_not_discussed": "Equivariance is closely related to physical theories, and similarly, symplectic Gaussian process regression [a] provides a kernel method-based approach that preserves physical properties. Please discuss the relationship between the prior study and the proposed method.\n\n[a] Rath et al., Symplectic Gaussian process regression of maps in Hamiltonian systems. Chaos: An Interdisciplinary Journal of Nonlinear Science, 31(5):053121, 2021.", "other_strengths_and_weaknesses": "Weakness\n- The pros/cons of $K_{\\int}$ and $K_{\\pi}$ are not clear. It is clear that $K_{\\pi}$ is superior to B in terms of computational efficiency, but does $K_{\\pi}$ have any limitations in terms of expressiveness? Currently, the paper seems to present $K_{\\pi}$ as a complete superset of $K_{\\int}$, so it would be helpful to clarify this point explicitly.", "other_comments_or_suggestions": "No other comments.", "questions_for_authors": "No other question.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qbtId454OD", "reviewer": "Reviewer_iBFz", "summary": "This paper introduces a novel class of integration-free equivariant kernels for Gaussian processes (GPs), addressing the computational inefficiency of traditional equivariant kernels that require group integrations. The key idea leverages fundamental regions to project inputs into a representative subset, enabling equivariant kernel construction without integration. Empirical validation includes molecular dipole moment prediction and ocean velocity data. The proposed kernels achieve up to 500× speedup over integration-based methods while maintaining or improving predictive accuracy (RMSE, LogS), demonstrating practical utility in scientific applications.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the referee very much for taking the time reviewing our work, highlighting the underlying rationale and the obtained numerical benefits within a probabilistic prediction and evaluation framework, and pointing out further aspects that will deepen our work. Two specific directions stand out, and we think that they are both very valuable for our work and follow-ups thereof: scaling up the approach via sparse GP modelling, and criteria to select/update A. \n\nThe referee's point on sparse GPs inspired us to extend our theoretical framework, for which we below establish how stochastic equivariance of a GP guarantees stochastic equivariance of a sparse version of it. This result may broaden the applicability of equivariant GP modeling to large datasets, like with our new large N-Methylformamide molecule dipole moment dataset (See first results at https://equivariantrf.github.io/Equivariant-Random-Fields/reviewer-discussion-html.html). While our preliminary results with training/test sets of moderate size are very promising indeed, we envision on a longer run to apply sparse GP modelling to the full data set comprising 20000 molecules of each 9 atoms (that would ideally be benchmarked against equivariant neural networks following up on referee HxJv's suggestions).  \n\nThe gist of the extension of equivariance properties to sparse GPs is to build upon Corollarly 5.1. Let us explain that next. \nA (centered) sparse GP based on $m<<n$ inducing locations $X_u \\\\in \\\\mathbb{R}^{m\\\\times d}$ possesses the following posterior distribution in terms of the inducing locations\n$Z\\\\mid \\\\mathcal{D}^n \\\\sim \\\\mathcal{N}(m_{\\\\mathcal{D}^n}^u, K_{\\\\mathcal{D}^n}^u),$ where for $\\\\boldsymbol{x},\\\\boldsymbol{x'}\\\\in D$\n\n\\\\begin{equation*}\n    m_{\\\\mathcal{D}^n}^u(\\\\boldsymbol{x})=K(\\\\boldsymbol{x},X_u)K(X_u)^{-1}m_{\\\\mathcal{D}^{n-m}}(X_u)\n\\\\end{equation*}\n\nand \n\\\\begin{equation*}\nK_{\\\\mathcal{D}^n}^u(\\\\boldsymbol{x},\\\\boldsymbol{x'})=K(\\\\boldsymbol{x},\\\\boldsymbol{x'})-K(\\\\boldsymbol{x},X_u)K(X_u)^{-1}(K(X_u)-K_{\\\\mathcal{D}^{n-m}}(X_u))K(X_u)^{-1}K(X_u,\\\\boldsymbol{x'}).\n\\\\end{equation*}\nHere, $m_{\\\\mathcal{D}^{n-m}}(X_u)$ and$ K_{\\\\mathcal{D}^{n-m}}(X_u)$ are the posterior mean and covariance of the field at the inducing points given the remaining observations $\\\\mathcal{D}^{n-m},$ given by \n$$ \nm_{\\mathcal{D}^{n-m}}(X_u) = K(X_u,X_{tr})K(X_{tr})^{-1}\\boldsymbol{z}_{tr} \n$$\n\nand  \n\n$$ \nK_{\\mathcal{D}^{n-m}}(X_u) = K(X_u)-K(X_u,X_{tr})K(X_{tr})^{-1}K(X_{tr},X_u).\n$$\n\nAnalogously to the proof of Corollary 5.1, the posterior distribution of the sparse version of a stochastically equivariant GP is stochastically equivariant as well, since for any $g,h \\\\in G,$\n\\\\begin{equation*}\n    m_{\\\\mathcal{D}^n}^u(g\\\\star \\\\boldsymbol{x})=K(g\\\\star \\\\boldsymbol{x},X_u)K(X_u)^{-1}m_{\\\\mathcal{D}^{n-m}}(X_u)= \\\\rho_g K(\\\\boldsymbol{x},X_u)K(X_u)^{-1}m_{\\\\mathcal{D}^{n-m}}(X_u)=\\\\rho_g m_{\\\\mathcal{D}^n}^u(\\\\boldsymbol{x}),\n\\\\end{equation*}\n\n\\\\begin{align*}\n&K_{\\\\mathcal{D}^n}^u(g\\\\star\\\\boldsymbol{x},h\\\\star\\\\boldsymbol{x'})\\\\\\\\\n=&K(g\\\\star\\\\boldsymbol{x},h\\\\star\\\\boldsymbol{x'})-K(g\\\\star\\\\boldsymbol{x},X_u)K(X_u)^{-1}(K(X_u) \n-K_{\\\\mathcal{D}^{n-m}}(X_u))K(X_u)^{-1}K(X_u,h\\\\star\\\\boldsymbol{x'})\\\\\\\\\n=&\\\\rho_gK(\\\\boldsymbol{x},\\\\boldsymbol{x'})\\\\rho_h^T-\\\\rho_gK(\\\\boldsymbol{x},X_u)K(X_u)^{-1}(K(X_u)-K_{\\\\mathcal{D}^{n-m}}(X_u))K(X_u)^{-1}K(X_u,\\\\boldsymbol{x'})\\\\rho_h^T\\\\\\\\\n=&\\\\rho_gK_{\\\\mathcal{D}^n}^u(\\\\boldsymbol{x},\\\\boldsymbol{x'})\\\\rho_h^T.\n\\\\end{align*}\n\nSimilarly, conditioning on a finite number of derivatives or linear forms (e.g., Fourier coefficients) will preserve stochastic equivariance. \n\nConcerning the choice of $A, s,\\Pi_s$, while we have no procedure to construct them in a generic case-independent fashion, our additional tests illustrate that connectedness is a desirable feature for $A$. Automatically exploring the set of possible $A, s,\\Pi_s$ appear as a fascinating problem and as a daunting task.   \nAs of now, we are unaware of general adaptive selection methods in the employed fundamental region approach and would welcome suggestions. Let us add that the choice of $K_A$ and the interplay with $s$ and $\\Pi_s$ also offers interesting degrees of freedom as we will further stress in the discussion. Our first thoughts at this stage is that one could use likelihood- or cross-validation-based approaches as a means to compare several candidates for a corresponding tuple (consisting of $A, s, $ and possibly $K_A$). This could be performed straightforwardly on a finite set of tuples, and could be extended to parametric families. But already in the two-dimensional example 4.3, there are many possible ways to perform these choices. Let us observe that our initial choice not only feature connectedness but also \"flatness\"; taking {(x,h(x)), x>0} with $h:(0,\\infty) \\to \\mathbb{R}$ a continuous non-decrasing mapping such that $h(0)=0$ would work, too. We would be happy to include a perspective on that!", "claims_and_evidence": "Computational efficiency: Figure 3 and Section 4.3 show a 45-hour vs. 55-second runtime comparison for integration-based vs. integration-free kernels validated on synthetic data.\n\nEquivariance guarantees: Theorem 3.1 and Corollary 5.1 link kernel design to stochastic equivariance, with posterior samples in Figure 4 confirming equivariant realizations.", "methods_and_evaluation_criteria": "Evaluation: RMSE and LogS are appropriate metrics for regression and probabilistic calibration. Baselines (e.g., Helmholtz kernel, double-integration kernels) are well motivated.", "theoretical_claims": "I didn't check the proofs because I'm not an expert in this field.", "experimental_designs_or_analyses": "NA", "relation_to_broader_scientific_literature": "If I understand correctly, the work builds on integration-based equivariant kernels and scalar invariance via fundamental regions. It extends these ideas to matrix-valued kernels and stochastic equivariance. I'm not sure if this idea could be borrowed into the sparse GP field.", "essential_references_not_discussed": "NA", "other_strengths_and_weaknesses": "NA", "other_comments_or_suggestions": "NA", "questions_for_authors": "1. Would adaptive section selection (e.g., optimizing A) improve robustness? I liken this paper to introducing inducing points in sparse GPs, where the positions of the inducing points can be optimized. Therefore, I wonder if the foundation region A could also be optimized.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: Four knowledgeable reviewers recommend Accept and I accept. The authors should incorporate several clarification comments mentioned by the different reviewers.
Decision: Accept (poster)