Optimal Rates for Regularized Conditional Mean Embedding Learning
Authors: Zhu Li
Dimitri Meunier
Mattes Mollenhauer
Arthur Gretton
Venue: NeurIPS 2022 Accept (2022)
Keywords: Kernel Methods
Learning Theory

Abstract: We address the consistency of a kernel ridge regression estimate of the  conditional mean embedding (CME), which is an embedding of the conditional distribution of $Y$ given $X$ into a target reproducing kernel Hilbert space $\mathcal{H}_Y$. The CME allows us to take conditional expectations of target RKHS functions, and has been employed in nonparametric causal and Bayesian inference.
We address the misspecified setting, where the target CME is
in the space of Hilbert-Schmidt operators acting from an input interpolation space between $\mathcal{H}_X$ and $L_2$, to $\mathcal{H}_Y$. This space of operators is shown to be isomorphic to a newly defined vector-valued interpolation space. Using this isomorphism, we derive a novel and adaptive statistical learning rate for the empirical CME estimator under the misspecified setting. Our analysis reveals that our rates match the optimal $O(\log n / n)$ rates without assuming $\mathcal{H}_Y$ to be finite dimensional. We further establish a lower bound on the learning rate, which shows that the obtained upper bound is optimal.

Optimal Learning Rates for Regularized Conditional Mean Embedding
Zhu Li
Dimitri Meunier'
Gatsby Computational Neuroscience Unit University College London zhu
Gatsby Computational Neuroscience Unit University College London
dimitri. meunierr .arr.210 21@ucl ac.uk
Mattes Mollenhauer
Arthur Gretton
Department of Mathematics and Computer Science Freie Universität Berlin
Gatsby Computational Neuroscience Unit
University College London arthur weethur.reettee www.ttrtttto......... trretton@gmee gma com
mattes mol www.mollllaarrrreem mww.eeeeeaueeeeee-ereei.... fu-berlin.
Abstract
We address the consistency of kernel ridge regression estimate of the conditional mean embedding (CME), which is an embedding of the conditional distribution of Y given X into target reproducing kernel Hilbert space Hy. The CME allows us take conditional expectations of target RKHS functions, and has been employed in nonparametric causal and Bayesian inference. We address the misspecified setting, where the target CME is in the space of Silbert-Schmidt operators acting from an input interpolation space between Hx and L2, to Hy. This space of operators shown be isomorphic to newly defined vector-valued interpolation space. Using this isomorphism, we derive novel and adaptive statistical learning rate for the empirical CME estimator under the misspecified setting. Our analysis reveals that our rates match the optimal Olog n/n) rates without assuming Hy to be finite dimensional. We further establish lower bound on the learning rate, which shows that the obtained upper bound is optimal.
Introduction
Approximation of the conditional expectation operator is central issue in the statistical learning
community, and many approaches have been proposed 42 20 21 [22]. Given random variables X and Y, the conditional expectation operator for function is defined
[x) (=) E((()
Conventional parametric models to approximate P often involve density estimation and expensive numerical analysis. Hence, recent studies attempt to explore a new framework to approximate P via kernel methods. Specifically, given kernels kx and ky with corresponding reproducing kernel Hilbert space H.x and Hy for X and Y respectively, we may define the conditional mean embedding (CME) as (xx EE(ky Y)JX
w)) and may employ the reproducing property to obtain [z) = (ff, (x))) N.y for any f E Hy. The advantage of the CME framework is that allows the straightforward evaluation of conditional expectations of any function in Hy. The CME framework has been applied successfully to many learning problems such as probabilistic inference
35
reinforcement learning 2X 16 and causal inference 25
denotes equal contribution.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Despite these successful applications, there have been two main challenges in establishing rigorous theory of CMEs. The first challenge, remarkably, has been in establishing a principled and sufficiently general definition of the conditional mean embedding itself. The CME was originally introduced as This definition has the benefit of elegance, and of a straightforward expression in terms of feature covariances and cross-covariances. A disadvantage E Hx, Vg E Hy. This strong 19 and IK Section 3.1] for illustrations and alternative requirements), and significantly restricts the class of distributions on which we can define
an operator mapping from Hx to Hy [1 371
is that the definition requires the conditional mean IE' (Y) X
assumption may be violated in practice (see [8
CME.
An alternative approach, due to 15 is to express the conditional mean embedding as the solution of a least- squares regression problem in a vector-valued RKHS [5] 6 In subsequent work, a rigorous measure-theoretic definition of the conditional mean embedding as the Hy-valued square integrable function F. is established in 229 18 which is the definition we will use in the present work. Both connect this CME definition to the original operator-mapping definition by means of a surrogate loss, which upper bounds the regression loss. A direct connection remained elusive until the work of 26 [9,, which show that under denseness assumptions, the CME can be arbitrarily well approximated by Hilbert-Schmidt operator from Hx to Hy, thus connecting the theretii- and measure- theorreicc definitions.
The second challenge has been in obtaining consistency results and the optimal learning rates for empirical estimates of the CME. An early consistency analysis of the sample estimator, due [6)]
requires very strong smoothness assumptions. A more refined analysis, due to [15 attains the minimax optimal learning rate Olog n2/1) for the sample estimator, but only in the case where Hy is finite dimensional. For the infinite dimensional RKHS, [31 and [ establish consistency in the well-specified case, with learning rates of O(n -1/0) and O(n-1/4 Nevertheless, the obtained rates are far from optimal and consistency under misspecified setting was not established. Recently,
41 obtains sharper rate under the misspecified case using the interpolation RKHS. The results of impose assumptions, however, which strongly limit their applicability (refer to Remark 5 for a rigorous discussion):
1. They require an explicit relation between the smoothness of the target CME and the size of the RKHS. In particular, when the kernel has slow eigenvalue decay (as in the case of Matérn kernels, for example), the setting is very close to the well-specified scenario.
2. They rely on the explicit construction of an interpolation RKHS. Unlike in 11] where a similar approach is based only on equivalence classes of functions (i.c., Sobolev-like spaces), this concept requires the embedding of the RKHS into the corresponding L'-space (or equivalently the integral operator) to be wnjective-which is generally not the case (see [40 for details). Counterexamples can easily be constructed when one considers degenerate pushforward measures on the RKHS in one or more coordinate directions (for example point masses). By contrast, the authors of [II] do not explicitly require the injectivity in the real-valued learning scenario. Moreover, in case where the chosen kernel has slow eigenvalue decay, the constructed interpolation RKHS is not well-defined.
Finally, to our knowledge, there is presently no result establishing a matching lower bound for the CME learning rate in the case where Hy may be infinite dimensional. Hence, whether the obtained upper rate is optimal remains unknown.
In the present work, we address the challenges mentioned above. Building on 29 interpolation space theory results of A0 [. we introduce an interpolation space consisting of vector-valued functions via a natural tensor product construction. This concept compatible with the recent measure- -ncsour--heooretic definition of the CME due to [29] and allows to prove convergence in the misspecified setting without the limitations of prior work. Based on this novel -allue interpolation space, we establish consistency and convergence rates of the CME sample estimator in the misspecified setting. In particular, under certain benign conditions, we obtain the optimal (11/n) learning rate up to a logarithmic factor. This matches with the current optimal analysis from 15 without the restrictive assumption of finite dimensional Hy. Thanks to our operator-theoree definition of the CME, and unlike 41 we do not require an a-priori relation between the rate of kernel eigenvalue decay and the smoothness of the conditional mean operator (i.e., our results apply generally in the misspecified setting). Finally, in Theorem
26
and the
provide novel lower bound on the
2

---

CME learning rate, which demonstrates that the obtained upper rate is optimal in the setting of a smooth CME operator.
2 Background
Throughout the paper, we consider two random variables X, Y defined respectively on the second countable locally compact Hausdorff spaces Ex and Ey endowed with their respective Borel o-field FEx and FEy We let F, IP) be the underlying probability space with expectation operator E. Let and v be the pushforward of IP under X and Y respectively. i..,, X ~ π and Y ~ v. We use the Markov kernel p: Ex x FEY - 1 to define the conditional distribution:
P|Y A|X - x] x1]= p(zz,d)), dy),
for all x E Ex and events A E FEy. We denote the space of real-valued Lebesgue square integrable functions on (Ex.Fxx) with respect to т as L Fxxxxxxxxx т) abbreviated L²(T) and similarly for y use (2 FEy, .V) abbreviated (z(v). Let B be separable Banach space with norm || - || and H a separable real Hilbert space with inner product (.-) We write [(B, B as the Banach space of bounded linear operators from B another Banach space B', equipped with the operator norm || B-B-B' When B = B', we simply write L(B) instead. We also let (Fx, FEx x x; B), abbreviated (T; B), the space of strongly FEx Fa measurable and Bochner p-integrable functions f: Ex + B for S S. 00. Finally, we denote the p-Schatten class S, H!') be the space of all compact operators C' from H to another Hilbert space H!' such that ([C]Is,(H.H)) = (I(o,())) lIl, is finite. Here || (C,()) lle, is the E, sequence space norm of the sequence of the strictly positive singular values of C' indexed by the countable set J. For p = 2, (H, H') is the Hilbert space of Hilbert-Schmidt operators from H to H'.
Tensor Product of Hilbert Spaces (II, Section 12): Denote H @ H' the tensor product of Hilbert spaces H, H'. The Hilbert space H & H' is the completion of the algebraic tensor product with respect to the norm induced by the inner product (z) @ 21 22 @ x4) H@H' = (I1, x2) H (41, x2) for 21,22 H and z z4 E H'' defined on the elementary tensors of H @ H' This definition extends to span(2 (z @ x' lx E H x' E H'} and finally to its completion. The space H @ H' is separable whenever both H and H' are separable. The element x: 8' E H H' is treated the linear rank-one operator &
x x': H' -> H defined by y' - (v/ x²) H'
x for y' E H'. Based on this identification, the tensor the space of Hilbert-Schmidt operators from
product space H @ H' is isometrically isomorphic H' to i.e., H @ H' = S2(H', H). We will hereafter not make the distinction between those two spaces and them as identical. If {ei)ier and (e/)iss are orthonormal basis in H and H', (e; o0 d))NErde is orthonormal basis in H & H'
Remark (MI) Theorem 12.6.1). Consider the Bochner space L2(T;; H) where H is a separable Hilbert space. One can show that L2(T;; H) is isometrically identified with the tensor product space 11 @ (T).
Reproducing Kernel Hilbert Spaces, Covariance Operators: We let 1x Ex x Ex +
R
be symmetric and positive definite kemell function and Hx be vector space of Ex - functions, endowed with a Hilbert space structure via an inner product (xx- xx is a reproducing 2x;2.Vx E Ex and Vf Hx, f(x) (f, (I. .) F A space Hx which possesses a reproducing kernel is called a reproducing kernel Hilbert space (R We denote the canonical feature map of Hx as (x(x) kx (x,))).. Similarly for Ey, we consider a RKHS Hy with symmetric and positive definite kernel ky By x By + I and canonical feature map denoted as Øy.
T
kernel of H if and only if: 1. Va E Ex,
We require some technical assumptions the previously defined RKHSs and kemels::
1. Hx and Hy are separable, this is satisfied if Ex and Ey are Polish spaces and kx, by are continuous 3
2. kx (.) and ky (. are measurable for 7-almost all Ex and v-almost all y Ey;
3. kx (x) I x2x for -lammsst all x E Ex and ky (v. y) S ney for v-almost all y E Ey.
Note that the above assumptions are not restrictive in practice, as well-known kernels such as the Gaussian. Laplacian and Matérn kernels satisfy all of the above assumptions on Rd We now
3

---

introduce some facts about the interplay between 7xx and (T(T). which has been extensively studied by B3 13..9 9 and 200 We first define the (not necessarily injective) embedding 1 H.x + (
mapping a function f Hxx to its z-equivalence class [ʃ]. The embedding is well-defined compact operator as long as Hilbert-Schhidt norm finite. fact, this requirement satisfied since its Hilbert-Schmidd norm can be computed as
a (1HxxxI2(1)) ||kx|leatm) =- (1 lix (x((x
1/2 < 00.
Ex
The adjoint operator S == I (2( + Hxx is an integral operator with respect to the kernel kx. ie. for f E (TT and x E Ex we have
[ JEx kx (x, (x.) dr (x!)
(s-
Next, we define the self-adjoint and positive semi-definite integral operators
Lx == Sx L2(x) + (T( and Cxx == Sy Hx - Hx
These operators are trace class and their trace norms satisfy
(xxlls,(<x((m)) = ||Cxxxlx,,ccc = 1 1 LL2(7) = 1| -x 11
Vector-valued RKHS also give brief overview of the vector-valued reproducing kemne Hilbert space (vRKHS). We refer the reader to 5 and 3 for more detail.
Definition 1. Let H be real Hilbert space and K Ex Ex - L(H) be an operator valued positive-semidefiiie (psd) kernel such that K (xxxx') K( xx) for all x, x' E Exx and for all T1,,,,, In Ex and h4, by E H,
(his K ((i,,, 20. O.
Fix K, Ex, and H. (+) K(,xz) defines a function from Ex to H. We now consider
Spre =- span (Kah Ex, h H}
with inner product Spe by linearly extending the expression
(K₂h, kw))g = (h, K (x, h')
(1)
Let G be the completion of Gie with respect this inner product. We call & the vRKHS induced by the kernel K. The space & is Hilbert space consisting of functions from Ex to H with the reproducing property
(F(x), (F
(2)
for all FE E H and x E Ex. For all F EG we obtain
[x())l & (1
& Fx.
Since the inner product given by Eq. 4D implies that K₂ is a bounded operator for all x E Ex. For all F E G and Ex, Eq. ) can be written as F(z) F.F. The linear operators K2 H g and K G H are bounded with
|K || [I(x,)))1
and we have Ky = K (x, x²) ,x), xx E Ex. the following, we will denote g as the vRKHS induced by the kernel Ex Ex + C(Hy) with
K K(x,x') = 1x x' Iday x. E Ex.
An important property of G is that elements in G are isometric Hchbert-Schmidr operators between Hx and Hy.

---

Theorem 1 (Theorem 4.4 in (261). Let Hx and Hy be real-valued RKHS with kernel kx and ky respectively. For fy Hy and 9x Hxx define the map V on the elementary tensors as
(fr gx)] (x) = (x (fr gx) (x)).
We then have that defines an isometric isomorphism between S2(Hx. Hy) and G through linearity and completion.
More details regarding Theorem | can be found in 22 Theorem 4.4]. The isometric isomorphism w induces the operator reproducing property stated below.
Corollary 1. For every function F there exists operator C = (F) S2 ((Hx,Hy) such that
Fx) Cox(#) E Hy,
for all E Ex with [[C]]ss.Hxxxxx)) = ||g and vice versa. Conversely, for any pair F Gnd Ce S2HHx, Hy), we have C' C==--<<0 as long as F(xr) (x)
The proof of Corollary Tis simple extension of Lemma 15 in 7 and Corollary 4.5 in 126 Corollary shows that the vRKHS G is generated via the space of Hilbert-Schhiin operators Hy
(F Ex - Hy| = Cox(), S2 Hy))
Conditional Mean Embedding: A particular advantage of kernel methods is its convenience of operating probability distributions, see [27 30 for examples. This is through the so called kernel mean
embedding 2 64 [77. Assuming the integrability condition JEx /kx (z, z)) < co (which is satisfied when the kernel is almost surely bounded), we define the kernel mean embedding μ (-) is easy to show that for each f E Hx. I f(x)dT (f MT.x)) Hx Replacing with the conditional distribution, we obtain the kernel conditional mean embedding as defined in [29 [8]
JEx xx (-, x d (x))..
Definition 2. The Hy-vvluud conditional mean embedding (CME) for the Markov kernel p(a, dy) is defined as
F.(x) = 1 Φv (y(p(e du) = =E (Y) F2x n; Hy) BY
(3)
By the reproducing property, we have [Y] Ex. The approximation of F. is a key concept in kernel methods. By [26] suppose we impose Assumptions 1-3 together with two additional assumptions: i) Hx S Cx(Ex) where Co the space of continuous functions vanishing at infinity and ii) Hx is dense in L2(T), then we have that G is dense in L2 [2x ग; Hy). ). As result, for any > O, there is an F E such that || F. 1r2 < 8. Hence, in the literature, we often assume the so-called well-specified case obtain a closed-form solution,
= x]
(fr, F. (x)) Hy. Vfv E Hy and x e
E. g.
It shown in 1& Theorem 5.3] and 26 Corollary 5.6 and Remark 5.8] that F. admits closed form expression under Eq. via
Fx(x) - CxV)
where Cyx = E[Φy (Y) @ 6x (X) and Ct denotes the pseudoinverse of C.
Remark 2. We point out that in the original derivations, the CME is written as F.(x) B3. However, C'xx is not globally defined Hf is infinite-dimensioonal xxxxx((x) is problematic,
xxxxxxxxxx 37
12
expect F, be defined for all Ex based on the Markov kernel p. In the well- well-specified scenario, 118) corrected this issue by defining the CME as xCxxCxx) (xy)" box (x:). It is shown that in this case, (C'xxxxx) is bounded (actu- ally Hilbert-Schmidt, see also (1)), and hence globally defined. The connection of this corrected operator-theorett perspective to the well-specified regression scenario was established in (26).
Hence the expression Cy xxx x
CxxCx
This satisfied if lx is bounded and kx (x()x) Co(Ex) for alamost all x E Ex.
5

---

Once we have the closed-fore form solution, a natural question to ask is how to estimate the CME. Indeed,
this has been extensively studied in [ 29 IT. Given a data set D [(x y:) independently and identically sampled from the joint distribution of X and Y, regularized estimate of F. is the solution of the following optimization problem:
CYIXA= = arg min
E lebv - Cox X|IC (Hxx
(5)
(ES,(Hx,Hr)
Cxxx,ax(-)) (-), where is the regularization parameter. Implicit in the construction, however, is the assumption F. E 9 that the solution is well-specified. We provide few remarks regarding this assumption:
A())= = V (Or x))()= ()
Remark 3. In the literature, the prevalent definition of well-speciffednnss is through
T [ = Hxx E Hy,
(6)
see e.g. 37 37 18] for details. However, this definition not equivalent to that Eq. 4 Specifically,
assuming F. E 9 implies that Eq. O holds. Nonetheless, the reverse is not true. In particular, there
exist concrete examples satisfying Eq. b.. but the corresponding operator representative of the CME
is not Hilben--Schmide (see Section D in Appendix for details). To avoid confusion, refer to Eq.
as the well-specified case hereafter.
ti) The conventional assumption Eq. 1 can actually be refined via the inclusion map I In particular. since Ly is an inclusion map from H.x to L2(TT) and E [ = Hx, we can apply 1, to E lf (X) = -]. In addition, we are only interested in the case where I (E (Yv(()) = + 0. As a result, the refined definition should be
E [ - - (ker I=)² Vfr Hr.
(7)
We now characterize the Hilbert spaces used to define the CME in the misspecified setting.
Real-valued Interpolation Space: We review the results of 40 [|] that set out the eigendecom- positions of Lx and Cxx, and apply these in constructing the interpolation spaces used for the misspecified setting. By the spectral theorem for self- self-adjoint compact operators, there exists an at most countable index set non-increasing sequence (#:)ee > 0, and a family (c))ez Hx. such that ((ei))ez is an orthonormal basis (ONB) of ran I, C L2 and (u./2et)ser ee)ser is an ONB of (ker ++ S Hxx and we have
I.x
Emit:eed)Laomlee iE7
Cxx Emiineaan
SEI
For 2 0, we define the a-interpolation space 40 by
[7 == Σ
2/2 [ea] (as)er E E
c L2(T),
GET
equipped with the a-power norm
1/2
Σ SET
a/2 a.j
[e:]
= [|(as))erlless)
Σ
2
[M]
More broadly, we sometimes need to deal with function of the form f + c where f E [A0] and E IR. For this, we follow the classical definition of the direct sum of two Hilbert spaces and define the a-power norm for f a
|| cllpx ||]|| + |1/l
(8)
For (a:)ier E E2(1), the a-interpolation space becomes Hilbert space with inner product defined as
Σ VeT
Σ a.b.
[A

---

(2(, Hy)
v L2(1T; Hy)
I+ S2 Mly
g
V
Figure 1: V and V are the bijective linear operators that define respective isomorphisms between each pair of spaces. The precise form of the isomorphisms is given in the appendix. Ix denotes the canonical embedding between the two Hilbert-Schmic spaces.
Moreover, (1e [ea] forms an ONB of [M| and consequently [M] is a separable Hilbert iEz
space. In the following, we use the abbreviation |la llrus- For 0 we have [H]]
ran T S (T( with || || [|))())) Moreover, for we have [H] ran /= and [H] is
isometrically isomorphic to the closed subspace (ker 1=²² of Hxx via La i.e. ||[II|| = |I || for
f E (ker -= For 0 < 3 < a, we have
|M - [1 + M S L2(x).
(9)
Moreover, under Assumptions 1-3, we further assume that supp(z)) = Ex and Hx is dense in (TT,, we have [H]] (T()..
3 Approximation of CME with Vector-valued Interpolation Space
In this section, we deal with the misspecified setting where F. & G. To do this, we first define the vector-valued interpolation space via the tensor product space. We now recall from Remark T that (2; Hy) is isomorphic to S2 (L2(T), Hy-) and we denote by v the isomorphism between the two spaces. Similarly, we have G = (2 Hy) and we denote by the isomorphism between both spaces in accordance with Theorem I This is summarized in Figure [I] The second chain of spaces is not isometric to the first but can be naturally embedded into the first as follows. Recall that we denote by x Hx (=) the embedding that maps each function its equivalent class,
T=( = [f]. We therefore naturally define the embedding I- S2x , Hy) + S2(T<2 (yy through (g f) = g [f) g @ [f] for all f E Hx. g E Hy, and obtain the extension the whole space by linearity and continuity. Therefore, for F E G we define FF] = T I, o v (F). In the rest of the paper, every embedding will be denoted using the notation
Strict notation would require us to write l due to dependence on the measure ग, but we omit the subscript for ease of notation.
Definition 3. Suppose that and Hy and let [H1& be the real-valued interpolation space associated to Hx with some a 2 Since [H]& C L2(x)), it is natural to define the vector-valued interpolation space [g]
given real-valued kernels kx and ky with associated RKHS Hx
[ == v (82(1H)) Hy)) {F (CC),, C' S2( Hy)).
[ is a Hilbert space equipped with the norm
|| Fla == ||CIls. Hy)
(F E [g]%,
where C= (=)--.F.. For a we retrieve,
Fl || (C))s)((2(()))H))
Remark The vector-valued interpolation space [g] allows us to study the CME the misspecified case. To see this, we note that by Eq. for < B < a we have
3 we have F. L2(Ex, FEx: n. Hy). In light of Eq. (9)
g [9] 1910 S L2(Ex,
Hy).
Again, once we further assume that supp(T)) = Ex and Hx dense in Le2(T), we have 1g (xxxx FFx, n; H.). Hence, while the well-specified case corresponds to F. E G, the misspecified case amounts to assuming that F., E [ for some < a < and relaxes the well-specified assumption in Eq. 4
7

---

4 Learning Rate for CME
In this section, we derive the learning rate for the difference between [Fx] and F, in the interpolation norm. We first state additional assumptions that are needed in our derivations. As our assumptions match those of I. we include the corresponding labels from 1 for ease of reference.
5. Recall that (pi)ser r are the eigenvalues of Cxx. For some constants c2 > 0 and p E [0, 1] and for all i E
14 S 028 o2/ 1/p
(EVD)
6. For E (p, 1}. the inclusion map 10:00 : [H] + (TT) is continuous, and there is constant A > 0 such that
| [M] + box (x) SA
(EMB)
7. There exists 0 < B S such that
F. E [g] We let Cyjx == (F-) E S2( Hy) and we call Cyyx the conditional mean
(SRC)
embedding operator.
EVD is a standard assumption on the eigenvalue decay of the integral operator (see more details (EMB) is referred the embedding property in [1I and it can be shown that it
4 I 4l).
implies Eie iir r 14 (T) I Aa for r-almost all x E Ex (ΑI) Theorem 9). Since we assume 1x to
be bounded, the embedding property always hold true when
1. Furthermore, EMB)) implies
a polynomial eigenvalue decay of order 1/a,, which is why we take a > p. SRC is justified by
24 D. It imposes Remark X and is often referred as the source condition in literature ((4 I 23
the smoothness assumption on the target CME operator F.. In particular, when 3 2 1, the source condition implies that F. has representative from G. indicating the well specified scenario. However, once we let 3 < 1, we are in the misspecified learning setting, which is the main interest in this manuscript. Finally, in computing the learning rate for real-valued regression, one often needs the so-called (MOM) condition on the Markov kernel p(z, dy) (see [4] [1 41 for more details). The generalization to our setting would amount to assume that there exists constants o, such that
E [1) - F. (2) xx=x] S 50l02 p
for r-almost surely all x E Ex and all q 2 2. The reason for requiring (MOM) in the scalar almost surely. In our setting, however,
regression setting that do not usually have YY| Assumption 3 implies 1 (v) - (x))ll I 2ky for 7-almost all x E Ex and v-almost all y E Fy. Therefore, (MOM) is automatically satisfied with = R 2Ky
Remark 5. We remark that in MY,, a variant of SRC condition is employed. In particular, instead of assuming F. [G]3, they impose the assumption that F. G S2 Hy), where H an RKHS with corresponding kernel defined as (-,x)) = E.mfe(()e(())) Comparing to [g] = 15((H)),, there are two shortcomings that arise when working with ga.
First, H denotes the interpolating RKHS consisting of continuous functions only, while [H1] is the interpolating Hilbert space, where elements are defined through an equivalence classes. Hence, by working with G the implicit assumption is that F. is continuous function. On the other hand, assuming F- E [ avoids the continuity requirement. In particular, we have H.x C [71) for any O. Therefore, our (SRC) condition applies to a more general setting.
any
Second, and more importantly, the construction of Ht relies on the condition that Σ, e2(2) ( < 20, as pointed out in [40). Failing this, the kernel 1ox associated with the interpolating RKHS H is unbounded, indicating H6x is not well-defined. Under (EVD), this effectively amounts require that 3 Z p. When kernel has slow eigenvalue decay (as for the Matérn kernel), p can be close to 1. Results obtained using 8 while requiring 3 > P are very close to the well-specified case. By contrast, [H]] is always well-defined as subspace of T22(m)) for any B > O, even if E, E,14 ( < co does not hold.
Remark 6. important to assume /3 in (SRC as our results do not apply when 3 = 0. The 3 = O setting arises for instance when Y 1 X, or when both Y = X and Hy Hx. In the former
8

---

case, is easy to see that the CME is the constant function (x) = py = SEy Φy (u)du(y)-
In the latter case, (x) (x(x), and the CME is the identity operator. These functions are not covered by [ for any 3 > 0 (see Appendix D details).
We now provide an upper bound on the learning rate.
Theorem Let Assumptions 1-3, EVD
EMB and
SRC with 0 < 3 2 hold, and let 0 7
with < 3,
the case B B+pS and An = + ((1/ log' (m)) for some r 1, there a constant
K > 0 independent of and such that
8-x
- P. ||AN--F.
S 2
log" n
is satisfied for sufficiently large n 2 1 with pm -probabbiility not less than 4e
2. the case 3 + + p.> a and An = (n-a-/) there a constant > independent of n 2 1 and 2 1 such that
| F. F.I 22
is satisfied for sufficiently large n 2 with Pa -probabbbiity not less than 1 4e
Theorem provides the finite sample 7-norm learning rate for the empirical CME estimator defined in Eq. 5). It states that the learning rate for [Ax] is governed by the interplay between p. and B. Intuitively, p describes the decay rate of the eigenvalues (Hi)igr, a determines the boundedness of the interpolation kernel (and has maximum value of 1 according to our assumption), 3 characterizes the smoothness the target CME operator.
To simplify the discussion, we may focus on the L2(Ex, qi Hy) learning rate, corresponding to 0. The exponent // max(ax + p) explicitly provides the learning rate for the CME operator. For example, if we have a S 3, we obtain a learning rate of //(33 + p). In particular, for a Gaussian kemne,, p and a arbitrarily close O, and our learning rate achieve fast O (1/n) rate up to logarithmic factor. If kernel with slow eigenvalue decay is used, such as the Matérn kernel, we can obtain the minimax optimal learning rate n 1/2 up to logarithmic factors if we have p S B. Finally, in the worst case where is close to 0, the learning rate can be arbitrarily slow.
5 Lower Bound
Our final theorem provides a lower bound for the convergence rate, which allows us to confirm the optimality of our learning rate. In deriving the lower bound, need extra assumption
8. For some constants
> 0 and (0, 1] and for all i E
c1/-1/p μ4 S 1/p
(EVD+)
Theorem 3. Let kx be kernel on Ex such that Assumptions 1-3 hold and a be a probability distribution on Ex such that (EV++) and {EMB) hold < S a < 1. Then for all < B < 0 < T < with y < 3 there exist constants Ko, K, > such that for all learning methods D FD (D = (.),,)))_))) all > O, and all sufficiently large 2 there is distribution P defined on Ex x Ey used to sample D, with marginal distribution π on Ex, such that (SRC) with respect to 3 is satisfied, and with 1 prrobability not less than 1 1/,
max mav(o.8)-1
|| F= 2 2 Kn
max a.
Theorem 3 states that under the assumtions of Theorem and EVD+). no learning method can achieve a learning rate faster than [ La2 norm. To our knowledge, this is the first analysis that demonstrates the lower rate for CME learning. In the context of regularized regression,

---

4 39 B provide similar lower bound on the learning rate. However, a key difference in our analysis that the oulput of the regression learning now lives in an infinite dimensional RKHS Hy, rather than in R. Our analysis reveals that in the case where I B. the obtained upper rate in Theorem 2 optimal, i.e., On 8+8) ). In particular, when 4xx is an exponentiated quadratic kernel on a compact
set Fxx C IR with Lipsehitz boundary, (EMB) is satisfied with any a (0, 1) I7 see Corollary 4.13]. As a result, the optimal rate is attained as long as 3 > O. We point out that finding the optimal rate for B < a remains challenge, and an open problem when the output TR.
Remark 7. Theorem G states that the upper bound and the lower bound match when > a. In particular, for exponential kernels such as Gaussian and Laplacian kernels with subgaussian distributions, the eigenvalues for the covariance operator have geometric decay rate. In these cases, is arbitrarily close to O. Hence, as long as B > O. we will have a < B. In other words, for commonly used kernels, CME learning can always obtain the optimal rate
Conclusion
In this paper, we provide rigorous theoretical foundation for approximating the CME operator, and study the statistical learning rate. Utilizing recently developed interpolation space techniques, we first define the vector-valued interpolation space [g] This allows to define the target CME operator in the larger interpolation space [G] in contrast to the well-specified setting where F. E G. By doing so, we are able to study the convergence rate of the empirical CME operator in the misspecified scenario. We then provide yynorm learning rate for the CME without any assumption on the interplay between and p, with matching lower bound. Our analysis shows that under appropriate conditions, we can obtain a fast Ologg n/n) convergence rate, which matches the rate obtained in the existing literature for finite dimensional Hy. In more challenging settings, we still obtain the minimax optimal rate O(n 1/22).
((12))
Looking beyond the present work, our current interpolation space setting indicates that the conver- gence rate can be arbitrarily slow if B 0. This prevents learning the constant function, which plays a crucial role in completing the theory of the CME, as pointed out by [8] Addressing this challenge an important direction of future research.
Acknowledgement The authors wish to thank Peter Orbanz and Bharath Sriperumbudur for fruit- ful discussions and proofreading. Zhu Li, Dimitri Meunier, and Arthur Gretton were supported by the Gatsby Charitable Foundation. Mattes Mollenhauer was supported by the Deutsche Forschungs- gemeinschaft (DFG) through grant EXC 2046 "MATH+"" Project Number 390685689, Project IN-8 "Infinite- Dimensioanl Supervised Least Squares Learning as Noncompact Regularized Inverse Problem"".
References
[1] J.-P. Aubin. Applied Functional Analysis. John Wiley & Sons, Inc., 2nd edition, 2000.
[2] A. Berlinet and C. Thomas- Agnan. Reproducing kernel Hilbert spaces in probability and statistics. Springer Science & Business Media, 2011.
[3] G. Blanchard and N. Mücke. Optimal rates for regularization of statistical inverse learning problems. Foundations of Computational Mathematics, 18(4):971-1013, 2018.
[4] A. Caponnetto and E. De Vito. Optimal rates for the regularized least-squares algorithm. Foundations of Computational Mathematics, ((3):331-3683 2007.
[5] C. Carmeli, E. De Vito, and A. Toigo. Vector valued reproducing kernel Hilbert spaces of integrable functions and Mercer theorem. Analysis and Applications, 4(04))377-403 2006.
[6] C. Carmeli, E. De Vito, A. Toigo, and V. Umanitá. Vector valued reproducing kernel Hilbert spaces and universality. Analysis and Applications, 8001):19-61, 2010..
[7] C. Ciliberto, L. Rosasco, and A. Rudi. A consistent regularization approach for structured prediction. Advances in Neural Information Processing Systems, 29, 2016.
10

---

[8] J. B. Conway. A course in functional analysis, volume 96. Springer, 2019.
[9] E. De Vito, L. Rosasco, and A. Caponnetto. Discretization enror analysis for tikhonov regular- ization. Analysis and Applications, 4(01))81199, 2006.
[10] R. DeVore, G. Kerkyacharian, D. Picard, and V. Temlyakov. Mathematical methods for supervised learning. IMI Preprints, 22:1-51, 2004.
[11] S. Fischer and I. Steinwart. Sobolev norm learning rates for regularized least-squares algorithms.
J. Mach. Learn. Res., 21:205-1, 2020.
[12] K. Fukumizu, F. R. Bach, and M. I. Jordan. Dimensionality reduction for supervised learning with reproducing kernel Hilbert spaces. Journal of Machine Learning Research, 5(Jan):73-99, 2004.
[13] K. Fukumizu, L. Song, and A. Gretton. Kernel bayes' rule: Bayesian inference with positive definite kernels. The Journal of Machine Learning Research, 14(1):3753-3785 2013.
[14] A. Gretton. RKHS in machine learning: Testing statistical dependence. Technical report, Technical report, UCL Gatsby Unit, 2018.
[15] S. Grünewääler, G. Lever, L. Baldassarre, S. Patterson, A. Gretton, and M. Pontil. Conditional mean embeddings as regressors- suppleeetty.. arXiv preprint 1rXiv:1205.4656, 2012.
[16] S. Grünewälder, G. Lever, L. Baldassarre, M. Pontil, and A. Gretton. Modelling transition dynamics in mdps with rkhs embeddings. In Proceedings of the 29th International Conference on Machine Learning, pages 535-542, New York, NY, USA, 2012. Omnipress.
[17] M. Kanagawa, P. Hennig, D. Sejdinovic, and B. K. Sriperumbudur. Gaussian processes and kernel methods: A review on connections and equivalences, arXiv preprint arXiv:1807.00 2018.
[18] I. Klebanov, Schuster, and Sullivan. A rigorous theory of conditional mean embeddings. SIAM Journal on Mathematics of Data Science, 2(3):583-606, 2020.
[19] I. Klebanov, B. Sprungk, and T. Sullivan. The linear conditional expectation in Hilbert space. Bernoulli, 27(4):2267-229, 2021.
[20] S. Klus, P. Koltai, and C. Schütte. On the numerical approximation of the Perron-Frobenius and Koopman operator. Journal of Computational Dynamics, 3(1):51-79, 2016.
[21] Klus, F. Nüske, P. Koltai, H. Wu, I. Kevrekidis, C. Schüttee and F. Noé. Data-driven model reduction and transfer operator approximation. Journal of Nonlinear Science, 28(3):985-1010, 2018.
[22] M. Korda and I. Mezić. On convergence of extended dynamic mode decomposition to the Koopman operator. Journal of Nonlinear Science, 28(2):687-710, 2018.
[23] J. Lin and V. Cevher. Optimal distributed learning with multi-pass stochastic gradient methods.
In International Conference Machine Learning, pages 3092-3101. PMLR, 2018.
[24] J. Lin, A. Rudi, L. Rosasco, and V. Cevher. Optimal rates for spectral algorithms with least-squares regression over hilbert spaces. Applied and Computational Harmonic Analy- sis, 48(3):868-890, 2020.
[25] Mitrovic, Sejdinovic, and Y. W. Teh. Causal inference via kernel deviance measures. Advances in Neural Information Processing Systems, 31, 2018.
[26] M. Mollenhauer and Koltai. Nonparametric approximation of conditional expectation opera- tors. arXiv preprint arXiv:2012. 12917, 2020.
[27] K. Muandet, K. Fukumizu, B. Sriperumbudur, and B. Schölkopf. Kernel mean embedding of distributions: A review and beyond. Foundations and Trends Machine Learning, 10(1-2):1- 141, 2017.
11

---

[28] Y. Nishiyama, A. Boularias, A. Gretton, and K. Fukumizu. Hilbert space embeddings of pomdps. In Proceedings of the Twenty-Eighth Conference on Uncertainty in Artificial Intelligence. UAI'12, page 653, Arlington, Virginia, USA, 2012. AUAT Press.
[29] J. Park and K. Muandet. A measure-theoretic approach to kernel conditional mean embeddings. Advances in Neural Information Processing Systems, 33:21247-21255 2020.
[30] D. Sejdinovic, B. Sriperumbudur, A. Gretton, and K. Fukumizu. Equivalence of distance-based and RKHS-basee statistics in hypothesis testing. The Annals of Statistics, pages 2013.
[31] R. Singh, M. Sahani, and A. Gretton. Kernel instrumental variable regression. Advances in Neural Information Processing Systems, 32, 2019.
[32] Smale and D.-X. Zhou. Shannon sampling and function reconstruction from point values. Bulletin of the American Mathematical Society, 41(3):279-38) -0,50, 2004.
[33] S. Smale and D.-X. Zhou. Shannon sampling II: Connections to learning theory. Applied and Computational Harmonic Analysis, 19(3):2853302 2005.
[34] A. Smola, A. Gretton, L. Song, and B. Schölkopf. A Hilbert space embedding for distributions. In International Conference on Algorithmic Learning Theory, pages 13-31. Springer, 2007.
[35] L. Song, K. Fukumizu, and A. Gretton. Kernel embeddings of conditional distributions: A unified kernel framework for nonparametric inference in graphical models. Signal Processing Magazine, IEEE, 30(4)098-1))))) 111, 2013.
[36] L. Song, A. Gretton, and C. Guestrin. Nonparametric tree graphical models. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pages 765-772. JMLR Workshop and Conference Proceedings, 2010.
[37] L. Song. J. Huang, A. Smola, and K. Fukumizu. Hilbert space embeddings of conditional distributions with applications to dynamical systems. In Proceedings of the 26th Annual International Conference on Machine Learning, pages 961-968. ACM, 2009.
[38] I. Steinwart and A. Christmann. Support vector machines. Springer Science & Business Media, 2008
[39] Steinwart, D. R. Hush, C. Scovel, al. Optimal rates for regularized least squares regression. In COLT, pages 79 93, 2009.
[40] I. Steinwart and C. Scovel. Mercer's theorem on general domains: On the interaction between measures, kernels, and RKHSs. Constructive Approximation, 35(3):363-417, 2012.
[41] Talwai, A. Shameli, and D. Simchi-Levi. Sobolev norm learning rates for conditional mean embeddings. In International Conference Artificial Intelligence and Statisties, pages 10427. PMLR, 2022.
[42] M. O. Williams, I. G. Kevrekidis, and C. W. Rowley. A data-driven approximation of the Koopman operator: Extending dynamic mode decomposition. Journal of Nonlinear Science, 1377-1333 2015.
12

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] Please see Page 10.
(c) Did you discuss any potential negative societal impacts of your work? [N/A] To the best of our knowledge, our research does not impose any negative societal impacts.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] Section 2 and
4 provides all the assumptions for our research.
(b) Did you include complete proofs of all theoretical results? [Yes] The proof of all the results are provided in the Appendix.
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or URL)? [N/A]
(b) Did you specify all the training details (c.g.. data splits, hyperparameters, how they were chosen)? [N/A]
(c) Did you report error bars (e.g.. with respect to the random seed after running experi- ments multiple times)? [N/A]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, cloud provider)? [N/A]
4. If you using existing assets (e.g., code, data, models) or curating/releasing assets...
(a) If your work uses existing assets, did you cite the creators? [N/A]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any assets either in the supplemental material as URL? [N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you're 're using/curating? [N/A]
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "sAd2EBipcz1", "reviewer": "Reviewer_AJfs", "summary": "The work focuses on estimation of the kernel conditional embedding operator in the so-called mis specified case. The recently proposed idea of interpolation spaces between H_X and L_2 is leveraged to define CME operator with milder conditions (depending on interpolant parameter \\beta). \n\nThe key contribution is theorem 4, where an upper bound on estimation error of the CME operator is presented (mis specified case). Though the derivation is technical, the final bound is insightful and depends on the interplay between \\beta, \\apha (smoothness parameter) and p (eigen value decay parameter). For larger \\beta (i.e., less mis specification), in particular, \\alpha\\le\\beta, which is shown to be optimal via theorem 5 that provides a matching lower bound. The upper bound however can be very loose when \\beta is tiny.\n\n", "questions": "Please verify minor typos:\nConsider five equations between eqn(14) and line 479. The fourth equation misses a C_YX term. But it is minor because it is back in the next equation.", "limitations": "Limitations have been discussed (case of low \\beta).", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Overall the paper is well-written and organized. Though technical, enough details are provided to be self-contained. The upper bound is interesting as it explicilty involves the mis-specification parameter. The lower bound on learning rate is also seems to fill a gap in existing literature.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "lqUBRPRQCj", "reviewer": "Reviewer_u4KM", "summary": "This paper presents novel theoretical results on learning empirical approximations of conditional mean embeddings (CMEs) from samples. The derivations focus on the misspecified case, where the true CME operator does not satisfy the usually strict assumptions found in the literature, but only milder ones which are more plausible in practical settings. The theoretical analysis show that the true CME can be approximated via an interpolation space between the reproducing kernel Hilbert space $\\mathcal{H}_X$ of functions over the conditioning inputs $X$ and the Hilbert space $L_2(\\pi)$ of square-integrable functions under the inputs marginal distribution $X \\sim \\pi$.The derived results present an optimal learning rate for the empirical CME approximation, which leads to a vanishing error bound with respect to the true CME operator.", "questions": "* I've missed some important details:\n - It is not specified whether the samples in the dataset for learning CME estimator need to be independent and identically distributed (i.i.d.) w.r.t. the joint the distribution of $X, Y$ or not. Would the results hold for a non-i.i.d. case?\n- The statement of Theorem 2 is a bit confusing regarding $\\alpha$. Does Thm. 2 hold for any $\\alpha > 0$? Or does it imply there is an $alpha > 0$ such that its result holds?\n- It is not clear to me how to get to the formulation of $C_{XX}$ in terms of $(\\mu_i)_i$ and $(e_i)_i$ in line 135.\n\n* Minor details:\n - In the proof sketch after Theorem 4, are we missing an identity operator next to $\\lambda$ in the definition of $F_{Y|X,\\lambda}$?\n", "limitations": "Most limitations are discussed in the paper.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The theoretical analysis is rigorous and improves on the understanding of CME learning in misspecified settings.\n- Assumptions are milder and seem easier to satisfy than the ones found in existing theoretical analyses in the literature.\n- The first lower bound on the learning rate for the CME estimator is derived.\n\nWeaknesses:\n- The derived results do not hold for the case of the constant function, i.e., when $Y$ is independent of $X$, which is a problem in the case of radial basis function and other characteristic kernels popular in kernel embeddings applications.\n- The paper is quite dense at some points and a few details get lost.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "dKaiod9Z5OC", "reviewer": "Reviewer_x4Bu", "summary": "The paper considers estimating the conditional mean embedding (CME). The authors propose an estimation of CME using an approximation in interpolation space.  Consistency and convergence rates are also established for the CME estimator in the misspecified setting.\n", "questions": "See the main comments.", "limitations": "N/A", "rating": 7, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "[Strengths]\n1. The presentation of the paper is clear, and estimating the CME is an important problem.\n2. The authors provide an operator-theoretic definition of the CME in the misspecified case where the target CME can be well approximated by Hilbert-Schmidt operators.\n3. A lower bound on the learning rate is established, which reveals that when the target CME operator is smooth, the obtained upper bound is optimal.\n\n[Weaknesses]\n1. It seems that most of the results follow from [8, 39] as well as direct adaption of the interpolation spaces. The background section could be reduced.\n2. I am not convinced about the advantages of adapting the interpolation space for approximation. How does the proposed approach compare with related work [39]?\n3. While most of the work is theoretical, it could be useful to provide numerical results to demonstrate the performance on real-life data.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "blBTW6QEkG", "reviewer": "Reviewer_nfGK", "summary": "This paper studies the conditional mean embedding and provides learning rate for the ridge regression based estimator of the CME.", "questions": "-", "limitations": "Not relevant", "rating": 9, "confidence": 2, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "This is a very technical paper and as such I have not verified all the details. Despite being technical the paper does a great job in being clear and precise about the statements.\n\nConditional mean embeddings (CME), although used frequently in machine learning, have only recently been defined and studied rigorously. This paper is part of that literature and it makes some valuable contributions. The most important of them is using interpolation space theory to study an estimator for the CME operator and provide optimal learning rates.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept