Original: Consistent Interpolating Ensembles via the Manifold-Hilbert Kernel
Revised: Consistent Interpolating Ensembles via the Manifold-Hilbert Kernel

Original: Abstract
Revised: Abstract

Original: Recent research in the theory of overparametrized learning has sought to establishgeneralization guarantees in the interpolating regime.
Revised: Recent research in the theory of overparametrized learning has sought to establishgeneralization guarantees in the interpolating regime.

Original: Such results have been established for a few common classes of methods, but so far not for ensemble methods.
Revised: Such results have been established for a few common classes of methods, but so far not for ensemble methods.

Original: We devise an ensemble classification method that simultaneously interpolates thetraining data, and is consistent for a broad class of data distributions.
Revised: We devise an ensemble classification method that simultaneously interpolates thetraining data, and is consistent for a broad class of data distributions.

Original: To thisend, we define the manifold-Hilbert kernel for data distributed on a Riemannianmanifold.
Revised: To thisend, we define the manifold-Hilbert kernel for data distributed on a Riemannianmanifold.

Original: We prove that kernel smoothing regression and classification usingthe manifold-Hilbert kernel are weakly consistent in the setting of Devroye et al.[20].
Revised: We prove that kernel smoothing regression and classification usingthe manifold-Hilbert kernel are weakly consistent in the setting of Devroye et al.[19].

Original: For the sphere, we show that the manifold-Hilbert kernel can be realizedas a weighted random partition kernel, which arises as an infinite ensemble ofpartition-based classifiers.
Revised: For the sphere, we show that the manifold-Hilbert kernel can be realizedas a weighted random partition kernel, which arises as an infinite ensemble ofpartition-based classifiers.

Original:  Introduction
Revised:  Introduction

Original: Ensemble methods are among the most often applied learning algorithms, yet their theoreticalproperties have not been fully understood [11].
Revised: Ensemble methods are among the most often applied learning algorithms, yet their theoreticalproperties have not been fully understood [11].

Original: Based on empirical evidence, Wyner et al.
Revised: Based on empirical evidence, Wyner et al.

Original: [41]conjectured that interpolation of the training data plays a key role in explaining the success of
Revised: [40]conjectured that interpolation of the training data plays a key role in explaining the success of

Original: AdaBoost and random forests.
Revised: AdaBoost and random forests.

Original: However, while a few classes of learning methods have been analyzedin the interpolating regime [5, 3], ensembles have not.
Revised: However, while a few classes of learning methods have been analyzedin the interpolating regime [5, 3], ensembles have not.

Original: Towards developing the theory of interpolating ensembles, we examine an ensemble classificationmethod for data distributed on the sphere, and show that this classifier interpolates the training dataand is consistent for a broad class of data distributions.
Revised: Towards developing the theory of interpolating ensembles, we examine an ensemble classificationmethod for data distributed on the sphere, and show that this classifier interpolates the training dataand is consistent for a broad class of data distributions.

Original: To show this result, we develop two additionalcontributions that may be of independent interest.
Revised: To show this result, we develop two additionalcontributions that may be of independent interest.

Original: First, for data distributed on a Riemannian manifold
Revised: First, for data distributed on a Riemannian manifold

Original: M , we introduce the manifold-Hilbert kernel K H M , a manifold extension of the Hilbert kernel [38].
Revised: M , we introduce the manifold-Hilbert kernel K H M , a manifold extension of the Hilbert kernel [37].

Original: Under the same setting as Devroye et al.
Revised: Under the same setting as Devroye et al.

Original: [20], we prove that kernel smoothing regression with K H Mis weakly consistent while interpolating the training data.
Revised: [19], we prove that kernel smoothing regression with K H Mis weakly consistent while interpolating the training data.

Original: Consequently, the classifier obtained bytaking the sign of the kernel smoothing estimate has zero training error and is consistent.
Revised: Consequently, the classifier obtained bytaking the sign of the kernel smoothing estimate has zero training error and is consistent.

Original: Second, we introduce a class of kernels called weighted random partition kernels.
Revised: Second, we introduce a class of kernels called weighted random partition kernels.

Original: These are kernelsthat can be realized as an infinite, weighted ensemble of partition-based histogram classifiers.
Revised: These are kernelsthat can be realized as an infinite, weighted ensemble of partition-based histogram classifiers.

Original: Ourmain result is established by showing that when M = S d , the d -dimensional sphere, the manifoldHilbert kernel is a weighted random partition kernel.
Revised: Ourmain result is established by showing that when M = S d , the d -dimensional sphere, the manifoldHilbert kernel is a weighted random partition kernel.

Original: In particular, we show that on the sphere, themanifold-Hilbert kernel is a weighted ensemble based on random hyperplane arrangements.
Revised: In particular, we show that on the sphere, themanifold-Hilbert kernel is a weighted ensemble based on random hyperplane arrangements.

Original: Thisimplies that the kernel smoothing classifier is a consistent, interpolating ensemble on S d .
Revised: Thisimplies that the kernel smoothing classifier is a consistent, interpolating ensemble on S d .

Original: To ourknowledge, this is the first demonstration of an interpolating ensemble method that is consistent for abroad class of distributions in arbitrary dimensions.
Revised: To ourknowledge, this is the first demonstration of an interpolating ensemble method that is consistent for abroad class of distributions in arbitrary dimensions.

Original: 1.1 Problem statement
Revised: 1.1 Problem statement

Original: Consider the problem of binary classification on a Riemannian manifold M .
Revised: Consider the problem of binary classification on a Riemannian manifold M .

Original: Let ( X, Y ) be randomvariables jointly distributed on M × {± 1 } .
Revised: Let ( X, Y ) be randomvariables jointly distributed on M × {± 1 } .

Original: Let D n := { ( X i , Y i ) } ni =1 be the (random) training 36 data consisting of n i.i.d copies of X, Y .
Revised: Let D n := { ( X i , Y i ) } ni =1 be the (random) training 36 data consisting of n i.i.d copies of X, Y .

Original: A classifier , i.e., a mapping from D n to a function(cid:98) f ( •∥ D n ) :
Revised: A classifier , i.e., a mapping from D n to a function(cid:98) f ( •∥ D n ) :

Original: M → {± 1 } , has the interpolating-consistent property if, when X has a continuousdistribution, both of the following hold: 1) (cid:98) f
Revised: M → {± 1 } , has the interpolating-consistent property if, when X has a continuousdistribution, both of the following hold: 1) (cid:98) f

Original: ( X i ∥ D n ) =
Revised: ( X i ∥ D n ) =

Original: Y i , for all i ∈ { 1 , . . . , n } , and 2)
Revised: Y i , for all i ∈ { 1 , . . . , n } , and 2)

Original: [Equation]
Revised: [Equation]

Original: Our goal is to find an interpolating-consistent ensemble of histogram classifiers , to be defined below.
Revised: Our goal is to find an interpolating-consistent ensemble of histogram classifiers , to be defined below.

Original: A partition on M , denoted by P , is a set of subsets of M such that P ∩ P ′ = ∅ for all P, P ′ ∈ P and41 M = (cid:83)
Revised: A partition on M , denoted by P , is a set of subsets of M such that P ∩ P ′ = ∅ for all P, P ′ ∈ P and

Original:  P ∈P P .
Revised: M = (cid:83) P ∈P P .

Original: Given x ∈ M , let P [ x ] denote the unique element P ∈ P such that x ∈ P .
Revised: Given x ∈ M , let P [ x ] denote the unique element P ∈ P such that x ∈ P .

Original: The setof all partitions on a space M is denoted Part ( M ) .
Revised: The setof all partitions on a space M is denoted Part ( M ) .

Original: The histogram classifier with respect to D n over
Revised: The histogram classifier with respect to D n over

Original: P is the sign of the function (cid:98) h ( •∥ D n , P ) :
Revised: P is the sign of the function (cid:98) h ( •∥ D n , P ) :

Original: M → R given by
Revised: M → R given by

Original: [Equation]
Revised: [Equation]

Original: where I is the indicator function.
Revised: where I is the indicator function.

Original: See Figure 1-left panels.

Original: Definition 1.1.
Revised: Definition 1.1.

Original: A weighted random partition (WRP) over M is a 3-tuple (Θ , P , α ) consisting of (i)parameter space of partitions : a set Θ where P θ
Revised: A weighted random partition (WRP) over M is a 3-tuple (Θ , P , α ) consisting of (i)parameter space of partitions : a set Θ where P θ

Original: ∈ Part ( M ) for each θ ∈ Θ , (ii) random partitions :a probability measure P on Θ , and (iii) weights : a nonnegative function α : Θ → R ≥ 0 .
Revised: ∈ Part ( M ) for each θ ∈ Θ , (ii) random partitions :a probability measure P on Θ , and (iii) weights : a nonnegative function α : Θ → R ≥ 0 .

Original: Example 1.2 (Regular partition of the d -cube) .
Revised: Example 1.2 (Regular partition of the d -cube) .

Original: Let M =
Revised: Let M =

Original: [0 , 1] d and Θ = { 1 , 2 . . .
Revised: [0 , 1] d and Θ = { 1 , 2 . . .

Original: =: N + .
Revised: =: N + .

Original: Foreach n ∈ N + , denote by P n the regular partition of M into n d d -cubes of side length 1 /n .
Revised: Foreach n ∈ N + , denote by P n the regular partition of M into n d d -cubes of side length 1 /n .

Original: For anyprobability mass function P on N + and weights α : N + → R ≥ 0 , the 3-tuple (Θ , P , α ) is a WRP.
Revised: For anyprobability mass function P on N + and weights α : N + → R ≥ 0 , the 3-tuple (Θ , P , α ) is a WRP.

Original: Below, WRPs will be denoted with 2-letter names in the sans-serif font, e.g., “ rp ” for a generic WRP,and “ ha ” for the weighted hyperplane arrangement random partition (Definition 5.1).
Revised: Below, WRPs will be denoted with 2-letter names in the sans-serif font, e.g., “ rp ” for a generic WRP,and “ ha ” for the weighted hyperplane arrangement random partition (Definition 5.1).

Original: The weightedrandom partition kernel associated to rp =
Revised: The weightedrandom partition kernel associated to rp =

Original: (Θ , P , α ) is defined as
Revised: (Θ , P , α ) is defined as

Original: [Equation]
Revised: [Equation]

Original: When α ≡ 1 , we recover the notion of unweighted random partition kernel introduced in [19].
Revised: When α ≡ 1 , we recover the notion of unweighted random partition kernel introduced in [18].

Original: Notethat the kernel is symmetric since I { x ∈ P θ
Revised: Notethat the kernel is symmetric since I { x ∈ P θ

Original: [ z ] } =
Revised: [ z ] } =

Original: I { z ∈ P θ [ x ] } .
Revised: I { z ∈ P θ [ x ] } .

Original: If K rp M < ∞ , then K rp M is a positive 56 definite (PD) kernel.
Revised: If K rp M < ∞ , then K rp M is a positive 56 definite (PD) kernel.

Original: When K rp
Revised: When K rp

Original: M can evaluate to ∞ , the definition of a PD kernel is not applicablesince the positive definite property is defined only for to kernels taking finite values [9].
Revised: M can evaluate to ∞ , the definition of a PD kernel is not applicablesince the positive definite property is defined only for to kernels taking finite values [9].

Original: Let sgn : R ∪{±∞} → {± 1 } be the sign function.
Revised: Let sgn : R ∪{±∞} → {± 1 } be the sign function.

Original: For a WRP, define the weighted infinite-ensemble 

Revised: For a WRP, define the weighted infinite-ensemble

Original: [Equation]
Revised: [Equation]

Original: Note that the equality on the right follows immediately from linearity of the expectation and thedefinition of (cid:98) h ( •∥ D n , P θ ) in Equation (2).See Figure 1-right panel.
Revised: Note that the equality on the right follows immediately from linearity of the expectation and thedefinition of (cid:98) h ( •∥ D n , P θ )] in Equation (2).

Original: Main problem .
Revised: Main problem .

Original: Find a WRP such that sgn ( (cid:98) u ( •∥ D n , K rp M )) has the interpolating-consistent property.
Revised: Find a WRP such that sgn ( (cid:98) u ( •∥ D n , K rp M )) has the interpolating-consistent property.

Original: 1.2 Outline of approach and contributions
Revised: 1.2 Outline of approach and contributions

Original: In the regression setting, we have ( X, Y ) jointly distributed on M × R .
Revised: In the regression setting, we have ( X, Y ) jointly distributed on M × R .

Original: Let m ( x ) :=
Revised: Let m ( x ) :=

Original: E [ Y | X = x ] .
Revised: E [ Y | X = x ] .

Original: Recall from Belkin et al.
Revised: Recall from Belkin et al.

Original: [7, Equation (7)] the definition of the kernel smoothing estimator with aso-called singular 1 kernel K : M × M → [0 , + ∞ ] :
Revised: [7, Equation (7)] the definition of the kernel smoothing estimator with aso-called singular 1 kernel K : M × M → [0 , + ∞ ] :

Original: [Equation][Equation]
Revised: [Equation]

Original: ] such thatx
Revised: that

Original: [Equation]
Revised: [Equation]

Original: We note that Equation (5) is referred as the Nadaraya-Watson estimate in [7].
Revised: We note that Equation (5) is referred as the Nadaraya-Watson estimate in [7].

Original: Now, we simply write(cid:98) m n ( x ) instead of (cid:98) m ( x ∥ D n , K ) when there is no ambiguity.
Revised: Now, we simply write(cid:98) m n ( x ) instead of (cid:98) m ( x ∥ D n , K ) when there is no ambiguity.

Original: Similarly, we write (cid:98) u n ( x ) instead of(cid:98) u
Revised: Similarly, we write (cid:98) u n ( x ) instead of(cid:98) u

Original: ( x ∥ D n , K ) from earlier.
Revised: ( x ∥ D n , K ) from earlier.

Original: Note that sgn ( (cid:98) m n ( x )) =
Revised: Note that sgn ( (cid:98) m n ( x )) =

Original: sgn ( (cid:98) u n ( x ))
Revised: sgn ( (cid:98) u n ( x ))

Original: if (cid:80) nj =1 K ( x, X j ) > 0 .
Revised: if (cid:80) nj =1 K ( x, X j ) > 0 .

Original: Observe that (cid:98) m n is interpolating by construction.
Revised: Observe that (cid:98) m n is interpolating by construction.

Original: Let µ X denote the marginal distribution of X .
Revised: Let µ X denote the marginal distribution of X .

Original: The
Revised: The

Original: L 1 -error of (cid:98) m n in approximating m is J n :=
Revised: L 1 -error of (cid:98) m n in approximating m is J n :=

Original: (cid:82) M | (cid:98) m n ( x ) − m ( x ) | µ X ( dx ) .
Revised: (cid:82) M | (cid:98) m n ( x ) − m ( x ) | µ X ( dx ) .

Original: For M = R d and the
Revised: For M = R d and the

Original: Hilbert kernel defined by K H R d ( x, z ) :
Revised: Hilbert kernel defined by K H R d ( x, z ) :

Original: =
Revised: =

Original: ∥ x − z
Revised: ∥ x − z

Original: ∥ − d , Devroye et al.
Revised: ∥ − d , Devroye et al.

Original: [20] proved L 1 -consistency forregression : J n → 0 in probability when Y is bounded and X is continuously distributed.
Revised: [19] proved L 1 -consistency for 73 regression : J n → 0 in probability when Y is bounded and X is continuously distributed.

Original: Our contributions.
Revised: Our contributions.

Original: Our primary contribution is to demonstrate an ensemble method with theconsistent-interpolating property.
Revised: Our primary contribution is to demonstrate an ensemble method with theconsistent-interpolating property.

Original: Toward this end, in Section 3, we introduce the manifold-Hilbertkernel K H M on a Riemannian manifold M .
Revised: Toward this end, in Section 3, we introduce the manifold-Hilbertkernel K H M on a Riemannian manifold M .

Original: When show that when M is complete, connected, andsmooth, kernel smoothing regression with K H M has the same consistency guarantee (Theorem 3.2) as
Revised: When show that when M is complete, connected, andsmooth, kernel smoothing regression with K H M has the same consistency guarantee (Theorem 3.2) as

Original: K H R d mentioned in the preceding paragraph.
Revised: K H R d mentioned in the preceding paragraph.

Original: In Section 5, we consider the case when M = S d , andshow that the manifold-Hilbert kernel K H S d is a weighted random partition kernel (Proposition 5.2).
Revised: In Section 5, we consider the case when M = S d , andshow that the manifold-Hilbert kernel K H S d is a weighted random partition kernel (Proposition 5.2).

Original: Devroye et al.
Revised: Devroye et al.

Original: [20, Section 7] observed that the L 1 -consistency of (cid:98) m n for regression implies theconsistency for classification of sgn ◦ (cid:98) u n .
Revised: [19, Section 7] observed that the L 1 -consistency of (cid:98) m n for regression implies theconsistency for classification of sgn ◦ (cid:98) u n .

Original: Furthermore, (cid:98) m n is interpolating for regression implies thatsgn ◦ (cid:98) u n is interpolating for classification.
Revised: Furthermore, (cid:98) m n is interpolating for regression implies thatsgn ◦ (cid:98) u n is interpolating for classification.

Original: These observations together with our results demonstratethe existence of a weighted infinite-ensemble classifier with the interpolating-consistent property.
Revised: These observations together with our results demonstratethe existence of a weighted infinite-ensemble classifier with the interpolating-consistent property.

Original: 1.3 Related work
Revised: 1.3 Related work

Original: Kernel regression.
Revised: Kernel regression.

Original: Kernel smoothing regression, or simply kernel regression, is an interpolatorwhen the kernel used is singular, a fact known to Shepard [38] in 1968.
Revised: Kernel smoothing regression, or simply kernel regression, is an interpolatorwhen the kernel used is singular, a fact known to Shepard [37] in 1968.

Original: Devroye et al.
Revised: Devroye et al.

Original: [20] showedthat kernel regression with the Hilbert kernel is interpolating and weakly consistent for data with adensity and bounded labels.
Revised: [19] showedthat kernel regression with the Hilbert kernel is interpolating and weakly consistent for data with adensity and bounded labels.

Original: Using singular kernels with compact support, Belkin et al.
Revised: Using singular kernels with compact support, Belkin et al.

Original: [7] showedthat minimax optimality can be achieved under additional distributional assumptions.
Revised: [7] showedthat minimax optimality can be achieved under additional distributional assumptions.

Original: Random forests.
Revised: Random forests.

Original: Wyner et al.
Revised: Wyner et al.

Original: [41] proposed that interpolation may be a key mechanism for thesuccess of random forests and gave a compelling intuitive rationale.
Revised: [40] proposed that interpolation may be a key mechanism for thesuccess of random forests and gave a compelling intuitive rationale.

Original: Belkin et al.
Revised: Belkin et al.

Original: [5] studiedempirically the double descent phenomenon in random forests by considering the generalizationperformance past the interpolation threshold.
Revised: [5] studiedempirically the double descent phenomenon in random forests by considering the generalizationperformance past the interpolation threshold.

Original: The PERT variant of random forests, introduced by
Revised: The PERT variant of random forests, introduced by

Original: Cutler and Zhao [18], provably interpolates in 1-dimension.
Revised: Cutler and Zhao [17], provably interpolates in 1-dimension.

Original: Belkin et al.
Revised: Belkin et al.

Original: [6] pose as an interestingquestion whether the result of Cutler and Zhao [18] extends to higher dimension.
Revised: [6] pose as an interestingquestion whether the result of Cutler and Zhao [17] extends to higher dimension.

Original: Many workhave established consistency of random forest and its variants under different settings [14, 10, 37].
Revised: Many workhave established consistency of random forest and its variants under different settings [13, 10, 36].

Original: However, none of these work addressed interpolation.
Revised: However, none of these work addressed interpolation.

Original: Boosting.
Revised: Boosting.

Original: For classification under the noiseless setting (i.e., the Bayes error is zero), AdaBoost isinterpolating and consistent (see Freund and Schapire [24, first paragraph of Chapter 12]).
Revised: For classification under the noiseless setting (i.e., the Bayes error is zero), AdaBoost isinterpolating and consistent (see Freund and Schapire [23, first paragraph of Chapter 12]).

Original: However,this setting is too restrictive and the result does not answer if consistency is possible when fitting thenoise.
Revised: However,this setting is too restrictive and the result does not answer if consistency is possible when fitting thenoise.

Original: Bartlett and Traskin [4] proved that AdaBoost with early stopping is universally consistent,however without the interpolation guarantee.
Revised: Bartlett and Traskin [4] proved that AdaBoost with early stopping is universally consistent,however without the interpolation guarantee.

Original: To the best of our knowledge, whether AdaBoost or anyother variant of boosting can be interpolating and consistent remains open.
Revised: To the best of our knowledge, whether AdaBoost or anyother variant of boosting can be interpolating and consistent remains open.

Original: Random partition kernels.
Revised: Random partition kernels.

Original: Breiman [15] and Geurts et al.
Revised: Breiman [14] and Geurts et al.

Original: [26] studied infinite ensembles ofsimplified variants of random forest and connections to certain kernels.
Revised: [25] studied infinite ensembles ofsimplified variants of random forest and connections to certain kernels.

Original: Davies and Ghahramani [19] formalized this connection and coined the term random partition kernel .
Revised: Davies and Ghahramani[18] formalized this connection and coined the term random partition kernel .

Original: Scornet [36] further developed the theory of random forest kernels and obtained upper bounds on the rate of convergence.
Revised: Scornet [35] furtherdeveloped the theory of random forest kernels and obtained upper bounds on the rate of convergence.

Original: However, it is not clear if these variants of random forests are interpolating.
Revised: However, it is not clear if these variants of random forests are interpolating.

Original: Previously defined (unweighted) random partition kernels are bounded, and thus cannot be singular.
Revised: Previously defined (unweighted) random partition kernels are bounded, and thus cannot be singular.

Original: On the other hand, the manifold-Hilbert kernel is always singular.
Revised: On the other hand, the manifold-Hilbert kernel is always singular.

Original: To bridge between ensemblemethods and theory on interpolating kernel smoothing regression, we propose weighted randompartitions (Definition 1.1), whose associated kernel (Equation 3) can be singular.
Revised: To bridge between ensemblemethods and theory on interpolating kernel smoothing regression, we propose weighted randompartitions (Definition 1.1), whose associated kernel (Equation 3) can be singular.

Original: Learning on Riemannian manifolds.
Revised: Learning on Riemannian manifolds.

Original: Strong consistency of a kernel-based classification method onmanifolds has been established by Loubes and Pelletier [30].
Revised: Strong consistency of a kernel-based classification method onmanifolds has been established by Loubes and Pelletier [29].

Original: However, the result requires the kernelto be bounded and thus the method is not guaranteed to be interpolating.
Revised: However, the result requires the kernelto be bounded and thus the method is not guaranteed to be interpolating.

Original: See Feragen and Hauberg[23] for a review of theoretical results regarding kernels on Riemannian manifolds.
Revised: See Feragen and Hauberg[22] for a review of theoretical results regarding kernels on Riemannian manifolds.

Original: Beyond kernel methods, other classical methods for Euclidean data have been extended to Riemannianmanifolds, e.g., regression [39], classification [42], and dimensionality reduction and clustering[43][32].
Revised: Beyond kernel methods, other classical methods for Euclidean data have been extended to Riemannianmanifolds, e.g., regression [38], classification [41], and dimensionality reduction and clustering [Figure] [Equation] [Figure]

Original: To the best of our knowledge, no previous works have demonstrated an interpolatingconsistent classifiers on manifolds other than R d .
Revised: To the best of our knowledge, no previous works have demonstrated an interpolatingconsistent classifiers on manifolds other than R d .

Original: In many applications, the data naturally belong to a Riemannian manifold.
Revised: In many applications, the data naturally belong to a Riemannian manifold.

Original: Spherical data arisefrom a range of disciplines in natural sciences.
Revised: Spherical data arisefrom a range of disciplines in natural sciences.

Original: See the influential textbook by Mardia and Jupp [31,
Revised: See the influential textbook by Mardia and Jupp [30,

Original: Ch.1§4].
Revised: Ch.1§4].

Original: For applications of the Grassmanian manifold in computer vision, see Jayasumana et al.[28] and the references therein.
Revised: For applications of the Grassmanian manifold in computer vision, see Jayasumana et al.[27] and the references therein.

Original: Topological data analysis [40] presents another interesting setting ofmanifold-valued data in the form of persistence diagrams [2, 29].
Revised: Topological data analysis [39] presents another interesting setting ofmanifold-valued data in the form of persistence diagrams [2, 28].

Original:  Background on Riemannian Manifolds
Revised:  Background on Riemannian Manifolds

Original: We give an intuitive overview of the necessary concepts and results on Riemannian manifolds.
Revised: We give an intuitive overview of the necessary concepts and results on Riemannian manifolds.

Original: Alonger, more precise version of this overview is in the Supplemental Materials Section A.1.
Revised: Alonger, more precise version of this overview is in the Supplemental Materials Section A.1.

Original: A smooth d -dimensional manifold M is a topological space that is locally diffeomorphic 2 to opensubsets of R d .
Revised: A smooth d -dimensional manifold M is a topological space that is locally diffeomorphic 2 to opensubsets of R d .

Original: For simplicity, suppose that M is embedded in R N for some N ≥ d , e.g., S d ⊆ R d +1 .
Revised: For simplicity, suppose that M is embedded in R N for some N ≥ d , e.g., S d ⊆ R d +1 .

Original: Let x ∈ M be a point.
Revised: Let x ∈ M be a point.

Original: The tangent space at x , denoted T x M , is the set of vectors that is tangentto M at x .
Revised: The tangent space at x , denoted T x M , is the set of vectors that is tangentto M at x .

Original: Since linear combinations of tangent vectors are also tangent, the tangent space T x Mis a vector space.
Revised: Since linear combinations of tangent vectors are also tangent, the tangent space T x Mis a vector space.

Original: Tangent vectors can also be viewed as the time derivative of smooth curves.
Revised: Tangent vectors can also be viewed as the time derivative of smooth curves.

Original: Inparticular, let x ∈ M .
Revised: Inparticular, let x ∈ M .

Original: If ϵ > 0 is an open set and γ : ( − ϵ, ϵ ) → M is a smooth curve such thatγ (0) =
Revised: If ϵ > 0 is an open set and γ : ( − ϵ, ϵ ) → M is a smooth curve such thatγ (0) =

Original: x , then dγdt (0)
Revised: x , then dγdt (0)

Original: ∈
Revised: ∈

Original: T x M .
Revised: T x M .

Original: A Riemannian metric on M is a choice of inner product ⟨· , ·⟩ x on T x M for each x such that⟨· , ·⟩ x varies smoothly with x .
Revised: A Riemannian metric on M is a choice of inner product ⟨· , ·⟩ x on T x M for each x such that⟨· , ·⟩ x varies smoothly with x .

Original: Naturally,
Revised: Naturally,

Original: ∥ z
Revised: ∥ z

Original: ∥ x := (cid:112) ⟨ z, z ⟩ x defines a norm on T x M .
Revised: ∥ x := (cid:112) ⟨ z, z ⟩ x defines a norm on T x M .

Original: The lengthof a piecewise smooth curve γ : [ a, b ] → M is defined by len ( γ ) :
Revised: The lengthof a piecewise smooth curve γ : [ a, b ] → M is defined by len ( γ ) :

Original: = (cid:82) b a ∥ ˙
Revised: = (cid:82) b a ∥ ˙

Original: γ ( t ) ∥ γ ( t ) dt .
Revised: γ ( t ) ∥ γ ( t ) dt .

Original: Definedist M ( x, ξ ) :
Revised: Definedist M ( x, ξ ) :

Original: = inf { len ( γ ) : γ is a piecewise smooth curve from x to ξ } , which is a metric on Min the sense of metric spaces (see Sakai [35, Proposition 1.1]).
Revised: = inf { len ( γ ) : γ is a piecewise smooth curve from x to ξ } , which is a metric on Min the sense of metric spaces (see Sakai [34, Proposition 1.1]).

Original: For x ∈ M and r ∈ (0 , ∞ ) , the openmetric ball centered at x of radius r is denoted B x
Revised: For x ∈ M and r ∈ (0 , ∞ ) , the openmetric ball centered at x of radius r is denoted B x

Original: ( r, M ) := { ξ ∈ M : dist M ( x, ξ ) < r } .
Revised: ( r, M ) := { ξ ∈ M : dist M ( x, ξ ) < r } .

Original: A curve γ : [ a, b ] → M is a geodesic if γ is locally distance minimizing and has constant speed, i.e.,∥ dγdt ( τ ) ∥
Revised: A curve γ : [ a, b ] → M is a geodesic if γ is locally distance minimizing and has constant speed, i.e.,∥ dγdt ( τ ) ∥

Original: γ ( τ ) is constant.
Revised: γ ( τ ) is constant.

Original: Now, suppose x ∈ M and v ∈
Revised: Now, suppose x ∈ M and v ∈

Original: T x M are such that there exists a geodesicγ : [0 , 1] → M where γ (0)
Revised: T x M are such that there exists a geodesicγ : [0 , 1] → M where γ (0)

Original: =
Revised: =

Original: x and dγdt (0) = v .
Revised: x and dγdt (0) = v .

Original: Define exp x ( v ) :
Revised: Define exp x ( v ) :

Original: = γ (1) , the element reached bytraveling along γ at time = 1 .
Revised: = γ (1) , the element reached bytraveling along γ at time = 1 .

Original: See Figure 2 for the case when M = S 2 .
Revised: See Figure 1 for the case when M = S 2 .

Original: For a fixed x ∈ M , the above function exp x , the exponential map , can be defined on an open subsetof T x M containing the origin.
Revised: For a fixed x ∈ M , the above function exp x , the exponential map , can be defined on an open subsetof T x M containing the origin.

Original: The Hopf-Rinow theorem ([21, Ch. 8, Theorem 2.8]) states that if
Revised: The Hopf-Rinow theorem ([20, Ch. 8, Theorem 2.8]) states that if

Original: M is connected and complete with respect to the metric dist M , then exp x can be defined on all of
Revised: M is connected and complete with respect to the metric dist M , then exp x can be defined on all of

Original: T x M .
Revised: T x M .

Original:  The Manifold-Hilbert kernel
Revised:  The Manifold-Hilbert kernel

Original: Throughout the remainder of this work, we assume that M is a complete, connected, and smooth
Revised: Throughout the remainder of this work, we assume that M is a complete, connected, and smooth

Original: Riemannian manifold of dimension d .
Revised: Riemannian manifold of dimension d .

Original: Definition 3.1.
Revised: Definition 3.1.

Original: We define the manifold-Hilbert kernel K H M : M × M → [0 , ∞ ] for each x, ξ ∈ Mby K H M ( x, ξ ) :
Revised: We define the manifold-Hilbert kernel K H M : M × M → [0 , ∞ ] for each x, ξ ∈ Mby K H M ( x, ξ ) :

Original: = dist M ( x, ξ )
Revised: = dist M ( x, ξ )

Original: − d
Revised: − d

Original: if x ̸ = ξ and K H M ( x, x ) := ∞ otherwise.[Equation] x [Equation]
Revised: if x ̸ = ξ and K H M ( x, x ) := ∞ otherwise.

Original: Let λ M be the Riemann–Lebesgue volume measure of M .
Revised: Let λ M be the Riemann–Lebesgue volume measure of M .

Original: Integration with respect to this measure isdenoted (cid:82) M fdλ M for a function f : M → R .
Revised: Integration with respect to this measure isdenoted (cid:82) M fdλ M for a function f : M → R .

Original: For details of the construction of λ M , see Amann and
Revised: For details of the construction of λ M , see Amann and

Original: Escher [1, Proposition 1.5].
Revised: Escher [1, Proposition 1.5].

Original: When M = R d , λ M is the ordinary Lebesgue measure and (cid:82) R d fdλ R d isthe ordinary Lebesgue integral.
Revised: When M = R d , λ M is the ordinary Lebesgue measure and (cid:82) R d fdλ R d isthe ordinary Lebesgue integral.

Original: For this case, we simply write λ instead of λ R d .
Revised: For this case, we simply write λ instead of λ R d .

Original: We now state our first main result, a manifold theory extension of Devroye et al.
Revised: We now state our first main result, a manifold theory extension of Devroye et al.

Original: [20, Theorem 1].
Revised: [19, Theorem 1].

Original: Theorem 3.2.
Revised: Theorem 3.2.

Original: Suppose that X has a density f
Revised: Suppose that X has a density f

Original: X with respect to λ M and that Y is bounded.
Revised: X with respect to λ M and that Y is bounded.

Original: Let
Revised: Let

Original: P Y | X be a conditional distribution of Y given X and m Y |
Revised: P Y | X be a conditional distribution of Y given X and m Y |

Original: X be its conditional expectation.
Revised: X be its conditional expectation.

Original: Let(cid:98)
Revised: Let(cid:98)

Original: m n ( x ) :
Revised: m n ( x ) :

Original: = (cid:98) m ( x ∥ D n , K H M ) .
Revised: = (cid:98) m ( x ∥ D n , K H M ) .

Original: Then
Revised: Then1.

Original: at almost all x ∈ M with f X ( x ) > 0 , we have (cid:98) m n ( x ) → m Y | X( x ) in probability,
Revised: at almost all x ∈ M with f X ( x ) > 0 , we have (cid:98) m n ( x ) → m Y | X

Original: [Equation]
Revised: ( x ) in probability,2. J n := (cid:82) M | (cid:98) m n ( x ) − m Y | X ( x ) |

Original:  in probability.
Revised: f X ( x ) dλ M ( x ) → 0 in probability.

Original: In words, the kernel smoothing regression estimate (cid:98) m n based on the manifold-Hilbert kernel isconsistent and interpolates the training data, provided X has a density and Y is bounded.
Revised: In words, the kernel smoothing regression estimate (cid:98) m n based on the manifold-Hilbert kernel isconsistent and interpolates the training data, provided X has a density and Y is bounded.

Original: As aconsequence, following the same logic as in Devroye et al.
Revised: As aconsequence, following the same logic as in Devroye et al.

Original: [20], the associated classifier sgn ◦ (cid:98) u nhas the interpolating-consistent property.
Revised: [19], the associated classifier sgn ◦ (cid:98) u nhas the interpolating-consistent property.

Original: Before proving Theorem 3.2, we first review key conceptsin probability theory on Riemannian manifolds.
Revised: Before proving Theorem 3.2, we first review key conceptsin probability theory on Riemannian manifolds.

Original: 3.1 Probability on Riemannian manifolds
Revised: 3.1 Probability on Riemannian manifolds

Original: Let B M be the Borel σ -algebra of M , i.e., the smallest σ -algebra containing all open subsets of M .
Revised: Let B M be the Borel σ -algebra of M , i.e., the smallest σ -algebra containing all open subsets of M .

Original: We recall the definition of M -valued random variables, following Pennec [33, Definition 2]:
Revised: We recall the definition of M -valued random variables, following Pennec [32, Definition 2]:

Original: Definition 3.3.
Revised: Definition 3.3.

Original: Let (Ω , P , A ) be a probability space with measure P and σ -algebra A .
Revised: Let (Ω , P , A ) be a probability space with measure P and σ -algebra A .

Original: A M -valuedrandom variable X is a Borel-measurable function Ω → M , i.e., X − 1 ( B ) ∈
Revised: A M -valued 174 random variable X is a Borel-measurable function Ω → M , i.e., X − 1 ( B ) ∈

Original: A for all B ∈ B M .
Revised: A for all B ∈ B M .

Original: Definition 3.4 (Density) .
Revised: Definition 3.4 (Density) .

Original: A random variable X taking values in M has a density if there exists anonnegative Borel-measurable function f : M → [0 , ∞ ] such that for all Borel sets B in M , we have
Revised: A random variable X taking values in M has a density if there exists anonnegative Borel-measurable function f : M → [0 , ∞ ] such that for all Borel sets B in M , we have

Original: Pr( X ∈ B ) = (cid:82) B fdλ M .
Revised: Pr( X ∈ B ) = (cid:82) B fdλ M .

Original: The function f is said to be a probability density function (PDF) of X .
Revised: The function f is said to be a probability density function (PDF) of X .

Original: Next, we recall the definition of conditional distributions, following Dudley [22, Ch. 10 §2]:
Revised: Next, we recall the definition of conditional distributions, following Dudley [21, Ch. 10 §2]:

Original: Definition 3.5 (Conditional distribution 3 ) .
Revised: Definition 3.5 (Conditional distribution 3 ) .

Original: Let ( X, Y ) be a random variable jointly distributed on
Revised: Let ( X, Y ) be a random variable jointly distributed on

Original: M × R .
Revised: M × R .

Original: Let P X ( · ) be the probability measure corresponding to the marginal distribution of X .
Revised: Let P X ( · ) be the probability measure corresponding to the marginal distribution of X .

Original: A conditional distribution for Y given X is a collection of probability measures P Y | X ( ·| x ) on R 182 indexed by x ∈ M satisfying the following:
Revised: A conditional distribution for Y given X is a collection of probability measures P Y | X ( ·| x ) on R 182 indexed by x ∈ M satisfying the following:

Original: For all Borel sets A ⊆ R , the function M ∋ x
Revised: For all Borel sets A ⊆ R , the function M ∋ x

Original: (cid:55)→ P Y | X ( A | x ) ∈ [0 , 1] is Borel-measurable.
Revised: (cid:55)→ P Y | X ( A | x ) ∈ [0 , 1] is Borel-measurable.

Original: For all A ⊆ R and B ⊆ M Borel sets, Pr( Y ∈ A, X ∈ B ) =
Revised: For all A ⊆ R and B ⊆ M Borel sets, Pr( Y ∈ A, X ∈ B ) =

Original: (cid:82) B P Y | X ( A | x ) P X ( dx ) .
Revised: (cid:82) B P Y | X ( A | x ) P X ( dx ) .

Original: The conditional expectation 4 is defined as m Y |
Revised: The conditional expectation 4 is defined as m Y |

Original: X ( x ) :=
Revised: X ( x ) :

Original: [Equation] | 
Revised: = (cid:82) R yP Y | X ( dy | x ) .

Original: [Equation] The existence of a conditional probability for a joint distribution ( X, Y ) is guaranteed by Dudley[22, Theorem 10.2.2].
Revised: The existence of a conditional probability for a joint distribution ( X, Y ) is guaranteed by Dudley[21, Theorem 10.2.2].

Original: When ( X, Y ) has a joint density f XY and marginal density f X , the abovedefinition gives the classical formula P Y | X
Revised: When ( X, Y ) has a joint density f XY and marginal density f X , the abovedefinition gives the classical formula P Y | X

Original: ( A | x ) = (cid:82) A f XY ( x, y ) /f
Revised: ( A | x ) = (cid:82) A f XY ( x, y ) /f

Original: X ( x ) dy when ∞ > f X ( x ) > 0 .
Revised: X ( x ) dy when ∞ > f X ( x ) > 0 .

Original: See the first example in Dudley [22, Ch. 10 §2].
Revised: See the first example in Dudley [21, Ch. 10 §2].

Original: 3.2 Lebesgue points on manifolds
Revised: 3.2 Lebesgue points on manifolds

Original: Devroye et al.
Revised: Devroye et al.

Original: [20] proved Theorem 3.2 when M = R d and, moreover, that part 1 holds for theso-called Lebesgue points , whose definition we now recall.
Revised: [19] proved Theorem 3.2 when M = R d and, moreover, that part 1 holds for theso-called Lebesgue points , whose definition we now recall.

Original: Definition 3.6.
Revised: Definition 3.6.

Original: Let f : M → R be an absolutely integrable function and x ∈ M .
Revised: Let f : M → R be an absolutely integrable function and x ∈ M .

Original: We say that x is a  Lebesgue point of f if f ( x ) =
Revised: We say that x is a 194 Lebesgue point of f if f ( x ) =

Original: lim r →
Revised: lim r →

Original: [Equation]
Revised: [Equation]

Original: For an integrable function, the following result states that almost all points are its Lebesgue points.
Revised: For an integrable function, the following result states that almost all points are its Lebesgue points.

Original: For the proof, see Fukuoka [25, Remark 2.4].
Revised: For the proof, see Fukuoka [24, Remark 2.4].

Original: Theorem 3.7 (Lebesgue differentation) .
Revised: Theorem 3.7 (Lebesgue differentation) .

Original: Let f : M → R be an absolutely integrable function.
Revised: Let f : M → R be an absolutely integrable function.

Original: Thenthere exists a set A ⊆ M such that λ M ( A )
Revised: Thenthere exists a set A ⊆ M such that λ M ( A )

Original: = 

Revised: =

Original: and every x ∈ M \ A is a Lebesgue point of f .
Revised: and every x ∈ M \ A is a Lebesgue point of f .

Original: Next, for the reader’s convenience, we restate Devroye et al.
Revised: Next, for the reader’s convenience, we restate Devroye et al.

Original: [20, Theorem 1], emphasizing theconnection to Lebesgue points.
Revised: [19, Theorem 1], emphasizing theconnection to Lebesgue points.

Original: The result will be used in our proof of Theorem 3.2 in the nextsection.

Original: Theorem 3.8 (Devroye et al.
Revised: Theorem 3.8 (Devroye et al.

Original: Let M
Revised: Let M

Original: =
Revised: =

Original: R
Revised: R

Original: d be the flat Euclidean space.
Revised: d be the flat Euclidean space.

Original: Then Theorem 3.2holds.
Revised: Then Theorem 3.2holds.

Original: Moreover, Part 1 holds for all x that is a Lebesgue point to both f X and m Y | X · f X .
Revised: Moreover, Part 1 holds for all x that is a Lebesgue point to both f X and m Y | X · f X .

Revised: The above result will be used in our proof of Theorem 3.2 below.

Original:  Proof of Theorem 3.2
Revised:  Proof of Theorem 3.2

Original: The focal point of the first subsection is Lemma 4.1 which shows the Borel measurability of extensionsof the so-called Riemannian logarithm.
Revised: The focal point of the first subsection is Lemma 4.1 which shows the Borel measurability of extensionsof the so-called Riemannian logarithm.

Original: The second subsection contains two key results regardingdensities of M -valued random variables transformed by the Riemannian logarithm.
Revised: The second subsection contains two key results regardingdensities of M -valued random variables transformed by the Riemannian logarithm.

Original: The finalsubsection proves Theorem 3.2 leveraging results from the preceding two subsections.
Revised: The finalsubsection proves Theorem 3.2 leveraging results from the preceding two subsections.

Original: 4.1 The Riemannian logarithm
Revised: 4.1 The Riemannian logarithm

Original: Throughout, x is assumed to be an arbitrary point of M .
Revised: Throughout, x is assumed to be an arbitrary point of M .

Original: Let U x M = { v ∈ T x M : ∥ v ∥ x = 1 } ⊆ 211 T x M denote the set of unit tangent vectors.
Revised: Let U x M = { v ∈ T x M : ∥ v ∥ x = 1 } ⊆ 211 T x M denote the set of unit tangent vectors.

Original: Define a function τ x : U x M → (0 , ∞ ] as follows 5 :
Revised: Define a function τ x : U x M → (0 , ∞ ] as follows 5 :

Original: [Equation]
Revised: [Equation]

Original: The tangent cut locus is the set
Revised: The tangent cut locus is the set

Original: ˜ C x ⊆ T x M defined by ˜
Revised: ˜ C x ⊆ T x M defined by ˜

Original: C x := { τ x ( u ) u : u ∈ U x M, τ x ( u ) < ∞} .
Revised: C x := { τ x ( u ) u : u ∈ U x M, τ x ( u ) < ∞} .

Original: Note that it is possible for τ x ( u ) = ∞ for all u ∈ U x M in which case ˜ C x is empty.
Revised: Note that it is possible for τ x ( u ) = ∞ for all u ∈ U x M in which case ˜ C x is empty.

Original: The cut locus isthe set C x:= exp x ( ˜ C x ) ⊆ M .
Revised: The cut locus isthe set [Equation] x

Original:  The tangent interior set is ˜ I x := { tu : 0 ≤ t < τ x ( u ) , u ∈ U x M } and the interior set is the set
Revised: [Equation] The tangent interior set is ˜ I x := { tu : 0 ≤ t < τ x ( u ) , u ∈ U x M } and the interior set is the set

Original: I x := exp x (˜ I x ) .
Revised: I x := exp x (˜ I x ) .

Original: Finally, define ˜ D
Revised: Finally, define ˜ D

Original: x := ˜
Revised: x := ˜

Original: I x ∪ ˜ C x .
Revised: I x ∪ ˜ C x .

Original: Note that for each z = tu ∈
Revised: Note that for each z = tu ∈

Original: ˜
Revised: ˜

Original: I x , we have
Revised: I x , we have

Original: [Equation]
Revised: [Equation]

Original: Consider the example where M = S 2 as in Figure 2.
Revised: Consider the example where M = S 2 as in Figure 1.

Original: Then τ x ( u ) =
Revised: Then τ x ( u ) =

Original: π for all u ∈ U x M .
Revised: π for all u ∈ U x M .

Original: Thus, thetangent interior set ˜ I
Revised: Thus, thetangent interior set ˜ I

Original: x = B 0 ( π, R 2 ) , the open disc of radius π centered at the origin.
Revised: x = B 0 ( π, R 2 ) , the open disc of radius π centered at the origin.

Original: When restricted to ˜ I x , the exponential map exp x | ˜ I x : ˜ I x
Revised: When restricted to ˜ I x , the exponential map exp x | ˜ I x : ˜ I x

Original: → I x is a diffeomorphism.
Revised: → I x is a diffeomorphism.

Original: Its functionalinverse, denoted by log x | I x , is called the Riemannian Logarithm [8, 44].
Revised: Its functionalinverse, denoted by log x | I x , is called the Riemannian Logarithm [8, 43].

Original: In previous works, log x | I xis only defined from I x to ˜ I x .
Revised: In previous works, log x | I xis only defined from I x to ˜ I x .

Original: The next result shows that the domain of log x | I x : I x → ˜ I x can beextended to log x : M → ˜ D x while remaining Borel-measurable.
Revised: The next result shows that the domain of log x | I x : I x → ˜ I x can beextended to log x : M → ˜ D x while remaining Borel-measurable.

Original: Lemma 4.1.
Revised: Lemma 4.1.

Original: For all x ∈ M , there exists a Borel measurable map log x : M → T x M such thatlog x ( M ) ⊆
Revised: For all x ∈ M , there exists a Borel measurable map log x : M → T x M such thatlog x ( M ) ⊆

Original: ˜ D x and exp x ◦ log x is the identity on M .
Revised: ˜ D x and exp x ◦ log x is the identity on M .

Original: Furthermore, for all x, ξ ∈ M , we havedist M ( x, ξ ) = ∥ log x ( ξ ) ∥ x .
Revised: Furthermore, for all x, ξ ∈ M , we have 225 dist [Equation]

Original: Proof sketch.
Revised: Proof sketch.

Original: The full proof of the lemma is provided in Section A.2 of the Supplemental Materials.
Revised: The full proof of the lemma is provided in Section A.2 of the Supplemental Materials.

Original: Below, we illustrate the idea of the proof using the example when M = S 2 as in Figure 2.
Revised: Below, we illustrate the idea of the proof using the example when M = S 2 as in Figure 1.

Original: Let x ∈ S 2 be the “northpole” (the blue point).
Revised: Let x ∈ S 2 be the “northpole” (the blue point).

Original: The tangent cut locus ˜ C x is the dashed circle in theleft panel of Figure 2.
Revised: The tangent cut locus ˜ C x is the dashed circle in theleft panel of Figure 1.

Original: The exponential map exp x is one-to-one on ˜ D
Revised: The exponential map exp x is one-to-one on ˜ D

Original: x
Revised: x

Original: except on the dashed circle,which all gets mapped to − x , the “southpole” (the orange point).
Revised: except on the dashed circle,which all gets mapped to − x , the “southpole” (the orange point).

Original: A consequence of the measurableselection theorem 6 is that log x can be extended to be a Borel-measurable right inverse of exp x byselecting z point on ˜ C x such that log x (− x ) =
Revised: A consequence of the measurableselection theorem 6 is that log x can be extended to be a Borel-measurable right inverse of exp x byselecting z point on ˜ C x such that log x ( x ) =

Original: z .
Revised: z .

Revised: 4.2 Random variable transforms

Original: Thus, we’ve shown that log x : M → T x M is Borel-measurable.
Revised: In the previous subsection, we showed that log x : M → T x M is Borel-measurable.

Original: Now, recall that T x M is equippedwith the inner product ⟨· , ·⟩ x , i.e., the Riemannian metric.
Revised: Now, recall that T x M is equipped with the inner product ⟨· , ·⟩ x , i.e., the Riemannian metric.

Original: Below, for each x ∈ M  choose anorthonormal basis on T x M with respect to ⟨· , ·⟩ .
Revised: Below, for each x ∈ M 236 choose an orthonormal basis on T x M with respect to ⟨· , ·⟩ .

Original: Then T x M is isomorphic as an inner product spaceto R d with the usual dot product.
Revised: Then T x M is isomorphic as an innerproduct space to R d with the usual dot product.

Original: Our next two results are “change-of-variables formulas” for computing the densities/conditionaldistributions of M -valued random variables after the log x transform.
Revised: Our first result of this subsection is a “change-of-variables formula” for computing the densitiesof M -valued random variables after the log x transform.

Original: Recall that λ M is the RiemannLebesgue measure on M and λ is the ordinary Lebesgue measure on R d = T x M .
Revised: Recall that λ M is the Riemann-Lebesguemeasure on M and λ is the ordinary Lebesgue measure on R d = T x M .

Original: Proposition 4.2.
Revised: Proposition 4.2.

Original: Let x ∈ M be fixed.
Revised: Let x ∈ M be fixed.

Original: There exists a Borel measurable function ν x : M → R withthe following properties:
Revised: There exists a Borel measurable function ν x : M → R with 242 the following properties:

Original: (i) Let X be a random variable on M with density f X and let Z :=
Revised: (i) Let X be a random variable on M with density f X and let Z :=

Original: log x ( X ) .
Revised: log x ( X ) .

Original: Then Z is a randomvariable on T x M with density f Z ( z ) :
Revised: Then Z is a randomvariable on T x M with density f Z ( z ) :

Original: = f X (exp x ( z )) ·
Revised: = f X (exp x ( z )) ·

Original: ν x (exp x ( z )) .(ii)
Revised: ν x (exp x ( z )) .(ii)

Original: Let f : M → R be an absolutely integrable function such that x is a Lebesgue point of f . Define f : T x M → R by h ( z ) :
Revised: Let f : M → R be an absolutely integrable function such that x is a Lebesgue point of f .Define f : T x M → R by h ( z ) :

Original: = f
Revised: = f

Original: (exp x ( z )) · ν x (exp x ( z )) .
Revised: (exp x ( z )) · ν x (exp x ( z )) .

Original: Then 0 ∈ T x M is a Lebesguepoint for h .
Revised: Then 0 ∈ T x M is a Lebesguepoint for h .

Original: Proposition 4.3.
Revised: Proof sketch. The full proof of the proposition is in Supplemental Materials Section A.3. The functionν x is the Jacobian of the change-of-variables formula for integrating (cid:82) ˜ B

Revised: f Z dλ where ˜ B ⊆ T x M isa Borel subset.

Revised: See Supplemental Materials Lemma A.4 for the exact definition of ν x .

Revised: Part (i) is asimple consequence of this change-of-variables formula, which says that (cid:82)

Revised: ˜ B

Revised: f Z dλ = (cid:82) exp x

Revised: ( ˜ B ) hdλ M .

Revised: For part (ii), the key observations are that (a) ν x

Revised: (exp x (0))

Revised: =

Original: Let ( X, Y ) have a joint distribution on M × R such that the marginal of Xhas a density f X on M  . 
Revised: ν x (0) = 1 and (b) the volumes of B x ( r, M ) and B 0 ( r, T x M ) are equal as r → 0 . More precisely, lim r → 0 λ M ( B x ( r,M )) λ ( B 0 ( r,T x M ))

Revised: = 1 .

Original:  Let P Y | X ( ·|· ) be a conditional distribution for Y given X .
Revised: Fromthese two observations, it is straightforward to directly verify Definition 3.6. Proposition 4.3. Let ( X, Y ) have a joint distribution on M × R such that the marginal of Xhas a density f X on M . Let P Y | X ( ·|· ) be a conditional distribution for Y given X .

Original: Let x ∈ M .
Revised: Let x ∈ M .

Original: Define Z := log x ( X ) and consider the joint distribution ( Z, Y ) on T p M × R .
Revised: Define Z := log x ( X ) and consider the joint distribution ( Z, Y ) on T p M × R .

Original: Then P Y | Z ( ·|· ) :
Revised: Then P Y | Z ( ·|· ) :

Original: =
Revised: =

Original: P Y | X ( ·| exp x ( · )) is a conditional distribution for Y given Z .
Revised: P Y | X ( ·| exp x ( · )) is a conditional distribution for Y given Z .

Original: Consequently, m Y | X ◦ exp x
Revised: Consequently, m Y | X ◦ exp x

Original: = m Y
Revised: = m Y

Original: | Z .
Revised: | Z .

Original: The above propositions are straightforward manifold-theoretic extensions of well-known results on

Original: Euclidean spaces.
Revised: Proof sketch.

Original: For completeness, the full proofs are in Supplemental Materials Section A.4.
Revised: The full proof of the Proposition is in Supplemental Materials Section A.4.

Revised: The idea isthe same as in the proof of Proposition 4.2, except that the probability density f

Revised: Z is replaced by anappropriate conditional probability density.

Original: 4.  Proof of Theorem 3. 
Revised: 4. Finishing up the Proof of Theorem 3.

Original: Fix x ∈ M such that x is a Lebesgue point of f X and m Y | X · f X .
Revised: Fix x ∈ M such that x is a Lebesgue point of f X and m Y | X · f X .

Original: Note that by Theorem 3.7, almostall x ∈ M has this property.
Revised: Note that by Theorem 3.7, almostall x ∈ M has this property.

Original: Next, let Z = log x ( X ) and f Z be as in Proposition 4.2-(i).
Revised: Next, let Z = log x ( X ) and f Z be as in Proposition 4.2-(i).

Original: Then1.
Revised: Then1.

Original: f
Revised: f

Original: Z =
Revised: Z =

Original: ( f X ◦ exp x ) · ( ν x ◦ exp x ) , and
Revised: ( f X ◦ exp x ) · ( ν x ◦ exp x ) , and

Original: [Equation]
Revised: [Equation]

Original: Now, proposition 4.2-(ii) implies that 0 is a Lebesgue point of both f Z and ( m Y | X ◦ exp x ) · f Z . {+
+}
Revised: Now, proposition 4.2-(ii) implies that 0 is a Lebesgue point of both f Z and ( m Y | X ◦ exp x ) · f Z . {+
+}

Original: Furthermore, by Proposition 4.3, we have m Y | X ◦ exp x
Revised: Furthermore, by Proposition 4.3, we have m Y | X ◦ exp x

Original: = m Y
Revised: = m Y

Original: | Z .
Revised: | Z .

Original: Thus, 0 is a Lebesgue point of f Zand m Y | Z · f Z .
Revised: Thus, 0 is a Lebesgue point of f Z 269 and m Y | Z · f Z .

Original: Now, let D n := { ( X i , Y i ) } i ∈ [ n ] .
Revised: Now, let D n := { ( X i , Y i ) } i ∈ [ n ] .

Original: Define Z i :=
Revised: Define Z i :=

Original: log x
Revised: log x

Original: ( X i ) , which are i.i.d copies of the randomvariable Z :=
Revised: ( X i ) , which are i.i.d copies of the randomvariable Z :=

Original: log x ( X ) , and let ˜ D n := { ( Z i , Y i ) } i ∈ [ n ] .
Revised: log x ( X ) , and let ˜ D n := { ( Z i , Y i ) } i ∈ [ n ] .

Original: Then we have
Revised: Then we have

Original: [Equation]
Revised: [Equation]

Original: where equations marked by (a) and (d) follow from Equation (5), (b) from Lemma 4.1, and (c) fromthe fact that the inner product space T x M with ⟨· , ·⟩ x is isomorphic to R d with the usual dot product.
Revised: where equations marked by (a) and (d) follow from Equation (5), (b) from Lemma 4.1, and (c) fromthe fact that the inner product space T x M with ⟨· , ·⟩ x is isomorphic to R d with the usual dot product.

Original: By Theorem 3.8, we have (cid:98) m (0 ∥ ˜ D n , K H R d ) →
Revised: By Theorem 3.8, we have (cid:98) m (0 ∥ ˜ D n , K H R d ) →

Original: m Y | Z (0) in probability.
Revised: m Y | Z (0) in probability.

Original: In other words, for all ϵ > 0 ,
Revised: In other words, for all ϵ > 0 ,

Original: [Equation]
Revised: [Equation]

Original: By Proposition 4.3, we have m Y | Z (0) =
Revised: By Proposition 4.3, we have m Y | Z (0) =

Original: m Y | Z (exp x (0))
Revised: m Y | Z (exp x (0))

Original: =
Revised: =

Original: m Y | Z ( x ) .
Revised: m Y | Z ( x ) .

Original: Therefore,
Revised: Therefore,(cid:110)

Original: [Equation]
Revised: [Equation]

Original: as events.
Revised: as events.

Original: Thus, (cid:98) m ( x ∥ D n , K H M )
Revised: Thus, (cid:98) m ( x ∥ D n , K H M )

Original: → m Y | X ( x ) converges in probability, proving Theorem 3.2 part 1.
Revised: → m Y | X ( x ) converges in probability, proving Theorem 3.2 part 1.

Original: As noted in Devroye et al.
Revised: As noted in Devroye et al.

Original: [20, §2], part 2 of Theorem 3.2 is an immediate consequence of part 1.
Revised: [19, §2], part 2 of Theorem 3.2 is an immediate consequence of part 1.

Original:  Application to the d -Sphere
Revised:  Application to the d -Sphere

Original: The d -dimensional round sphere is S d := { x ∈ R d +1 : x 21 + · · · + x 2 d +1 = 1 } .
Revised: The d -dimensional round sphere is S d := { x ∈ R d +1 : x 21 + · · · + x 2 d +1 = 1 } .

Original: Here, a round sphereassumes that S d has the arc-length metric :
Revised: Here, a round sphereassumes that S d has the arc-length metric :

Original: [Equation]
Revised: [Equation]

Original: Let S be a set and σ : M → S be a function.
Revised: Let S be a set and σ : M → S be a function.

Original: The partition induced by σ is defined by { σ − 1 ( s ) : s ∈
Revised: The partition induced by σ is defined by { σ − 1 ( s ) : s ∈

Original: Range ( σ ) } .
Revised: Range ( σ ) } .

Original: For example, when M = S d and W ∈ R ( d +1) × h , then the function σ W : S d → {± 1 } hdefined by σ W ( x ) = sgn ( W ⊤ x ) induces a hyperplane arrangement partition.
Revised: For example, when M = S d and W ∈ R ( d +1) × h , then the function σ W : S d → {± 1 } hdefined by σ W ( x ) = sgn ( W ⊤ x ) induces a hyperplane arrangement partition.

Original: Let N = { 1 , 2 , . . . }
Revised: Let N = { 1 , 2 , . . . }

Original: and N 0 = N ∪ { 0 } denote the positive and non-negative integers.
Revised: and N 0 = N ∪ { 0 } denote the positive and non-negative integers.

Original: Definition 5.1 (Random hyperplane arrangement partition) .
Revised: Definition 5.1 (Random hyperplane arrangement partition) .

Original: Let d ∈ N and M = S d .
Revised: Let d ∈ N and M = S d .

Original: Let q < 0 be anegative number, and let H be a random variable with probability mass function p H : N 0 → [0 , 1]such that p H ( h ) > 0 for all h .
Revised: Let q < 0 be anegative number, and let H be a random variable with probability mass function p H : N 0 → [0 , 1]such that p H ( h ) > 0 for all h .

Original: Define the following weighted random partition ha :=
Revised: Define the following weighted random partition ha :=

Original: (Θ , P , α ) 
Revised: (Θ , P , α ) :1.

Original: The parameter space Θ = (cid:70) ∞
Revised: The parameter space Θ = (cid:70) ∞

Original: h =

Revised: h =

Original: R ( d +1)
Revised: R ( d +1)

Original: × h is the disjoint union of all ( d + 1) × h matrices.
Revised: × h is the disjoint union of all ( d + 1) × h matrices.

Original: Element of Θ are matrices θ = W ∈ R ( d +1)
Revised: Element of Θ are matrices θ = W ∈ R ( d +1)

Original: × h where the number of columns h ∈{ 0 , 1 , 2 , . . .
Revised: × h where the number of columns h ∈{ 0 , 1 , 2 , . . .

Original: } varies.
Revised: } varies.

Original: By convention, if h = 0 , the partition P θ =
Revised: By convention, if h = 0 , the partition P θ =

Original: P W is the trivial partition{ S d } .
Revised: P W is the trivial partition{ S d } .

Original: If h > 0 , P W is the partition induced by x (cid:55)→ sgn ( W ⊤ x ) .2.
Revised: If h > 0 , P W is the partition induced by x (cid:55)→ sgn ( W ⊤ x ) .2.

Original: The probability P is constructed by the procedure where we first sample h ∼ p H ( h ) , thensample the entries of W ∈ R d × h i.i.d according to Gaussian (0 , 1) .
Revised: The probability P is constructed by the procedure where we first sample h ∼ p H ( h ) , thensample the entries of W ∈ R d × h i.i.d according to Gaussian (0 , 1) .3.

Original: For θ ∈ Θ , define α ( θ ) :
Revised: For θ ∈ Θ , define α ( θ ) :

Original: = π q p H ( h ) − 1 ( − 1) h (cid:0) qh (cid:1) , where (cid:0) qh (cid:1) :
Revised: = π q p H ( h ) − 1 ( − 1) h (cid:0) qh (cid:1) , where (cid:0) qh (cid:1) :

Original: = 1 h !
Revised: = 1 h !

Original: (cid:81)
Revised: (cid:81)

Original: h − 1 j =0 ( q − j ) 
Revised: h − 1 j =0 ( q − j ) .

Original: Note that ( − 1) h (cid:0) qh (cid:1)
Revised: Note that ( − 1) h (cid:0) qh (cid:1)

Original: = 1 h !
Revised: = 1 h !

Original: (cid:81)
Revised: (cid:81)

Original: h − 1 j =0 ( q − j ) > 0 when q < 0 .
Revised: h − 1 j =0 ( q − j ) > 0 when q < 0 .

Original: Theorem 5.2.
Revised: Theorem 5.2.

Original: Let ha =
Revised: Let ha =

Original: (Θ , P , α ) be as in Definition 5.1.
Revised: (Θ , P , α ) be as in Definition 5.1.

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original:  When q = − d , we have K ha S d = K H S d where the right hand side is the manifold-Hilbert kernel.
Revised: + [Equation] : otherwise. When q = − d , we have K ha S d = K H S d where the right hand side is the manifold-Hilbert kernel.

Original: Proof of Theorem 5.2.
Revised: Proof of Theorem 5.2.

Original: Before proceeding, we have the following useful lemma:
Revised: Before proceeding, we have the following useful lemma:

Original: Lemma 5.3.
Revised: Lemma 5.3.

Original: Let rp =
Revised: Let rp =

Original: (Θ , P , α ) be a WRP.
Revised: (Θ , P , α ) be a WRP.

Original: Let H be a random variable.
Revised: Let H be a random variable.

Original: Let θ ∼ P .
Revised: Let θ ∼ P .

Original: Suppose that forall x, z ∈ M , the random variables α ( θ ) and I { x ∈ P θ [ z ] } are conditionally independent given H .
Revised: Suppose that forall x, z ∈ M , the random variables α ( θ ) and I { x ∈ P θ [ z ] } are conditionally independent given H .

Original: Then we have K rp M ( x, z ) =
Revised: Then we have K rp M ( x, z ) =

Original: E H (cid:104) α
Revised: E H (cid:104) α

Original: ( H ) ·
Revised: ( H ) ·

Original: E θ ∼ P [ I { x ∈ P θ [ z ] }| H ] (cid:105) where α ( h ) :
Revised: E θ ∼ P [ I { x ∈ P θ [ z ] }| H ] (cid:105) where α ( h ) :

Original: =
Revised: =

Original: E θ ∈ P [ α ( θ ) | H
Revised: E θ ∈ P [ α ( θ ) | H

Original: = h ]for a realization h of H .
Revised: = h ]for a realization h of H .

Original: The lemma follows immediately from the Definition of K rp M ( x, z ) in Equation 3 and the conditionalindependence assumption.
Revised: The lemma follows immediately from the Definition of K rp M ( x, z ) in Equation 3 and the conditionalindependence assumption.

Original: Now, we proceed with the proof of Theorem 5.2.
Revised: Now, we proceed with the proof of Theorem 5.2.

Original: Let ϕ :
Revised: Let ϕ :

Original: = ∠ ( x, z ) /π .
Revised: = ∠ ( x, z ) /π .

Original: Let H ∼ p H and θ ∼ P be the random variables in Definition 5.1.
Revised: Let H ∼ p H and θ ∼ P be the random variables in Definition 5.1.

Original: Note thatby construction, the following condition is satisfied: for all x, z ∈ M , the random variables α ( θ )and I { x ∈ P θ [ z ] } are conditionally independent given H .
Revised: Note thatby construction, the following condition is satisfied: for all x, z ∈ M , the random variables α ( θ ) and I { x ∈ P θ [ z ] } are conditionally independent given H .

Original: In fact, α ( θ ) =
Revised: In fact, α ( θ ) =

Original: π q p H ( h ) − 1 ( − 1) h (cid:0) qh (cid:1) isconstant given H = h .
Revised: π q p H ( h ) − 1 ( − 1) h (cid:0) qh (cid:1) isconstant given H = h .

Original: Hence, applying Lemma 5.3, we have
Revised: Hence, applying Lemma 5.3, we have

Original: [Equation]
Revised: [Equation]

Original: Next, we claim that Pr { x ∈ P θ [ z ] | H = h } =
Revised: Next, we claim that Pr { x ∈ P θ [ z ] | H = h } =

Original: (1 − ϕ ) h .
Revised: (1 − ϕ ) h .

Original: When h = 0 , x ∈ P θ [ z ] is always true since
Revised: When h = 0 , x ∈ P θ [ z ] is always true since

Original: P θ = { S d } is the trivial partition.
Revised: P θ = { S d } is the trivial partition.

Original: In this case, we have Pr { x ∈ P θ
Revised: In this case, we have Pr { x ∈ P θ

Original: [ z ] | H = h } = 

Revised: [ z ] | H = h } =

Original: = (1 − ϕ ) 0 .
Revised: = (1 − ϕ ) 0 .

Original: 302 When h > 0 , we recall a result of Pinelis [34]:
Revised: When h > 0 , we recall a result of Pinelis [33]:

Original: Lemma 5.4.
Revised: Lemma 5.4.

Original: Let x, z ∈ S d .
Revised: Let x, z ∈ S d .

Original: Let w ∈ R
Revised: Let w ∈ R

Original: d
Revised: d

Original: +1 be a random vector whose entries are sampled i.i.daccording to Gaussian (0 , 1) .
Revised: +1 be a random vector whose entries are sampled i.i.daccording to Gaussian (0 , 1) .

Original: Then Pr { sgn ( w ⊤ x ) = sgn ( w ⊤ z ) } = 1 − ( ∠ ( x, z ) /π ) .
Revised: Then Pr { sgn ( w ⊤ x ) = sgn ( w ⊤ z ) } = 1 − ( ∠ ( x, z ) /π ) .

Original: Let W =
Revised: Let W =

Original: [ w 1 , . . .
Revised: [ w 1 , . . .

Original: , w h ] be as in Definition 5.1 where w j denotes the j -th column of W .
Revised: , w h ] be as in Definition 5.1 where w j denotes the j -th column of W .

Original: Thenby construction, w j is distributed identically as w in Lemma 5.4.
Revised: Thenby construction, w j is distributed identically as w in Lemma 5.4.

Original: Furthermore, w j and w j ′ areindependent for j, j ′ ∈
Revised: Furthermore, w j and w j ′ areindependent for j, j ′ ∈

Original: [ h ] where j ̸ = j ′ .
Revised: [ h ] where j ̸ = j ′ .

Original: Thus, the claim follows from
Revised: Thus, the claim follows from

Revised: Pr {

Original: [Equation]
Revised: [Equation]

Original:  where equality (a) follows from Definition 5.1, (b) from W ∈ R ( d +1)
Revised: − [Equation] where equality (a) follows from Definition 5.1, (b) from W ∈ R ( d +1)

Original: × h having i.i.d standard
Revised: × h having i.i.d standard

Original: Gaussian entries given H = h , and (c) from Lemma 5.4.
Revised: Gaussian entries given H = h , and (c) from Lemma 5.4.

Original: Putting it all together, we have
Revised: Putting it all together, we have

Original: [Equation]
Revised: [Equation]

Original: h = (cid:18) q h cid:19) ( ϕ − 1) h = ∠ ( x, z ) q . For the last step, we used the fact that for all q ∈ R the binomial series (1 + t ) q = (cid:80) ∞
Revised: For the last step, we used the fact that for all q ∈ R the binomial series (1 + t ) q = (cid:80) ∞

Original: h =0 (cid:0) qh (cid:1) t hconverges absolutely for | t | < 1 (when ϕ ∈ (0 , 1] ) and diverges to + ∞ for t = − 1 (when ϕ = 0 ).
Revised: h [Equation] (cid:0) qh (cid:1) t hconverges absolutely for | t | < 1 (when ϕ ∈ (0 , 1] ) and diverges to + ∞ for t = − 1 (when ϕ = 0 ).

Original: Corollary 5.5.
Revised: Corollary 5.5.

Original: Let q := − d
Revised: Let q := − d

Original: and K ha S d be as in Theorem 5.2.
Revised: and K ha S d be as in Theorem 5.2.

Original: The infinite-ensemble classifiersgn ( (cid:98) u ( •∥ D n , K ha S d )) (see Equation 4 for definition) has the interpolating-consistent property.
Revised: The infinite-ensemble classifiersgn ( (cid:98) u ( •∥ D n , K ha S d )) (see Equation 4 for definition) has the interpolating-consistent property.

Original: Proof.
Revised: Proof.

Original: As observed in Devroye et al.
Revised: As observed in Devroye et al.

Original: [20, Section 7], for an arbitrary kernel K , the L 1 -consistency of(cid:98) m ( •∥ D n , K ) for regression implies the consistency for classification of sgn ( (cid:98) u ( •∥ D n , K )) .
Revised: [19, Section 7], for an arbitrary kernel K , the L 1 -consistency of(cid:98) m ( •∥ D n , K ) for regression implies the consistency for classification of sgn ( (cid:98) u ( •∥ D n , K )) .

Original: Furthermore, (cid:98) m ( •∥ D n , K ) is interpolating for regression implies that sgn ( (cid:98) u ( •∥ D n , K )) is interpolating forclassification.
Revised: Furthermore, (cid:98) m ( •∥ D n , K ) is interpolating for regression implies that sgn ( (cid:98) u ( •∥ D n , K )) is interpolating forclassification.

Original: While the argument there is presented in the R d case, the argument holds in the moregeneral manifold case mutatis mutandis .
Revised: While the argument there is presented in the R d case, the argument holds in the moregeneral manifold case mutatis mutandis .

Original: Thus, by Theorem 3.2, we have sgn ( (cid:98) u ( •∥ D n , K H S d )) is consistent for classification, i.e., Equation (1)holds.
Revised: Thus, by Theorem 3.2, we have sgn ( (cid:98) u ( •∥ D n , K H S d )) is consistent for classification, i.e., Equation (1)holds.

Original: It is also interpolating since (cid:98) m ( •∥ D n , K ) is interpolating.
Revised: It is also interpolating since (cid:98) m ( •∥ D n , K ) is interpolating.

Original: By Proposition 5.2, we have
Revised: By Proposition 5.2, we have

Original: K ha S d = K H S d .
Revised: K ha S d = K H S d .

Original: Thus sgn ( (cid:98) u ( •∥ D n , K ha S d )) is an ensemble method having the interpolating-consistentproperty.
Revised: Thus sgn ( (cid:98) u ( •∥ D n , K ha S d )) is an ensemble method having the interpolating-consistentproperty.

Original:  Discussion
Revised:  Discussion

Original: We have shown that using the manifold-Hilbert kernel in kernel smoothing regression, also known as Nadaraya-Watson regression, results in a consistent estimator that interpolates the training data on a Riemannian manifold M .We proposed weighted random partition kernels , a generalization of theunweighted analogous definition by Davies and Ghahramani [19] which provided a framework foranalyzing ensemble methods such as random forest via kernels.
Revised: We have shown that using the manifold-Hilbert kernel in kernel smoothing regression, also knownas Nadaraya-Watson regression, results in a consistent estimator that interpolates the training dataon a Riemannian manifold M .

Original: When M = S d is the sphere, weshowed that the manifold-Hilbert kernel is a weighted random partition kernel, where the randompartitions are induced by random hyperplane arrangements.
Revised: Furthermore, when M = S d is the sphere, we showed that themanifold-Hilbert kernel is a weighted random partition kernel, where the random partitions areinduced by random hyperplane arrangements.

Original: This demonstrates an ensemble methodthat has the interpolating-consistent property.
Revised: This demonstrates an ensemble method that has theinterpolating-consistent property.

Original: One limitation of this work is the lack of rate of convergence of the ensemble methods.

Original: The analogousresult for the Nadaraya-Watson regression have been obtained by Belkin et al.

Original: However, it is notclear if the kernels used in [7] are weighted random partition kernels. Resolving this is an interestingfuture direction. Similar to PERT [18], another limitation of this work is that the base classifiers in the ensemble aredata-independent.
Revised: A limitation of this work is that the random hyperplane arrangement partition is data-independent.

Original: Such ensemble methods in these line of work(including ours) are easier to analyzethan the data- dependent ensemble methods used in practice.
Revised: Thus, the resulting ensemble method considered in this work are easier to analyze than popularensemble methods used in practice.

Original: See Biau and Scornet [11] and [12]for an in-depth discussion. We believe our work offers one theoretical basis towards understandinggeneralization in the interpolation regime of ensembles of histogram classifiers over data-dependentpartitions, e.g., decision trees à la CART [16].
Revised: Nevertheless, we believe our work offers one theoretical basistowards understanding generalization in the interpolation regime of ensembles of histogram classifiersover data-dependent partitions, e.g., decision trees à la CART [15].

Original: Checklist
Revised: Checklist

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
Revised: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?

Original: contributions and scope? [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes](b) Did you describe the limitations of your work?

Original: [Yes] See the Section 6.
Revised: [Yes] See the Section 6.

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [N/A]
Revised: [N/A]

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to

Original: them?
Revised: them?

Original: [Yes]
Revised: [Yes]

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] This is the
Revised: [Yes] This is the

Original: opening sentence of our first main technical section, Section 3.
Revised: opening sentence of our first main technical section, Section 3.(b)

Original: (b) Did you include complete proofs of all theoretical results?
Revised: Did you include complete proofs of all theoretical results?

Original: [Yes]
Revised: [Yes]3.

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?
Revised: mental results (either in the supplemental material or as a URL)?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
Revised: (c) Did you report error bars (e g., with respect to the random seed after running experiments multiple times)?

Original: [N/A]
Revised: [N/A]

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e g., typeof GPUs, internal cluster, or cloud provider)?

Original: [N/A]
Revised: [N/A]

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
Revised: If you are using existing assets (e g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A]
Revised: [N/A]

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: using/curating?
Revised: using/curating?

Original: [N/A]
Revised: [N/A]

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable
Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable

Original: information or offensive content?
Revised: information or offensive content?

Original: [N/A]
Revised: [N/A]

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, if
Revised: (a) Did you include the full text of instructions given to participants and screenshots, if

Original: applicable?
Revised: applicable?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?
Revised: Board (IRB) approvals, if applicable?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation? [N/A]
Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount

Original: Supplemental Materials: Interpolating Ensembles via the Manifold-
Revised: spent on participant compensation?

Original: Hilbert

Original: Kernel

Original: A.1 Basics of Riemannian Manifolds

Original: In this section, we review the main concepts from Riemannian manifold theory essential to this work.

Original: Our main references are Sakai [35] and Do Carmo [21].

Original: Throughout, d ∈ N denotes the dimension.

Original: We use the word smooth to mean infinitely differentiable.

Original: Manifolds.

Original: A smooth manifold M of dimension d is a Hausdorff, second countable topologicalspace together with an atlas : a set Atlas :

Original: = { ( U α , φ α ) } α ∈

Original: A where 1).

Original: { U α } α ∈

Original: A is an open coverof M , 2).

Original: for each α ∈ A , φ α : U α → φ α ( U α ) ⊆

Original: R d is a homeomorphism onto its image, and 3).φ α ◦ φ − 1 β : φ β ( U α ∩ U β ) → φ − 1 α ( U α ∩ U β ) is smooth for each pair α, β ∈ A .

Original: An element ( U, φ )of Atlas is called a chart .

Original: Smooth maps.

Original: A real-valued function f : M → R is a smooth function if f ◦ φ − 1 is smooth(in the elementary calculus sense) for all charts ( U, φ ) .

Original: The set of all smooth functions is denoted

Original: Fn ( M ) , which forms an R -vectorspace.

Original: Let N be another smooth manifold with atlas B .

Original: A functionΦ : M → N is a smooth map if g ◦ Φ ∈ Fn

Original: ( M ) for all g ∈ Fn ( N ) .

Original: Tangent space.

Original: Let x ∈ M .

Original: A derivation at x is a linear function v : Fn ( M )

Original: → R satisfyingthe product rule : v [ fg ] = f ( x ) v [ f ] + g ( x ) v [ g ] for all f, g ∈ Fn ( M ) .

Original: The tangent space at x ,denoted T x M , is the vector space of all derivations at x .

Original: Elements of T x M are referred to as tangentvectors at x .

Original: For a given chart ( U, φ ) where x ∈ U , define a derivation at x , denoted ∂ i | x , byf (cid:55)→ d ( f ◦ φ − 1 )

Original: dz i ( φ ( x )) where ddz i is the i -th partial derivative in ordinary calculus.

Original: It is a fact that{ ∂ i | x : i = 1 , . . .

Original: , d } is a basis for T x M .

Original: Although the above definition of a tangent vector is abstract, it can be concretely interpreted in termsof derivative along a curve.

Original: Let a < t 0 < b be real numbers.

Original: A curve through x is a smooth mapγ : ( a, b ) → M such that γ ( t 0 )

Original: =

Original: x .

Original: Then Fn ( M ) ∋ f

Original: (cid:55)→ ddt f ( γ ( t )) | t = t 0 ∈ R defines a derivation 496 at x .

Original: Oftentimes, this derivation is denoted ˙ γ ( t 0 )

Original: [Equation]

Original: Riemannian metric.

Original: The tangent bundle is the set TM :

Original: = (cid:83) x T x M , which itself is a smoothmanifold of dimension 2 d .

Original: A vector field on M is a smooth map V : M → TM such that

Original: V ( x ) ∈ T x M for all x ∈ M .

Original: The set of all vectors fields on M is denoted Vf ( M ) .

Original: A Riemannian metric on M is a choice of an inner product ⟨· , ·⟩ x (and thus, a norm ∥ · ∥ x ) on T x M 501 for each x ∈ M such that the function M → R given by x (cid:55)→ ⟨ V ( x ) , U ( x ) ⟩ x is smooth for all 502 V ,

Original: U ∈

Original: Vf ( M ) .

Original: As shorthands, when x is clear from context, we drop the subscripts and simply write⟨· , ·⟩ and ∥ · ∥ instead.

Original: Choosing an orthonormal basis for T x M with respect to ⟨· , ·⟩ x for each x , wecan identify T x M with R d with the ordinary dot inner product.

Original: Let x ∈ M and ( U, φ ) be a chart such that x ∈ U .

Original: Define g ij ( x ) = ⟨ ∂ i | x , ∂ j | x ⟩ x .

Original: Denote by G ( x )the d × d positive definite matrix [ g ij ( x )] ij .

Original: Below, we will refer to the function G : U → R d × d as thecoordinate representation of the Riemannian metric.

Original: Define g ij ( x ) :=

Original: [ G ( x ) − 1 ] ij .

Original: The Christoffelsymbols with respect to ( U, φ ) are defined by Γ kij :

Original: = 12 (cid:80) dℓ

Original: =1 g kℓ ( ∂ i | x g jℓ + ∂ j | x g iℓ − ∂ ℓ | x g ij ) .

Original: Notethat g kℓ , g kℓ , G , Γ kij , and ∂ i | x g jℓ are all functions with domain U .

Original: Geodesics.

Original: Fix a chart ( U, φ ) .

Original: Consider a smooth curve γ : [ a, b ] → U .

Original: Let ζ i ( t ) :

Original: =

Original: [ φ ( γ ( t ))]

Original: i bethe i -th component functions.

Original: The curve γ is a geodesic if ζ is a solution to the following systemof second order ordinary differential equations (ODEs): d 2 ζ

Original: i dt 2 + (cid:80) dj,ℓ

Original: =

Original: 1 Γ ijℓ ◦ γ dζ j dt dζ ℓ dt = 0 for alli = 1 , . . .

Original: , d at all time t ∈ [ a, b ] .

Original: Geodesics are minimizers of the so-called energy functional E ( γ ) =

Original: 12 (cid:82) b a ∥ ˙

Original: γ ( t ) ∥ 2 γ ( t ) dt .

Original: The abovesystem of ODEs are the analog of the “first derivative test” for local minimizers of E .

Original: Thus, geodesicsare defined independently of the choice of the chart.

Original: Exponential map.

Original: For x ∈ M and v ∈ T x M , there exists ϵ > 0 and a unique geodesic curveγ v : [ − ϵ, ϵ ]

Original: → M such that γ v (0) =

Original: x and ˙ γ v (0) = v .

Original: This follows from the existence anduniqueness of the solution to an ODE given initial conditions where the ODE is as discussed above.

Original: A

Original: Note that although geodesics are previously defined in U where ( U, φ ) is a chart, they can be extendedoutside of U using additional charts.

Original: Let x ∈ M and v ∈ T x

Original: M be fixed and let γ v : [ − ϵ, ϵ ]

Original: → M be as in the preceding paragraph.

Original: If∥ v ∥ x ≤ ϵ , then define exp x ( v ) :

Original: = γ v (1) .

Original: A fundamental fact is that exp x , known as the exponential 524 map at x , can be defined on an open set of T x M containing the origin.

Original: Distance function.

Original: Let x, ξ ∈ M and a < b be real numbers.

Original: A piecewise smooth curve from xto ξ is a piecewise smooth map γ : [ a, b ] → M

Original: such that γ ( a ) =

Original: x and γ ( b ) = ξ .

Original: Assume that

Original: M is connected.

Original: Then for all x, ξ ∈ M , there exists a piecewise smooth curve from x to ξ .

Original: Thelength of γ is defined as len ( γ ) :

Original: = (cid:82) b a ∥ ˙

Original: γ ( t ) ∥ γ ( t ) dt .

Original: Define dist M ( x, ξ ) := inf { len ( γ ) : γ is apiecewise smooth curve from x to ξ } , which is a metric on M in the sense of metric spaces (see [35,

Original: Proposition 1.1]).

Original: For x ∈ M and r ∈ (0 , ∞ ) , the open ball centered at x of radius r is denoted

Original: B x ( r, M ) :

Original: = { z ∈ M : dist M ( x, z ) < r } .

Original: Complete Riemannian manifolds.

Original: A Riemannian manifold is complete if it is a complete metricspace under the metric dist M .

Original: The Hopf-Rinow theorem ([21, Ch. 8, Theorem 2.8]) states that if Mis connected and complete, then the exponential exp x can be defined on the entire T x M .

Original: A.2 Proof of Lemma 4.

Original: This section uses definitions and notations introduced in Section 4.1.

Original: In particular, recall the cutlocus C x , the tangent cut locus ˜ C x , the interior set I x and the tangent interior set ˜ I x .

Original: The proof of

Original: Lemma 4.1 is presented towards the end of the section.

Original: At this point, we compile some facts fromvarious sources about the cut locus.

Original: Lemma A.1.

Original: For all x ∈ M , we have

Original: C x is a closed subset of M (Hebda [27, Proposition 1.2]).

Original: I x ∩ C x = ∅

Original: and I x ∪ C x = M

Original: (Sakai [35, Ch II, Lemma 4.4 (1)])

Original: I x is an open subset of M (immediate from 1 and 2 above)

Original: exp x : ˜ I x

Original: → I x is a diffeomorphism ([35, Ch II, Lemma 4.4 (2)])

Original: λ M ( C x ) = 0 , where λ M is the Riemann-Lebesgue measure ([35, Lemma 4.4 (3)])

Original: τ x is continuous and inf u ∈

Original: U x M τ x ( u ) > 0 ([35, Ch II, Propositions 4.1 (2) and 4.13 (1)])

Original: While the following lemma is elementary, we provide a proof since we could not find one in theliterature.

Original: Lemma A.2.

Original: For all x ∈ M , the (topological) closure of ˜ I x in T x M is ˜ D x .

Original: Furthermore, for allx ∈ M , we have exp x ( ˜ D x ) =

Original: M .

Original: Proof of Lemma A.2.

Original: Take a convergent sequence { t i u i } i ∈ N ⊆

Original: ˜

Original: I x where u i ∈

Original: U x M and 0 ≤ t

Original: i <

Original: τ x ( u i ) .

Original: Let v ∗ =

Original: lim i t i u i .

Original: Our goal is to show that v ∗ ∈

Original: ˜ D x = ˜

Original: I x ∪ ˜ C x .

Original: Since U x M is compact, we may assume that u ∗ :=

Original: lim i u

Original: i exists after passing to a subsequence ifnecessary.

Original: Furthermore, ∥ t i u i ∥ x

Original: = t i implies that t ∗ :

Original: = lim i t i exists as well (i.e., t ∗ < ∞ ).

Original: Hence,v ∗ =

Original: t ∗ u ∗ .

Original: Consider the case that τ x ( u ∗ )

Original: = ∞ .

Original: Then 0 ≤ t ∗ < τ x ( u ∗ ) implies that v ∗ =

Original: t ∗ u ∗ ∈ ˜

Original: I x .

Original: For theother case that t ( u ) < ∞ , we first note that t

Original: i u i ∈ ˜

Original: I x implies that t i <

Original: τ x ( u i ) .

Original: Taking the limit ofboth sides, we have t ∗ =

Original: lim

Original: i t i ≤ lim

Original: i τ x ( u i ) =

Original: τ x ( u ∗ ) .

Original: Note that the last limit can be exchangedsince τ x is continuous (Lemma A.1 part 6).

Original: Thus, either t ∗ <

Original: τ x ( u ∗ ) in which case v ∗ ∈ ˜

Original: I x , ort ∗ =

Original: τ x ( u ∗ ) in which case v ∗ =

Original: τ x ( u ∗ ) u ∗ ∈ ˜

Original: C x .

Original: For the “furthermore” part, note that

Original: [Equation]

Original: where the last equality is Lemma A.1 part 2.

Original: Proof of Lemma 4.1.

Original: Denote by cl( T x M ) the set of closed subsets of T x M .

Original: Define ψ : M →cl( T x M ) by ψ ( ξ ) :

Original: = { x ∈ ˜ D

Original: x : exp x ( x ) = ξ } = exp − 1 x ( ξ ) ∩ ˜

Original: D x .

Original: Note that ψ ( ξ ) is a closed set by

Original: Lemma A.2.

Original: A

Original: We claim that ψ is weakly-measurable , i.e., for every open set ˜ U ⊆ T x M , the subset of M definedby { ξ ∈ M : ψ ( ξ ) ∩ ˜

Original: U ̸

Original: = ∅} is Borel.

Original: To see this, note that

Original: [Equation]

Original: As inner product spaces, T x M and R d are isomorphic (see Section A.1-Riemannian metric).

Original: Since,

Original: T x M and R d are homeomorphic as topological spaces, R d being locally compact implies T x M islocally compact as well.

Original: Thus, we can write ˜ U

Original: = (cid:83) i ∈ N ˜ K i as a countable union of compact sets˜ K i ⊆

Original: T x M .

Original: Furthermore, ˜ D x ∩ ˜ U

Original: =

Original: (cid:83) i ∈ N ˜ D x ∩ ˜ K

Original: i and so exp x ( ˜ D x ∩ ˜ U )

Original: =

Original: (cid:83) i ∈ N exp x

Original: ( ˜ D x ∩ ˜ K i ) .

Original: Since exp x is continuous, exp x ( ˜ D x ∩ ˜ K i ) is a compact subset of M , and hence closed and bounded bythe Hopf-Rinow theorem ([21, Ch. 8, Theorem 2.8]).

Original: Thus, exp x ( ˜ D x ∩ ˜ U ) =

Original: (cid:83) i ∈ N exp x

Original: ( ˜ D x ∩ ˜ K i )is a countable union of closed sets, which is Borel.

Original: This proves the claim that ψ is weakly Borelmeasurable.

Original: By the Kuratowski–Ryll-Nardzewski measurable selection theorem (see [13, Theorem 6.9.3]), thereexists a Borel measurable function M → T x M , which we denote by log x , such that log x ( ξ ) ∈ 578 ψ ( ξ ) = exp −

Original: [Equation]

Original: for all ξ ∈ M , as desired.

Original: By construction, log x ( ξ ) ∈ exp − 1 x ( ξ ) for all ξ ∈ M ,and so exp x (log x ( ξ )) =

Original: ξ is immediate.

Original: For the “furthermore” part, let ξ ∈ M be arbitrary and let z := log x ( ξ ) ∈

Original: ˜ D x . Let { z i } ⊆

Original: ˜

Original: I x be a 581 sequence such that lim

Original: i z i

Original: = z .

Original: By Equation (6), we have dist M ( x, exp x

Original: ( z i ))

Original: =

Original: ∥

Original: z i ∥ x .

Original: By continuity of dist M and exp x , we have dist M ( x, ξ ) =

Original: dist M ( x, exp x ( z )) =

Original: lim i dist M ( x, exp x

Original: ( z i )) .

Original: To conclude, we have lim

Original: i dist M ( x, exp x

Original: ( z i ))

Original: =

Original: lim i ∥

Original: z i ∥ x = ∥

Original: z ∥ x = ∥ log x ( ξ ) ∥ x , as desired.

Original: A.3 Proof of Proposition 4.

Original: Recall from Section A.1-Riemannian metric, given a chart ( U, φ ) , one can define the matrix-valuedfunction G : U → R d × d referred to earlier as the coordinate representation of the Riemannian metric.

Original: Now, Lemma A.1 part 3 states that I x is an open neighborhood of x .

Original: Furthermore, ˜ I x is an opensubset of T x M , which is identified with R d using an orthonormal basis (see Section A.1-Riemannianmetric).

Original: Hence, { ( I x , log x | I x ) } x ∈ M is an atlas of M (see Section A.1-Manifolds).

Original: Definition A.3.

Original: The chart ( I x , log x | I x ) is called a normal coordinate system at x .

Original: Let G : I x →

Original: R d × d be the coordinate representation of the Riemannian metric for this chart.

Original: To emphasize thedependency on x , we write G x := G .

Original: Denote by G ⊥ x : M → R d × d the zero extension of G x to therest of M , i.e., G ⊥ x ( ξ ) =

Original: G x ( ξ ) for ξ ∈ I x and G ⊥ x ( ξ ) is the zero matrix for ξ ̸∈ I x .

Original: The normal coordinate system has the property that G x ( x ) =

Original: G ⊥ x

Original: ( x ) is the identity matrix.

Original: This isthe result of Sakai [35, Ch.

Original: II §2 Exercise 4].

Original: Lemma A.4 (Change-of-Variables) .

Original: Let x ∈ M be fixed.

Original: Define the function ν x : M → R byν x ( ξ ) =

Original: (cid:112) | det G ⊥ x ( ξ ) | where G ⊥ x is as in Definition A.3.

Original: Then ν x is Borel-measurable.

Original: Furthermore,ν x satisfies the following property: Let f : M

Original: → R be an absolutely integrable function.

Original: Define thefunction

Original: [Equation]

Original: by

Original: [Equation]

Original: (exp x ( z )) .

Original: Then (i) h (0) =

Original: f

Original: ( x ) and (ii) for all Borel set ˜ B

Original: ⊆ T x M we have (cid:82) B fdλ M = (cid:82)

Original: ˜ B hdλ where

Original: B := exp x ( ˜ B ∩ ˜ I x ) .

Original: Proof of Lemma A.4.

Original: We first show that ν x is Borel-measurable.

Original: Recall that G ⊥ x : M → R d × d is thezero extension of G x : I x

Original: → R , which is by definition smooth (see Section A.1-Riemannian metric).

Original: In particular, G x : I x → R is continuous and so (cid:112) det( G x ( • )) is Borel-measurable.

Original: Now, note that 606 (cid:112) det( G ⊥ x ( • )) is the zero extension of (cid:112) det( G x ( • )) from I x to M .

Original: Hence, (cid:112) det( G ⊥ x ( • )) , whichis ν x by definition, is Borel-measurable.

Original: A

Original: Next, we prove the “Furthermore” part (i).

Original: Note that exp x (0) =

Original: x .

Original: Moreover, G ⊥ x ( x ) =

Original: G x ( x )is the identity matrix as asserted after Definition A.3 (see Sakai [35, Ch.

Original: II §2 Exercise 4]).

Original: Thus,h (0) =

Original: f (exp x (0)) (cid:112) | det G ⊥ x (exp x (0))

Original: | = f ( x ) √ 1 = f ( x ) , as desired.

Original: For the “Furthermore” part (ii), we first note that ˜ B

Original: = ( ˜ B ∩ ˜ I x ) ∪ ( ˜ B ∩ ˜

Original: C x ) expresses ˜ B as a disjointunion.

Original: Thus, B = exp x ( ˜ B ) =

Original: exp x ( ˜ B ∩ ˜ I x ) ∪ exp( ˜ B ∩ ˜ C x ) expresses B as a disjoint union as well.

Original: Moreover, exp( ˜ B ∩ ˜ C x ) ⊆

Original: exp( ˜ C x ) =

Original: C x , which has λ M -measure zero (Lemma A.1 part 5).

Original: Recall that λ is the shorthand for the ordinary Lebesgue measure λ R d

Original: (see paragraph right after

Original: Definition 3.1).

Original: Now, we directly compute to obtain the formula

Original: [Equation]

Original: fdλ

Original: [Equation]

Original: ∵ Amann and Escher [1, Ch XII, Thm 1.10] =

Original: [Equation]

Original: fdλ

Original: [Equation]

Original: fdλ M

Original: [Equation]

Original: M , as desired.

Original: Proposition A.5.

Original: Let x ∈ M be fixed.

Original: Let X be a random variable on M with density f X wherethe underlying probability space is (Ω , P , A ) (see Definition 3.3).

Original: Define Z :=

Original: log x ( X ) .

Original: Then Zis a random variable on T x M such that for all events E ∈ A and Borel sets

Original: ˜ B ⊆ T x M we have

Original: Pr( E ∩ { Z ∈ ˜ B }

Original: [Equation]

Original: =

Original: Pr

Original: [Equation]

Original: Proof of Proposition A.5.

Original: To start with, we have

Original: [Equation]

Original: Since exp x : ˜ I x → I x is a diffeomorphism (Lemma A.1-part 4) with inverse log x , we have

Original: [Equation]

Original: as sets.

Original: On the other hand,

Original: [Equation]

Original: Finally, Pr( X ∈ C x ) =

Original: (cid:82) C x

Original: f X dλ M = 0 since C x has λ M -measure zero (Lemma A.1-part 5).

Original: Proof of Proposition 4.2 part (i).

Original: Recall that λ is the shorthand for the ordinary Lebesgue measureλ R d (see paragraph right after Definition 3.1).

Original: Let E = Ω in Proposition A.5.

Original: Then we have

Original: [Equation]

Original: Part

Original: [Equation]

Original: ∵ f X is the density of X

Original: [Equation]

Original: ∵ Lemma A.

Original: [Equation]

Original: ∵ Definition of f Z

Original: A

Original: By assumption, f X is Borel-measurable.

Original: By Lemma A.4, ν x is Borel-measurable.

Original: Since exp x iscontinuous, we have that both f X ◦ exp x

Original: and ν x ◦ exp x are Borel-measurable.

Original: This proves that f Z is 629 Borel-measurable.

Original: Hence, the integrand is Borel-measurable and a density function for Z .

Original: Proof of Proposition 4.2 part (ii).

Original: Recall that λ is the shorthand for the ordinary Lebesgue measure λ R d

Original: (see paragraph right after Definition 3.1).

Original: By Lemma A.1 part 6, we have τ ∗ x :=

Original: inf u ∈

Original: U x M τ x ( u ) > 0 .

Original: Now, let r ∈

Original: (0 , τ ∗ x ) .

Original: By the definition of r , we have B x

Original: ( r, M ) ⊆

Original: ˜

Original: I x .

Original: Hence letting z = log x ( ξ ) for ξ ∈ B x ( r, M ) , by Equation (6) we have

Original: [Equation]

Original: Thus,log x (

Original: [Equation]

Original: [Equation]

Original: [Equation]

Original: (9) and

Original: [Equation]

Original: ( r

Original: [Equation]

Original: Thus, by Lemma A.4, we have

Original: [Equation]

Original: Before proceeding, we need the following lemma:

Original: Lemma A.6.

Original: For all x ∈ M , we have lim r → 0 λ M ( B x ( r,M )) λ

Original: ( B 0 ( r,T x M ))

Original: = 1 .

Original: Proof of Lemma A.6.

Original: Let ω d := π d/ 2 / Γ( d 2 + 1) be the volume of the unit ball in R d where Γ is thegamma function.

Original: Then λ ( B 0 ( r, T x M )) =

Original: ω d r d .

Original: Next, [35, Ch II.5 Exercise 3] states that

Original: [Equation]

Original: where S x ∈ R is a constant that depends only on x (it is the scalar curvature of M at x ).

Original: By simplealgebra, the above yields

Original: [Equation]
Revised: [N/A]

Original: In particular, we have lim

Original: [Equation]

Original: = 0 , as desired.

Original: Now we continue with the proof of Proof of Proposition 4.2 part (ii).

Original: We observe that

Original: [Equation]

Original: ∵ x is a Lebesgue point of f

Original: [Equation]

Original: ∵ definition of h and equation (11)

Original: [Equation]

Original: Since f ( x ) = h (0) (Lemma A.4) , we’ve shown that

Original: [Equation]

Original: Thus,

Original: A

Original: A.4 Proof of Proposition 4.

Original: Recall that λ is the shorthand for the ordinary Lebesgue measure λ R d

Original: (see paragraph right after

Original: Definition 3.1).

Original: Let A ⊆ R and ˜ B ⊆ T x M be Borel subsets.

Original: Then

Original: [Equation]

Original: ∵ Definition of P Y | Z = z

Original: [Equation]

Original: ∵ Lemma A.4 and Proposition 4.2 (ii)

Original: [Equation]

Original: ∵ Proposition 4.

Original: [Equation]

Original: This proves that P Y | Z ( ·|· ) is a conditional probability for Y given Z .

Original: A