An analysis of Ermakov-Zolotukhin quadrature using kernels

1 Introduction . Integrals appear in many scientific fields as quantities of interest per se . For example , in statistics , they represent expectations [ 27 ] , while in mathematical finance , they represent the prices of financial products [ 17 ] . Unfortunately , integrals that can be written in closed form are exceptional . In general , their values are only known through approximations . For this reason , numerical integration is at the heart of many tasks in applied mathematics and statistics . Among all the possible approximation schemes , quadratures are the most practical since they approximate the integral of a function by a finite mixture of its evaluations . In this work , we focus on quadrature rules that take the form∫ X f ( x ) g ( x ) dω ( x ) ≈ ∑ i∈ [ N ] wif ( xi ) , ( 1 ) where the nodes xi are independent of f and g , while the weights wi depend only on g. The nodes and the weights of a quadrature may be seen as degrees of freedom that the practitioner may tune in order to achieve a given level of approximation error . The design of quadratures gave birth to a rich literature from Gaussian quadrature [ 15 ] to Monte Carlo methods [ 25 ] to quadratures based on determinantal point processes ( DPPs ) [ 1 ] . These latter form a large class of probabilistic models of repulsive random subsets that make numerical integration possible in a variety of domains with strong theoretical guarantees . In particular , central limit theorems with asymptotic convergence rates that scale better than the typical Monte Carlo rate O ( N−1/2 ) were proven for several DPP based quadratures : when the integrand is a C1 function [ 1 ] or even when the integrand is non-differentiable [ 10 ] . Moreover , it is possible to design quadrature rules based on DPPs with non-asymptotic guarantees and with rates of convergence that adapt to the smoothness of the integrand . This is the case of the quadrature proposed by Ermakov and Zolotukhin in [ 14 ] and recently revisited in [ 16 ] , and the optimal kernel quadrature [ 3 ] . In this work , we study the quadrature rule proposed by Ermakov and Zolotukhin ( EZQ ) through the lens of kernel methods . We start by comparing the weights of EZQ to the weights of the optimal 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . kernel quadrature ( OKQ ) , and we prove that they both belong to a broader class of quadrature rules that we call kernel based interpolation quadrature . Then , we study the approximation quality of EZQ in reproducing kernel Hilbert spaces ( RKHSs ) . This is done by proving a general tractable formula of the expected value of the squared worst-case integration error for functions that belong to the unit ball of an RKHS when the nodes follow the distribution of a determinantal point process . This formula involves principally the eigenvalues of the integral operator , and converges to 0 at a slightly slower rate than the optimal rate . Interestingly , this analysis yields a better upper bound for the optimal kernel quadrature with DPPs proposed initially in [ 3 ] . Comparably to the theoretical guarantees given in [ 14 , 16 ] , our theoretical guarantees are independent of the choice of the test function . This facilitates the comparison of EZQ with other quadratures such as OKQ . The rest of the article is organized as follows . Section 2 reviews the work of [ 14 ] and recall key concepts on kernel based quadrature . In Section 3 , we present the main results of this work and their consequences . A sketch of the proof of the main theorem is given in Section 4 . We illustrate the theoretical results by numerical experiments in Section 5 . Finally , we give a conclusion in Section 6 . Notation and assumptions . We use the notation N∗ = Nr { 0 } . We denote by ω a Borel measure supported on X , and we denote by L2 ( ω ) the Hilbert space of square integrable real-valued functions on X with respect to ω , equipped with the inner product 〈· , ·〉ω , and the associated norm ‖.‖ω . For N ∈ N∗ , we denote by ω⊗N the tensor product of ω defined on XN . Moreover , we denote by F the RKHS associated to the kernel k : X × X → R that we assume to be continuous and satisfying the condition ∫ X k ( x , x ) dω ( x ) < +∞ . In particular , we assume the Mercer decomposition k ( x , y ) = ∑ m∈N∗ σmφm ( x ) φm ( y ) , ( 2 ) to hold , where the convergence is pointwise , and σm and φm are the corresponding eigenvalues and eigenfunctions of the integral operator Σ defined for f ∈ L2 ( ω ) by Σf ( · ) = ∫ X k ( · , y ) f ( y ) dω ( y ) . ( 3 ) We assume that the sequence σ = ( σm ) m∈N∗ is non-increasing and its elements are non-vanishing , and we assume that the corresponding eigenfunctions φm are continuous . Note that the φm are normalized : ‖φm‖ω = 1 for m ∈ N∗ . In particular , ( φm ) m∈N∗ is an o.n.b . of L2 ( ω ) , and every element f ∈ F satisfies ∑ m∈N∗ 〈f , φm〉2ω σm < +∞ . ( 4 ) Moreover , for every N ∈ N∗ , we denote by EN the eigen-subspace of L2 ( ω ) spanned by φ1 , . . . , φN . For any kernel κ : X × X → R , and for x ∈ XN , we define the kernel matrix κ ( x ) : = ( κ ( xi , xj ) ) i , j∈ [ N ] ∈ RN×N . Finally , we denote in bold fonts the corresponding kernel matrices : K ( x ) for the kernel k , KN ( x ) for the kernel kN , K⊥N ( x ) for the kernel k ⊥ N , κ ( x ) for the kernel κ ... Similarly , for any function µ : X → R and for x ∈ XN , we define the vector of evaluations µ ( x ) : = ( µ ( xi ) ) i∈ [ N ] ∈ RN . 2 Related work . In the section , we review some results that are relevant to the contribution . 2.1 Ermakov-Zolotukhin quadrature . The quadrature rule proposed by Ermakov and Zolotukhin in [ 14 ] deals with integrals that write∫ X f ( x ) φm ( x ) dω ( x ) , ( 5 ) where f ∈ L2 ( ω ) , and ( φm ) m∈N∗ is an orthonormal family with respect to the measure ω . Its construction goes as follows . Let N ∈ N∗ and let x ∈ XN such that the matrix ΦN ( x ) : = ( φn ( xi ) ) ( n , i ) ∈ [ N ] × [ N ] is non-singular . For n ∈ [ N ] , define IEZ , n ( f ) = ∑ i∈ [ N ] ŵEZ , ni f ( xi ) , ( 6 ) where ŵEZ , n : = ( ŵEZ , ni ) i∈ [ N ] ∈ RN is given by ŵEZ , n = ΦN ( x ) −1en , with en is the vector of RN with the n-th coordinate is 1 and the rest are 0 1 . We can prove easily that this quadrature is exact ∀f ∈ Span ( φn ) n∈ [ N ] , ∑ i∈ [ N ] ŵEZ , ni f ( xi ) = ∫ X f ( x ) φn ( x ) dω ( x ) . ( 7 ) A quadrature rule that satisfies ( 7 ) is typically called an interpolatory quadrature rule . Now , if f /∈ Span ( φn ) n∈ [ N ] , the authors studied the expected value and the variance of IEZ , n ( f ) when x = ( x1 , . . . , xN ) is taken to be a random variable in XN that follows the distribution of density pDPP ( x1 , . . . , xN ) : = 1 N ! Det2 ΦN ( x ) , ( 8 ) with respect to the product measure ω⊗N defined on XN . As it was observed in [ 16 ] , the nodes of the quadrature follow the distribution of the determinantal point process of reference measure ω and marginal kernel κN defined by κN ( x , y ) = ∑ n∈ [ N ] φn ( x ) φn ( y ) . We refer to [ 20 ] for further details on determinantal point processes . Now , we recall the main result of [ 14 ] . Theorem 1 . Let x be a random subset of X that follows the distribution of DPP of kernel κN and reference measure ω . Let f ∈ L2 ( ω ) , and n ∈ [ N ] . Then EDPPIEZ , n ( f ) = ∫ X f ( x ) φn ( x ) dω ( x ) , ( 9 ) and VDPPIEZ , n ( f ) = ∑ m≥N+1 〈f , φm〉2ω . ( 10 ) Theorem 1 shows that the IEZ , n ( f ) is an unbiased estimator of ∫ X f ( x ) φn ( x ) dω ( x ) , and its variance depends on the coefficients 〈f , φm〉ω for m ≥ N + 1 . Consequently , the expected squared error of the quadrature is equal to the variance of IEZ , n ( f ) and it is given by EDPP ∣∣∣∣ ∫ X f ( x ) φn ( x ) dω ( x ) − ∑ i∈ [ N ] ŵEZ , ni f ( xi ) ∣∣∣∣2 = ∑ m≥N+1 〈f , φm〉2ω . ( 11 ) The identity ( 11 ) gives a theoretical guarantee for an interpolatory quadrature rule when the nodes follow the distribution of the DPP defined by ( 8 ) . Compared to existing work on the literature [ 11 , 22 , 23 , 24 ] , ( 11 ) is generic and applies to any orthonormal family . Now , we may observe that the expected squared error in ( 11 ) depends strongly on the function f . This makes the comparison between EZQ and other quadratures , based on some test function f , tricky : the choice of f may favor ( or disfavor ) EZQ . In order to circumvent this difficulty , we suggest to study a figure of merit that is independent of the choice of the function f . This is possible using kernels through the study of the worst-case integration error on the unit ball of an RKHS . The definition of this quantity will be recalled in the following section . 2.2 The worst integration error in kernel quadrature . The use of the kernel framework in the context of numerical integration can be tracked back to the work of Hickernell [ 18 , 19 ] , who introduced the use of kernels to the quasi Monte Carlo community . Their use was popularized in the machine learning community by [ 28 , 9 ] . In this framework , the quality of a quadrature is assessed by the worst-case integration error on the unit ball of an RKHS F associated to some kernel k : X × X → R+ . This quantity is defined as follows sup f∈F , ‖f‖F≤1 ∣∣∣ ∫ X f ( x ) g ( x ) dω ( x ) − ∑ i∈ [ N ] wif ( xi ) ∣∣∣ . ( 12 ) 1The dependency of the vector ŵEZ , n on x was dropped for simplicity . This quantity reflects how good is the quadrature uniformly on the unit ball of F . Interestingly , this quantity has a closed formula ∥∥∥∥µg − ∑ i∈ [ N ] wik ( xi , . ) ∥∥∥∥ F , ( 13 ) where µg = Σg is the so-called embedding of g in the RKHS F . We shall use in Section 3.2 the equivalent expression ( 13 ) of the worst-case integration error , to derive a closed formula of EDPP sup f∈F , ‖f‖F≤1 ∣∣∣ ∫ X f ( x ) g ( x ) dω ( x ) − ∑ i∈ [ N ] ŵEZ , ni f ( xi ) ∣∣∣2 . ( 14 ) By now , we observe that the weights ŵEZ , ni of EZQ are non-optimal in the sense that they do not minimize ( 13 ) . By definition , the optimal kernel quadrature for a given configuration x , such that the kernel matrixK ( x ) is non-singular , is the quadrature with nodes the xi and weights the ŵi that minimize ( 13 ) . We specify in Section 3.1 the subtle difference between the quadrature of Ermakov and Zolotukhin and the optimal kernel quadrature . Before that , we review the existing constructions of the optimal kernel quadrature in the following section .

Summary: The paper gives a kernel quadrature interpretation for certain interpolatory quadrature rules specified by the property that they integrate exactly all functions in certain finite-dimensional spaces spanned by $L^2$-orthonormal functions. Error estimates in expectation are proved when the integration nodes are sampled from a DPP. The authors call these quadratures Ermakov-Zolotukhin quadratures (EZQ).