An analysis of Ermakov-Zolotukhin quadrature using kernels

1 Introduction . Integrals appear in many scientific fields as quantities of interest per se . For example , in statistics , they represent expectations [ 27 ] , while in mathematical finance , they represent the prices of financial products [ 17 ] . Unfortunately , integrals that can be written in closed form are exceptional . In general , their values are only known through approximations . For this reason , numerical integration is at the heart of many tasks in applied mathematics and statistics . Among all the possible approximation schemes , quadratures are the most practical since they approximate the integral of a function by a finite mixture of its evaluations . In this work , we focus on quadrature rules that take the form∫ X f ( x ) g ( x ) dω ( x ) ≈ ∑ i∈ [ N ] wif ( xi ) , ( 1 ) where the nodes xi are independent of f and g , while the weights wi depend only on g. The nodes and the weights of a quadrature may be seen as degrees of freedom that the practitioner may tune in order to achieve a given level of approximation error . The design of quadratures gave birth to a rich literature from Gaussian quadrature [ 15 ] to Monte Carlo methods [ 25 ] to quadratures based on determinantal point processes ( DPPs ) [ 1 ] . These latter form a large class of probabilistic models of repulsive random subsets that make numerical integration possible in a variety of domains with strong theoretical guarantees . In particular , central limit theorems with asymptotic convergence rates that scale better than the typical Monte Carlo rate O ( N−1/2 ) were proven for several DPP based quadratures : when the integrand is a C1 function [ 1 ] or even when the integrand is non-differentiable [ 10 ] . Moreover , it is possible to design quadrature rules based on DPPs with non-asymptotic guarantees and with rates of convergence that adapt to the smoothness of the integrand . This is the case of the quadrature proposed by Ermakov and Zolotukhin in [ 14 ] and recently revisited in [ 16 ] , and the optimal kernel quadrature [ 3 ] . In this work , we study the quadrature rule proposed by Ermakov and Zolotukhin ( EZQ ) through the lens of kernel methods . We start by comparing the weights of EZQ to the weights of the optimal 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . kernel quadrature ( OKQ ) , and we prove that they both belong to a broader class of quadrature rules that we call kernel based interpolation quadrature . Then , we study the approximation quality of EZQ in reproducing kernel Hilbert spaces ( RKHSs ) . This is done by proving a general tractable formula of the expected value of the squared worst-case integration error for functions that belong to the unit ball of an RKHS when the nodes follow the distribution of a determinantal point process . This formula involves principally the eigenvalues of the integral operator , and converges to 0 at a slightly slower rate than the optimal rate . Interestingly , this analysis yields a better upper bound for the optimal kernel quadrature with DPPs proposed initially in [ 3 ] . Comparably to the theoretical guarantees given in [ 14 , 16 ] , our theoretical guarantees are independent of the choice of the test function . This facilitates the comparison of EZQ with other quadratures such as OKQ . The rest of the article is organized as follows . Section 2 reviews the work of [ 14 ] and recall key concepts on kernel based quadrature . In Section 3 , we present the main results of this work and their consequences . A sketch of the proof of the main theorem is given in Section 4 . We illustrate the theoretical results by numerical experiments in Section 5 . Finally , we give a conclusion in Section 6 . Notation and assumptions . We use the notation N∗ = Nr { 0 } . We denote by ω a Borel measure supported on X , and we denote by L2 ( ω ) the Hilbert space of square integrable real-valued functions on X with respect to ω , equipped with the inner product 〈· , ·〉ω , and the associated norm ‖.‖ω . For N ∈ N∗ , we denote by ω⊗N the tensor product of ω defined on XN . Moreover , we denote by F the RKHS associated to the kernel k : X × X → R that we assume to be continuous and satisfying the condition ∫ X k ( x , x ) dω ( x ) < +∞ . In particular , we assume the Mercer decomposition k ( x , y ) = ∑ m∈N∗ σmφm ( x ) φm ( y ) , ( 2 ) to hold , where the convergence is pointwise , and σm and φm are the corresponding eigenvalues and eigenfunctions of the integral operator Σ defined for f ∈ L2 ( ω ) by Σf ( · ) = ∫ X k ( · , y ) f ( y ) dω ( y ) . ( 3 ) We assume that the sequence σ = ( σm ) m∈N∗ is non-increasing and its elements are non-vanishing , and we assume that the corresponding eigenfunctions φm are continuous . Note that the φm are normalized : ‖φm‖ω = 1 for m ∈ N∗ . In particular , ( φm ) m∈N∗ is an o.n.b . of L2 ( ω ) , and every element f ∈ F satisfies ∑ m∈N∗ 〈f , φm〉2ω σm < +∞ . ( 4 ) Moreover , for every N ∈ N∗ , we denote by EN the eigen-subspace of L2 ( ω ) spanned by φ1 , . . . , φN . For any kernel κ : X × X → R , and for x ∈ XN , we define the kernel matrix κ ( x ) : = ( κ ( xi , xj ) ) i , j∈ [ N ] ∈ RN×N . Finally , we denote in bold fonts the corresponding kernel matrices : K ( x ) for the kernel k , KN ( x ) for the kernel kN , K⊥N ( x ) for the kernel k ⊥ N , κ ( x ) for the kernel κ ... Similarly , for any function µ : X → R and for x ∈ XN , we define the vector of evaluations µ ( x ) : = ( µ ( xi ) ) i∈ [ N ] ∈ RN . 2 Related work . In the section , we review some results that are relevant to the contribution . 2.1 Ermakov-Zolotukhin quadrature . The quadrature rule proposed by Ermakov and Zolotukhin in [ 14 ] deals with integrals that write∫ X f ( x ) φm ( x ) dω ( x ) , ( 5 ) where f ∈ L2 ( ω ) , and ( φm ) m∈N∗ is an orthonormal family with respect to the measure ω . Its construction goes as follows . Let N ∈ N∗ and let x ∈ XN such that the matrix ΦN ( x ) : = ( φn ( xi ) ) ( n , i ) ∈ [ N ] × [ N ] is non-singular . For n ∈ [ N ] , define IEZ , n ( f ) = ∑ i∈ [ N ] ŵEZ , ni f ( xi ) , ( 6 ) where ŵEZ , n : = ( ŵEZ , ni ) i∈ [ N ] ∈ RN is given by ŵEZ , n = ΦN ( x ) −1en , with en is the vector of RN with the n-th coordinate is 1 and the rest are 0 1 . We can prove easily that this quadrature is exact ∀f ∈ Span ( φn ) n∈ [ N ] , ∑ i∈ [ N ] ŵEZ , ni f ( xi ) = ∫ X f ( x ) φn ( x ) dω ( x ) . ( 7 ) A quadrature rule that satisfies ( 7 ) is typically called an interpolatory quadrature rule . Now , if f /∈ Span ( φn ) n∈ [ N ] , the authors studied the expected value and the variance of IEZ , n ( f ) when x = ( x1 , . . . , xN ) is taken to be a random variable in XN that follows the distribution of density pDPP ( x1 , . . . , xN ) : = 1 N ! Det2 ΦN ( x ) , ( 8 ) with respect to the product measure ω⊗N defined on XN . As it was observed in [ 16 ] , the nodes of the quadrature follow the distribution of the determinantal point process of reference measure ω and marginal kernel κN defined by κN ( x , y ) = ∑ n∈ [ N ] φn ( x ) φn ( y ) . We refer to [ 20 ] for further details on determinantal point processes . Now , we recall the main result of [ 14 ] . Theorem 1 . Let x be a random subset of X that follows the distribution of DPP of kernel κN and reference measure ω . Let f ∈ L2 ( ω ) , and n ∈ [ N ] . Then EDPPIEZ , n ( f ) = ∫ X f ( x ) φn ( x ) dω ( x ) , ( 9 ) and VDPPIEZ , n ( f ) = ∑ m≥N+1 〈f , φm〉2ω . ( 10 ) Theorem 1 shows that the IEZ , n ( f ) is an unbiased estimator of ∫ X f ( x ) φn ( x ) dω ( x ) , and its variance depends on the coefficients 〈f , φm〉ω for m ≥ N + 1 . Consequently , the expected squared error of the quadrature is equal to the variance of IEZ , n ( f ) and it is given by EDPP ∣∣∣∣ ∫ X f ( x ) φn ( x ) dω ( x ) − ∑ i∈ [ N ] ŵEZ , ni f ( xi ) ∣∣∣∣2 = ∑ m≥N+1 〈f , φm〉2ω . ( 11 ) The identity ( 11 ) gives a theoretical guarantee for an interpolatory quadrature rule when the nodes follow the distribution of the DPP defined by ( 8 ) . Compared to existing work on the literature [ 11 , 22 , 23 , 24 ] , ( 11 ) is generic and applies to any orthonormal family . Now , we may observe that the expected squared error in ( 11 ) depends strongly on the function f . This makes the comparison between EZQ and other quadratures , based on some test function f , tricky : the choice of f may favor ( or disfavor ) EZQ . In order to circumvent this difficulty , we suggest to study a figure of merit that is independent of the choice of the function f . This is possible using kernels through the study of the worst-case integration error on the unit ball of an RKHS . The definition of this quantity will be recalled in the following section . 2.2 The worst integration error in kernel quadrature . The use of the kernel framework in the context of numerical integration can be tracked back to the work of Hickernell [ 18 , 19 ] , who introduced the use of kernels to the quasi Monte Carlo community . Their use was popularized in the machine learning community by [ 28 , 9 ] . In this framework , the quality of a quadrature is assessed by the worst-case integration error on the unit ball of an RKHS F associated to some kernel k : X × X → R+ . This quantity is defined as follows sup f∈F , ‖f‖F≤1 ∣∣∣ ∫ X f ( x ) g ( x ) dω ( x ) − ∑ i∈ [ N ] wif ( xi ) ∣∣∣ . ( 12 ) 1The dependency of the vector ŵEZ , n on x was dropped for simplicity . This quantity reflects how good is the quadrature uniformly on the unit ball of F . Interestingly , this quantity has a closed formula ∥∥∥∥µg − ∑ i∈ [ N ] wik ( xi , . ) ∥∥∥∥ F , ( 13 ) where µg = Σg is the so-called embedding of g in the RKHS F . We shall use in Section 3.2 the equivalent expression ( 13 ) of the worst-case integration error , to derive a closed formula of EDPP sup f∈F , ‖f‖F≤1 ∣∣∣ ∫ X f ( x ) g ( x ) dω ( x ) − ∑ i∈ [ N ] ŵEZ , ni f ( xi ) ∣∣∣2 . ( 14 ) By now , we observe that the weights ŵEZ , ni of EZQ are non-optimal in the sense that they do not minimize ( 13 ) . By definition , the optimal kernel quadrature for a given configuration x , such that the kernel matrixK ( x ) is non-singular , is the quadrature with nodes the xi and weights the ŵi that minimize ( 13 ) . We specify in Section 3.1 the subtle difference between the quadrature of Ermakov and Zolotukhin and the optimal kernel quadrature . Before that , we review the existing constructions of the optimal kernel quadrature in the following section .

Summary: The authors demonstrate how two kinds of kernel quadrature are related (Ermakov-Zolotukhin quadrature and optimal kernel quadrature) by defining a generalisation that encapsulates both. They then give worst case bounds for Ermakov-Zolotukhin quadrature, which are worse than for the optimal one. However, they then build on this result to tighten the known bounds of optimal kernel quadrature when using a determimental point process to distribute the nodes.  Result is certainly relevant and valuable, though of very niche interest. Still a lot more interesting than most "0.1% better" NN papers though! Paper itself is as clear as a wall of maths can be (and has a good intro), but while the precision is critical I will always object to this mode of presentation: maths written without intuition or examples works only for those who already understand the solution. For everyone else it creates an unnecessary challenge to understanding. But maybe this doesn't matter given the niche nature of the result.  To allow a reader to apply an appropriate weight to this review, it should be known that I have not understood this paper to the standard I would usually obtain. There is too much, I haven't had the time and I remain extremely burned out (or whatever this is; "brain replaced with tired cloud" fits just as well) from the extreme workload of the last year. Saying that, given the shear mathematical denseness within this paper I'm not convinced giving it a proper review in the tight time frame of a conference review is actually a realistic prospect.