A Quadrature Rule combining Control Variates and Adaptive Importance Sampling
Anonymous Author(s)
AfﬁliationAddress email
Abstract
Driven by several successful applications such as in stochastic gradient descentor in Bayesian computation, control variates have become a major tool for Monte
Carlo integration.
However, standard methods do not allow the distribution of theparticles to evolve during the algorithm, as is the case in sequential simulationmethods.
Within the standard adaptive importance sampling framework, a simpleweighted least squares approach is proposed to improve the procedure with controlvariates.
The procedure takes the form of a quadrature rule with adapted quadratureweights to reﬂect the information brought in by the control variates.
The quadraturepoints and weights do not depend on the integrand, a computational advantage incase of multiple integrands.
Moreover, the target density needs to be known only upto a multiplicative constant.
Our main result is a non-asymptotic bound on the probabilistic error of the procedure.
The bound proves that for improving the estimate’saccuracy, the beneﬁts from adaptive importance sampling and control variatescan be combined.
The good behavior of the method is illustrated empirically onsynthetic examples and real-world data for Bayesian linear regression.
Introduction
In recent years, sequential simulation has emerged as a leading approach to compute multidimensionalintegrals.
A key object in sequential simulation is the sequence of distributions, called the policy,from which to generate the random variables, called particles, used to approximate the integrals of interest.
The policy is designed to evolve in the course of the algorithm to mimic the target density,which may itself be known only up to a proportionality constant.
While the design of algorithms withadaptive policies has been of major interest recently, only a few studies have focused on using controlvariates to reduce the variance.
This paper provides a new method to incorporate control variateswithin standard sequential algorithms.
The proposed approach signiﬁcantly improves the accuracy ofthe initial algorithm, both theoretically and in practice.
The sequential framework.
Consider the problem of approximating the integral (cid:82) gf d λ =
R d g ( x ) f ( x ) d x , where λ is the d -dimensional Lebesgue measure, f is a probability density on R dand the integrand g is a real-valued function on R d .
For instance, one may think of f as the posteriordensity in Bayesian inference.
Let ( q i ) i ≥ 0 be the policy of the algorithm, i.e., a sequence of probability densities which evolves adaptively depending on previous outcomes.
The particles ( X i )
i ≥ 1are generated sequentially—at iteration i , particle X i is drawn from q i − 1 .
The integral (cid:82) gf d λ isestimated by the normalized sum (cid:0)(cid:80) ni
=1 w i g
( X i ) (cid:1) / (cid:0)(cid:80) ni
=
w i (cid:1) , where w i = f
( X i ) /q
i −
( X i ) arethe importance weights.
The normalization (cid:80) ni
=1 w i allows to deal with situations where the targetdensity f is known only up to a proportionality constant.
Such an algorithm is part of the adaptive importance sampling (AIS) framework.
Many different wayshave been investigated to update the densities
q i adaptively.
Early works that inspired such sequentialschemes include [13, 21, 28] where the sampling policy is chosen out of a parametric family.
Theparametric approach has been further extended by the Population Monte Carlo framework [4, 5, 25].
Various asymptotic results have been obtained in [6, 10, 32].
In [7, 9, 22, 37], nonparametricimportance sampling based on kernel smoothing is studied.
The latter bears resemblance to sequential
Monte Carlo methods [8, 6], in which the target distribution f changes in the course of the algorithm.
Control variates.
Let h
= ( h 1 , . .
, h m ) (cid:62) be a vector of real-valued functions on R d such that foreach k , the integral (cid:82) h k f d λ is known.
Without loss of generality, suppose that (cid:82) hf
d λ = 0 .
Thefunctions h
k are called control variates and can be obtained in different ways.
In Bayesian statistics,
Stein control variates [27] are constructed by applying the second-order Stein operator to functionssatisfying certain regularity conditions [26].
Other control variates might be created by re-weightinga function h ∗ that satisﬁes (cid:82) h ∗ d λ = 0 via h
= h ∗ /f .
The use of control variates is a well studiedvariance-reduction technique [14, 29].
The beneﬁts can be established theoretically in terms oferror bounds [27, 23], weak convergence [33], the excess risk [2] and even uniform error boundsover large classes of integrands [31].
In practice, the control variates framework has led to efﬁcientprocedures in reinforcement learning [17, 24] and optimization [36], to name a few.
Importancesampling and control variates in case of a Gaussian target density is explored in [19].
The procedurein [20] incorporates control variates and is said to involve adaptive importance sampling, but in factthe particles are always sampled from the uniform distribution on the unit cube.
To the best of ourknowledge, the existing control variate methods do not account for sequential changes in the particledistribution as is the case in AIS.
AISCV estimate.
The proposed approach to use control variates within the sequential AIS frameworkrelies on the ordinary least squares expression of control variates (see for instance [33]).
To takecare of the policy changes, some re-weighting must be applied.
The AISCV estimate of the integral 59 (cid:82)
gf
d λ is deﬁned as the ﬁrst coordinate of the solution to the weighted least squares problem
[Equation]
with w i the importance weights from before.
The AISCV estimate ˆ α n has several interestingproperties: (a) whenever g is of the form α + β (cid:62) h for some α ∈ R and β ∈ R m , the error is zero,i.e., ˆ α n = α = (cid:82)
gf d λ ; (b) the estimate takes the form of a quadrature rule ˆ α n = (cid:80) ni
=1 v n,i g ( X i ) ,for quadrature weights v n,i that do not depend on the function g and that can be computed by a singleweighted least squares procedure; and (c) it can be computed even when f is known only up to amultiplicative constant.
Point (a) suggests that when the linear combinations of the functions h k spana rich function class, the integration error is likely to be small.
Point (b) implies that multiple integralscan be computed just as easily as a single one.
Point (c) shows that the approach is applicable for
Bayesian computations.
In addition, the control variates can be brought into play in a post-hocscheme, after generation of the particles and importance weights, and this for any AIS algorithm.
Main
result.
The main theoretical result of the paper is a probabilistic, non-asymptotic bound onˆ
α n − α .
Under appropriate conditions, the bound scales as τ/ √ n , where τ 2 is the scale constant in asub-Gaussian tail condition on the error variable ε = g − α − β (cid:62) h for ( α, β ) =
arg min a,b (cid:82) ( g −a − b (cid:62) h )
f d λ .
Note that ε has the smallest possible variance one could get using control variates h .
As a consequence, when the space of control variates is well suited for approximating g , the AISCVestimate will be highly accurate.
Also, our bound depends only on the linear function space spannedby the control variates h 1 , . .
, h m , not on the particular basis chosen in that space.
The results relyon martingale theory, in particular on a concentration inequality for norm-subGaussian martingalesin [18].
In the course of the proof, we develop a novel bound on the smallest eigenvalue of certainrandom matrices, extending an inequality from [35] to the martingale case.
Outline.
Section 2 introduces the general framework of adaptive importance sampling and controlvariates.
Next, Section 3 presents the AISCV estimate and the associated quadrature rule.
Section 4contains the statements of the theoretical results while Section 5 gathers practical considerations,including the construction of control variates.
Numerical experiments are presented in Section 6.
Mathematical proofs and additional numerical details are available in the supplementary material.
Preliminaries on Monte Carlo integration
The aim of this section is to present the required mathematical framework for Monte Carlo integrationand the variance reduction methods of interest, namely adaptive importance sampling and the controlvariate technique.
Recall that g : R
d
→ R is an integrand and f a probability density on R d .
The aimis to compute E f
[ g ] = (cid:82) gf d λ .
Adaptive importance sampling.
In adaptive importance sampling (AIS), E f [ g ] is estimated by aweighted mean over a sample of random particles X 1 , . . .
, X n in R d .
Since appropriate samplingdensities naturally depend on g and f , we generally cannot simulate from them.
They are thenapproximated in an adaptive manner by a family of tractable densities ( q i ) i ≥ 0 that often evolvetowards a density q opt that optimizes some criterion.
While the starting density q 0 is ﬁxed, the densityq i for i ≥ 1 is determined in function of the particles X 1 , . . .
, X i already sampled; think for instanceof a parametric family, where the parameter of q i is a function of X 1 , . . .
, X i .
Given the particles
X 1 , . . .
, X i , the next particle, X i +1 , is then drawn from q i .
Formally, let ( X i )
i ≥ 1 be a sequence ofrandom vectors on R d deﬁned on some probability space (Ω , F , P ) .
The distribution of the sequence( X i )
i ≥ 1 is speciﬁed by its policy as deﬁned below.
Deﬁnition 1 (Policy) .
A policy is a random sequence of probability density functions ( q i ) i ≥ 0 on R dadapted to the σ -ﬁeld ( F i ) i ≥ 0 deﬁned by F 0 = { ∅ , Ω } and F i = σ ( X 1 , . . .
, X i ) for i ≥ 1 .
Thesequence ( q i ) i ≥ 0 is the policy of ( X i )
i ≥ 1 whenever X i has density q i − 1 conditionally on F i − 1 .
The (normalized) adaptive importance sampling estimate of E f
[ g ] is then deﬁned as
[Equation]
where w i = f ( X i ) q i − 1 ( X i ) for i = 1 , . . .
, n.
[Equation]
The sampling weights w i reﬂect the fact that X i has been sampled from q i − 1 rather than from f .
The division by (cid:80) ni =1 w i rather than by n has two beneﬁts: ﬁrst, the integration is exact for constantintegrands, and second, f needs to be known only up to a proportionality constant, an advantage for
Bayesian inference.
Since updating the density q i at each iteration may be computationally expensive, it is customaryto hold it ﬁxed over a pre-determined number of iterations.
Writing n = n 1 + · · · + n T in terms ofpositive integers ( n t )
Tt =1 called the allocation policy , the AIS estimate then becomes
[Equation]
where w t,i = f
( X t,i ) q t ( X t,i ) (2) for t = 1 , . . .
, T and i = 1 , . .
. , n t .
At stage t , the particles X t, 1 , . . .
, X t,n t are sampled independently from q t − 1 , while all particles sampled up to and including stage t are used to determine thesampling density q t for stage t
+ 1 .
It is easy to see that the two formulations of the AIS estimateare equivalent: (1) arises from (2) by setting n t = 1 for all t , while (2) can be obtained from (1) byconstructing the policy in such a way that the densities q i do not change within integer intervals ofthe form { 0 , . . .
, n 1 − 1 } , { n 1 , . . . , n 1 + n 2 − 1 } , and so on.
While the shorter representation (1) ismore convenient for theoretical purposes, formulation (2) is the one used in practice (see Section 6).
Interestingly, the AIS estimate (1) may be seen as a weighted least-squares estimate minimizing theloss function a (cid:55)→ (cid:80) ni =1 w i ( g ( X i ) − a ) 2 .
This perspective is key to understand control variates.
Control variates.
The control variates method is a variance reduction technique that consistsin incorporating a new piece of information—the known values of the integrals of some controlfunctions—in a basic Monte Carlo framework.
Control variates are simply functions h 1 , . .
, h m ∈
L 2 ( f ) with known integrals.
Without loss of generality, assume that E f [ h j ] = 0 for all j = 1 , . . .
, m .
Let h
= ( h 1 , . .
, h m ) (cid:62) denote the R m -valued function with the m control variates as elements.
Forany coefﬁcient vector β ∈ R m , we have E f [ g − β (cid:62) h ] = 0 .
Given an independent random sample X 1 , . . .
, X n from f , any β ∈ R m therefore results in an unbiased estimator of E f
[ g ] by
[Equation]
Provided the m × m covariance matrix
G
=
E f [ hh (cid:62) ] is invertible, there is a unique coefﬁcient vectorβ ∗ ∈ R m for which the variance of I (cv) n ( g ) is minimal and it is given by
[Equation]
This vector being generally unknown, it needs to be estimated from the particles X 1 , . . .
, X n .
Castingthe problem in an ordinary least squares framework leads to the control variate estimate
[Equation]
where
[Equation]
The estimator I (cv) n ( g ) is well-deﬁned provided the minimizer ˆ α (cv) n to (5) is unique.
This is the caseif and only if there does not exist b ∈ R m such that b (cid:62) h
( X i )
=
for all i = 1 , . . . , n .
The asymptotic distribution of I (cv) n ( g ) as n → ∞ is the same as if the variance-minimizing vectorβ ∗ were used in (3).
In particular, the asymptotic variance of I (cv) n ( g ) is σ 2 m ( g ) /n
where
[Equation]
Interestingly, when using only the ﬁrst (cid:96) out of m control variates, where (cid:96) ∈ { 0 , 1 , . . . , m } , wehave σ 2 m ( g ) ≤ σ 2 (cid:96) ( g ) .
In terms of asymptotic variance, it therefore never harms to add more controlvariates.
Their construction will be addressed in Section 5.1.
Combining adaptive importance sampling with control variates
AISCV estimator.
Consider the same integration problem E f
[ g ] = (cid:82)
gf
d λ as in Section 2.
Withthe idea of performing variance reduction when calculating integrals with respect to the posteriordensity in Bayesian inference, we incorporate control variates into the AIS estimate.
Let the particles( X i )
i ≥ 1 be generated according to a policy ( q i ) i ≥ 0 as in Deﬁnition 1.
Let h
= ( h 1 , . .
, h m ) (cid:62) be avector of control variates, i.e., h j ∈ L 2 ( f ) and E f
[ h j ] = 0 for every j = 1 , . . .
, m .
Combining (1)and (3), the proposed estimate takes the form
[Equation]
where β ∈ R m remains to be determined.
To do so, the ordinary least-squares problem in (5) isreplaced by a weighted one, yielding the novel AISCV estimator
[Equation]
where
[Equation]
The estimator is well-deﬁned only if the minimizer ˆ α n is unique—the minimizer ˆ β n need not be.
Wewill come back to this in the next paragraph.
As in (2), the policy may be divided into T stages in order to reduce the number of times the samplerneeds to be updated.
Stage t
= 1 , . .
, T has length n t , with (cid:80)
Tt =1 n t = n .
Within each stage, thesampling density remains constant.
In practice, this leads to the AISCV estimate in Algorithm 1.
Quadrature rule.
The AIS estimate (1) is a quadrature rule with quadrature points
X i and quadratureweights proportional to the sampling weights w i .
The AISCV estimate (7) has the same property, butwith adapted quadrature weights.
Let e n = ( e n,i ) i =1 ,...,n be the vector of residuals resulting fromthe weighted least-squares regression of the constant vector 1 n = (1 , . .
, 1) (cid:62) ∈ R n on the controlvariates but without intercept:
[Equation]
where
[Equation]
Even though the vector ˆ β n ( 1 n ) is not necessarily unique, the weighted least squares ﬁt(ˆ β n ( 1 n ) (cid:62) h
( X i ))
i =1 ,...,n always is.
According to the next proposition, the quadrature weights areproportional to ( w i e n,i ) i
=1 ,...,n .
Require: integrand g , target density f (up to a proportionality constant), number of stages T ∈ N ∗ , allocation policy ( n t )
Tt =1 , initial density q 0 , update rule for the sampling policy 1: for t = 1 , . .
, T do 2:
Generate an independent random sample X t, 1 , . . .
, X t,n t from q t −3:
Compute the vector of weights ( w t,i ) n t i =1 where w t,i = f
( X t,i ) /q
t − 1 ( X t,i ) 4:
Construct the matrix of control variates H t = (cid:0)
h j ( X t,i ) (cid:1)
j
=1 ,...,m i =1 ,...,n t 5:
Evaluate the integrand in the particles: ( g ( X t,i )) n t i = 6:
Update the sampler q t based on all previous particles ( X s,i : s = 1 , . .
. , t ; i = 1 , . .
. , n s ) 7: end for 8: Compute (ˆ α T , ˆ β T ) =
arg min ( a,b ) ∈ R × R
m
(cid:110)(cid:80)
Tt =1 (cid:80) n t i =1 w t,i (cid:0) g ( X t,i ) − a − b (cid:62) h
( X t,i ) (cid:1) 2 (cid:111) 9: return I (aiscv) n ( g ) =
ˆ α T .
Proposition 1 (AISCV quadrature rule) .
The minimizer ˆ α n in (7) is unique if and only if e n (cid:54) = 0 in(8) .
In that case, the AISCV estimate is
I (aiscv) n ( g ) =
ˆ α n = (cid:80) ni
=1 w i e n,i g ( X i ) (cid:80) ni
=1 w i e n,i .
If e n = 0 , then there exists b ∈ R m such that b (cid:62) h
( X i )
=
for all i = 1 , . . .
, n .
In that case, theminimizer ˆ α n in (7) is not unique and the AISCV estimate is not well-deﬁned.
To remedy this, onecan for instance reduce the number of control variates.
This issue already occurs with the ordinarycontrol variate estimator in (3).
Rather than requiring a different weighted least squares problem for every integrand g as in (7),the quadrature rule in (9) only involves a single weighted least squares problem (8), whatever g .
Given the quadrature weights, calculating the AISCV estimate for a novel integrand only requires theevaluations of that function on the sampled particles, making the whole procedure a post-hoc scheme.
The steps in case the sampling policy is divided into T stages are given in Algorithm 2, which givesthe same result as Algorithm 1, but with less effort if multiple integrands g are into play.
Algorithm
Require: integrand g , T ∈ N ∗ , allocation policy ( n t )
Tt =1 , weights ( w t )
Tt =1 with w t = ( w t,i ) n t i =1 , matrices ( H t )
Tt =1 with H t = (cid:0)
h
j ( X t,i ) (cid:1) j
=1 ,...,m i =1 ,...,n t , particles ( X t
i t
T i
. , n t )
t
: Compute
Theoretical properties of the AISCV estimate
Here we point out several theoretical properties of the novel AISCV estimate.
A ﬁrst point is that theintegration rule is exact on the linear span of the control variates and the constant function.
Proposition 2 (Exact integration) .
For integrands of the form g
= α + β (cid:62) h for α ∈ R and β ∈ R m ,the AISCV estimate is exact: I (aiscv) n ( g ) =
α = E f
[ g ] .
A second property is that we may apply arbitrary invertible linear transformations to the controlvariates without changing the AISCV estimate.
This can be advantageous computationally, to makethe underlying weighted least squares problem more stable numerically.
Also, it means that without loss of generality, we may assume that the control variates are uncorrelated and have unit variance,which simpliﬁes the theoretical performance analysis.
Proposition 3 (Invariance) .
If the matrix A ∈ R m × m is invertible, then the AISCV estimate basedon the control variates Ah is the same as the one based on h .
Our main result is a non-asymptotic bound on the error of the AISCV estimate for (cid:82)
gf d λ when 185 (cid:82) g
2 f d λ is ﬁnite.
First, we introduce some assumptions and deﬁnitions.
The ﬁrst condition that is required concerns the policy given by the AIS part of the algorithm.
It issupposed that any element from the policy should dominate the function f .
Assumption 1 (Dominated measures) .
There exists c ≥ 1 such that, for all x ∈ R d and for anyi = 1 , . . .
, n , we have f ( x ) ≤ c · q i ( x ) .
This assumption represents a safe approach to importance sampling, as the policy will always allow tosample in places where f is positive.
A well-known and well-spread [16, 29, 9] technique to achievesuch a defensive strategy is to a use mixture density q i = (1 − η )
f i + ηq 0 where η ∈ (0 , 1) andwhere q 0 has sufﬁciently heavy tails to dominate f .
Such a mixture allows to choose the densitiesf i with some ﬂexibility using in principle any AIS algorithm.
Second, the control variates shall belinearly independent and bounded.
Assumption 2 (Control variates) .
We have sup x : f ( x ) > 0 | h j ( x ) | < ∞ for all j = 1 , . . .
, m .
Thematrix G = (cid:82) hh (cid:62) f
d λ is invertible.
The previous condition allows to deﬁne the standardized vector of control variates as (cid:126) =
G − 1 / 2 h .
By Proposition 3, this change does not affect the AISCV estimate.
The orthonormal control variates (cid:126) will play a key role through the following quantity
[Equation]
The quadratic form (cid:107) (cid:126) ( x ) (cid:107) 22 = h ( x ) (cid:62) G − 1 h ( x ) is referred to as the leverage function in ordinarylinear regression as it quantiﬁes the inﬂuence of a training point x on the prediction of the observedresponse.
It is invariant with respect to invertible linear transformations of the control variate vector.
Assumption 2 and the fact that the integrand g is square integrable with respect to f allows to deﬁnethe residual function ε = g − (cid:82) gf d λ − h (cid:62) β ∗ where β ∗ has been introduced in (4) as a minimizerof the residual variance.
Since we work in the space L 2 ( f ) , we assume without loss of generality thatg and h vanish outside { x : f ( x ) > 0 } and we put ε ( x ) = 0 for x ∈ R d such that f ( x ) = 0 .
Theresidual function ε should satisfy the following tail condition.
Assumption 3 (Residual tail) .
There exists τ > 0 such that, for all t > 0 and all integer
i ≥ 1 , wehave P [ | w i ε ( X i ) | > t | F i − 1 ] ≤ 2 exp (cid:0) − t 2 / (2 τ 2 ) (cid:1) .
The previous assumption concerns both the function ε and the policy sequence ( q i ) i ≥ 0 .
Since 209 E [ w i ε ( X i ) | F i − 1 ] = 0 , it is implied by the so-called sub-Gaussian condition [3] that 210 E
[exp( λw i ε ( X i )) | F i − 1 ] ≤ exp (cid:0) − λ 2 τ 2 / 2 (cid:1) for any λ ∈ R .
In the proof of Theorem 1, Assumption 3 allows to derive concentration bounds on residual-based sums using recent results from[18, 23].
We are now in position to state our main result on the error of the AISCV estimate.
Theorem 1 (Concentration inequality for AISCV estimate) .
If Assumptions 1, 2 and 3 hold, then, forany δ ∈ (0 , 1) and for all n ≥ C 1 c 2 B log(10 m/δ ) , we have, with probability at least 1 − δ , that
[Equation]
where C 1 , C 2 , C 2 are universal constants speciﬁed in the proof.
Remark 1 (Understanding τ ) .
The quantity τ in Assumption 3 is related to the conditional variance 217 E [ w 2 i ε 2 ( X i ) | F i − 1 ] .
They actually coincide when w i ε ( X i ) is Gaussian.
For a policy satisfying
Assumption 1, E [ w 2 i ε 2 ( X i ) | F i − 1 ] ≤ cσ 2 m which for certain combinations of integrands and controlfunctions scales as m − s/d [33] where the parameter s represents the degree of smoothness of g .
Remark 2 (Convergence rates) .
Consider an asymptotic regime where the number of control variatesm tends to inﬁnity with the sample size n .
The AISCV estimate improves upon the AIS method( m = 0) , which has rate 1 / √ n , as soon as τ + τB log( m ) / √ n → 0 .
To recover the same order ofan oracle estimate with rate τ/ √ n , one must have B log( m )
=
O ( √ n ) as n → ∞ .
Practical considerations
This section presents ways to build control variates using either families of polynomials or generalfunctions based on Stein’s method, with a highlight on computations in the Bayesian framework.
5.1 Control variate constructions
Orthogonal polynomials.
When the target density f can be decomposed as a product of univariatedensities f = p 1 ⊗ · · · ⊗ p d , multidimensional control functions may be constructed based onunivariate ones.
This happens for instance for the uniform distribution over the unit cube [0 , 1] dor with uncorrelated Gaussian distributions on R d .
Such univariate control variates may be easilyconstructed using families of polynomials [12], such as Legendre polynomials for the uniformdistribution on [0 , 1] and Hermite polynomials for the Gaussian distribution on R .
This technique canalso be used when f is dominated by another density f ∗ having the said product form by transformingzero-mean control variates h ∗ with respect to f ∗ via h
= h ∗ f ∗ /f .
Let ( h 1 , . . .
, h k ) be a vector of univariate control functions with respect to a density p , i.e., E p [ h j ] =
0for all j = 1 , . .
k .
Let h
= 1 denote the constant function equal to one.
For a multi-index(cid:96) =
( (cid:96) 1 , . . .
, (cid:96) d ) in { 0 , . . .
, k } d \ { (0 , . . . , 0) } , multivariate controls with respect to p ⊗ d are built byforming tensor products of the form h (cid:96) ( x 1 , . .
, x d ) =
h (cid:96) 1 ( x 1 ) · · · h (cid:96) d ( x d ) , yielding a total numberof m = ( k + 1) d − 1 control functions.
Alternative approaches yielding smaller control spacesconsist of imposing (cid:96) j = 0 for all but a small number of coordinates j = 1 , . . .
, d or by the constraint(cid:96) 1 + · · · + (cid:96) d ≤ Q for some Q ≥ 1 .
Stein control variates.
In the general case where one has only access to the evaluations of f , controlvariates may be constructed using Stein’s method.
The technique relies on the gradient ∇ x log f ( x )which can either be directly computed (see the example of Bayesian regression below) or whichmay be available through automatic differentiation provided in popular API’s such as Tensorﬂowand PyTorch [1, 30].
Let ∆ x = ∇ (cid:62) x ∇ x denote the Laplace operator.
By deﬁnition, the second-order
Stein operator L [34, 15] associated to the density f is deﬁned by:
[Equation]
The transformation guarantees that E f [ L ϕ ] = 0 for all ϕ with weak regularity conditions [26].
Therefore, we can build inﬁnitely many control variates h ϕ
=
L ϕ from given functions ϕ .
One simple wayis to let ϕ be a polynomial with bounded total degree: for a degree vector
α =
( α 1 , . .
. , α d ) ∈ N dwith
α 1 + · · · + α d ≤ Q , deﬁne ϕ
α ( x ) =
x α 1 1 · · · x α d d .
Given the dimension d and the total degree
Q , there are m = (cid:0) d
+ Qd (cid:1) − 1 such degree vectors, yielding the associated control variates h α
= h ϕ α .
For fast computation, note that, writing φ α ( x ) =
ϕ α ( x ) 1 d , D 1 ( x ) =
diag( α 1 /x 1 , . .
. , α d /x d )and D 2 ( x ) =
diag( α 1 ( α 1 − 1) /x 21 , . .
. , α d ( α d − 1) /x 2 d ) , we have ∇ x ϕ α ( x ) =
D 1 ( x ) φ α ( x ) and∆ x ϕ α ( x ) = 1 (cid:62)
d ( D 2 ( x ) φ α ( x )) .
In practice, all combinations of α are stored in a matrix A ∈ N m × d .
5.2 Bayesian inference
Given data D and a parameter of interest θ ∈ Θ ⊂ R d , posterior integrals take the form 259 (cid:82)
R d g ( θ ) p ( θ | D )
d θ, where p ( θ | D ) ∝ (cid:96) ( D | θ )
π ( θ ) is the posterior distribution, proportional to aprior π ( θ ) and a likelihood function (cid:96) ( D | θ ) .
For instance, when g ( θ ) = θ , the integral aboverecovers the posterior mean.
Stein control variates involve the computation of the gradient of the logposterior ∇ θ log p ( θ | D ) , which implicitly relies on the score function ∇ θ log (cid:96) ( D | θ ) .
We point outtwo common examples—linear and logistic regression—where these functions are easy to compute.
Bayesian linear regression.
Consider a linear regression problem comprised of observations
X ∈ R N × d with labels y ∈ R N .
In the Gaussian ﬁxed design setting, the predictor x i producesthe response y i
=
x (cid:62) i θ + ε i where ε 1 , . .
ε N ∼ N (0 , σ 2 ) are centered Gaussian noises.
Thelikelihood (cid:96) ( X, y | θ ) is proportional to ( σ 2 ) − N/ 2 exp (cid:0) − ( y − Xθ ) (cid:62) ( y − Xθ ) / (2 σ 2 ) (cid:1) , yielding thescore function ∇ θ log (cid:96) ( X, y | θ ) =
X (cid:62) ( y − Xθ ) / (2 σ 2 ) .
Bayesian logistic regression.
Next, consider the logistic regression problem comprised of observations X ∈ R N × d with associated binary labels y ∈ { 0 , 1 } N .
Letting σ ( s )
= 1 / (1 + e − s )
denote thesigmoid function
, the likelihood function is (cid:96) ( X, y | θ ) =
(cid:81)
Ni =1 σ ( θ (cid:62) x i )
y i (1 − σ ( θ (cid:62) x i ))
1 − y i .
Thescore function is simply ∇ θ log (cid:96) ( X, y | θ ) =
X (cid:62) ( y − σ ( Xθ )) .
Numerical illustration
To compare the ﬁnite-sample performance of the AIS and AISCV estimators, we ﬁrst present in
Section 6.1 synthetic data examples involving the integration problem over the unit cube [0 , 1] d andthen with respect to some Gaussian mixtures as in [4].
The goal is to compute (cid:82)
gf d λ for vectors ofintegrands g : R d → R p .
We consider various dimensions d > 1 and several choices for the numberof control variates m .
Section 6.2 deals with real-world datasets in the context of Bayesian inference.
For ease of reproducibility, the code, numerical details and additional results are available in thesupplementary material.
Parameters.
In all simulations, the sampling policy is taken within the family of multivariate
Student t distributions of degree ν denoted by { q µ, Σ 0 : µ ∈ R d } with Σ 0 = σ 0 I d ( ν − 2) /ν
andν > 2 , σ 0 > 0 .
Similarly to [32], the mean µ t is updated at each stage t = 1 , . .
, T by the generalizedmethod of moments (GMM), leading to µ t =
( (cid:80) ts =1 (cid:80) n s i =
1 w s,i X s,i ) /
( (cid:80) ts =1 (cid:80) n s i =1 w s,i ) .
Theallocation policy is ﬁxed to n t = 1000 and the number of stages is T ∈ { 5; 10; 20; 30; 50 } .
Thedifferent Monte Carlo estimates are compared by their mean squared error (MSE) obtained over 100independent replications.
6.1 Synthetic examples
Integration on [0 , 1] d .
We seek to integrate functions g with respect to the uniform densityf ( x ) = 1 for x ∈ [0 , 1] d in dimensions d ∈ { 4; 8 } .
We rely on Legendre polynomials forthe control variates.
Consider the integrands g 1 ( x ) = 1 + sin (cid:16) π (2 d − 1 (cid:80) di
=1 x i − 1) (cid:17) , g 2 ( x ) =(cid:81) di
=1 (2 /π ) 1 / 2 x − 1 i e − log( x i ) 2 / 2 and g 3 ( x )
= (cid:81) di
=1 log(2)2 1 − x i , all of which integrate to 1 on [0 , 1] d .
None of the integrands is a linear combination of the control variates.
The policy parameters areµ
= (0 . 5 , . . .
) ∈ R d , ν = 8 , and σ 0 = 0 .
The control variates are built out of tensor productsof Legendre polynomials where the degree (cid:96) j equals 0 for all but two coordinates, leading to a totalnumber of m =
kd + k 2 d ( d − 1) / 2 control variates.
The maximum degree in each variable is k = 6 ,yielding m = 240 and m = 1056 control variates in dimensions d = 4 and d = 8 respectively.
Figure 1 presents the boxplots of the AIS and AISCV estimates.
The error reduction obtained thanksto the control variates is huge: the AISCV estimate has a mean squared error smaller than the one ofthe AIS estimate by a factor at least 10 and up to 100 (see Table 1 in the supplement).
[Figure]
g g
Gaussian mixture f and Stein control variates.
In this setting we assume we only have access tothe evaluations of the target density f .
We consider the classical example introduced in [4] wheref is a mixture of two Gaussian distributions.
The control variates are built using Stein’s method(Section 5.1) out of polynomials of total degree at most Q ∈ { 2; 3 } , leading to a number of controlvariates m ∈ { 14; 34 } in dimension d = 4 and m ∈ { 44; 164 } in dimension d = 8 respectively.
Weconsider two cases: an isotropic and an anisotropic one.
Isotropic case.
Let f Σ ( x ) = 0 .
5Φ Σ ( x − µ )+0 .
5Φ Σ ( x + µ ) where µ = (1 , . .
, 1) (cid:62) / 2 √ d, Σ
= I d /dand Φ Σ is the multivariate normal density function with zero mean and covariance matrix Σ .
The
Euclidean distance between the two mixture centers is 1 , independently of d .
The initial density q 0 isthe multivariate Student t distribution with mean (1 , − 1 , 0 , . . .
√ d and variance (5 /d )
I d .
The initial mean value differs from the null vector to prevent the naive algorithm using the initial densityfrom having good results due to the symmetrical set-up.
Anisotropic case.
In this case, the mixture is unbalanced and each Gaussian is anisotropic.
Thetarget density is f V ( x ) = 0 .
75Φ V ( x − µ ) + 0 .
25Φ V ( x + µ ) where µ = (1 , . .
, 1) (cid:62) / 2 √ d and
V = diag(10 , 1 , . .
/d .
The initial density q 0 is the same as for the isotropic case.
Figure 2 presents boxplots of the logarithm of the mean squared error (cid:107) ˆ I ( g ) − I ( g ) (cid:107) 22 .
Once again,the AISCV estimators are the clear winners with a mean squared error smaller by a factor up to 1000for the anisotropic case (see Table 2 in the supplement).
[Figure] [Figure]
Sample Size (x1e3) {+
+} L o g o f M e a n Sq u a r e d
E rr o r
||
I ( g )
I ( g ) ||
AISAISCV2AISCV (d) f V ( d = 8)
6.2 Real-world examples
We place ourselves in the framework of Bayesian linear regression (Section 5.2) with features X ∈ R N × d and continuous responses y ∈ R N .
The posterior distribution p ( θ | D ) involves a Gaussian prior π ( θ ) ∼ N ( µ a , Σ a ) and a likelihood function (cid:96) ( D | θ ) proportional to( σ 2 ) − N/ 2 exp (cid:0) − ( y − Xθ ) (cid:62) ( y − Xθ ) / (2 σ 2 ) (cid:1) .
The noise level is ﬁxed and taken sufﬁciently largeat σ = 50 to account for general priors.
The posterior distribution is Gaussian too: N ( µ b , Σ b ) withµ b = Σ b ( σ − 2 X (cid:62)
y + Σ − 1 a µ a ) and Σ b = ( σ − 2 X (cid:62) X + Σ − 1 a ) − 1 .
The integrand is g ( θ )
= (cid:80) di[-=
-]
=1 θ 2 i variates are built with the Stein operator (Section 5.1) out of monomials with totaldegree Q ∈ { 1; 2 } , leading to the AISCV1 and AISCV2 estimators respectively.
Datasets and parameters.
Classical datasets from [11] are considered : housing ( N = 506; d =13; m ∈ { 12; 104 } ); abalone ( N = 4177; d = 8; m ∈ { 7; 44 } ); red wine ( N = 1599; d = 11; m ∈{ 10; 77 } ); and white wine ( N = 4898; d = 11; m ∈ { 10; 77 } ).
The initial density is the multivariate
Student t distribution with ν = 10 degrees of freedom, zero mean and covariance matrix Σ b .
[Figure]
Results.
Figure 3 presents the boxplots of the relative error (ˆ I ( g ) − I ( g )) /I
( g ) , revealing the beneﬁtsof control variates even with polynomials of degree Q = 1 .
When Q = 2 , the error of the AISCV2estimate is virtually zero (see Table 3 in the supplement), in line with Proposition 2.
The meansquared error of the AISCV1 estimate is smaller than that of the AIS estimate by a factor rangingbetween 2 and 10 .
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes]
(c) Did you discuss any potential negative societal impacts of your work?
[N/A]
The
theoretical results presented in this paper do not present any foreseeable societal
consequence.
(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] see Section 4.(b)
Did you include complete proofs of all theoretical results?
[Yes] see supplementary
material.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes] see supplemen- tary material.
(b) Did you specify all the training details (e
g., data splits, hyperparameters, how theywere chosen)?
[Yes] see supplementary material Appendix D.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] see numerical experiments and supplementary material.
The different ﬁgures and tables report the boxplots and numerical values obtained over100 independent runs
(d) Did you include the total amount of compute and the type of resources used (e.g.,type of GPUs, internal cluster, or cloud provider)?
[Yes] see supplementary material
Appendix D.
If you are using existing assets (e g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[N/A]
(b) Did you mention the license of the assets?
[N/A]
(c ) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identiﬁable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects..
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]