BR-SNIS: Bias Reduced Self-Normalized Importance Sampling
Abstract
Importance Sampling (IS) is a method for approximating expectations under a target distribution using independent samples from a proposal distribution and the associated importance weights.
In many applications, the target distribution is known only up to a normalization constant, in which case self-normalized IS (SNIS) can be used.
While the use of self-normalization can have a positive effect on the dispersion of the estimator, it introduces bias.
In this work, we propose a new method, BR-SNIS, whose complexity is essentially the same as that of SNIS and which signiﬁcantly reduces bias without increasing the variance.
This method is a wrapper in the sense that it uses the same proposal samples and importance weights as SNIS, but makes clever use of iterated sampling–importance resampling (i-SIR) to form a bias-reduced version of the estimator.
We furnish the proposed algorithm with rigorous theoretical results, including new bias, variance and high-probability bounds, and these are illustrated by numerical examples.
Introduction Background and previous work:
Importance sampling [17, 1] (IS) is a classical Monte Carlo technique for estimating expectations under some given probability distribution (the target ) on the basis of a sample of draws from a different distribution (the proposal ).
In the modern era of artiﬁcial intelligence and statistical machine learning, characterized by large computational resources and
Bayesian inference, IS technologies are enjoying a revival; see, e.g. , [37, 23] and [11] for a recent survey.
The method is not only relevant to situations where sampling from the target is intractable; it can also be used to achieve variance reduction [24].
When the proposal is dominating the target— in the sense that the support of the latter is contained in the support of the former—unbiased estimation can be achieved by assigning each draw an importance weight given by the likelihood ratio between the target and the proposal.
In the very common case where the target is known only up to a normalizing constant, consistent estimation can still be achieved by simply normalizing each importance weight by the total weight of the sample; however, since such self-normalized importance sampling (SNIS) involves ratios of random variables, the procedure can only be implemented at the cost of bias, which can be signiﬁcant in some applications.
More precisely, let ( X , X ) be some state space and π (d x ) ∝ w ( x ) λ (d x ) a given target probability distribution, where w and λ are a positive weight function and a proposal probability distribution on ( X , X ) , respectively, such that the normalizing constant λ ( w ) =
(cid:82) w ( x ) λ (d x ) (this will be our generic notation for Lebesgue integrals) of π is ﬁnite.
The SNIS estimator is given by
[Equation]
where X 1: M = ( X 1 , . .
, X M ) are independent draws from λ , and can be used to approximate π ( f )
= (cid:82) f ( x ) π (d x ) for any test function f such that π ( | f | ) < ∞ .
The estimator (1) can be calculated without knowledge of the normalizing constant λ ( w ) , which is intractable in general.
The SNIS estimator is known to be biased; provided that λ ( w 2 ) < ∞ , the bias and mean-squared error (MSE) of the SNIS estimator (1) over bounded test functions f satisfying (cid:107) f (cid:107) ∞ ≤ 1 are given respectively (see [1, Theorem 2.1]) by
[Equation]
where κ [ π , λ ] = λ ( w 2 ) /λ 2 ( w ) .
Although IS is primarily intended to approximate integrals in the form π ( f ) , it can also be used to generate unweighted samples being approximately distributed according to π .
In this paper, we consider iterated sampling importance resampling (i-SIR), proposed in [46]; see [4, 27, 26, 5].
The i-SIR can be seen as an iterative application of the sampling importance resampling (SISR) algorithm proposed by [40]; the k -th iteration is deﬁned as follows.
Given a state
Y k ∈ X , (i) set X 1 k +
=
Y k and draw X 2: N k +1 independently from the proposal distribution λ ; (ii) compute, for i ∈ { 1, . . .
, N } , the normalized importance weights {+ω iN , k +
+}
= w ( X ik +1 ) / (cid:80) N(cid:96)
=1 w ( X (cid:96)k +1 ) ; (iii) select Y k +1 from the set X 1: N k +1 by choosing X ik +1 with probability ω iN , k +1 .
In the following,
Y k +1 and X 1: N k +1 will be referred to as the state and the candidate pool , respectively.
Following [46] (see Section 2.1), i-SIR may be viewed (up to an irrelevant permutation of the samples) as a two-stage
Gibbs sampler targeting an extended probability distribution ϕ N on an enlarged state space including the state as well as the candidate pool.
As this extended distribution allows π as a marginal with respect to the state, one can expect the marginal distribution of the generated states ( Y k ) k ∈ N , forming themselves a Markov chain, to approach the target π of interest as k tends to inﬁnity.
This paper:
In i-SIR, the only function of the candidate pool is to guide the states selected at stage (iii) towards the target.
Thus, since all rejected candidates are discarded, the approach results generally in a large waste of computational work.
Thus, in the present paper we propose to recycle all the generated samples by incorporating all the proposed candidates X 1: N k into the estimator rather than only the selected candidate Y k .
We proceed in three steps.
First, we show that under the stationary distribution ϕ N of the process ( Y k , X 1: N k ) k ∈ N generated by i-SIR, the expectation of Π N f
( X 1: N k ) (given by (1)) equals π ( f ) for every valid test function f
(see Theorem 2).
Second, we establish that since i-SIR is nothing but a systematic-scan Gibbs sampler, the two processes ( X 1: N k ) k ∈ N and ( Y k ) k ∈ N are interleaving (see Theorem 5); thus, if ( Y k ) k ∈ N is uniformly geometrically ergodic, so is ( X 1: N k )
k ∈ N with the same mixing rate κ N .
Third, as the main result of the present paper, we establish a novel O ( κ kN /N )
bound on the bias of the estimator Π N f
( X 1: N k ) (see Theorem 3), where the exponentially diminishing factor κ kN indicates a drastic bias reduction vis-à-vis the standard IS estimator (1) based on i.i.d.
samples.
As a consequence, approximating π ( f ) by the average of (Π N f
( X 1: N k )) [-k ` = k 
-]k(cid:96)
= k 0 +1 , where the “burn-in” period k 0 should be chosen proportionally to the mixing time of the process, yields an estimator whose bias can be furnished with a bound which is, roughly, proportional to κ k 0 N and inversely proportional to the total number M = kN of samples generated in the algorithm (see Theorem 4).
To complete the theoretical analysis of these estimators, we also equip the same with variance bounds.
The procedure of recycling, as described above, all the samples generated in the i-SIR and to incorporate, at negligible computational cost, the same into the ﬁnal estimator, will from now on be referred as BR-SNIS.
Finally, we test numerically the proposed estimators and illustrate how a signiﬁcant bias reduction relatively to the standard i-SIR can be obtained at basically no cost.
To sum up, our contribution is twofold, since we
– propose a new algorithm, BR-SNIS, which makes better use of the available computational resources by recycling the candidate pool generated at each iteration of i-SIR.
– furnish the proposed algorithm with rigorous theoretical results, including novel bias, variance, and high-probability bounds which support our claim that sample recycling may lead to drastic bias reduction without impairing the variance.
Main results 2.1 Statements
The i-SIR algorithm can be interpreted as a systematic-scan two-stage Gibbs sampler, alternately sampling from the full conditions of an extended target ϕ N on the product space of states and candidate pools.
Once the extended target ϕ N is properly deﬁned, these full conditionals can be retrieved from a dual representation of ϕ N presented in Theorem 1.
In order to deﬁne ϕ N , we introduce the Markov kernel (see Appendix A.1 for comments)
[Equation]
on X × X (cid:15) N , which describes probabilistically the sampling operation (i) in i-SIR.
Using the kernel Λ N we may now deﬁne properly the extended target ϕ N as the probability law
[Equation]
on ( X N +1 , X (cid:15) ( N +1) ) .
Note that since for every A ∈ X , ϕ N (1 A × X N ) =
π ( A ) , the target π coincides with the marginal of ϕ N with respect to the state.
Moreover, it is easily seen that Λ N provides the conditional distribution, under ϕ N , of the candidate pool given the state.
Deﬁning the kernels
[Equation]
d y ) = Γ [Equation] x
[Equation]
[Equation]
[Equation]
Γ N
[Equation] X [Equation] x 1: N ) (3) on X × X (cid:15) N , the marginal distribution π N of ϕ N with respect to x 1: N is given by
[Equation]
It is interesting to note that the marginal π N has a probability density function, proportional to Γ N 1 X ( x 1: N )
=
(cid:80) Ni =1 w ( x i ) /N , with respect to the product measure λ (cid:15) N .
Using (4), we immediately obtain the following result.
Theorem 1 (duality of extended target) .
For every N ∈ N ∗ ,
[Equation]
Note that the second identity of the dual representation (5) provides also the conditional distribution, under ϕ N , of the state given the candidates.
Consequently, i-SIR is a systematic scan two-stage
Gibbs sampler which generates a Markov chain ( X k , Y k )
k ∈ N with time-homogeneous Markov kernel
P N (( y k , x 1: N k ), d( y k
+1 , x 1: N k +1 )) = Λ N ( y k , d x 1: N k +1 ) Π N ( x 1: N k +1 , d y k +1 ) on X N +1 × X (cid:15) ( N +1) .
Note that the law P N ( y k , x 1: N k , · ) does not depend on x 1: N k , which means that only the state Y k needs to be stored from one iteration to the other. Thus, ( Y k ) k ∈ N is a Markov chain with Markov transition kernel [Equation]
[Equation]
y
[Equation]
+
[Equation]
(where integration is w.r.t.
x 1: N k +1 ) on X × X .
The kernel (6) was analyzed in [5].
Given some probability distribution ξ on ( X N +1 , X (cid:15) ( N +1) ) , we denote by P ξ the law of the canonical Markov chain ( X k , Y k ) k ∈ N with kernel P N and initial distribution ξ .
Our ﬁrst results establishes the unbiasedness of the estimator Π N f
( X 1: N ) under ϕ N .
Theorem 2.
For every N ∈ N ∗ and π -integrable function f ,
[Equation]
The proof of Theorem 2 is postponed to Appendix A.3.
Next, we present theoretical bounds on the discrepancy, in terms of bias, MSE and covariance, between Π N f ( X 1: N k ) and π ( f ) , for bounded target functions f , when the i-SIR chain is initialized according to an arbitrary distribution ξ .
We will work under the following assumption.
A1.
It holds that ω = (cid:107) w (cid:107) ∞ /λ ( w ) < ∞ .
Under A 1, the state and candidate-pool Markov chains ( Y k ) k ∈ N and ( X 1: N ) k ∈ N can be shown to be uniformly geometrically ergodic with mixing rate and mixing-time upper bound
[Equation]
respectively; see Theorem 6 below for details.
Here the mixing time τ mix , N grows logarithmically with the sample size N .
The exact value of τ mix , N is likely to be grossly pessimistic, but we conjecture that the logarithmic dependence in the minibatch size holds true.
In addition, under A 1 we deﬁne the constants
[Equation]
With these deﬁnitions, the following holds true.
Theorem 3.
Assume A 1.
Then for every initial distribution ξ on ( X N +1 , X (cid:15) ( N +1) ) , bounded measurable function f on ( X , X ) such that (cid:107) f (cid:107) ∞ ≤ 1 , N ≥ 2 , and ( k , (cid:96) ) ∈ ( N ∗ ) 2 ,
[Equation]
ii
[Equation]
where constants are given in (7) and (8) .
It is worth noting that the bias decreases inversely with the number of candidates and exponentially with the number of iterations (the mixing time of the chain also depends on N ).
The MSE is also inversely proportional to the number of candidates N .
In the light of the previous results, it is natural to consider an estimator formed by an average across the IS estimators (Π N f
( X 1: N k ))
k ∈ N associated with the candidate pools generated at the different i-SIR iterations.
To mitigate the bias, we remove a “burn-in” period whose length k 0 should be chosen proportional to the mixing time τ mix , N of the Markov chain ( Y k ) k ∈ N (which turns out to coincide with that or the chain ( X 1: N k ) k ∈ N ; see Section 2.2).
This yields the estimator
[Equation]
of π ( f ) .
The total number of samples (generated by the proposal λ ) underlying this estimator is
M = ( N − 1) k .
Importantly, all the importance weights included in the estimators are obtained as a by-product of the i-SIR schedule; thus, it is, for a given budget of simulations ( i.e. , under the constraint that ( k − k 0 ) N is constant), possible to compute Π ( k 0 , k ), N ( f ) for different values of k 0 , k and N with a negligible computational cost.
We denote by υ =
( k − k 0 ) /k
the ratio of the number of candidate pools used in the estimator to the total number of sampled such pools.
Note that this type of estimator was already suggested by [47] and also appears in [42].
Our ﬁnal main result provides bounds on the bias and the MSE of the estimator (9) as well as a high-probability bound for the same.
Deﬁne ζ bias = 4 τ mix , N ς bias / 3 , ζ mse i =
ς mse ( i +1) ∧ 2 1 { 0,2 } ( i ) + (8 / 3) τ mix , N ς cov i , i ∈ { 0, 1, 2 } , ζ mse = ζ mse+ ζ mse 1 ( N − 1) − 1 / 4 + ζ mse 2 ( N − 1) − 1 , and MSE is M = (4 /M ) κ
[ π , λ ] , see (2).
Theorem 4.
Assume A 1.
Then the following holds true for every initial distribution ξ on ( X N +1 , X (cid:15) ( N +1) ) , bounded measurable function f on ( X , X ) such that (cid:107) f (cid:107) ∞ ≤ 1 , and N ≥ 2 .
(i) (cid:12)(cid:12)
E ξ [Π ( k 0 , k ), N ( f )] − π ( f ) (cid:12)(cid:12) ≤ ζ bias ( υM ) − 1 4 − k 0 /τ mix , N (ii) E ξ [ { Π ( k 0 , k ), N ( f ) − π ( f ) } 2 ] ≤ MSE is υM + ζ mse ( υM ) − 1 ( N − 1) − 1 /(iii)
For every δ ∈ (0, 1) , | Π ( k 0 , k ), N ( f ) − π ( f ) | ≤ ς hpd ( υM ) − 1 / 2 (log(4 /δ )) 1 / 2 with probability at least 1 − δ , where ς hpd = 664 ω .
Bootstrap: As established in Theorem 4, the bias of the BR-SNIS estimator decreases exponentially with the burn-in period k 0 , leading to potentially signiﬁcant bias reduction with respect to SNIS.
Still, using a large k 0 is done at a price of increased overall MSE (mainly through the term MSE is υM in
Theorem 4(ii), which is directly related to k 0 via υ ).
A natural way to reduce the variance is to use bootstrap.
More precisely, we ﬁrst apply a random permutation to the samples and re-compute BRSNIS on the basis of the bootstrapped samples.
After this, we produce a ﬁnal estimator by averaging over the bootstrapped BR-SNIS replicates.
In most applications, the major computational bottleneck consists of sampling from λ and evaluating w and f at the samples; thus, the additional operations that this bootstrap approach entails are computationally cheap.
Therefore, in our experiments, we use bootstrap in combination with the choice k 0 =
k − 1 (in order to minimize the bound in Theorem 4(i)).
2.2 Elements of proofs Ergodic properties of i-SIR:
The systematic scan two-stage Gibbs sampler is a well-studied
MCMC algorithmic structure, and we summarize its most important properties in Theorem 5 below; see [30, 3] and [39, Chapter 9] as well as the references therein.
In particular, as shown in [30], the state and candidate-pool Markov chains ( Y k ) k ∈ N and ( X 1: N k ) k ∈ N satisfy a duality property referred to as interleaving (Theorem 5(iii)).
Theorem 5.
Assume that for every x ∈ X , w ( x ) > 0 , λ ( w ) < ∞ and that there exists a set C ∈ X such that λ ( C ) > 0 and sup x ∈ C w ( x ) /λ ( w ) < ∞ .
Then,
(i) the Markov kernel P N is Harris recurrent and ergodic with unique invariant distribution ϕ N .
(ii) the Markov kernel P N is π -reversible, Harris recurrent and ergodic.
(iii) the two Markov chains ( Y k ) k ∈ N and ( X 1: N k ) k ∈ N are conjugate of each other with the interleaving property,
i
e., for every initial distribution ξ and k ∈ N , under P ξ , (a) X 1: N k and X 1: N k +1 are conditionally independent given Y k , (b) Y k and Y k +1 are conditionally independent given X 1: N k +1 ; (c) moreover, under P ϕ N , ( Y k , X 1: N k − 1 ) and ( Y k , X 1: N k ) are identically distributed.
The ergodic behavior of the i-SIR algorithm has been studied in many works; see [26, 28, 5] in particular.
The analysis is particularly simple under the assumption that the importance weight function w is bounded, as imposed by A 1.
Recall that the total variation-distance between two probability measures ξ and ξ (cid:48) on ( X , X ) is given by d TV ( ξ , ξ (cid:48) )
=
sup g :osc( g ) ≤ 1 { ξ ( g ) − ξ (cid:48) ( g ) } , where osc( g ) = sup
( x , x (cid:48) ) ∈ X 2 | g ( x ) − g ( x (cid:48) ) | denotes the oscillator norm of a measurable function g .
The following result establishes the uniform geometric ergodicity of the state chain ( Y k ) k ∈ N .
Theorem 6.
Assume A 1.
Then for every N ≥ 2 , y ∈ X and k ∈ N , d TV ( P kN ( y , · ), π ) ≤ κ kN , where κ N is given in (7) .
The proof is given in [28, 5], but we provide it in Appendix A.5 for completeness.
For uniformly ergodic Markov chains, it is often more appropriate to work with the mixing time
[Equation]
(where τ mix , N is given in (7)), i.e. , the number of time steps required for the distribution of the chain to be within a certain total variation distance from its stationary distribution [2, 15].
An interesting consequence of the interleaving property is that if the Markov chain ( Y k ) k ∈ N is (geometrically) ergodic, then the Markov chain ( X 1: N k ) k ∈ N is (geometrically) ergodic as well with the same mixing time; see [39, Corollary 9.14]).
Bias of the BR-SNIS estimator:
As the BR-SNIS estimator Π N f ( X 1: N k ) (where Π N is deﬁned in (3)) is made up by a ratio of the two unnormalized estimators Γ N f ( X 1: N k ) and Γ N 1 X ( X 1: N k ) , a key ingredient in the proof of Theorem 3 is to bound the bias and the p th order moments of statistics deﬁned as ratios of sums of random variables that are not necessarily independent.
The basic idea is to reduce the study of these relations to the analysis of the moments of the numerator and the denominator of these statistics and to exploit their concentration around the respective (conditional and unconditional) means.
The main results that we will use in the rest of the paper are summarized in Appendix B.
Lemma 7.
For every initial distribution ξ on ( X N +1 , X (cid:15) ( N +1) ) , k ∈ N ∗ , and bounded measurable function f : X → R , it holds that
(i) for every y ∈ X , Λ N Γ N f ( y ) =
(1 − 1 /N ) λ ( wf ) + (1 /N ) w ( y ) f
( y ) , (ii) E ξ (cid:2) Γ N f ( X 1: N k ) (cid:12)(cid:12) Y k − 1 (cid:3) = Λ N Γ N f ( Y k − 1 ) , P ξ -a.s.
, (iii) E ξ (cid:2) { Γ N f ( X 1: N k ) − Λ N Γ N f ( Y k − 1 ) } 2 (cid:12)(cid:12) Y k − 1 (cid:3) = ( N − 1) /N 2 λ ( { wf − λ ( wf ) } 2 ), P ξ -a.s.
We now have all the elements that allow us to determine the ﬁrst important result of this work, namely the bias and the MSE of the estimator Π
N f ( X 1: N k ) of π ( f ) .
Proof of Theorem 3.
We establish the bias bound in (i) and postpone the proof of the bounds on the
MSE and the covariance in (ii) and (iii) to the supplement.
Deﬁne the measure ξ ( A ) =
ξ ( A × X ) , A ∈
X , and the kernel P N = Λ N Π N on X × X .
Consequently, P N f ( Y k − 1 ) = E ξ [Π N f
( X 1: N k ) | Y k − 1 ] and Λ N Γ N f ( Y k − 1 ) =
E ξ [Γ N f ( X 1: N k ) | Y k − 1 ] , P ξ -a.s.
Since ( Y k ) k ∈ N is, under P ξ , a Markov chain with initial distribution ξ and Markov kernel P N (see (6)), it holds that
[Equation]
Consequently, the proof is concluded by establishing that for every k ∈ N ∗ ,
[Equation]
On the other hand, since by Theorem 2, π ( P N f ) =
π ( f ) , we may use Theorem 6 to obtain the bound | ξ P k − 1 N P N f − π ( f ) | =
| ξ P k − 1 N P N f − π ( P N f ) | ≤ κ k − 1 N osc( P N f ).
Finally, we establish (10) by bounding osc( P N f ) .
Note that
[Equation]
N
[Equation]
f )
[Equation]
∞ , where, for every y ∈ X , using Theorem 11,
[Equation]
Now, since Λ N Γ N 1 X ( y ) ≥ (1 − 1 /N ) λ ( w ) , we get, using Lemma 7,
[Equation]
On the other hand, using the elementary inequality a/b − c/d = a ( d − b ) /bd
+ ( a − c ) /d , we get, as π ( f ) =
λ ( wf ) /λ ( w ) ,
[Equation]
Finally, the bound (10) is established by noting that
[Equation]
2.3 Related works
The ﬁrst use of the IS method, then as a variance reduction technique, dates back to the ’50s; see [13, 22] and the references therein.
Today, the renewed interest in IS parallels the ﬂurry of activity in the probabilistic ML community and its ever-increasing computational demands; thus, it is impossible to fully present the literature.
We therefore limit ourselves to describing results that have inspired our work, and refer the readers to the recent reviews [1, 11] for additional references.
There is clearly a plethora of modern ML applications where the standard SNIS estimator may be substantially improved using the BR-SNIS method.
To mention just a selection of examples, SNIS plays a key role for a robust off-policy selection strategy BY [23] (extending [43, 32]), Bayesian problems (see, e.g. , [1, Section 3]), Bayesian transfer learning [18, 31], variational autoencoders [9], inference of energy-based models [25], patch-based image restoration [37] and many more.
In stochastic-approximation procedures, where a statistical estimator or algorithm is employed repeatedly to produce mean-ﬁeld estimates, controlling its bias becomes critical [44, 19].
Thus, it is natural to aim at minimizing the bias for a given computational budget, provided that the variance does not explode.
For this reason, bias reduction (or unbiasedness) in stochastic simulation has been the subject of extensive research during the last decades; see [12, 16].
The present paper contributes to this line of research.
Despite long-standing interest in SNIS, there are only few theoretical results.
For example, [1,
Theorem 2.1] provides bounds on the bias and variance of SNIS, results that we extend to BR-SNIS in Theorem 3.
Moreover, [32, Proposition D.3] provides a suboptimal variance bound based on a bound for the second-order moment.
This result can be compared to the sophisticated sub-Gaussian concentration bound for BR-SNIS obtained in Theorem 4 (a result that can be obtained for SNIS using the same proof mechanism; see Appendix A.8).
Finally, [23] obtains a semi-empirical sub-Gaussian concentration inequality using the Efron-Stein estimate of variance and the Harris inequality.
As an MCMC sampling method, the i-SIR algorithm that has been applied successfully in many situations.
It was recently used—under the alternative name conditional importance sampling —in [34] for Markovian score climbing .
In the same work, it is mentioned that it is possible to “RaoBlackwellize” the gradient of the score using the proposed candidates, which is in line with the recycling argument underpinning the estimator suggested by us, but without theoretical justiﬁcations.
In its most basic form, the i-SIR algorithm appeared in the pioneering work of [46].
The same idea played a key role in the development of the particle Gibbs sampler [4, 5, 35], which extends i-SIR principles to sequential Monte Carlo methods .
An approach very similar to BR-SNIS can be taken also in this context; however, casting BR-SNIS into the framework of particle Gibbs methods is a non-trivial problem which is the subject of ongoing work.
Experimental results
In this section we compare numerically the performances of BR-SNIS and SNIS in three different settings: mixture of Gaussians, Bayesian logistic regression and variational autoencoders (VAE).
We leave to the supplementary material (Appendix C.1)
the detailed numerical veriﬁcation of the bounds established in Section 2.
Mixture of Gaussian distributions: We start with an example where the target distribution π is a mixture of two Gaussian distributions of dimension d = 7 , as shown in Figure 2a.
The proposal distribution is a Student distribution with ν = 3 degrees of freedom.
The test function is f = 1 A − 1 B , where A and B are a d -dimensional rectangle intersecting each of the modes of π (see Appendix C. for precise deﬁnitions).
We verify the positive effect of bootstrap in Figures 1a and 1b by computing the bias and the MSE over 1000 chains for N = 129 for several k .
The purple, green, and red curves correspond to a number of bootstrap rounds of 1, 21 , and 201 , respectively.
We illustrate the decay of the mean Sliced Wasserstein distance (according to [7]) with k for different values of N ( N =purple, N = 32 green, N = 64 orange, and N = 128 red) in Figure 1c.
The decay of the Wassertein distance is directly linked to the mixing time of the i-SIR kernel (see (7)), and hence allows us to represent the effective mixing time of the chain.
Moreover, we represent the theoretical slopes as dashed lines.
This illustrates that the effective value of τ mix , N is smaller than its theoretical bound.
The bias and MSE for SNIS with M = 25600 are shown in black dashed lines.
We compare the bias (Figure 2b) and MSE (Figure 2c) of BR-SNIS and SNIS for a ﬁxed budget with a total number of M = 16384 samples.
We run the experiments 10 6 times; we compute the bias and MSE over batches of 10 4 replications using the true value of π ( f ) computed above (the boxplots in Figure 2 are therefore obtained over 100 replications).
For the algorithm BR-SNIS, we used N ∈ { 129, 513 } , k
=
k max − 1 and k max = M/ ( N − 1) bootstrap rounds.
As can be seen from Figure 2b, BR-SNIS signiﬁcantly reduces bias (by a factor of almost 10) w.r.t.
standard SNIS for both conﬁgurations, while MSE increases only slightly (at around 20% ), as can be seen in Figure 2c.
The code used for this experiment is available at 1 .
We also show in Appendix C.
that k [-
-]0 = {+(cid:98) 0.
+} k max (cid:99) can lead to about 3 times less bias w.r.t.
standard SNIS while only augmenting the MSE of 10% .
We have also compared in BR-SNIS to zero bias estimators based on SNIS such as [33], the results are in shown in Appendix C.1.
Bayesian Logistic regression: We consider posterior inference in a Bayesian logistic regression model.
Let D train =
( x i , y i )
Ti =1 be a dataset, where each x i ∈ R d is a vector of covariates and y i ∈ {− 1, 1 } is a binary response.
Let p ( y i | x i ; θ ) = { 1 + exp
− y i x (cid:62) i θ ) } − 1 be the probability of the i th observation at θ ∈ Θ ⊆ R d and π 0 (d θ ) be a prior distribution for θ .
The Bayesian posterior is
[Figure] [Figure]
given
[Equation]
For numerical illustration, we use the heart failure clinical records ( d = 13 , T = 299 ), breast cancer detection ( d = 30 , T = 569 ), and Covertype ( d = 55 , T = 4 · 10 4 ) datasets from the UCI machine learning repository.
For Covertype, we use Cover type 1 (Spruce/Fir) and Cover type(Lodgepole Pine) classes to deﬁne a binary classiﬁcation problem.
As a prior, we use a Gaussian distribution N(0, τ − 2 I ) with τ 2 = 5 · 10 − 2 .
The importance distribution λ is Gaussian with mean and diagonal covariance learned by variational inference; see Appendix C.2 for details.
The boxplots for bias in Figure 3 were constructed in the same way as those in Figure 2.
We compare two test .
[Table]
[-5.5e0.
 +/- 6.7eSNIS, M =0.0026 +/- 0. 4.3e-5 +/--]functions, f ( θ )
= θ , corresponding to evaluation of the posterior mean, and f ( θ ) =
p ( y | x , θ ) , where ( x , y ) ∈ D test .
This last function allows us to compute a TV distance for the predictive distribution.
Indeed, in a classiﬁcation context, one can compute the TV distance between any two predictive distributions p and ˆ p as
[Equation]
where we compare the predictive distribution p ( y | x
[Table]
D train ) = (cid:82) p ( y | x , θ ) π ( θ )
d θ and ˆ p is the estimation of this quantity, provided in the experiments by SNIS or BR-SNIS.
From Figure 3 we can see that for each dataset we have a constant decrease in bias, while the variance increases only slightly.
We plot the bias in other components of θ and provide further numerical details in Appendix C.2.
(a) Heart     0.
(b) Breast      0.
[Figure]
[Table]
Generative Model:
We now extend our methodology to the more complex deep latent generative models (DLGM).
A DLGM deﬁnes a family of probability densities p θ ( x ) over an observation space x ∈ R P by introducing a latent variable z ∈ R d , deﬁning the joint density function p θ ( x , z ) (with respect to Lebesgue measure) and aiming to ﬁnd a parameter θ maximizing the marginal log-likelihood of the model p θ ( x ) = (cid:82) p θ ( x , z )d z .
Under simple technical assumptions, by Fisher’s identity,
[Equation]
In most cases, the conditional density p θ ( z | x ) =
p θ ( x , z ) /p
θ ( x ) is intractable and can only be sampled.
The variational autoencoder [21] is based on the introduction of an additional parameter φ and a family of variational distributions q φ ( z | x ) .
The joint parameters { θ , φ } are then inferred by maximizing the evidence lower bound (ELBO) deﬁned by
[Equation]
This basic setup has been further developed and improved in many directions.
Here we consider the importance weighted autoencoder (IWAE) [8], which relies on SNIS to design a tighter ELBO on the log-likelihood.
The objective of the IWAE is given by
[Equation]
where w θ , φ , x ( z ) =
p θ ( x , z ) /q φ ( z | x ) denote the importance weights.
However, writing, following [8, Eq.
[Equation]
where ω ( i ) θ , φ , x = w θ , φ , x ( z i ) / (cid:80) Mj
=
1 w θ , φ , x ( z j ) are normalized importance weights, yields an expression of the gradient that corresponds exactly to the biased SNIS approximation of (13).
Thus, the optimization problem will suffer from bias.
We hence propose to use BR-SNIS for learning IWAE.
The proposed algorithm proceeds in two steps, which are repeated during the optimization (details are given in Appendix C.3)
• First, update the parameter φ as in the IWAE algorithm (using the reparameterization trick and following the methodology of [8]) according to φ ( t +1)
=
φ ( t ) − η ∇ φ L M ( θ ( t ) , φ ( t ) ) .
• Second, update the parameter θ by estimating (13) using BR-SNIS for π ( z )
= p θ ( x , z ) , f ( z ) = ∇ θ log p θ ( x , z ) and λ ( z ) =
q φ ( z | x ) .
We refer to this model as BR-IWAE.
As an illustration, we train the model using the binarized MNIST dataset [41], where x ∈ { 0, 1 } 784 are binarized digits images in dimension 784.
For both for the encoder q φ and the decoder p θ , we use a convolutional neural network (more details are given in
Appendix C.3).
For comparison, we estimate the log-likelihood using the VAE, IWAE and BR-IWAE approaches, and the result is reported in Table 2.
All models are run for 100 epochs, using the
Adam optimizer [20] and a learning rate of 10 − 4 .
The complete experimental details are given in
Appendix C.3.
Conclusion
In this paper, we have introduced a novel method, BR-SNIS, which improves over SNIS when it comes to producing close to unbiased estimates of expectations taken w.r.t.
to distributions known only up to a normalizing constant, a ubiquitous problem in machine learning and statistics.
The high performance of BR-SNIS is supported theoretically by non-asymptotic bias, variance and high-probability bounds.
We illustrate our method on various examples, which show the practical advantages of BR-SNIS over SNIS.
Finally, BR-SNIS is naturally adapted to other IS based methods, for example [45], which use a Hamiltonian (gradient-based) transform [36] as part of the IS proposal.
The extension of BR-SNIS to [45] would produce an Hamiltonian based sampler able to recycle all samples, contrarily to other classical Hamiltonian-based methods [36, 14].
BR-SNIS can also be extended to Particle Markov chain Monte Carlo methods such as Particle Gibbs with Ancestor sampling [29].
Acknowledgement
The work of G. Cardoso is supported through the Investment of the Future grant, ANR-10-IAHU04, from the government of France through the Agence National de la Recherche.
The work of S. Samsonov was supported by SCAI Project-ANR-19-CHIA-0002 and within the framework of the HSE University Basic Research Program.
Part of this work has been carried out under the auspice of the Lagrange Center in Mathematics and Computing.
The work of J. Olsson is supported by the Swedish Research Council, Grant 2018-05230.
The work of E. Moulines is supported by
ANR CHIA 002 – SCAI.