Nonlinear MCMC for Bayesian Machine Learning
Abstract
We explore the application of a nonlinear MCMC technique ﬁrst introduced in [1] to problems in Bayesian machine learning.
We provide a convergence guarantee in total variation that uses novel results for long-time convergence and large-particle (“propagation of chaos”) convergence.
We apply this nonlinear MCMC technique to sampling problems including a Bayesian neural network on CIFAR10.
Introduction
Characterizing uncertainty is a fundamental problem in machine learning.
It is often desirable for a machine learning model to provide a prediction and a measure of how “certain” the model is about that prediction.
Having access to a robust measure of uncertainty becomes particularly important in real-world, high risk scenarios such as self-driving cars [2–4], medical diagnosis [5, 6], and classifying harmful text [7].
However, despite the need for uncertainty in machine learning predictions, it is well known that traditional ML training, i.e. based on optimizing an objective function, frequently does not provide robust uncertainty measures [8], yielding overconﬁdent predictions for popular neural networks such as ResNets [9].
An appealing alternative to the traditional optimization paradigm for ML is the
Bayesian probabilistic framework, due to its relatively simple formulation and extensive theoretical grounding; see for example [10].
From the probabilistic perspective of machine learning [10], one combines a prior P ( θ ) over the parameter space θ ∈ Θ and a likelihood of the data given model parameters P ( D| θ ) using Bayes’ rule to obtain a posterior over the parameters P ( θ |D ) ∝ P ( D| θ ) P ( θ ) .
The “traditional” approach in machine learning is to optimize the posterior (or the likelihood) to obtain θ ∗ ∈ arg max P ( θ |D ) and generate predictions via P ( y | x, D ) =
P ( y | x, θ ∗ ) .
However, if we adopt the Bayesian approach, the posterior characterizes the uncertainty about the parameters of the model (i.e. epistemic uncertainty), which can propagate to uncertainty about a prediction by integration : P ( y | x,
[Equation]
This paper studies the problem of how to approximate this integration with samples from P ( θ |D ) .
1.1 Contributions
• Our main contribution is the novel analysis of a modiﬁcation of the general nonlinear Markov Chain Monte Carlo (MCMC) sampling method from [1] to obtain quantitative convergence guarantees in both the number of iterations and the number of samples.
• We apply the general results from above to determine the convergence of two speciﬁc nonlinear MCMC samplers.
• In experiments, we compare these nonlinear MCMC samplers to their linear counterparts, and ﬁnd that nonlinear MCMC provides additional ﬂexibility in designing sampling algorithms with as good, or better, performance than the linear variety.
1.2 Background
Bayesian ML & MCMC.
In Bayesian machine learning, the “computationally difﬁcult” step is integration since the integral ´ P ( y | x, θ ) P (d θ |D ) is not analytically solvable except for rare cases.
In practice, one typically uses a Monte Carlo approximation such as ˆ
[Equation]
P [Equation] y | x, θ i ) , where θ i iid ∼ P ( θ |D ) where the expected error in this approximation is well known to converge to zero like O (1 / √ N ) by the Central Limit Theorem (CLT).
There are various approaches to sampling from P ( θ |D ) , but
Markov chain Monte Carlo (MCMC) is perhaps the most widely used.
The basic idea of MCMC is to use a Markov transition kernel T with stationary measure P ( θ |D ) to simulate a Markov chain θ n +1 ∼ T ( θ n , • ) that converges rapidly to P ( θ |D ) .
In this case, we can estimate ˆ
[Equation]
where n sim is some large number of simulation steps and { θ in } ∞ n =0 are independent Markov chains governed by T .
The basic problem is then to design an efﬁcient transition kernel T .
There is a vast body of literature studying various choices of T ; some well known choices are the Metropolis-Hastings algorithm [11, 12], the Gibbs sampler [13], the Langevin algorithm [14–17], Metropolis-Adjusted Langevin [18, 19], and Hamiltonian Monte Carlo [20–24].
However, there are various challenges in Bayesian ML that make applying these samplers difﬁcult in practice.
One challenge is that the posterior P ( θ |D ) can be highly multimodal [25, 26], which makes it difﬁcult to ensure that the Markov chain θ n explores all modes of the target distribution.
One can combat this issue by employing auxiliary samplers that explore a more “tractable” variation of P ( θ |D ) [27, 28].
Other methods that empirically improve posterior sampling quality include tempering [29–32], RMSProp-style preconditioning [33], or adaptive MCMC algorithms [34, 35] such as the popular No U-Turn Sampler [36].
Nonlinear MCMC.
Another class of powerful MCMC algorithms, which is less-studied in the context of Bayesian ML, arises from allowing the transition kernel T to depend on the distribution of the Markov chain as in θ n +1 ∼ T Distribution( θ n ) ( θ n , • ) .
This approach gives rise to so-called nonlinear MCMC since { θ n } is no longer a true Markov chain.
Nonlinear Markov theory is a rich area of research [37–42] and has strong connections to nonlinear ﬁltering problems [43, 44], sequential Monte Carlo [45, 46], and nonlinear Feynman-Kac models [47].
One can replace Distribution( θ n ) , which is often intractable, with an empirical estimate Distribution( θ n ) ≈ 1 N (cid:80) Ni =1 δ θ in to obtain interacting particle MCMC (iMCMC, or iPMCMC) methods; see for example [48, 49, 1, 50, 51].
Our view is that nonlinear MCMC offers some appealing features that traditional linear MCMC lack.
One such feature is the ability to leverage global information about the state space Θ contained in Distribution( θ in ) to improve exploration of Θ — a central issue in Bayesian ML.
Another feature is the increased ﬂexibility of nonlinear MCMC algorithms, which can be leveraged to correct biases that are introduced by other design decisions in MCMC for Bayesian ML such as tempering.
These features will be explored empirically in Section 4.
However, the theoretical analysis of nonlinear Markov MCMC presents an added difﬁculty in that the particles { θ 1 n , . .
, θ Nn } of an interacting particle system are now statistically dependent .
This means that, in addition to studying the large-time behaviour which is classical in MCMC [52, 53], one must study the large-particle behaviour separately to obtain Monte Carlo estimates since the CLT does not apply.
One such large-particle behaviour is the propagation of chaos [41], which studies the tendency of interacting particles to become independent as the number of particles, N , increases; see [41].
We will need both of these elements — long-time and propagation of chaos behaviour — to properly characterize the convergence of nonlinear MCMC.
Other Sampling Methods.
Finally, let us mention that MCMC is certainly not the only way to obtain Monte Carlo sample estimates in Bayesian ML; some popular examples include MC dropout [8], black-box variational inference [54], and normalizing ﬂows [55, 56].
1.3 Common Notation
Let P ( R d ) be the set of probability measures on the meaurable space ( R d , B ( R d )) .
For µ ∈ P ( R d ) and f ∈ B b ( R d ) :
= { f : R d → R | f is bounded } , we will denote µ ( f ) :
=
´ f d µ .
If K :
R d × B ( R d ) → [0 , 1] is a Markov kernel 2 then we will denote Kf ( x ) :
= ´ f ( y ) K
( x, d y ) and µK (d y ) :
=
´ µ (d x ) K ( x, d y ) .
Finally, for y := { y 1 , . .
, y N } ⊂ R d , we will denote the empirical measure of y as m ( y ) :
= 1 N (cid:80) Ni =
δ y i ∈ P ( R d ) .
Nonlinear MCMC
In this section, we outline the family of MCMC algorithms that will be studied in rest of the work.
We will use general notation for simplicity but this difference is merely cosmetic; the “target distribution” π in this section corresponds directly to P ( θ |D ) from the previous section.
2.1 Nonlinear Jump Interaction Markov Kernels
To specify a MCMC algorithm, we must specify the Markov transition kernel.
The family of nonlinear
Markov kernels that we will be studying was introduced in [1] and is a mixture of a linear kernel, denoted K , and a nonlinear jump-interaction kernel indexed by a probability measure η , denoted J η , to obtain
[Equation]
where ε ∈ ]0 , 1[ is the mixture hyperparameter.
The Markov kernel K η will be the main object of interest throughout this paper.
We will give speciﬁc examples of J η in Section 2.2, which were also introduced in [1].
Desipite buidling on the constructions of [1], this work proceeds in some different substantially directions; see Appendix A.1 for more details.hi
Mean Field System.
Now we show how the kernel K η can be used to construct a Markov chain.
Following [1], we use an auxiliary Markov chain { Y n } with transition kernel Q on the same state space as K η (i.e. R d ) to obtain the nonlinear Markov chain { ( Y n , X n ) } ∞ n =0 deﬁned by
[Equation]
where µ 0 , η 0 ∈ P ( R d ) are the initial distributions and ∼ denotes “sample from”.
One should interpret this as a sequence of steps where ﬁrst we sample the auxiliary state Y n +1 from Q , then we obtain the distribution of Y n +1 denoted η n +1 , and we use this distribution to index the primary kernel K η n +1 and obtain a sample X n +1 .
We sample X n +1 with probability (1 − ε ) from the linear kernel K , and with probability ε it will “jump” according to J η n +1 ( X n , • ) .
Because the Markov dynamics depend on Distribution( Y n ) , we call this a “mean ﬁeld system”.
Interacting Particle System.
One issue with the mean ﬁeld system (2) is the fact that computing
Distribution( Y n +1 ) is generally impossible except in special cases.
Hence, to get a viable simulation algorithm, we must approximate Distribution( Y n ) , and we do this by replacing Distribution( Y n ) with its empirical measure estimated from a set of N particles Y n := { Y 1 n , . . .
, Y Nn } as follows:
[Equation]
2.2 Application to MCMC
Now we detail how to apply K η and the Markov chains (2) and (3) to MCMC.
In particular, we
As is usually the case in probabilistic inference problems, we will assume that the target distribution π is known only up to a normalizing constant and that it has a density, also denoted π .
We also make the simplifying assumption that Q has an invariant measure η (cid:63) (also with density denoted η (cid:63) )
i.e. η (cid:63) Q = η (cid:63) .
This is not burdensome; in practice we can, and will, obtain Q from a linear MCMC algorithm for some choice of η (cid:63) .
In fact, being able to choose η (cid:63) is a powerful design parameter of our methods as we will see in Section 4.
We will also assume that the linear kernel K is π -invariant, i.e. πK = π .
To see how we can ensure that π is K η -invariant, consider the fact that we will design Q s.t. η n :=
Distribution( Y n ) converges to η (cid:63) .
This means we will eventually be sampling from the kernel
K η (cid:63) and we already have π -invariance of K .
Therefore, if we arrange for J η (cid:63) to be π -invariant, π will be invariant for K η (cid:63) since
[Equation]
Intuitively, if the auxiliary chain converges to a steady state and the jumps in that steady state preserve π (and K preserves π ), then so will K η (cid:63) .
Now the remaining task is to design nonlinear interaction kernels J η that will yield good performance.
We detail two choices below.
Boltzmann-Gibbs Interaction.
The ﬁrst choice of J η we will investigate, from [1], relies on the
Boltzmann-Gibbs transformation [47], which we now explain.
Let G : R d → ]0 , ∞ [ be a potential function; then the Boltzmann-Gibbs (BG) transformation is a nonlinear mapping Ψ G : P ( R d ) →
P ( R d ) deﬁned by
[Equation]
x ) µ ( G ) µ (d x ) or equivalently
[Equation]
for any f ∈ B b ( R d ) and whenever µ ( G ) (cid:54) = 0 .
This transformation has many interesting properties and been extensively studied in [47] and related works.
To use the BG transformation in MCMC, we will assume that the densities π and η (cid:63) are positive 3 and make the choice that G ( x ) will be the function
[Equation]
With this choice, we get an interaction kernel J BGη ( x, d y ) :
= Ψ G ( η )(d y ) .
We can easily see that η [Equation] G
[Equation]
= [Equation] and Ψ
[Equation]
i.e. Ψ G is the multiplicative “change of measure” from η (cid:63) to π .
Hence the ﬁrst nonlinear Markov kernel we will investigate is
[Equation]
From the remarks above, is clear that π is K [Equation] invariant.
Accept-Reject Interaction.
The second choice of jump interaction we will study, also introduced in [1], is a type of accept-reject interaction related to the Metropolis-Hastings algorithm.
For the same choice of potential function G in (4), we can deﬁne the acceptance ratio
[Equation]
[-1 ∧ G ( y ) G ( x ) = 
 ∧ π ( y ) η (cid:63) ( x ) η (cid:63)-] ( y ) π ( x ) and the quantity A η ( x ) :
= ˆ α ( x, y ) η (d y ) for η ∈ P ( R d ) .
Hence we can deﬁne the accept-reject interaction kernel as[-
-]
[Equation]
We can interpret this jump interaction as: starting in state x , we jump to a new state distributed according to η (d y ) with probability α ( x, y )
(i.e. accept the proposed jump) and remain the in current state with probability 1 − A η ( x ) (i.e. reject the proposed jump).
This is a form of “adaptive
Metropolis-Hastings” in which the proposal distribution evolves over time as the distribution of the auxiliary Markov chain.
Hence we obtain the accept-reject nonlinear jump interaction kernel
[Equation]
We note that π is also J ARη (cid:63) -invariant; see Proposition 3 in Appendix G for a simple calculation.
Given f ∈ B b ( R d ) , we can also write this as J ARη f ( x ) = ´
[ f ( y ) − f ( x )]
α ( x, y ) η
(d y )
+ f ( x )
Simulation.
Let us note brieﬂy that using both K BGη and K ARη in (3) produce interacting particle systems that can, and will, be simulated.
The simulation is relatively straightforward, see Appendix A for pseudocode implementing the nonlinear MCMC algorithms we have now constructed.
Convergence Analysis
We will now study whether the nonlinear MCMC algorithms based on K η from Section 2 — i.e., the interacting particle system (3) with the restrictions on K, Q, J η from Section 2.2 — will actually converge to the target distribution π .
In other words, we would like to estimate (cid:107) µ Nn − π (cid:107) for some suitable notion of distance on P ( R d ) , where µ Nn :=
Distribution( X 1 n ) is the distribution of a single particle (it doesn’t matter which particle as the X in are exchangeable ).
The nonlinear nature of K η makes this analysis more difﬁcult than of a linear MCMC method.
We break the problem into two parts: one studying the convergence of the mean-ﬁeld system (2) as the number of steps n → ∞ , and one studying the convergence of the interacting particle system (3) to the mean ﬁeld system as the number of particles N → ∞ .
This will allow us to apply the triangle inequality as follows:
[Equation]
where µ n :=
Distribution( X n ) is the distribution of the mean-ﬁeld system.
Crucially, our analysis of the large-particle limit is uniform in the number of steps n , which will allow us to establish bounds above that hold as n → ∞ .
The actual result is contained in Theorem 1.
While our analysis does not rely on heavy mathematical machinery, to state the full set of conditions and results for long-time and large-particle convergence — each of which is a substantial result in its own right — would occupy too much space in the main text.
Instead, we will state the main result in Theorem 1, which is essentially a corollary of the long-time and large-particle analyses
Theorems 2 and 3 in Appendices E and F respectively, and below we will sketch the general arguments used in those appendices.
The proofs of the main results are in Appendix F.2.
Note that our analysis, and the results we obtain, are novel and not found in [1]; see Appendix D for an elaboration.
The following result is stated in terms of the total variation metric, deﬁned here for µ, ν ∈ P ( R d ) as (cid:107) µ − ν (cid:107) tv :=
sup (cid:107) f (cid:107) ∞ ≤ 1 | µ ( f ) − ν ( f ) | where (cid:107) • (cid:107) ∞ is the sup-norm on B b ( R d ) .
Theorem 1.
[Convergence of Nonlinear MCMC] Under suitable conditions on K η and Q , there exist ﬁxed constants C 1 , C 2 , C 3 > 0 , a function R : [0 , ∞ [ → [1 , ∞ [ , and ρ > 0 s.t.
[Equation]
Let us make a couple of remarks: • This result shows that, to control the approximation error (cid:107) µ Nn − π (cid:107) tv , it does not necessarily sufﬁce to run the MCMC algorithm for a large number of steps n , since if n → ∞ but N < ∞ then our bound on (cid:107) µ Nn − π (cid:107) (cid:54)→ 0 .
However, this approximation cannot lead to arbitrarily bad results: Theorem 1 provides a quantitative upper bound on how much the MCMC algorithm can be biased.
This behaviour is supported empricially; in Figure 5 of Appendix C.1.4 we provide a clear illustration of how changing N signiﬁcantly affects the bias of our nonlinear MCMC methods while having no effect on the bias of linear MCMC, as expected.
• This result uses total variation, which is a strong metric that represents a worst-case over all bounded functions f (up to rescaling by (cid:107) f (cid:107) ∞ ).
It is entirely possible that, for many choices of practical f , the approximation will be better as we will see empirically.
• The constant ρ is, roughly speaking, the maximum of the rate of convergence for Q and that for K .
Hence if K, Q are chosen to be efﬁcient samplers with fast convergence, this will result in ρ (cid:28) 1 and hence µ Nn will also converge quickly.
• In our speciﬁc samplers K BGη and K AR , we will see (in Appendix G that R is a monotonically increasing function that is lower-bounded by 1 .
Hence as N → ∞ , 1 /N R (1 /N )
→as expected.
A corollary of Theorem 1 is that that we regain a Monte Carlo estimate for the interacting particle system.
This result is essentially due to [41] Theorem 2.2.
Corollary 1.
[Adapted from [41], Theorem 2.2] Suppose that Theorem 1 applies to K η .
Let
X n :=
{ X 1 n , . . .
, X Nn } be the interacting particle system from (3) .
Then for every n ∈ N and f ∈ B b ( R d ) we have
[Equation]
This corollary directly relates to the application of Bayesian ML we are interested in, where we would have f ( θ )
=
P ( y | x, θ ) .
3.1 Long-Time Bounds
There are two main ingredients in the general result on long-time convergence: ergodicity of K and
Q , and Lipschitz regularity of the interaction kernel η (cid:55)→ J η .
These, along with other technical conditions, produce an estimate of the form (cid:107) µ n − π (cid:107) ≤ C 2 ρ n + C 3 nρ n where (cid:107) • (cid:107) is a weighted total variation norm.
The full statement is in Theorem 2 of Appendix E.
Ergodicity of K and Q .
A fundamental requirement of our results is that the linear building blocks of K η must converge to their respective stationary measures in an appropriate metric.
This type of result is now standard in the Markov chain literature, and we use a result from [57] for K and a result from [1] for Q .
The former is actually able to ensure that K is a contraction on P ( R d ) w.r.t.
a suitable weighted total variation; we use this feature repeatedly in our analysis.
Lipschitz Regularity of J η .
We also need that η (cid:55)→ J η is Lipschitz -continuous w.r.t.
a weighted total vartiation norm on Markov kernels (the Lipschitz constant does not have to be < 1 ).
This regularity is used to translate the convergence of η n → η (cid:63) (as guaranteed by the ergodicity of Q ) into convergence of J η → J η (cid:63) with the Lipschitz estimate (cid:107) J η n − J η (cid:63) (cid:107) (cid:46) (cid:107) η n − η (cid:63) (cid:107) .
In Appendix G, we verify this analytically for J BGη and J ARη , see Lemma 5 and [1] Proposition 5.3.
3.2 Large-Particle Bounds
To study the large-particle behaviour, we would like to measure how close subset of q ∈ { 1 , . . .
, N } interacting particles { X 1 n , . . .
X qn } ⊂ { X 1 n , . .
, X Nn } =: X n from (3) is to being i.i.d.
according to the mean-ﬁeld measure µ n .
This analysis was pioneered in [41] under the name “propagation of chaos” and formalizes the intuition that, as N → ∞ , the inﬂuence of any individual particle → 0 .
To state this more precisely, ﬁrst note that if we had random variables Z i iid ∼ η ∈ P ( R d ) then the joint distribution of Z :
=
{ Z 1 , . . .
, Z N } would be η ⊗ N .
Hence, as N → ∞ for the interacting particles X n at time n , we expect the distribution of { X 1 n , . . .
, X qn } , denoted µ q,Nn , to get closer to the distribution of i.i.d.
mean ﬁeld particles from (2), denoted µ ⊗ qn .
In other words, we expect (cid:107) µ q,Nn − µ ⊗ qn (cid:107) tv → 0 as N → ∞ .
The full statement of this result is Theorem 3 of Appendix F.
The main condition in our propagation of chaos result is another type of regularity for η (cid:55)→ J η which basically requires that, if one approximates a distribution η ∈ P ( R d ) by its empirical measure m ( Y )
where Y :=
{ Y 1 , . . .
Y N } and Y
i iid ∼ η , then J m ( Y ) → J η as N → ∞ .
More precisely, there should be a function R : [0 , ∞ [ → [1 , ∞ [ , which is ideally nondecreasing, s.t.
| E [ J ⊗ q m ( Y ) f ( x )] − J ⊗ q η f ( x ) | (cid:46) q 2 N R ( q 2 /N )
∀ x ∈ R d and f ∈ B b ( R d ) with “oscillations” osc( f ) ≤ 1 .
The expectation is taken over η ⊗ N , and “oscillations” are deﬁned precisely in Appendix D.
This inequality is essentially a total variation regularity since we can alternately write (cid:107) µ − ν (cid:107) tv = sup {| µ ( f ) − ν ( f ) | | f ∈ B b ( R d ) , osc( f ) ≤ 1 } [47].
3.3 Analysis of Speciﬁc Interaction Kernels
The main result Theorem 1 is in terms of conditions on a general K η (i.e. general choices of K, Q, J η ).
To apply this result to the samplers in Section 2, we must establish if these conditions hold for K BGη and K ARη .
Fortunately this can be done analytically; in Appendix G, we present conditions under which the Lipschitz regularity (see Lemma 5 and [1] Proposition 5.3) and large-particle regularity (see Corollaries 3 and 5) hold.
These results, particularly for K BGη , are interesting and rely on novel techniques for controlling the various quantities, sometimes improving over previous methods.
Due to space constraints, the results for K BGη and K ARη are in Corollaries 2 and 4 from Appendix G.
Experiments
In this section, we detail two experiments designed to explore how one might apply the nonlinear
MCMC methods developed in the previous sections to Bayesian machine learning.
5 Let us state explicitly that our aim is not to achieve state-of-the-art with these experiments, nor do we claim that this method will necessarily lead to state-of-the-art results on a particular task.
Rather, the aims of these experiments are: to demonstrate that nonlinear MCMC can be applied successfully to large-scale problems; to compare linear vs nonlinear methods to understand what beneﬁts and drawbacks nonlinear MCMC offers compared to linear MCMC in practice; and to develop some recipes for choosing the various hyperparameters and samplers that determine a nonlinear MCMC method.
4.1 Two-Dimensional Toy Experiments
First, we use a toy setting of two-dimensional distributions to compare the relative beneﬁts of linear and nonlinear MCMC.
A beneﬁt of this simple setting is that the multimodal toy distributions can be exactly sampled.
This gives us the opportunity to quantify the quality of our samples via an unbiased estimator of the Maximum Mean Discrepancy (MMD) metric on P ( R d ) [58].
This approach stands in contrast to many previous works, which use simplistic distributions (e.g. Gaussians) with analytically tractable statistics to measure quality.
See Appendix C.1 for an overview of our methodology.
Setup.
Our setup will compare the Metropolis Adjusted Langevin Algorithm (MALA) [18] (see
Appendix B for an overview of MALA) with the nonlinear BG and AR samplers using MALA for the kernels K, Q in our nonlinear setup from Section 2.
This will allow us to examine the effects of the nonlinearity in K ARη and K BGη while controlling for the type of sampler used and its hyperparameters.
The main difference between the linear and nonlinear algorithms, aside from the interaction itself, is the extra “design knob” to control in the form of the choice of the auxiliary density η (cid:63) .
Below, we show how one can use η (cid:63) to incorporate additional insight to guide the sampling, such as regions of the state space to explore.
In our experiments, we choose η (cid:63) to be a centered, 2-dimensional Gaussian with a large variance ( Σ = 4 I 2 for the circular MoG and two-rings densities, and Σ = [-
-]20 I 2 for the grid MoG).
This conveys “coarse-grained” information of roughly where the support of the target density is located – in this case, a neighbourhood of (0 , 0) .
See Table 2 in Appendix C.1 for a full account of our experimental settings.
Results.
From Figure 1, we see that having a simple auxiliary density with good coverage of the support of the target distribution is quite helpful.
In all three examples, one or both of the nonlinear samplers outperformed the equivalent linear sampler in the empirical MMD metric.
For the two most challenging densities, the “two rings” and “grid MoG” distributions, the improved exploration is particularly evident.
We also include an analysis of the runtime of the algorithms in Appendix C.1.3. Comparison With [1]. We also compared the performance of our methods with those of [1] in this two-dimensional toy setting.
See Appendix C.1.4 for an overview of the results; they support all of the theoretical considerations and design principles we have introduced in this paper. [Figure]
4.2 CIFAR 4.2.1 Setup
To examine the properties of the nonlinear sampler outside of a toy setting, we have also implemented a Bayesian neural network on the CIFAR10 dataset.
We use a likelihood P ( y | x, θ ) parameterized by a ResNet-18 convolutional neural network [61] and a Gaussian prior P ( θ ) on the parameters of this neural network which are combined to form a posterior P ( θ |D train )
∝ P ( θ ) (cid:81) ( x i ,y i ) ∈D train P ( y i | x i , θ ) .
The goal is to sample θ i , i = 1 , . . .
, N from this posterior.
See Table 4 in Appendix C.2 for a full account of the experimental settings.
To deal with the fact that this sampling problem is very high-dimensional ( d ≈ 11M) and |D train | is large ( |D train | = 60 , 000) we use a variety of techniques:
We sample minibatches (cid:98) D of size 256 to obtain the surrogate target density P ( θ | (cid:98) D ) [17].
We use an RMSProp-like “preconditioned” Langevin algorithm, called RMS-Langevin or RMS-ULA, as in [33] for the auxiliary sampler Q ; see Appendix B for details on this sampler.
As shown in [33], this sampler is biased.
We use tempering, wherein we aim to sample from π or η ∝ P ( θ |D train ) 1 /τ where τ is a small number.
This substantially improves mixing for hard-to-sample distributions such as P ( θ |D train ) at the cost of bias since we are no longer sampling from the true posterior [32].
Using our nonlinear algorithm presents a novel opportunity to correct the bias introduced by tempering.
For our experiments, we pick η (cid:63) ∝ P ( θ |D train ) 1 /τ and π = P ( θ |D train ) .
This means that the auxiliary chain Y n explores a tempered version of the target, whereas the target chain X n (in theory) explores the true target distribution.
This is a novel strategy that is made possible by being able to select η (cid:63) almost independently of the target π .
We study the case when π is tempered as well.
For our experiments, we use the RMS-Langevin sampler as the baseline, and we also use it for the auxiliary sampler Q .
For the target sampler, it is not possible to use the RMS-Langevin algorithm because the smoothed square-gradient estimate is incompatible with the discontinuities (i.e. jumps) introduced by the nonlinear interaction.
Instead, for the linear sampler K we use the unadjusted
Langevin algorithm, ULA, [16] (see Appendix B).
We investigate both test accuracy and calibration error [9] to assess performance.
The tempered results are using τ = 10 − 4 .
All results between linear and nonlinear are statistically signiﬁcantly different ( p (cid:28) 0 . 05 ).
See Appendix C.2 for an overview of expected calibration error.
We also compute the maximum calibration error in Appendix C.2.
All Expected Calibration Error numbers are multiplied by 10 2 in this table.
[Table]
4.2.2 Results
Linear vs Nonlinear.
From Table 1, we see that the linear (RMS-Langevin) sampler has slightly higher, but comparable test accuracy to the nonlinear samplers.
This is likely because RMS-Langevin algorithm has better stability around the regions of high probability due to its adaptive stepsize scaling.
However, from Figure 2, we see that for both the tempered and non-tempered cases, the nonlinear interaction appears to beneﬁt during early exploration.
This is an expected and desired property of these nonlinear samplers, which incorporate global information about the sampler state (in this case, the relative potential G of each auxiliary chain’s state) and are able to emphasize those states with higher probability.
However, the linear method is able to eventually explore the relevant regions of the state space, and the difference disappears.
See Appendix C.2.5 for a comparison linear vs nonlinear performance scaled by the number of gradient evaluations.
Tempering Vs Non-Tempering.
The linear MCMC sampler is always tempered in our experiments so there should not be any statistically signiﬁcant difference in the linear case.
For the nonlinear sampler, the tempered version has slightly higher accuracy; this trend is also observed in [32].
On the other hand, the calibration errors are the same for both tempered and non-tempered variants.
This is somewhat surprising, given the aggressive tempering used, and one would expect that this reduces the variance of the posterior estimate.
This observation can perhaps be explained by the fact that we are using N = 10 samples which may not be enough to accurately change the tempered auxiliary distribution η (cid:63) into the non-tempered primary distribution π in the jump interaction.
Calibration.
Considering the expected calibration error (ECE) [9], in Table 1 we see that the nonlinear method has statistically signiﬁcantly lower ECE ( p (cid:28) 0 . 05) compared to the linear method.
We hypothesize that this is due to a tension between the RMS scaling of the gradient which improves the efﬁciency of each MCMC step but at the cost of bias, which may be measurable in the form of calibration error.
The Langevin algorithm is also biased, but is generally known to have good convergence properties [16] and much less is known about the RMS-Langevin variant.
By using our nonlinear setup, we are able to aggressively explore the auxiliary distribution without sacriﬁcing calibration on the target distribution.
Conclusion
In this paper, we have studied the theoretical and empirical properties of nonlinear MCMC methods.
We have obtained powerful theoretical results to characterize the convergence of our MCMC methods, and we have applied these methods to Bayesian neural networks.
The results on BNNs are comparable to, but not better than, the linear methods we studied.
We hypothesize that this is because more investigation into choosing the best auxiliary density η (cid:63) is required; our choice is simplistic and may not be optimal.
This hypothesis is supported by our toy experiments, which show signiﬁcant improvement when η (cid:63) is able to incorporate some additional insight into the problem.
How to do this in high dimensions is an exciting direction for future research.
Broader Impact.
There are beneﬁts and drawbacks to the nonlinear MCMC methods we describe.
The beneﬁts are mainly that properly accounting for uncertainty in machine learning will lead to better real-world outcomes for high-value scenarios such as self-driving cars or medical imaging.
The drawbacks are that MCMC methods require O ( N ) storage and computations relative to the O (1) for deterministic methods; in fact, our nonlinear method would require 2 N resources compared with
N for a linear method (see also Appendices C.1.3 and C.2.5).
If our algorithms were applied to a large swath of ML “as is”, this would mean a substantial increase in the energy consumption required for experimentation and deployment, worsening an already substantial issue in the ﬁeld
A Pseudocode & Numerical Implementation Details A.1 Simulation Algorithm
The method of simulating K η is where this work and [1] diverge: we will provide an algorithm that uses a ﬁxed number of samples N ∈ N to simulate the IPS (3), whereas [1] investigates an algorithm in which the empirical measure is formed from all the past samples .
While the algorithm in [1] yields asymptotically unbiased estimate of the target measure as n → ∞ (for K AR • at least, see [1]), their MCMC algorithm’s memory complexity increases linearly with time.
This behaviour is not well-suited to large-scale software implementations, which typically favour ﬁxed-size “batches” of computations.
Additionally, a practitioner cannot add more samples to increase the sampling accuracy in a ﬁxed time frame.
By contrast, Algorithm 1 below uses a ﬁxed number of particles N throughout the lifetime of the algorithm.
Theorem 1 indicates that this produces a biased estimate of the target measure π , but that this bias can be controlled both in the number of particles, N , and the number of steps, n .
In exchange, our algorithm can be efﬁciently implemented in vectorized computing frameworks, as we demonstrate in Section 4.
In Appendix C.1.4, we investigate the differences in empirical performance between the algorithms in [1] and Algorithm 1; those ﬁndings unambiguously support this discussion.
Bernoulli
Additionally, let us note that the decision to use an auxiliary Markov chain is highly pragmatic.
It is possible to develop “autonomous” nonlinear MCMC algorithms of the form (cid:101) µ n +1 = K (cid:101) µ n (see e.g. [47] Ch. 5) with strong theoretical guarantees but bad empirical performance.
A primary reason for this is sample degeneracy ; due to the properties of nonlinear interaction, in an autonomous nonlinear
Markov kernel, quite often a single particle X in will be given a large potential G ( X in ) which results in the jump interaction being concentrated on a single point.
From this point onwards, the algorithm will be unable to generate enough diversity within its particles to correctly estimate the variance of the target measure.
Due to the close relationship of nonlinear MCMC and nonlinear ﬁltering, these issues have been studied in many settings such as [62, 48] and using auxiliary dynamics is indeed a common solution.
A.2 Efﬁcient Software Implementation
In Algorithm 1, we gave a high-level pseudocode implementation of the nonlinear sampler.
In this implementation, for the sake of notational clarity, we used for loops to sample the individual particles.
However, with modern single instruction multiple data (SIMD) computing frameworks such as GPU accelerators, this is highly inefﬁcient; one can, and should, parallelize all for loops except the outer “time” loop in Algorithm 1.
We have done this using the variety of powerful tools provided by the JAX library [63] within Python such as vmap which allows for automatic vectorization, jit compilation, and seamless targeting of GPU accelerators.
vmap is particularly useful for automatic batch-wise and sample-wise vectorization in the case of stochastic gradient MCMC while writing functions in-terms of single inputs and outputs.
We have also leveraged the library of linear MCMC algorithms provided by the jax-bayes library https://github.com/jamesvuc/jax-bayes .
All experiments were run on a single Titan RTX 3090 GPU; the code can be found at https://github.com/jamesvuc/nonlinear-mcmc-paper .
B Linear MCMC Sampling Algorithms
Markov chain Monte Carlo algorithms form an integral part of Bayesian inference.
In this section, we will review the basic MCMC algorithms used in this paper.
In each case, we assume a C 1 , strictly positive target density π known only up to a normalizing constant.
B.1 Unadjusted Langevin Algorithm
Consider the (overdamped) continuous Langevin diffusion [64]
[Equation]
where B t is a standard Brownian motion.
This is a fundamental stochastic process with far reaching consequences in many areas of math; in particular, the Langevin diffusion is a Markov process with π as a stationary measure [64].
We obtain the unadjusted Langevin algorithm, or ULA, by applying an Euler-Maruyama discretization with stepsize δ > 0 to the SDE above yielding [64]
[Equation]
This is a popular and well-studied algorithm for MCMC, although it is known to be biased in the sense that the sationary measure of this Markov chain is not π [65].
However, this bias converges to 0 as δ → 0 [16].
B.2 Metropolis-Adjusted Langevin Algorithm
The Metropolis-Adjusted Langevin algorithm (MALA) [18, 19], like all basic Metropolis-Hastings
MCMC methods, consists of two steps: a proposal step and an accept step.
In the proposal step, a candidate next state starting from the current state X n is sampled according to the Langevin dynamics above, i.e. (cid:101)
X n {++
+} =
X n + δ ∇ log π ( X n ) √ 2 δZ n .
We will write q ( y | x ) for the proposal distribution; in this case q ( y | x ) =
N ( x + δ ∇ log π ( x ); 2 δ )( y ) .
However, unlike ULA, MALA consists of a second step that will accept the propsal (cid:101) X n +1 with probability
[Equation]
In other words
[Equation]
B.3 RMS-Unadjusted Langevin Algorithm
The ULA is a powerful technique for “black-box” MCMC in which one only has access to gradient information about the target density.
However, for very high dimensional problems in which π is highly anisotropic, it can be inefﬁcient to simulate an isotropic diffusion such as the Langevin algorithm.
A simple and effective technique, borrowed from the optimization literature in which the same phenomenon can cause problems, is to precondition the dynamics as follows.
First we maintain an exponentially smoothed squared-gradient estimate r n deﬁned as
[Equation]
where the gradient squared is meant elementwise.
Then, as in RMSProp [66], we can construct an adaptive stepsize by dividing by the square root of r n as follows
[Equation]
and then using this stepsize in the Langevin update
[Equation]
This has the effect of using gradient information to scale the stepsize of original Langevin diffusion independently along each dimension, which likely reduces the negative impacts of anisotropy and substantially increases mixing rate.
As studied in in [33], this algorithm, which we call RMS-ULA, is biased but this bias can be controlled with δ .
B.4 RMS-MALA.
Similar to the progression from ULA to MALA, we can “metropolize” the RMS-ULA algorithm to correct the bias.
This follows the same structure as the MALA, except that the proposal depends on r
[Equation]
B.5 Practical Considerations for Bayesian Neural Networks: Reversibility & Tempering.
In practical settings, the high dimensional, anisotropic characteristics and limited computation budget of Bayesian neural networks (BNNs) necessitate some modiﬁcations to the above algorithms.
The ﬁrst is that the Metropolis-Hastings (MH) step is rarely used except in simple settings.
This is because, while ensuring that that π is the invariant measure, the reversibility of the MH step is too inefﬁcient when we cannot affort to reject many samples.
Because they cannot “backtrack”, nonreversible dynamics are generally much more efﬁcient than reversible dynamics [67, 68, 64, 69].
This has led to the recent interest in using piecewise deterministic Markov processes [70–73] and other nonreversible MCMC dynamics such as [74] for high dimensional Bayesian inference.
See [69], Section 3.1 for an interesting dicussion on nonreversibility and efﬁciency.
The second modiﬁcation is tempering.
A tempered version of a distribution π is π β ∝ π β .
This is a pragmatic solution to speed up exploration of a target distribution when the noise component of
Langevin-like algorithms causes slow or unstable dynamics.
In the limit as β → ∞ , sampling from π ∞ is equivalent to the maximum a posteriori estimate (this is simulated annealing [75]).
In practice, tempering can be achieved by simply scaling the noise by √ τ
[Equation]
which targets π β where β = 1 /τ .
8 See also [32] for a discussion of tempering in BNNs.
C Experimental Details
No hyperparameter sweeps were used in the any of these experiments.
Hyperparameters were chosen based on reasonable guesses and minimal manual tuning.
C.1 2d Toy Experiments C.1.1 Maximum Mean Discrepancy
We use the maximum mean discrepancy (MMD) metric [58] as a way to quantitatively evaluate our
MCMC algorithms.
The MMD metric is an integral probability metric of the form
[Equation]
where H is a reproducing kernel Hilbert space (RHKS) associated to a positive-deﬁnite kernel function k : R d × R d → R ; see [58] for details.
For our purposes, the important features of the
MMD metric are that 1) it is similar to the total variation metric, which optimizes over the unit ball in
B b ( R d ) instead of H ; and 2) the MMD metric (cid:107) µ − ν (cid:107) MMD can be efﬁciently empirically estimated from samples of µ, ν .
This is because, according to Lemma 6 in [58], we have
[Equation]
which depends only on the kernel function k
[Equation]
and an unbiased estimator of (cid:107) µ − ν (cid:107) 2 MMD is
[Equation]
j = k ( X i , Y j )
+
N ν ( N ν − 1)
N ν (cid:88) i,j =1 , j (cid:54) =
i k
( Y i , Y j ) where X i iid ∼ µ for i = 1 , . . . , N µ and Y i iid ∼ ν for i = 1 , . .
. , N ν .
If we treat µ as the distribution µ n of a (linear or nonlinear) MCMC algorithm at step n and ν as π , then we can apply this estimator provided we can sample from π exactly.
This is not feasible for complex high-dimensional distributions, but it is possible for the toy distributions we have chosen, which are all mixtures or transformations of Gaussians.
C.1.2 Experimental Setup
Refer to Table 2 for the experimental settings.
Additionally, we used 10 , 000 samples from π and the kernel
[Equation]
to estimate the MMD as in the previous section.
[Table]
η
U ([ − 7 . 5 , 7 . 5] × [ − 7 . 5 , 7 . 5])
[Table]
η
U ([ − 7 . [Table] 7 . 5] × [ − 7 . 5 , 7 . 5])
[Table]
ε
[Table]
[Table]
C.1.3 Runtime Analysis
We also report on the performance vs runtime of the linear and nonlinear algorithms measured in terms of number of gradient executions and the wallclock time.
In general, the nonlinear methods will require 2 × the number of gradient executions as the linear methods due to the requirement that we sample from the auxiliary chain in addition to the primary chain.
In Figure 3, we show the MMD-squared metric plotted against the number of gradient [Figure] [Table] [Figure]
evaluations. Since we ran all algorithms for a ﬁxed number of steps, this amounts to using 2 × the gradient evaluations for the nonlinear methods, although not 2 × the steps for a given particle.
Despite this, the nonlinear methods offer better performance per gradient evaluation. In Table
[Table]
we also report the wallclock time for each of the methods and each of the distributions.
We omit the ﬁrst twenty iterations in our calculation since these can be slow while JAX compiles the program and we only want to measure the steady-state execution performance. [Table]
Thanks to a very helpful suggestion from an Anonymous Reviewer, we conducted a comparison of the nonlinear MCMC algorithms proposed in this paper with those proposed in the “parent” work of this paper, i.e. [1]. Below, we detail the implementation details as they are nontrivial, and the results we obtained. See also Figure 4 for the visual results.
Implementation Details. As detailed in Appendix A.1, the key difference between algorithms in this work and those in [1] is how one constructs the empirical measure η Nn for indexing K η Nn in (3). Recall, ours uses a ﬁxed batch of samples, and the algorithm from [1] uses all past samples from a single trajectory. In other words, our approach uses η Nn = m ( { Y 1 n , . . .
, Y Nn } ) whereas [1] uses η nn = m ( { Y 0 , . . , Y n } ) . This decision to use a dynamic number of samples is well-motivated from a theoretical point of view in [1], but is highly suboptimal from an implementation standpoint.
The reason is that using a dynamic number of samples, i.e. n in the case of [1], results in frequent memory reallocations which are costly operations in common software frameworks and especially on GPUs. Indeed, modern software frameworks and accelerators have extensive optimizations for ﬁxed-size (i.e. batched) workloads and this approach is contrary to this paradigm. In fact, this work was motivated by the need to understand how ﬁxed sample sizes would perform in the setting introduced by [1]. Nevertheless, to implement the algorithm in [1] as a benchmark, we were faced with the classical memory-speed trade-off in software engineering.
On the one hand, one can disable jit-compilation in JAX and accept the memory-allocation costs to use a variable-size array to store the samples. Alternately, one can aggressively use more memory to speed up the computation by pre-allocating enough device memory for all future samples and apply masking to do computations on a subset of those samples. The former approach yielded a slowdown of ∼ 140 × on the 2-dimensional examples, which was deemed unacceptable. Hence we used the pre-allocation method at the cost of allocating a N × n × d - float32 tensor.
For small d , i.e. 2 in our case, this was acceptable at 2 , 000 × 10 , 000 × 2 × 4 = 152MB. However, for larger-scale problems such as the CIFAR experiments below, this would be completely intractable with either approach. Bias-Implementation Trade-off. The results of our comparison experiment unequivocally support the bias-speed trade-off we expect in our work.
From Figure 4, we see that the algorithms from [1] are considerably more stable and have lower variance and bias.
This is explainable by comparing the effective number of samples each method uses – ours uses N = 2 , 000 throughout, but theirs effectively uses N = 10 , 000 at the end of simulation.
In our main result Theorem 1, we show that the bias of our algorithm decays like 1 /N for ﬁxed timestep n , therefore using more samples in our method would result in a less-biased approximation of π .
On the other hand, as shown in [1], their algorithm is asymptotically unbiased .
Despite this desirable property, the discussion above clearly shows why accepting some bias is required for practical applications of this type of algorithm.
In Figure 5, we show the results of a study matching the N = 10 , 000 samples used by the algorithms from [1] in our own algorithms.
The results align completely with the predictions made by Theorem 1; i.e. the bias of our methods decreases signiﬁcantly.
We see it is also comparable to that of the methods in [1] for many of the samplers and distributions used in these experiments.
We note that this study is a qualitative example of the fundamental difference between nonlinear and linear MCMC algorithms: increasing N directly impacts the convergence rate of our algorithms but has no effect on the convergence of the linear ones.
This is due to the interacting (i.e. nonlinear) nature of our MCMC methods, and is supported by all the theoretical results we have developed so far.
Convergence Rate.
From Figure 4, we see empirically that our algorithms have better convergence rates than those in [1].
While the results in [1] are asymptotic and hence make no claim about rate of convergence, this aligns with our intuition of how these samplers work.
In particular, using past samples — including from the very beginning of simulation — can slow down the convergence since one expects that future samples will be of higher quality than past samples for a converging algorithm.
In [1], they use a burn-in period but of course this does nothing to help with convergence rate.
On the other hand, Theorem 1 upper-bounds convergence by the “pseudo-geometric” rate O ( nρ n ) for ρ < 1 .
C.2 CIFAR10 Experiments C.2.1 Experimental Setup
In this experiment, we implement a Bayesian neural network for image classiﬁcation on the CIFAR dataset [76].
The neural network architecture we use is a standard ResNet-18 architecture [61] using the default settings implemented by the DeepMind haiku library [77].
The likelihood function
P ( y | x, θ ) is the standard crossentropy on the logits y given an input image x
and parameters θ .
We also use standard data augmentation techniques including standardizing the images by the mean and variance, random crops and random ﬂips.
See the Table 4 for the experimental parameters.
See Figure 6 for the plot of CIFAR10 test accuracy during sampling for the non-tempered case.
C.2.3 Calibration Analysis
Calibration of a classiﬁer is a measure of how well that classiﬁer’s logits represent probabilities [9].
We use this as a necessary but not sufﬁcient test for correct sampling, since we know a priori that the Bayesian posterior gives true probabilities.
We will use the methodology in [9], particularly “expected calibration error” and “maximum calibration error” to study the calibration of our Bayesian neural network, which we describe below.
To measure calibration, assume ﬁrst that we have a test set of data D test =
{ ( x 1 , y 1 ) , . .
, ( x N test , y N test ) } ⊂ X × { 0 , . . .
, C } where X is the input space and there are [Figure]
Boltzmann [Table] [Table] [Table]
θ 0 is the initialized set of parameters from the Haiku implementation
[Table]
[Table]
dataset
[Table]
P ( θ )
N (0 , 1 × 10 −
[Table]
d ) C ∈ Z > 1 labels. Suppose (cid:98) p : X → ∆ C − 1 is a supposed probability distribution representing p ( y | x ) . If (cid:98) p were a good representation of p ( y | x )
[Table]
we would expect that the values of (cid:98) p should correlate with the accuracy of (cid:98) p , i.e. when (cid:98) p is α % certain, then α % of the time (cid:98) p is correct.
More formally, perfect calibration is when
[Equation]
p [ y i ] = α ) = α.
[Figure]
We can empirically estimate this relationship between predicted probability and true probability using histograms.
Speciﬁcally, given predicted probabilities (cid:98) p i
= (cid:98) p ( y i | x ) , we can form bins B k , k = 1 , . . .
, M on [0 , 1] and estimate the accuracy in the k -th bin as A k and the average probability in that bin as α k .
If (cid:98) p represents a true probability distribution, then A k ≈ α k for each k = 1 , . . .
, M .
The expected calibration error (ECE) is then
[Equation]
We can also study the maximum calibrarion error
[Equation]
In Table [Table] we report the ECE and MCE for the CIFAR10 experiment, where now
[Equation]
[Table]
C.2.4 Distribution Shift
One useful outcome of using the probabilistic paradigm for ML is robustness to distribution shift.
This is well-documented [8, 9]; traditional ML models tend to be overconﬁdent even in settings where they have no hope of making good predictions.
In short, traditional ML models lack the ability to say “I don’t know” in a way that probabilistic ML models do not.
This is particularly salient in real-world systems such as self-driving cars, where the distribution of images is highly nonstationary and the costs for overconﬁdent predictions are high.
In Figure 7 below, we show how the predictive entropy differs between in-domain examples, i.e. the CIFAR10 test set, and out-of-domain examples.
For the out-of-domain examples, we use the SVHN dataset [78] which also contains 32 × 32 RGB images, but this time of housing numbers (i.e. the digits 0-9) rather than objects such as airplanes or dogs.
The entropy (a measure of uncertainty) is calculated as
[Equation]
where p ∈ ∆ 9 is the vector of probabilities for a prediction p ( y | x, D train ) .
In the traditional ML case, the probabilities are made with p ( y | x, θ ∗ ) .
We average H ( p ) over the test data for the in-domain and out-of-domain examples.
The optimized method was trained to the same test accuracy ( 85% ) using essentially the same hyperparameters (learning rate, etc).
C.2.5 Runtime Analysis
In Figure C.2.5, we plot the performance of the tempered target vs the number of gradient evaluations.
As described in Appendix C.1.3, the nonlinear algorithm uses 2 × the gradient computations.
When plotting the eval performance vs the number of computations, we see that the linear algorithm is signiﬁcantly more efﬁcient.
Further work into designing nonlinear samplers that achieve better performance (such as in the two-dimensional examples above) is required to make this trade-off worthwhile in practice. [Figure]
D Notation & Assumptions
The remaining sections will be devoted to proving the theoretical results contained in this paper.
They should be read in sequence.
Note that, while some of the assumptions found below appear in [1], our analysis and the results we obtain are new.
In particular, [1] studies a different MCMC algorithm based on K η , see Appendix A for a discussion of algorithm differences.
Moreover, they obtain an asymptotic “strong law of large numbers” result whereas we obtain nonasymptotic mean ﬁeld long-time convergence and uniform propagation of chaos results, which are qualitatively different results using different proof techniques.
D.1 Notation and Deﬁnitions D.1.1 Probability Spaces, Measures, and Kernels
We will be working on the measurable space ( R d , B ( R d )) with B ( R d ) the Borel σ -algebra 9 ; let
P ( R d ) denote the space of probability measures on B ( R d ) .
Let µ ∈ P ( R d ) , f : R d → R be a measurable function, and, K : R d × B ( R d ) → [0 , 1] be a Markov kernel.
Throughout the sequel, we will write
[Equation]
An obvious consequence of this is the notation µK ( f )
=
´ f [Equation] y [Equation]
µ
[Equation]
d [Equation] K ( x, d y ) . We will also need the following subset of P ( R d ) for our results: let G, U : R d → [0 , ∞ [ and ﬁx constants m G , M U > 0 ; then we deﬁne
[Equation]
We will abbreviate P m G ,M U ( R d )
=
P m,M ( R d ) when there is no chance of confusion.
We will also frequently use the following notation for the empirical measure associated to y :
= { y 1 , . .
, y N } ⊂ R d :
[Equation]
with δ x ∈ P ( R d ) the Dirac measure.
Additionally, we will need the notion of tensor products for functions and measures: let q ∈ N , f i : R d → R and µ i ∈ P ( R d ) for i = 1 , . . .
, q .
Then for x = ( x 1 , . . .
, x q ) ∈ ( R d ) q and measurable g : ( R d ) q → R f 1 ⊗ · · · ⊗ f q ( x ) : = f 1 ( x 1 ) · · · f q ( x q ) , µ 1 ⊗ · · · ⊗ µ q ( g ) : = ˆ g
[Equation]
, x q ) µ 1 (d x 1 ) · · · µ q (d x q ) . When f = f 1 = · · · = f q and µ = µ 1 = · · · = µ q , we will write f ⊗ q and µ ⊗ q respectively.
Finally, we write µ (cid:28) ν to mean that ν dominates µ
and we write d µ d ν for the Radon Nikodym derivative.
The notation µ ∼ ν means that µ (cid:28) ν
and ν (cid:28) µ .
D.1.2 Norms
Denote by B b ( R d ) the set of bounded measurable functions f : R d → R .
We will be working with a family of norms on functions, and its dual norms on probability measures, which is parameterized by functions of the form U : R d → [1 , ∞ [ .
For such a U , we deﬁne
[Equation]
( x ) for f : R d → R .
The dual norm to (cid:107) f (cid:107) U corresponds to the weighted total variation distance; for µ, ν ∈ P ( R d ) we have
[Equation]
Note that we could replace the condition (cid:107) f (cid:107) U ≤ 1 by | f | ≤ U .
For V : R d → [0 , ∞ [ , we will often work with V β ( x ) :
= 1 + βV ( x ) for β > 0 ; in this case, we will write (cid:107) f (cid:107) β := (cid:107) f (cid:107) V β and (cid:107) • (cid:107) tv,β := (cid:107) • (cid:107) tv,V β .
We will also need the deﬁnition of the maximum oscillation of f : R d → R , which we denote and deﬁne as osc( f ) :
= sup {| f ( x ) − f ( y ) | | x, y ∈ R d } .
For Markov kernels K, Q on R d , we obtain the weighted kernel distance
[Equation]
tv,U U ( x ) . It is worth noting the special case U ≡ 1 which corresponds to the usual total variation distance; in this case we will write (cid:107) f (cid:107) ∞ and (cid:107) µ − ν (cid:107) tv in place of (cid:107) f (cid:107) U and (cid:107) µ − ν (cid:107) tv,U respectively.
Lastly, we will use the notation (cid:15) ( K ) to denote the contraction coefﬁcient of a Markov kernel K deﬁned as (cid:15) ( K ) :
= inf { c ∈ [0 , 1] | (cid:107) µK − νK (cid:107) tv ≤ c (cid:107) µ − ν (cid:107) tv ∀ µ, ν ∈ P} see, e.g. [47] Ch 4.
D.2 Assumptions
Assumption 1 (Drift and Minorization) .
Let K, Q : R d × B ( R d ) → [0 , 1] be Markov kernels.
K1 K satisﬁes the drift criterion
[Equation]
for a ∈ ]0 , 1[ , b > 0 , and V : R d → [0 , ∞ [ with lim (cid:107) x
[Equation]
V ( x ) = ∞ .
K2 K satisﬁes the following uniform minorization condition on the level sets of V : there exists γ ∈ ]0 , 1[ , ν ∈ P ( R d ) , and R > 2 b/ (1 − a ) s.t. [Equation] x | V [Equation]
x )
[Equation]
Q1 Q satisﬁes the drift criterion
[Equation]
for ξ ∈ ]0 , 1[ , c > 0 , and U : R d → [1 , ∞ [ with lim (cid:107) x [Equation] U ( x ) = ∞ .
Q2 Q satisﬁes the following uniform minorization condition on the level sets of U : there exists ζ ∈ ]0 , 1[ , ν [Equation] ∈ P [Equation]
[Equation]
The requirements in Assumption 1 are from [57] and are by now fairly standard in the Markov chain literature.
They will in particular imply the following properties for K and Q respectively:
Proposition 1 (Basic Properties of K, Q ; [57]&[1]) .
Suppose Assumptions 1-K1,K2 hold.
Then there exists γ ∈ ]0 , 1[ and β > 0 s.t. ∀ µ, ν ∈ P ( R d ) , we have
[Equation]
Suppose that Assumptions 1-Q1,Q2 hold.
Let η n = η 0 Q n for η 0 ∈ P ( R d ) , and r ∈ ]0 , 1] .
[-Ifη 
-]If η 0 ( U r ) < ∞ , then for r ∈ ]0 , 1[ there are constants M ( r ) > 0 , δ ∈ ]0 , 1[ s.t.
[Equation]
Proof.
This is directly from [57].
This uses Lemma which shows Assumption 1 Q
,Q
implies the assumptions
in Lemma C.1 from [1]
Then we combine this with Lemma
Both lemmas can be found in
Appendix E.
Throughout the remainder of this paper, β and γ will be ﬁxed and we will adopt the notation
[Equation]
Additionally, references to M ( r ) and δ are meant in the sense of the above proposition.
Next, we introduce some criteria that ensure that the kernels K, Q are “compatible” in-terms of their drift criteria from Assumption 1.
Assumption 2 (Compatibility) .
C1 There is r ∗ ∈ ]0 , 1] s.t.
V β ( x ) ≤ U ( x ) r ∗ ∀ x ∈ R
d
with V β as above.
C2 G satisﬁes the lower bound compatibility criterion with U : for every R >[-
-]
[Equation]
Assumption 2-C1 is also present in [1] and ensures that V and U are sufﬁciently “comparable”.
Assumption 2-C2 is a novel assumption that we will use to obtain an a priori lower bound on η n ( G ) .
Finally, some straightforward boundedness assumptions on G .
Assumption (Assumptions on G )
G1 G is bounded, i.e. (cid:107) G (cid:107) ∞ < ∞ .
G2 G is bounded in the weighted supremum norm for V β ; i.e. (cid:107) G (cid:107) β < ∞ .
Not all of the results below depend on all of the assumptions in this section.
We will make clear which of these assumptions are invoked in each result.
E Long-Time Convergence
We will now state the main long-time convergence result for the mean ﬁeld system.
We will revisit this theorem at the end of this section and provide a full proof.
We will often use the equivalent “distribution ﬂow” interpretation of system (2) deﬁned as
[Equation]
for our proofs.
Theorem 2.
Suppose that Assumption 1 and Assumption 2-C1 hold.
Suppose also that J η satisﬁes
[Equation]
for some constant C J > 0 and that
[Equation]
for (cid:101) a ∈ ]0 , 1[ , (cid:101) b > 0 and all η ∈ P m G ,M U for suitably chosen constants m G , M U .
Suppose δ is from (7) and set
Case 1: ρ := (1 − ε ) γ if J η ( x, d y ) doesn’t depend on x ; or Case 2: ρ :=
(1 − ε ) γ + ε (cid:107) J η (cid:63) V (cid:107) V if J η ( x, d y ) does depend on x .
If µ 0 ( V ) , η 0 ( U ) < ∞ then there exists a constant C > 0 s.t. [Equation]
− π (cid:107) tv [Equation] β + Cn max( ρ, [Equation]
n . In particular, if ρ < 1 , we have lim n →∞ µ n = π in V β -total variation.
E.1 Results About Weighted Total Variation
Lemma 1.
Suppose that a Markov kernel P satisﬁes a drift condition
[Equation]
for V : R d → [0 , ∞ [ , V ( x ) → ∞ as (cid:107) x (cid:107) → ∞ , and the minorization condition with (cid:15) ∈ ]0 , 1[ , ν ∈
P ( R d ) , and C :=
{ x | V ( x ) ≤ R } s.t.
[Equation]
holds for some R > 2 K/ (1 − a ) .
Then there exists a ∈ ] a, 1[ , b > 0 , ν ∈ P ( R d ) , S ∈ B ( R d ) s.t. PV ( x ) ≤ aV ( x ) + b 1 { x ∈ S [Equation] , inf
[Equation]
S
[Equation]
[Equation] (cid:15)
[Equation]
A ) .
Proof.
This argument merely an elaboration on the remark from the end of [57].
Let a ∈ ] a, 1[ such that R > b/ ( a − a ) and S
= { x | V ( x ) ≤ R } .
If x / ∈ S then
[Equation]
If x ∈ S then PV ( x ) ≤ aV ( x ) + b ≤ aV ( x ) + b .
Hence the choices are clear from ﬁxing R as above.
Lemma 2.
Let P be a Markov kernel with invariant measure π and suppose there are constants ρ ∈ ]0 , 1[ , C > 0 , and r ∈ ]0 , 1] s.t.
(cid:107) P n f − π ( f ) (cid:107) V r ≤ Cρ n (cid:107)
f (cid:107) V r for any f : R d → R with (cid:107) f (cid:107) V r < ∞ .
Then if µ 0 ( V r ) < ∞ and µ n is the ﬂow of P then there is a constant C (cid:48) >
0 s.t.
[Equation]
Proof.
Consider
[Equation]
Lemma 3.
Let P, Q be Markov kernels and suppose that P satisﬁes a drift condition
[Equation]
for V : R d → [1 , ∞ [ .
Then
[Equation]
and
[Equation]
Proof.
For the ﬁrst part:
[Equation]
and
[Equation]
For the second part [Equation] let (cid:107) f (cid:107) V ≤[-
-]
[Equation]
E.2 Proof of Theorem
Theorem 2.
Suppose that Assumption 1 and Assumption 2-C1 hold.
Suppose also that J η satisﬁes
[Equation]
for some constant C J > 0 and that
[Equation]
for (cid:101) a ∈ ]0 , 1[ , (cid:101) b > 0 and all η ∈ P m G ,M U for suitably chosen constants m G , M U .
Suppose δ is from (7) and set
Case 1: ρ := (1 − ε ) γ if J η ( x, d y ) doesn’t depend on x ; or
Case 2: ρ :=
[Equation]
1 − [Equation] [Equation] V if J η ( x, d y ) does depend on x .
If µ 0 ( V ) , η 0 ( U ) < ∞ then there exists a constant C > 0 s.t.
[Equation]
In particular, if ρ < 1 , we have lim n →∞ µ n = π in V β -total variation.
Proof.
Case 1: For n = 1 , consider
[Equation]
because [-
 [Equation] (cid:48) (cid:107) tv,β ≤ (cid:107) η − η (cid:48) (cid:107)-]from Assumption 2-C1 we have
[Equation]
Hence assume true for n − 1 .
Then for n :
[Equation]
hence Case 1 is done.
Case 2: We proceed the same way as Case 1 except we need to use
[Equation]
using Lemma 3.
Note that since K • satisﬁes a uniform drift criterion for V [Equation] using linearity we have
[Equation]
Hence for n = 1 :
[Equation]
Hence assume true for n − 1 .
Then for n :
[Equation]
[Table]
hence [Table] is done [Table]
F Large-Particle Convergence
We study the behaviour of the interacting particle system (3) as N → ∞ .
The behaviour of this system is probabilistically different from that of the mean ﬁeld system (2) because the particles in (3) are coupled whereas the particles in (2) are independent .
The fact that we recover independence in the limit of a collection of intercheangeable particles is a remarkable feature of interacting particle systems [41].
F.1 Result
For a collection of N particles, under suitable assumptions on K • , we will show that any ﬁxed-size q -block of particles of { X 1 n , . . .
, X qn } becomes independent as N → ∞ , and moreover this trend towards independence happens uniformly in time.
This phenomenon is called the uniform propagation of chaos property 11 .
Let us describe the dynamics of the distribution of the IPS (3).
From (3), each X in evolves according to
[Equation]
which indicates that, given Y n : = ( Y 1 n , . . , Y Nn ) , X in is sampled independently.
Letting X q,Nn :=
( X 1 n , . . .
, X qn ) and µ q,Nn :=
Distribution( X q,Nn ) ∈ P (( R d ) q ) , for measurable f : ( R d ) q
→ R one has
[Equation]
where the expectation is taken over the distribution of Y n =
( Y 1 n , . . .
, Y Nn ) , which is η ⊗ N n .
We will use this decomposition to derive a uniform propagation of chaos result in the following theorem.
Theorem 3.
Let N ∈ N , q ∈ { 1 , . . . , N } , and consider the interacting particle system X n := ( X 1 n , . . .
, X Nn )
, Y n :=
( Y 1 n , . . .
, Y Nn ) from (3) .
Let µ q,Nn :=
Distribution( X 1 n , . . .
, X qn ) , and let µ n be the distribution of the (independent) mean ﬁeld system (2) , with X i 0 ∼ µ 0 .
Suppose that ∀ x ∈ ( R d ) q , η ∈ P ( R d ) , and f ∈ B b (( R d ) q ) with osc ( f ) ≤ [-1 
 [Equation]-]1(cid:12)(cid:12)(cid:12) E [ J ⊗ qm ( Y ) f ( x )] − J ⊗ q η f ( x ) (cid:12)(cid:12)(cid:12) ≤ cq 2 N R ( q 2 /N ) , where Y =
{ Y 1 , . . .
, Y N } , Y i ∼ η.
Suppose ﬁnally that µ q,N [Equation] 0 for any 1 ≤ q ≤ N .
If (Case 1) : J η ( x, • ) doesn’t depend on x , or if (Case 2) :
J η ( x, • ) does depend on x but additionally that (cid:15) ( K ) < 1 , then there exists a ﬁxed constant C > 0 s.t.
[Equation]
Proof.
Case 1 : We ﬁrst claim that
[Equation]
Let f ∈ B b (( R d ) q ) s.t.
osc ( f ) ≤ 1 .
Consider the following expression for general n :
[Equation]
where we have used the assumption on J .
Thus if n = 1 in the above expression, the base case holds since µ q,N= µ ⊗ q 0 .
Now, if the claim holds for n , then for n + 1 we have
[Equation]
Thus the claim holds for n + 1 .
Since (1 − ε ) (cid:15) ( K ⊗ q ) < 1 , we have the result.
Case 2 :
We ﬁrst claim that
[Equation]
Let f ∈ B b (( R d ) q ) with osc ( f ) ≤ 1 .
We proceed by induction.
Note that:
[Equation]
for the ﬁrst term, we will use the decomposition (cid:12)
[Equation]
E
[Equation]
Then, using the assumption on J , we have
[Equation]
and also we know we can use
[Equation]
since (cid:15) ( J η n ) ≤ 1 , with osc( h ) ≤ 1 .
Hence putting these together, we see
[Equation]
Now if n = 0 in the above expression, the base case holds since µ q,N= µ ⊗ q 0 .
If it is true for n , then for n + 1 we have
[Equation]
Hence if (cid:15) ( K ⊗ q ) < 1 , i.e. (cid:15) ( K ) < 1 , the result holds.
F.2 Proof of Main Results (Theorem 1 & Corollary 1)
Theorem 1.
[Convergence of Nonlinear MCMC] Under suitable conditions on K η and Q , there exist ﬁxed constants C 1 , C 2 , C 3 > 0 , a function R : [0 , ∞ [ → [1 , ∞ [ , and ρ > 0 s.t.
[Equation]
Proof.
This follows straightforwardly from the above discussion, the only technicality is converting the results from Theorem 2 to the un-weighted total variation.
But note that since {(cid:107) f (cid:107) ∞ ≤ 1 } ⊂ {(cid:107) f (cid:107) β ≤ 1 } we have
[Equation]
so we’re done.
Corollary 1.
[Adapted from [41], Theorem 2.2] Suppose that Theorem 1 applies to K η .
Let
X n :=
{ X 1 n , . . .
, X Nn } be the interacting particle system from (3) .
Then for every n ∈ N and f ∈ B b ( R d ) we have
[Equation]
Proof.
Let f ∈ B b ( R d ) and consider
[Equation]
using interchangeability of the X in s. Since the strong propagation of chaos in Theorem 3 clearly implies weak propagation of chaos, i.e. for any bounded f | µ q,Nn ( f ) − µ ⊗ qn ( f ) | → 0 , we see that the expression above → {+
+} and we have L 2 -convergence which implies weak convergence.
G Analysis of the Kernels K BGη and K ARη G.1 Analysis of K BGη G.1.1 Statement of Results
A key difﬁculty that arises when working with Ψ G ( η ) =
G d η η ( G ) is obtaining a uniform lower bound the denominator η ( G ) .
This is important for deriving uniform regularity results and for generally establishing conditions under which the measures { Ψ G ( η n ) } ∞ n =0 are well-deﬁned.
An effective approach used in [47] and related works is to assume the uniform lower bound G ( x ) > (cid:15) > 0 ∀ x ∈ R d for some (cid:15) > 0 .
While this assumption is self-contained in that it works for any choice of η , it eliminates ubiquitous families of measures such as Gaussians, e.g. G ( x ) = exp( −(cid:107) x (cid:107) 2 / 2) and d η = d x .
We will see below how to relax G ( x ) > ε > 0 using the structure of our problem.
In particular, we can use the Lyapunov function U for Q to control the probability that the state Y n ∼ η n will venture “far away” from the center of the state space.
This leads to the intuition that, if G is sufﬁciently “compatible” with U , i.e. the function G stays away from zero in the “center” of the state space as determined by U , then the expectation η n ( G ) =
E Y n ∼ η n [ G ( Y n )] can be lower-bounded using the same Lyapunov function.
This insight is encoded in the compatibility criterion Assumption 2-C2, and we can use it to prove the following a priori lower bound on η n ( G ) .
Lemma 4.
Suppose that Assumption 1-Q1 and Assumption 2-C2 hold, and let η n = η n − 1 Q with η 0 ( U ) < ∞ .
Then we have the lower bound
[Equation]
where R ∗ is ﬁxed and doesn’t depend on n .
This lemma essentially says that G should be bounded away from zero on the level sets of U .
This is not a strong condition — if G is bounded away from zero on compact sets and U is continuous, the lemma applies.
As an example, if G = exp( −(cid:107) x (cid:107) 2 / 2) and U ( x ) = c (cid:107) x (cid:107)
+ 1 then this result applies.
The constant R ∗ arises since R (cid:55)→ θ ( R ) is nonincreasing and R (cid:55)→ 1 − 1 R is increasing so we can optimize this bound as a function of R to get R ∗ (which may not be unique, but the value the bound attains will be).
Together with the next lemma, we can obtain the desired convergence from
Theorem 2 for the BG interaction.
Lemma 5.
Let η, η (cid:48) ∈ P
( R d ) , and suppose that Assumption 3 holds, i.e. (cid:107) G (cid:107) ∞ , (cid:107) G (cid:107) β < ∞ .
Then (cid:107) Ψ G ( η ) − Ψ G ( η (cid:48) ) (cid:107) tv,
[Equation]
≤
[Equation]
Clearly, we will use our knowledge that η n ∈ P m G ,M U ( R d ) ∀ n to make the Lipschitz constant in
Lemma 5 uniform over P m G ,M U .
Proposition 2.
Suppose that Assumption 1, 2, and Assumption 3-G1 hold.
Then the drift criterion (10) holds uniformly over all η ∈ P m,M ( R d ) .
Now we can apply Theorem 2 to obtain convergence.
Corollary 2.
Suppose that Assumptions 1, 2, 3 hold.
If η 0 ( G ) > 0 , η 0 ( U ) , µ 0 ( V ) < ∞ , then
Theorem 2 holds for K BG , i.e. the ﬂow µ n converges to π in V β -total variation as long as ρ = (1 − ε ) γ < 1 .
Proof.
Lemma 4 and Lemma 5 imply the regularity condition (9) for constants determined by those lemmas and by noting that, due to Asssumption 2-C1, η ( V β ) ≤ η ( U r ∗ ) ≤ η ( U ) r ∗ < ∞ by Jensen’s inequality.
Additionally, Proposition 2 implies the condition (10).
Hence, since we are in “Case 1” of
Theorem 2, the result follows.
G.1.2 Proofs
Lemma 4.
Suppose that Assumption 1-Q1 and Assumption 2-C2 hold, and let η n = η n − 1 Q with η 0 ( U ) < ∞ .
Then we have the lower bound
[Equation]
where R ∗ is ﬁxed and doesn’t depend on n .
Proof.
[Equation]
But by Markov’s inequality
[Equation]
so
[Equation]
and hence
[Equation]
Now
[Equation]
and iterating this procedure gives
[Equation]
where we have used the sum of the geometric series to obtain c/ (1 − ξ ) .
Now, θ ( R ) is nonincreasing w.r.t.
R and 1 − 1 /R is increasing w.r.t.
R , so optimizing to get R ∗ and integrating we obtain
[Equation]
Lemma 5.
Let η, η (cid:48) ∈ P
( R d ) , and suppose that Assumption 3 holds, i.e. (cid:107) G (cid:107) ∞ , (cid:107) G (cid:107) β < ∞ .
Then
[Equation]
Proof.
Let (cid:107) f (cid:107) β ≤ 1 .
then |
[Equation]
[Equation]
[Equation]
x
[Equation]
d
[Equation]
ˆ
[Equation]
d
[Equation]
since
[Equation]
so putting these together
[Equation]
Using symmetry completes the proof
[Equation]
Proposition 2.
Suppose that Assumption 1, 2, and Assumption 3-G1 hold.
Then the drift criterion (10) holds uniformly over all η ∈ P m,M ( R d ) .
Proof.
Let η ∈ P m,M ( R d ) and consider
[Equation]
Theorem 4 ([47] Thm 8.7.1 pp.283) .
Suppose that G has bounded oscillations, let N ≥ q ≥ 1 , and
Y =
( Y 1 , . . .
Y N ) , Y i ∼ η .
Then for any f ∈ B b (( R d ) q ) with osc( f ) ≤ 1 , we have
[Equation]
where
[Equation]
In particular, by picking G ≡ 1 , we obtain
[Equation]
Corollary 3.
Suppose that Assumption 3 G1 holds (i.e. G is bounded) the compatibility criterion
Assumption 2-C2 with η n ( G ) ≥ m .
Then K BG • satisﬁes the uniform propagation of chaos in Casewith
[Equation]
Proof.
This follows from Theorem 3 and Theorem 4 above, noting 1) that (cid:107) G (cid:107) ∞ implies osc( G ) < ∞ , and 2) that we can obtain a bound on R G,η n independent of η n by applying Lemma 4 to obtain η n ( G ) ≥ m so
[Equation]
G.2 Analysis of K ARη G.2.1 Statement of Results
First, the result showing that π is K ARη (cid:63) -invariant.
Proposition 3.
K η (cid:63) is π -invariant.
Proof.
We have
[Equation]
since the integrals with f ( y ) and f ( x ) in the ﬁrst term are equal.
We will establish equvalent results from Section G.1but for K AR • .
This will not require assumption 2C2 since the interaction is well-deﬁned for η ∈ P ( R d ) as α ( x, y ) is in fact bounded .
K AR is the main subject of study for [1], and in fact the regularity and uniform drift conditions were established there.
Note that the statement η ∈ P 0 , ∞ is vaccuous since G ( x ) > 0 =
⇒ η
( G ) > 0 , and the second says that η ( V ) < ∞ .
Lemma 6 ([1]) .
Let η, η (cid:48) ∈ P 0 , ∞ ( R d ) .
Then
[Equation]
Lemma 7.
Suppose that Assumption 1, 2-C2, hold.
Then the uniform drift criterion (10) holds for η ∈ P 0 ,M ( R d ) .
Proposition 4.
Let J =
J AR , and Y =
{ Y 1 , . . .
, Y N }
where Y i iid ∼ η .
Then for any f ∈ B b (( R d ) q with osc( f ) ≤ 1 and x ∈ ( R d ) q , we have
[Equation]
Corollary 4.
Suppose that Assumption 1, 2-C2 hold.
If η 0 ( U ) , µ 0 ( V ) < ∞ , then Theorem 2 holds for
K BG , i.e. the ﬂow µ n converges to π in V β -total variation as long as ρ =
(1 − ε ) γ + ε (cid:107) J
η
V
β
(cid:107) < 1 .
G.2.2 Proofs
Lemma 7.
Suppose that Assumption 1, 2-C2, hold.
Then the uniform drift criterion (10) holds for η ∈ P 0 ,M ( R d ) .
Proof.
Let η ∈ P 0 ,M ( R d ) and consider
[Equation]
Proposition 4.
Let J =
J AR [Equation] and Y =
{ Y 1 , . . .
, Y N }
where Y i iid ∼ η .
Then for any f ∈ B b (( R d ) q with osc( f ) ≤ 1 and x ∈ ( R d ) q , we have
[Equation]
Proof.
Starting with the ﬁrst term, for each ﬁxed x
[Equation]
Now
[Equation]
so automatically osc( ϕ f ( x, • )) ≤ 2 .
Hence using Lemma[-
-]
[Equation]
Corollary 5.
If (cid:15) ( K ) < 1 , then K AR • satisﬁes the uniform propagation of chaos in Case 2 of
Theorem 3 with R ≡ 1 .