Nonlinear MCMC for Bayesian Machine Learning
Anonymous Author(s)
AfﬁliationAddress email
Abstract
We explore the application of a nonlinear MCMC technique ﬁrst introduced in [1]to problems in Bayesian machine learning.
We provide a convergence guarantee intotal variation that uses novel results for long-time convergence and large-particle(“propagation of chaos”) convergence.
We apply this nonlinear MCMC techniqueto sampling problems including a Bayesian neural network on CIFAR10.
Introduction
Characterizing uncertainty is a fundamental problem in machine learning.
It is often desirable for amachine learning model to provide a prediction and a measure of how “certain” the model is aboutthat prediction.
Having access to a robust measure of uncertainty becomes particularly importantin real-world, high risk scenarios such as self-driving cars [2–4], medical diagnosis [5, 6], andclassifying harmful text [7].
However, despite the need for uncertainty in machine learning predictions, it is well known thattraditional ML training, i.e. based on optimizing an objective function, frequently does not providerobust uncertainty measures [8], yielding overconﬁdent predictions for popular neural networks suchas ResNets [9].
An appealing alternative to the traditional optimization paradigm for ML is the
Bayesian probabilistic framework, due to its relatively simple formulation and extensive theoreticalgrounding; see for example [10].
In the probabilistic perspective of machine learning, one combines a prior P ( θ ) over the parameterspace θ ∈ Θ and a likelihood of the data given model parameters P ( D| θ ) via Bayes’ rule to obtaina posterior over the parameters P ( θ |D )
∝ P ( D| θ ) P ( θ ) .
The “traditional” approach in machinelearning is to optimize the posterior (or the likelihood) to obtain θ ∗ ∈ arg max P ( θ |D ) and generatepredictions via P ( y | x, D ) =
P ( y | x, θ ∗ ) .
However, if we adopt the Bayesian approach, the posteriorcharacterizes the uncertainty about the parameters of the model (i.e. epistemic uncertainty), whichcan propagate to uncertainty about a prediction by integration : P ( y | x, D
[Equation]
=
[Equation]
This paper studies the problem of how to approximate this integration with samples from P ( θ |D ) .
1.1 Contributions
• Our main contribution is the novel analysis of a modiﬁcation of the general nonlinear Markov
Chain Monte Carlo (MCMC) sampling method from [1] to obtain quantitative convergenceguarantees in both the number of iterations and the number of samples.
• We apply the general results from above to determine the convergence of two speciﬁcnonlinear MCMC samplers.
• In experiments, we compare these nonlinear MCMC samplers to their linear counterparts,
and ﬁnd that nonlinear MCMC provides additional ﬂexibility in designing sampling
algo-
rithms with as good, or better, performance than the linear variety.
1.2 Background
Bayesian ML & MCMC.
In Bayesian machine learning, the “computationally difﬁcult” step isintegration since the integral ´ P ( y | x, θ ) P (d θ |D ) is not analytically solvable except for rare cases.
In practice, one typically uses a Monte Carlo approximation such as
[Equation]
where θ i iid ∼ P ( θ |D ) .
where the expected error in this approximation is well known to converge to zero like O (1 / √ N )by the Central Limit Theorem (CLT).
There are various approaches to sampling from P ( θ |D ) , but
Markov chain Monte Carlo (MCMC) is perhaps the most widely used.
The basic idea of MCMC isto use a Markov transition kernel T with stationary measure P ( θ |D ) , and simulate a Markov chainθ n +1 ∼ T ( θ n , • ) that converges rapidly to this stationary measure.
Therefore we can estimate
[Equation]
where n sim is some large number of simulation steps and { θ in } ∞ n =0 are independent Markov chainsgoverned by T .
The basic problem is then to design an efﬁcient transition kernel T .
There is a vast body of literaturestudying various choices of T ; some well known choices are the Metropolis-Hastings algorithm[11, 12], the Gibbs sampler [13], the Langevin algorithm [14–17], Metropolis-Adjusted Langevin[18, 19], and Hamiltonian Monte Carlo [20–24].
However there are various challenges in Bayesian ML that make applying these samplers difﬁcultin practice.
One challenge is that the posterior P ( θ |D ) can be highly multimodal [25, 26], whichmakes it difﬁcult to ensure that the Markov chain θ n explores all modes of the target distribution.
One can combat this issue by employing auxiliary samplers that explore a more “tractable” variationof P ( θ |D ) [27, 28].
Other methods that empirically improve posterior sampling quality includetempering [29–32], RMSProp-style preconditioning [33] or adaptive MCMC algorithms [34, 35]such as the popular No U-Turn Sampler [36].
Nonlinear MCMC.
Another class of powerful MCMC algorithms, which is less-studied in thecontext of Bayesian ML, arises from allowing the transition kernel T to depend on the distributionof the Markov chain as in θ n +1 ∼ T Distribution( θ n ) ( θ n , • ) .
This approach gives rise to so-callednonlinear MCMC since { θ n } is no longer a true Markov chain.
Nonlinear Markov theory is a richarea of research [37–42] and, in discrete-time, has strong connections to nonlinear ﬁltering problems[43, 44], sequential Monte Carlo [45, 46], and nonlinear Feynman-Kac models [47].
One canreplace Distribution( θ n ) , which is often intractable, with an empirical estimate Distribution( θ n ) ≈1 N (cid:80) N i =1 δ θ in to obtain interacting particle MCMC (iMCMC, or iPMCMC) methods; see for example[48, 49, 1, 50, 51].
Our view is that nonlinear MCMC offers some appealing features that traditional linear MCMClack.
One such feature is the ability to leverage global information about the state space containedin Distribution( θ in ) to improve exploration of the state space Θ — a central issue in Bayesian ML.
Another feature is the increased ﬂexibility of nonlinear MCMC algorithms, which can be leveragedto correct biases that are introduced by other design decisions in MCMC for Bayesian ML such astempering.
These features will be explored empirically in Section 4.
However, the theoretical analysis of nonlinear Markov MCMC presents an added difﬁculty in that theparticles { θ 1 n , . .
, θ Nn } of an interacting particle system are now statistically dependent .
This meansthat, in addition to studying the large-time behaviour which is classical in MCMC [52, 53], one muststudy the large-particle behaviour separately to obtain Monte Carlo estimates since the CLT does notapply.
One such large-particle behaviour is the propagation of chaos [41], which studies the tendencyof interacting particles to become independent as the number of particles, N , increases; see [41].
Wewill need both of these elements — long-time and propagation of chaos behaviour — to properlycharacterize the convergence of nonlinear MCMC.
Other Sampling Methods.
Finally, let us mention that MCMC is certainly not the only way toobtain Monte Carlo sample estimates in Bayesian ML; some popular examples include MC dropout[8], black-box variational inference [54], and normalizing ﬂows [55, 56].
1.3 Common Notation
Let P ( R d ) be the set of probability measures on the meaurable space ( R d , B ( R d )) .
If µ ∈ P ( R d )and f ∈ B b ( R d ) :=
{ f : R d → R | f is bounded } , we will denote µ ( f ) :
=
´ f d µ .
If K :
R d × B ( R d ) → [0 , 1] is a Markov kernel 2 then we will denote Kf ( x ) :
= ´ f ( y ) K
( x, d y ) andµK (d y ) :
=
´ µ (d x ) K ( x, d y ) .
Finally, for y := { y 1 , . .
, y N } ⊂ R d , we will denote the empiricalmeasure of y as m ( y ) :
= 1 N (cid:80) Ni =
δ y i .
Nonlinear MCMC
In this section, we outline the family of MCMC algorithms that will be used in rest of the work.
Wewill use general notation for simplicity but this difference is merely cosmetic; the “target distribution”π in this section corresponds directly to P ( θ |D ) from the previous section.
2.1 Nonlinear Jump Interaction Markov Kernels
To specify a MCMC algorithm, we must specify the Markov transition kernel.
The family of nonlinear
Markov kernels that we will be studying was introduced in [1] and is a mixture of a linear kernel,denoted K , and a nonlinear jump-interaction kernel J η indexed by a probability measure η , to obtain
[Equation]
where ε ∈ ]0 , 1[ is the mixture hyperparameter.
The Markov kernel K η will be the main object ofinterest throughout this paper.
We will give speciﬁc examples of J η in Section 2.2.
Furthermore, foran expanded description of how our setup differs from [1], see Appendix A.1.
Mean Field System.
Now we show how the kernel K η can be used to construct a Markov chain.
Following [1], we use an auxiliary Markov chain { Y n } with transition kernel Q on the same statespace as K η (i.e. R d ) to obtain the nonlinear Markov chain { ( Y n , X n ) } ∞ n =0 deﬁned by
[Equation]
where µ 0 , η 0 are the initial distributions and ∼ denotes “sample from”.
One should interpret thisas a sequence of steps where ﬁrst we sample the auxiliary state Y n +1 from Q , then we obtain thedistribution of Y n +1 denoted η n +1 , and we use this distribution to index the primary kernel K η n +1 105 and obtain a sample X n +1 .
We sample X n +1 with probability (1 − ε ) from the linear kernel K , andwith probability ε it will “jump” according to J η n +1 ( X n , • ) .
Because the Markov dynamics dependon Distribution( Y n ) , we call this a “mean ﬁeld system”.
Interacting Particle System.
One issue with the mean ﬁeld system (2) is the fact that computing
Distribution( Y n +1 ) is generally impossible except in special cases.
Hence, to get a viable simulationalgorithm, we must approximate Distribution( Y n ) , and we do this by replacing Distribution( Y n )with its empirical measure estimated from a set of N samples Y n :=
{ Y 1 n , . . .
, Y Nn } as follows:
[Equation]
2.2 Application to MCMC
Now we detail how to apply K η and the Markov chains (2), and (3) to MCMC.
In particular, we mustunderstand how to choose Q, K, J η such that K η will be invariant w.r.t.
a target distribution π .
As is usually the case in probabilistic inference problems, we will assume that the target distributionπ is known only up to a normalizing constant and that it has a density, also denoted π .
We also makethe simplifying assumption that Q has an invariant measure η (cid:63) (also with density denoted η (cid:63) )
i.e.η (cid:63) Q = η (cid:63) .
This is not burdensome; in practice we can, and will, obtain Q from a linear MCMCalgorithm for some choice of η (cid:63) .
In fact, being able to choose η (cid:63) is a powerful design parameter ofour methods as we will see in Section 4.
We will also assume that the linear kernel K is π -invariant,i.e. πK = π .
To see how we can ensure that π is K η -invariant, consider the fact that we will design Q s.t.η n :=
Distribution( Y n ) converges to η (cid:63) .
This means we will eventually be sampling from the kernel
K η (cid:63) and we already have π -invariance of K .
Therefore, if we arrange for J η (cid:63) to be π -invariant, πwill be invariant for K η (cid:63) since
[Equation]
Intuitively, if the auxiliary chain converges to a steady state and the jumps in that steady state preserveπ (and K preserves π ), then so will K η (cid:63) .
Now the remaining task is to design nonlinear interactionkernels J η that will yield good performance.
We detail two choices below.
Boltzmann-Gibbs Interaction.
The ﬁrst choice of J η we will investigate, from [1], relies on the
Boltzmann-Gibbs transformation [47], which we now explain.
Let G : R d → ]0 , ∞ [ be a potentialfunction; then the Boltzmann-Gibbs (BG) transformation is a nonlinear mapping Ψ G : P ( R d ) →
P ( R d ) deﬁned byΨ G ( µ )(d x ) :
= G ( x ) µ ( G ) µ (d x ) or equivalently
[Equation]
µ [Equation] d
[Equation]
for any f ∈ B b ( R d ) and whenever µ ( G ) (cid:54) = 0 .
This transformation has many interesting propertiesand been extensively studied in [47] and related works.
To use the BG transformation in MCMC, we will assume that the densities π and η (cid:63) are positive 3 andmake the choice that G ( x ) will be the function
[Equation]
With this choice, we get an interaction kernel J BGη ( x, d y ) :
= Ψ G ( η )(d y ) .
We can easily see thatη (cid:63) ( G ) =
ˆ π η (cid:63) d η (cid:63) =
ˆ d π = 1 and Ψ G ( η (cid:63) )
(d x ) = G ( x ) η (cid:63) ( G ) η
(cid:63) (d x ) =
π ( x ) η (cid:63) ( x ) η (cid:63) (d x ) =
π (d x ) , i.e. Ψ G is the multiplicative “change of measure” from η (cid:63) →
π .
Hence the ﬁrst nonlinear Markovkernel we will investigate is
[Equation]
From the remarks above, is clear that π is K η (cid:63) -invariant.
Accept-Reject Interaction.
The second choice of jump interaction we will study, also introducedin [1], is a type of accept-reject interaction related to the Metropolis-Hastings algorithm.
For thesame choice of potential function G in (4), we can deﬁne the acceptance ratio
[Equation]
1 ∧ G ( y ) {+G ( x ) = 
+}
∧ π ( y ) η (cid:63) ( x ) η (cid:63) ( y ) π ( x ) and the quantity A η ( x ) :
= ˆ α ( x, y ) η (d y ) for η ∈ P ( R d ) .
Hence we can deﬁne the accept-reject interaction kernel as
[Equation]
We can interpret this jump interaction as: starting in state x , we jump to a new state distributedaccording to η (d y ) with probability α
( x, y ) (i.e. accept the proposed jump) and remain the incurrent state with probability 1 − A η ( x ) (i.e. reject the proposed jump).
This is a form of “adaptive
Metropolis-Hastings” in which the proposal distribution evolves over time as the distribution of theauxiliary Markov chain.
Hence we obtain the accept-reject nonlinear jump interaction kernel
[Equation]
We note that π is also J ARη (cid:63) -invariant; see Proposition 3 in Appendix G for a simple calculation.
Given f ∈ B b ( R d ) , we can also write this as J ARη f ( x ) = ´
[ f ( y ) − f ( x )]
α ( x, y ) η
(d y )
+ f ( x )
Simulation.
Let us note brieﬂy that using both K BGη and K ARη in (3) produce interacting particlesystems that can, and will, be simulated.
The simulation is relatively straightforward, see Appendix Afor pseudocode implementing the nonlinear MCMC algorithms we have now constructed.
Convergence Analysis
We will now study whether the nonlinear MCMC algorithms based on K η from Section 2 — i.e.,the interacting particle system (3) with the restrictions on K, Q, J η from Section 2.2 — will actuallyconverge to the target distribution π .
In other words, we would like to estimate (cid:107) µ Nn − π (cid:107) for somesuitable notion of distance on P ( R d ) , where µ Nn :=
Distribution( X 1 n ) is the distribution of a singleparticle (it doesn’t matter which particle, as the X in are exchangeable ).
The nonlinear nature of K η makes the analysis more difﬁcult than for a linear MCMC method.
Webreak the problem into two parts: one studying the convergence of the mean-ﬁeld system (2) as thenumber of steps n → ∞ , and one studying the convergence of the interacting particle system (3) tothe mean ﬁeld system as the number of particles N → ∞ .
This will allow us to apply the triangleinequality as follows:
[Equation]
where µ n :=
Distribution( X n ) is the distribution of the mean-ﬁeld system.
Crucially, our analysisof the large-particle limit is uniform in the number of steps n , which will allow us to establish boundsabove that hold as n → ∞ .
The actual result is contained in Theorem 1.
While our analysis does not rely on heavy mathematical machinery, to state the full set of conditionsand results for long-time and large-particle convergence — each of which is a substantial result in itsown right — would occupy too much space in the main text.
Instead, we will state the main result,which is essentially a corollary of the long-time and large-particle analyses Theorems 2 and 3 in
Appendices E and F respectively, and below we will sketch the general arguments used in thoseappendices.
The proofs of the main results are in Appendix F.2.
Note that our analysis, and the resultswe obtain, are novel and not found in [1]; see Appendix D for an elaboration.
The following is stated in terms of the total variation metric, deﬁned here for µ, ν ∈ P ( R d ) as(cid:107) µ − ν (cid:107) tv :=
sup (cid:107) f (cid:107) ∞ ≤ 1 | µ ( f ) − ν ( f ) | where (cid:107) • (cid:107) ∞ is the sup-norm on B b ( R d ) .
Theorem 1.
[Convergence of Nonlinear MCMC] Under suitable conditions on K η and Q , thereexist ﬁxed constants C 1 , C 2 , C 3 > 0 , a function R : [0 , ∞ [ → [1 , ∞ [ , and ρ > 0 s.t.
[Equation]
This result shows that, to control the approximation error (cid:107) µ Nn − π (cid:107) tv , it does not necessarily sufﬁceto run the MCMC algorithm for a large number of steps n , since if n → ∞ but N < ∞ then(cid:107) µ Nn − π (cid:107) (cid:54)→ 0 .
However, this approximation cannot lead to arbitrarily bad results: Theorem 1provides a quantitative upper bound on how much the MCMC algorithm can be biased.
Note also thatthis result uses total variation, which is a strong metric that represents a worst-case over all boundedfunctions f (up to rescaling by (cid:107) f (cid:107) ∞ ).
It is entirely possible that, for many choices of practical f ,the approximation will be better as we will see empirically.
A corollary of Theorem 1 is that that we regain a Monte Carlo estimate for the interacting particlesystem.
This result is essentially due to [41] Theorem 2.2.
Corollary 1.
[Adapted from [41], Theorem 2.2] Suppose that Theorem 1 applies to K η .
Let X n := { X 1 n , . . .
, X Nn } be from (3) .
Then for every n ∈ N and f ∈ B b ( R d ) we have
[Equation]
This corollary directly relates to the application of Bayesian ML we are interested in, where we wouldhave f ( θ )
=
P ( y | x, θ ) .
3.1 Long-Time Bounds
There are two main ingredients in the general result on long-time convergence: ergodicity of K and
Q , and Lipschitz regularity of the interaction kernel η (cid:55)→ J η .
These, along with other technicalconditions, produce an estimate of the form (cid:107) µ n − π (cid:107) ≤ C 2 ρ n + C 3 nρ n where (cid:107) • (cid:107) is a weightedtotal variation norm.
The full statement is in Theorem 2 of Appendix E.
Ergodicity of K and Q .
A fundamental requirement of our results is that the linear building blocksof K η must converge to their respective stationary measures in an appropriate metric.
This type ofresult is now standard in the Markov chain literature, and we use a result from [57] for K and aresult from [1] for Q .
The former is actually able to ensure that K is a contraction on P ( R d ) w.r.t.
asuitable weighted total variation; we use this feature repeatedly in our analysis.
Lipschitz Regularity of J η .
We also need that η (cid:55)→ J η is Lipschitz -continuous w.r.t.
a weightedtotal vartiation norm on Markov kernels (the Lipschitz constant does not have to be < 1 ).
Thisregularity is used to translate the convergence of η n → η (cid:63) (as guaranteed by the ergodicity of Q ) intoconvergence of J η → J η (cid:63) with the Lipschitz estimate (cid:107) J η n − J η (cid:63) (cid:107) (cid:46) (cid:107) η n − η (cid:63) (cid:107) .
In Appendix G,we verify this analytically for J BGη and J ARη , see Lemma 5 and [1] Proposition 5.3.
3.2 Large-Particle Bounds
To study the large-particle behaviour, we would like to measure how close a group of q ∈ { 1 , . . .
, N }interacting particles { X 1 n , . . .
X qn } ⊂ { X 1 n , . .
, X Nn } =: X n from (3) is to being i.i.d.
accordingto the mean-ﬁeld measure µ n .
This analysis was pioneered in [41] under the name “propagation ofchaos”, and formalizes the intuition that, as N → ∞ , the inﬂuence of any individual particle → 0 .
To state this precisely, ﬁrst note that if we had random variables Z i iid ∼ η ∈ P ( R d ) then the jointdistribution of Z :
=
{ Z 1 , . . .
, Z N } would be η ⊗ N .
Hence, for the interacting particles X n attime n , we expect the distribution of { X 1 n , . . .
, X qn } , denoted µ q,Nn , to get closer to the distributionof i.i.d.
mean ﬁeld particles from (2), denoted µ ⊗ qn , as N → ∞ .
In other words, we expect(cid:107) µ q,Nn − µ ⊗ qn (cid:107) tv → 0 as N → ∞ .
The full statement of this result is Theorem 3 of Appendix F.
The main condition in our propagation of chaos result is another type of regularity for η (cid:55)→ J η whichbasically requires that, if one approximates a distribution η ∈ P ( R d ) by its empirical measure m ( Y )
where Y :=
{ Y 1 , . . .
Y N } and Y
i iid ∼ η , then J m ( Y ) → J η as N → ∞ .
More precisely, thereshould be a nondecreasing function R : [0 , ∞
→ [1 , ∞ [ s.t.| E [ J ⊗ qm ( Y ) f ( x )] − J ⊗ q η f ( x ) | (cid:46) q 2 N R ( q 2 /N )
∀ x ∈ R d and f ∈ B b ( R d ) with “oscillations” osc( f ) ≤ 1 .
The expectation is taken over η , and “oscillations” are deﬁned precisely in Appendix D. This inequalityis analogous to the Lipschitz regularity from above since we can alternately write (cid:107) µ − ν (cid:107) tv =sup {| µ ( f ) − ν ( f ) | | f ∈ B b ( R d ) , osc( f ) ≤ 1 } [47].
3.3 Analysis of Speciﬁc Interaction Kernels
The main result Theorem 1 is in terms of conditions on a general K η (i.e. general choices of K, Q, J η ).
To apply this result to the samplers in Section 2, we must establish if these conditions hold for K BGηand K ARη .
Fortunately, this can be done analytically; in Appendix G, we present conditions underwhich the Lipschitz regularity (see Lemma 5 and [1] Proposition 5.3) and large-particle regularity(see Corollaries 3 and 5) hold.
These results, particularly for K BGη , are interesting and rely on noveltechniques for controlling the various quantities, sometimes improving over previous methods.
Dueto space constraints, the results for K BGη and K ARη are in Corollaries 2 and 4 from Appendix G.
Experiments
In this section, we detail two experiments designed to explore how one might apply the nonlinear
MCMC methods developed in the previous sections to Bayesian machine learning.
Let us state explicitly that our aim is not to achieve state-of-the-art with these experiments, nor do we claim thatthis method will necessarily lead to state-of-the-art results on a particular task.
Rather, the aims ofthese experiments are: to demonstrate that nonlinear MCMC can be applied successfully to large-scaleproblems; compare linear vs nonlinear methods to understand what beneﬁts and drawbacks nonlinear
MCMC offers compared to linear MCMC; and to develop some recipes for choosing the varioushyperparameters and samplers that determine a nonlinear MCMC method.
4.1 Two-Dimensional Toy Experiments
We use a toy setting of two-dimensional distributions to compare the relative beneﬁts of linear andnonlinear MCMC.
A beneﬁt of this low-dimensional setting is that the multimodal toy distributionscan be exactly sampled.
This gives us the opportunity to quantitatively measure the quality of oursamples via an unbiased estimator of the Maximum Mean Discrepancy (MMD) metric on P ( R d )[58].
This approach stands in contrast to many previous works, which use very simple distributions(e.g.
Gaussians) with analytically tractable statistics to measure quality.
See Appendix C.1 for a briefoverview of our methodology.
Setup.
Our setup will compare the Metropolis Adjusted Langevin Algorithm (MALA) [18] (see
Appendix B for an overview of MALA) with the nonlinear BG and AR samplers using MALA for thekernels K, Q in our nonlinear setup from Section 2.
This will allow us to examine the effects of thenonlinearity in K ARη and K BGη while controlling for the type of sampler used and its hyperparameters.
The main difference between the linear and nonlinear algorithms, aside from the interaction itself, isthe extra “design knob” to change in the form of the choice of the auxiliary density η (cid:63) .
We show howone can use η (cid:63) to incorporate additional insight to guide the sampling, such as regions of the statespace to explore.
In our experiments, we choose η (cid:63) to be a centered, 2-dimensional Gaussian with alarge standard deviation ( Σ = 4 I 2 for the circular MoG and two-rings densities, and Σ =
I 2 forthe grid MoG).
This conveys “coarse-grained” information of roughly where the support of the targetdensity is located – in this case, a neighbourhood of (0 , 0) .
See Table 2 in Appendix C.1 for a fullaccount of our experimental settings.
Results.
From Figure 1, we see that having a simple auxiliary density with good coverage of thesupport of the target distribution is quite helpful.
In all three examples, one or both of the nonlinearsamplers outperformed the equivalent linear sampler in the empirical MMD metric.
For the two mostchallenging densities, the “two rings” and “grid MoG” distributions, the improved exploration isevident, particularly for the BG interaction.
However, the BG interaction also displays a substantiallyhigher variance and can be less stable over time than the AR interaction.
See Appendix C.1.3 for ananalysis of the runtime of the algorithms.
4.2 CIFAR
4.2.1 Setup
To examine the properties of the nonlinear sampler outside of a toy setting, we have also implementeda Bayesian neural network on the CIFAR10 dataset.
We use a likelihood P ( y | x, θ ) parameterized bya ResNet-18 convolutional neural network [61] and a Gaussian prior P ( θ ) on the parameters of thisneural network which are combined to form a posterior P ( θ |D train )
∝ P ( θ ) (cid:81) ( x i ,y i ) ∈D train P ( y i | x i , θ ) .
The goal is to sample θ i , i = 1 , . . .
, N from this posterior.
See Table 4 in Appendix C.2 for a fullaccount of the experimental settings.
To deal with the fact that this sampling problem is very high-dimensional ( d ≈ 11M) and |D train | islarge ( |D train | = 60 , 000) we use a variety of techniques:
We sample minibatches (cid:98) D of size 256 to obtain the surrogate target density P ( θ | (cid:98) D ) [17].
We use an RMSProp-like “preconditioned” Langevin algorithm, called RMS-Langevin or RMS-ULA, as in [33] for the auxiliary sampler Q ; see Appendix B for details on this sampler.
As shown in [ , this sampler is biased.
We use tempering, wherein we aim to sample from π or η ∝ P ( θ |D train ) 1 /τ where τ is asmall number.
This substantially improves mixing for hard-to-sample distributions such asP ( θ |D train ) at the cost of bias since we are no longer sampling from the true posterior [32].
[Figure]
Using our nonlinear algorithm presents a novel opportunity to correct the bias introduced by tempering.
For our experiments, we pick η (cid:63) ∝ P ( θ |D train ) 1 /τ and π = P ( θ |D train ) .
This means that the auxiliarychain Y n explores a tempered version of the target, whereas the target chain X n (in theory) exploresthe true target distribution.
This is a novel strategy that is made possible by being able to select η (cid:63)independently of the target π .
We study the case when π is tempered as well.
For our experiments, we use the RMS-Langevin sampler as the baseline, and we also use it for theauxiliary sampler Q .
For the target sampler, it is not possible to use the RMS-Langevin algorithmbecause the smoothed square-gradient estimate is incompatible with the discontinuities (i.e. jumps)introduced by the nonlinear interaction.
Instead, for the linear sampler K we use the unadjusted
Langevin algorithm, ULA, [16] (see Appendix B).
We also use the Boltzmann-Gibbs interactionsince it is nonreversible, whereas the accept-reject interaction is reversible and this is not efﬁcient forthese high-dimensional settings (this is the same reason for using the ULA instead of MALA).
Wewill investigate both test accuracy and calibration error [9] to assess performance.
The tempered results are using τ = 10 − 4 .
All results between linear and nonlinear are statistically signiﬁcantly different ( p (cid:28) 0 . 05 ).
See Appendix C.2 for an overview of expected calibration error.
We also compute the maximum calibration error in Appendix C.2.
[Table]
4.2.2 Results
Linear vs Nonlinear.
From Table 1, we see that the linear (RMS-Langevin) sampler has slightlyhigher, but comparable test accuracy to the nonlinear Boltzmann-Gibbs sampler.
This is likely because RMS-Langevin algorithm has better stability around the regions of high probability due to its adaptivestepsize scaling.
However, from Figure 2, we see that for both the tempered and non-tempered cases,the nonlinear interaction appears to beneﬁt during early exploration.
This is an expected and desired
[Figure]
property of these nonlinear samplers, which incorporate global information about the sampler state(in this case, the relative potential G of each auxiliary chain’s state) and are able to emphasize thosestates with higher probability.
However, the linear method is able to eventually explore the relevantregions of the state space, and the difference disappears.
See Appendix C.2.4 for a comparison linearvs nonlinear performance scaled by the number of gradient evaluations.
Tempering Vs Non-Tempering.
The linear MCMC sampler is always tempered in our experimentsso there should not be any statistically signiﬁcant difference in the linear case.
For the nonlinearsampler, the tempered version has slightly higher accuracy; this trend is also observed in [32].
On theother hand, the calibration errors are the same for both tempered and non-tempered variants.
This issomewhat surprising, given the aggressive tempering used, and one would expect that this reducesthe variance of the posterior estimate.
This observation can perhaps be explained by the fact that weare using N = 10 samples which may not be enough to accurately change the tempered auxiliarydistribution η (cid:63) into the non-tempered primary distribution π in the jump interaction.
Calibration.
Considering the expected calibration error (ECE) [9], in Table 1 we see that thenonlinear method has statistically signiﬁcantly lower ECE ( p (cid:28) 0 . 05) compared to the linear method.
We hypothesize that this is due to a tension between the RMS scaling of the gradient which improvesthe efﬁciency of each MCMC step but at the cost of bias, which may be measurable in the formof calibration error.
The Langevin algorithm is also biased, but is generally known to have goodconvergence properties [16] and much less is known about the RMS-Langevin variant.
By using ournonlinear setup, we are able to aggressively explore the auxiliary distribution without sacriﬁcingcalibration on the target distribution.
Conclusion
In this paper, we have studied the theoretical and empirical properties of nonlinear MCMC methods.
We obtain powerful theoretical results for the convergence of our MCMC methods, and we applythese methods to Bayesian neural networks.
The results on BNNs are comparable to, but not betterthan, the linear methods we studied.
We hypothesize that this is because more investigation intochoosing the best auxiliary density η (cid:63) is required; our choice is simplistic and may not be optimal.
This hypothesis is supported by the toy experiments, which show signiﬁcant improvement when η (cid:63) isable to incorporate some additional insight into the problem.
How to do this in high dimensions is anexciting direction for future research.
Broader Impact.
There are beneﬁts and drawbacks to the nonlinear MCMC methods we describe.
The beneﬁts are mainly that properly accounting for uncertainty in machine learning will lead tobetter real-world outcomes for high-value scenarios such as self-driving cars or medical imaging.
The drawbacks are that MCMC methods require O ( N ) storage and computations relative to the O (1)for deterministic methods; in fact, our nonlinear method would require 2 N resources compared with N for a linear method (see also Appendices C.1.3 and C.2.4).
If applied to a large swath of ML,this would mean a substantial increase in the energy consumption required for experimentation anddeployment, worsening an already substantial issue in the ﬁeld.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes]
(c) Did you discuss any potential negative societal impacts of your work?
[Yes]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results.
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] Appendix D
(b) Did you include complete proofs of all theoretical results?
[Yes] Appendix E, F, G
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes] Code is in the
supplementary materials.
Pseudocode is in Appendix A.
(b) Did you specify all the training details (e
g., data splits, hyperparameters,
how they
were chosen)?
[Yes] In Appendix C
c) Did you report error bars (e
g., with respect to the random seed after running experi-
ments multiple times)?
[Yes] In Appendix C
(d) Did you include the total amount of compute and the type of resources used (e g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] In Appendix A
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a ) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[N/A]
(c)
Did you
include
any
new
assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using /curating?
[N/A]
(e) Did
you discuss
whether
the data you are using/ curating contains personally
identiﬁable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A]
(b) Did you describe any potential participant risks with links to Institutional Review
Board (IRB) approvals , if applicable?
[N/A
](c) Did you include the estimated hourly wage paid
to
participants and the total amount
spent on participant compensation?
[N/A]
A Pseudocode & Numerical Implementation DetailsA.1 Simulation Algorithm
The method of simulating K η is where this work and [1] diverge: we will provide an algorithm thatuses a ﬁxed number of samples N ∈ N to simulate the IPS (3), whereas [1] investigates an algorithmin which the empirical measure is formed from all the past samples .
While the algorithm in [1] yields a true estimate of the target measure in the limit as n → ∞ (for
K AR • at least, see [1]), their MCMC algorithm’s storage complexity increases linearly with time.
This behaviour is not well-suited to large-scale software implementations, which typically favourﬁxed-size “batches” of computations.
Additionally, a practitioner cannot horizontally scale theiralgorithm to improve accuracy (e.g. by simply adding more samples) in a ﬁxed time frame.
By contrast, Algorithm 1 below and the ensuing analysis use a ﬁxed number of particles N throughoutthe lifetime of the algorithm.
We will see that this produces a biased estimate of the target measure π ,but that this bias can be controlled both in the number of particles, N , and the number of steps, n .
Inexchange, our algorithm can be efﬁciently implemented in vectorized computing frameworks, as wewill demonstrate in Section 4.
Additionally, let us note that the decision to use an auxiliary Markov chain is highly pragmatic.
It ispossible to develop “autonomous” nonlinear MCMC algorithms of the form (cid:101) µ n +1 = K (cid:101) µ n (see e.g.[47] Ch. 5) with strong theoretical guarantees but bad empirical performance.
A primary reason forthis is sample degeneracy : due to the properties of nonlinear interaction, in an autonomous nonlinear
Markov kernel, quite often a single particle X in will be given a large potential G ( X in ) which resultsin the jump interaction being concentrated on a single point.
From this point onwards, the algorithmwill be unable to generate enough diversity within its particles to correctly estimate the variance ofthe target measure.
Due to the close relationship of nonlinear MCMC and nonlinear ﬁltering, theseissues have been studied in many settings such as [62, 48] and using auxiliary dynamics is indeed acommon solution.
A.2 Efﬁcient Software Implementation
In Algorithm 1, we gave a high-level pseudocode implementation of the nonlinear sampler.
Inthis implementation, for the sake of notational clarity, we used for loops to sample the individual particles.
However, with modern single instruction multiple data (SIMD) computing frameworkssuch as GPU accelerators, this is highly inefﬁcient.
Hence one can, and should, paralellize all forloops except the outer “time” loop in Algorithm 1.
We have done this using the variety of powerful tools provided by the Jax library [63] within Python,such as vmap which allows for automatic vectorization, and seamless targeting of GPU accelerators.
This becomes particularly useful for automatic batch-wise and sample-wise vectorization in the caseof stochastic gradient MCMC while writing functions in-terms of single inputs and outputs.
Wehave also leveraged the library of linear MCMC algorithms provided by the < REDACTED FOR
ANONYMITY > library < REDACTED FOR ANONYMITY >.
All experiments were run on a single
Titan X GPU; the code can be found at < REDACTED FOR ANONYMITY >.
B Linear MCMC Sampling Algorithms
Markov chain Monte Carlo algorithms form an integral part of Bayesian inference.
In this section,we will review the basic MCMC algorithms used in this paper.
In each case, we assume a C 1 , strictlypositive target density π known only up to a normalizing constant.
B.1 Unadjusted Langevin Algorithm
Consider the (overdamped) continuous Langevin diffusion [64]
[Equation]
where B t is a standard Brownian motion.
This is a fundamental stochastic process with far reachingconsequences in many areas of math; in particular, the Langevin diffusion is a Markov process with πas a stationary measure [64].
This fact has long been exploited by the molecular dynamics literature[14].
We obtain the unadjusted Langevin algorithm, or ULA, by applying an Euler-Maruyamadiscretization to the SDE above yielding [64]
[Equation]
This is a popular and well-studied algorithm for MCMC, although it is known to be biased in thesense that the sationary measure of this Markov chain is not π [65].
However, this bias converges to0 as δ → 0 [16].
B.2 Metropolis-Adjusted Langevin Algorithm
The Metropolis-Adjusted Langevin algorithm (MALA) [18, 19], as all basic Metropolis-Hastings
MCMC methods, consists of two steps: a proposal step and an accept step.
Starting from state X n ,in the proposal step, a candidate next state is sampled according to the Langevin dynamics above(cid:101)
X n +1 =
X n + δ ∇ log π ( X n ) √ 2 δZ n .
We will write q ( y | x ) for the proposal distribution; in this caseq ( y | x ) =
N ( x + δ ∇ log π ( x ); 2 δ )( y ) .
However, unlike the ULA, MALA consists of a second stepwhich will accept or reject the propsal (cid:101) X n +1 with probability
[Equation]
In other words
[Equation]
B.3 RMS-Unadjusted Langevin Algorithm
The ULA is a powerful technique for “black-box” MCMC in which one only has access to gradientinformation about the target density.
However, for very high dimensional problems in which πis highly anisotropic, it can be inefﬁcient to simulate an isotropic diffusion such as the Langevinalgorithm.
A simple and effective technique, borrowed from the optimization literature in which the same phenomenon can cause problems, is to precondition the dynamics as follows.
First we maintainan exponentially smoothed squared-gradient estimate r n deﬁned as
[Equation]
where the gradient squared is meant elementwise.
Then, as in RMSProp [66], we can construct anadaptive stepsize by dividing the square root of r n as follows(cid:98)
[Equation]
and then using this stepsize in the Langevin update
[Equation]
This has the effect of using gradient information to scale the stepsize of original Langevin diffusionindependently along each dimension, which likely reduces the negative impacts of anisotropy andsubstantially increases mixing time.
As studied in in [33], this algorithm, which we call RMS-ULA,is biased but this bias can be controlled with δ .
B.4 RMS-MALA.
Similar to the transition from ULA to MALA, we can “metropolize” the RMS-ULA algorithm tocorrect the bias.
This follows the same structure as the MALA, except that the proposal depends on
[Equation] [Equation]
B.5 Practical Considerations for Bayesian Neural Networks: Reversibility & Tempering.In
practical settings,
dimensional, anisotropic characteristics and limited computation budgetof Bayesian neural networks (BNNs) necessitate some modiﬁcations to the above algorithms.
The ﬁrst is that the Metropolis-Hastings (MH) step is rarely used except in simple settings.
Thisis because, while ensuring that that π is the invariant measure, the reversibility of the MH step istoo inefﬁcient when we cannot affort to reject many samples.
Because they cannot “backtrack”,nonreversible dynamics are generally much more efﬁcient than reversible dynamics [67, 68, 64, 69].
This has led to the recent interest in using piecewise deterministic Markov processes [70–73] andother nonreversible MCMC dynamics such as [74] for high dimensional Bayesian inference.
See[69], Section 3.1 for an interesting dicussion on nonreversibility and efﬁciency.
The second modiﬁcation is tempering.
A tempered version of a distribution π is π β ∝ π β .
This isa pragmatic solution to speed up exploration of a target distribution when the noise component of
Langevin-like algorithms causes slow or unstable dynamics.
In the limit as β → ∞ , sampling fromπ ∞ is equivalent to the maximum a posteriori estimate (this is simulated annealing [75]).
In practice,tempering can be achieved by simply scaling the noise by √ τ
[Equation]
which targets π β where β = 1 /τ .
7 See also [32] for a discussion of tempering in BNNs.
[Table]
[Table] [Table]
[Table]
C Experimental Details
No hyperparameter sweeps were used in the any of these experiments.
Hyperparameters were chosenbased on reasonable guesses and minimal manual tuning.
C.1 2d Toy ExperimentsC.1.1 Maximum Mean Discrepancy
We use the maximum mean discrepancy (MMD) metric [58] as a way to quantitatively evaluate our
MCMC algorithms.
The MMD metric is an integral probability metric of the form
[Equation]
where H is a reproducing kernel Hilbert space (RHKS) associated to a positive-deﬁnite kernelfunction
k : R d × R d → R ; see [58] for details.
For our purposes, the important features of the
MMD metric are that 1) it is similar to the total variation metric, which optimizes over the unit ball in
B b ( R d ) instead of H ; and 2) the MMD metric (cid:107) µ − ν (cid:107) MMD can be efﬁciently empirically estimatedfrom samples of µ, ν .
This is because, according to Lemma 6 in [58], we have
[Equation]
which depends only on the kernel function k , and an unbiased estimator of (cid:107) µ − ν (cid:107) 2 MMD is
[Equation] MMD ≈
[Equation]
N µ (cid:88)
i,j =1 , j (cid:54) =
i k
( X i , X j ) −
N µ N ν
N µ (cid:88) i =
N ν (cid:88)
j = k ( X i , Y j )
+
N ν ( N ν − 1)
N ν (cid:88) i,j =1 , j (cid:54) =
i k
( Y i , Y j ) where X i iid ∼ µ for i = 1 , . . . , N µ and Y i iid ∼ ν for i = 1 , . .
. , N ν .
If we treat µ as the distribution µ n of a (linear or nonlinear) MCMC algorithm at step n and ν as π ,then we can apply this estimator provided we can sample from π exactly.
This is not feasible forcomplex high-dimensional distributions, but it is possible for the distributions we have chosen, whichare all mixtures or transformations of Gaussains.
C.1.2 Experimental Setup
Refer to Table 2 for the experimental settings.
Additionally, we used 10 , 000 samples from π and thekernel
[Table]
, y ) =
exp( −(cid:107) x − y (cid:107) 2 ) + exp( − 2 (cid:107) x − y (cid:107)
[Table]
to estimate the MMD as in the previous section.
[Table]
η (cid:63)
N (0 , σ 2 I 2 ) with σ = 4 for circular
MoG and two rings and σ = 20 for grid
[Table]
N
[Table]
η
U ([ − 7 . 5 , 7 . 5] × [ − 7 . 5 , 7 . 5])
[Table]
η
U ([ − 7 .
[Table]
C.1.3 Runtime Analysis
We also report on the performance vs runtime of the linear and nonlinear algorithms measured interms of number of gradient executions and the wallclock time.
In general, the nonlinear methods will require two times the number of gradient executions as thelinear methods due to the requirement that we sample from the auxiliary chain in addition to theprimary chain.
In Figure 3, we show the MMD-squared metric plotted against the number of gradientevaluations.
Since we ran all algorithms for a ﬁxed number of steps, this amounts to using 2 × thegradient evaluations for the nonlinear methods, although not 2 × the steps for a given particle.
Despitethis, the nonlinear methods offer better performance per gradient evaluation.
[Figure]
In Table 3, we also report the wallclock time for each of the methods and each of the distributions.
We see that, despite using 2 × the gradient evaluations, the effective use of JAX and its vectorizationand jit-compilation features allows us to achieve sublinear wallclock time, at least for this operatingpoint of our GPU accelerator.
We omit the ﬁrst twenty iterations in our calculation since these can beslow while JAX compiles the program and we measure only the steady-state execution performance.
Tgt Density
Sampler
[Table]
deviation on 5 trials).
We measure the steady-state wall time by omitting the ﬁrst 20 iterations to account for the JAX jit-compilation time at the start of the program.
C.2 CIFAR10 ExperimentsC.2.1 Experimental Setup
In this experiment, we implement a Bayesian neural network for image classiﬁcation on the CIFAR10dataset [76].
The neural network architecture we use is a standard ResNet-18 architecture [61] usingthe default settings implemented by the DeepMind haiku library [77].
The likelihood function
P ( y | x, θ ) is the standard crossentropy on the logits y given an input image x
and parameters θ .
Wealso use standard data agumentation techniques including standardizing the images by the mean andvariance, random crops and random ﬂips.
See the Table 4 for the experimental parameters.
[Table]
C.2.2 Calibration Analysis
Calibration of a classiﬁer is a measure of how well that classiﬁer’s logits represent probabilities [9].
We use this as a necessary but not sufﬁcient test for correct sampling, since we know a priori thatthe Bayesian posterior will have much better calibration than an optimized version.
We will use themethodology in [9], particularly “expected calibration error” and “maximum calibration error” tostudy the calibration of our Bayesian neural network, which we describe below [Table]
To measure calibration, assume ﬁrst that we have a test set of data D test =
{ ( x 1 , y 1 ) , . .
, ( x N test , y N test ) } ⊂ X × { 0 , . . .
, C } where X is the input space and there are
C ∈ Z > 1 labels.
Suppose (cid:98) p : X → ∆ C − 1 is a supposed probability distribution representing p ( y | x ) .
We would expect that the probabilities output by (cid:98) p correlate with the accuracy of (cid:98) p , i.e. when (cid:98) p is α %certain, then α % of the time (cid:98) p is correct.
More formally, perfect calibration is when
P
[Equation]
y
[Equation]
=
[Equation] = [Equation] We can empirically estimate this relationship between predictied probability and true probability usinghistograms.
I.e. given predicted probabilities (cid:98) p i
= (cid:98) p ( y i | x i ) , we can form bins B k , k = 1 , . . .
, M on[0 , 1] , we can estimate the accuracy in the k -th bin as A k and the average probability in that bin as α k .
If (cid:98) p represents a true probability distribution, then A k ≈ α k for each k = 1 , . .
, M .
The expectedcalibration error (ECE) is then
[Equation]
We can also study the maximum calibrarion error
[Equation]
In Table 5, we report the ECE and MCE for the CIFAR10 experiment, where now
[Equation]
C.2.3 Distribution Shift
One useful outcome of using the probabilistic paradigm for ML is robustness to distribution.
Thisis well-documented [8, 9]; traditional ML models tend to be overconﬁdent even in settings where [Table] [Table] they have no hope of good predictions.
In short, traditional ML models lack the ability to say “Idon’t know” in a way that probabilistic ML models do not.
This is particularly salient in real-worldsystems such as self-driving cars, where the distribution of images is highly nonstationary and thecosts for overconﬁdent predictions are high.
In Figure 4 below, we show how the predictive entropy on in-domain examples, i.e. the CIFAR10 testset, and out-of-domain examples differs.
For the out-of-domain examples, we use the SVHN dataset[78] which also contains 32 × 32 RGB images, but this time of housing numbers (i.e. the digits 0-9)rather than objects such as airplanes or dogs.
The entropy (a measure of uncertainty) is calculated as
[Equation]
where p ∈ ∆ 9 is the vector of probabilities for a prediction p ( y | x, D train ) .
In the traditional ML case,the probabilities are made with p ( y | x, θ ∗ ) .
We average H ( p ) over the test data for the in-domain andout-of-domain examples.
The optimized method was trained to the same test accuracy ( 85% ) usingessentially the same hyperparameters (learning rate, etc).
C.2.4 Runtime Analysis
In Figure C.2.4, we plot the performance of the tempered target vs the number of gradient evaluations.
As described in Appendix C.1.3, the nonlinear algorithm uses 2 × the gradient computations.
Whenplotting the eval performance vs the number of computations, we see that the linear algorithm issigniﬁcantly more efﬁcient.
Further work into designing nonlinear samplers that achieve betterperformance (such as in the two-dimensional examples above) is required to make this tradeoffworthwhile in practice.
D Notation & Assumptions
The remaining sections will be devoted to proving the theoretical results contained in this paper.
Theyshould be read in sequence.
Note that, while some of the assumptions found below appear in [1], our analysis and the results weobtain are new.
In particular, [1] studies a different MCMC algorithm based on K η , see Appendix A for a discussion of algorithm differences.
Moreover, they obtain an asymptotic “strong law of largenumbers” result whereas we obtain nonasymptotic mean ﬁeld long-time convergence and uniformpropagation of chaos results, which are qualitatively different results using different proof techniques.
D.1 Notation and Deﬁnitions
D.1.1 Probability Spaces, Measures, and Kernels
We will be working on the measurable space ( R d , B ( R d )) with B ( R d ) the Borel σ -algebra 8 ; let
P ( R d ) denote the space of probability measures on B ( R d ) .
Let µ ∈ P ( R d ) , f : R d → R be ameasurable function, and, K : R d × B ( R d ) → [0 , 1] be a Markov kernel.
Throughout the sequel,we will write
[Equation]
An obvious consequence of this is the notation µK ( f )
=
´ f ( y ) µ (d x ) K ( x, d y ) .
We will alsoneed the following subset of P ( R d ) for our results: let G, U : R d → [0 , ∞ [ and ﬁx constantsm G , M U > 0 ; then we deﬁne
[Equation]
We will abbreviate P m G ,M U ( R d )
=
P m,M ( R d ) when there is no chance of confusion.
We will also frequently use the following notation for the empirical measure associated to y :
={ y 1 , . .
, y
[Equation]
[Equation]
with δ x ∈ P ( R d ) the Dirac measure.
Additionally, we will need the notion of tensor products forfunctions and measures: let q ∈ N , f i : R d → R and µ i ∈ P ( R d ) for i = 1 , . . .
, q .
Then forx =
( x 1 , . . .
, x q ) ∈ ( R d ) q and measurable g : ( R d ) q → R
[Equation]
When f = f 1 = · · · = f q and µ = µ 1 = · · · = µ q , we will write f ⊗ q and µ ⊗ q respectively.
Finally,we write µ (cid:28) ν to mean that ν dominates µ
and we write d µ d ν for the Radon Nikodym derivative.
Thenotation µ ∼ ν means that µ (cid:28) ν
and ν (cid:28) µ .
D.1.2 Norms
Denote by B b ( R d ) the set of bounded measurable functions f : R d → R .
We will be working with afamily of norms on functions, and their dual norms on probability measures, which is parameterizedby functions of the form U : R d → [1 , ∞ [ .
For such a U , we deﬁne
[Equation]
U ( x ) for f : R d → R .
The dual norm to (cid:107) f (cid:107) U corresponds to the weighted total variation distance; forµ, ν ∈ P ( R d ) we have
[Equation]
Note that we could replace the condition (cid:107) f (cid:107) U ≤ 1 by | f | ≤ U .
For V : R d → [0 , ∞ [ , we willoften work with V β ( x ) :
= 1 + βV ( x ) for β > 0 ; in this case, we will write (cid:107) f (cid:107) β := (cid:107) f (cid:107) V β and(cid:107) • (cid:107) tv,β := (cid:107) • (cid:107) tv,V β .
We will also need the deﬁnition of the maximum oscillation of f , which wedenote and deﬁne as osc( f ) :
= sup {| f ( x ) − f ( y ) | | x, y ∈ R d } .
For Markov kernels K, Q on R d , we obtain the weighted kernel distance
[Equation]
It is worth noting the special case U ≡ 1 which corresponds to the usual total variation distance; inthis case we will write
[Equation]
Lastly, we will use the notation (cid:15) ( K ) to denote the contraction coefﬁcient of a Markov kernel Kdeﬁned as (cid:15) ( K ) :
= inf { c ∈ [0 , 1] | (cid:107) µK − νK (cid:107) tv ≤ c (cid:107) µ − ν (cid:107) tv ∀ µ, ν ∈ P} see, e.g. [47] Ch 4.
D.2 Assumptions
Assumption 1 (Drift and Minorization) .
Let K, Q : R d × B ( R d ) → [0 , 1] be Markov kernels.
K1 K satisﬁes the drift criterion
[Equation]
for a ∈ ]0 , 1[ , b > 0 , and V : R d → [0 , ∞ [ with lim (cid:107) x
[Equation] V ( x ) = ∞ .
K2 K satisﬁes the following uniform minorization condition on the level sets of V : there existsγ ∈ ]0 , 1[ and ν ∈ P ( R d ) and R > 2 b/ (1 − a ) s.t.
[Equation]
Q1 Q satisﬁes the drift criterion
[Equation]
for ξ ∈ ]0 , 1[ , c > 0 , and U : R d → [1 , ∞ [ with lim (cid:107) x (cid:107)→∞ U ( x ) = ∞ .
Q2 Q satisﬁes the following uniform minorization condition on the level sets of U : there existsζ ∈ ]0 , 1[ and ν (cid:48) ∈ P ( R d ) and R (cid:48) > 2 c/ (1 − ξ ) s.t.
[Equation]
The requirements in Assumption 1 are from [57] and are by now fairly standard in the Markov chainliterature.
They will in particular imply the following properties for K and Q respectively:
Proposition 1 (Basic Properties of K, Q ; [57]&[1]) .1.
Suppose Assumptions 1-K1,K2 hold.
Then there exists γ ∈ ]0 , 1[ and β > 0 s.t. ∀ µ, ν ∈
P ( R d ) , we have
[Equation]
Suppose that Assumptions 1-Q1,Q2 hold.
Let η n = η 0 Q n for η 0 ∈ P ( R d ) , and r ∈ ]0 , 1] .
Ifη
( U r ) < ∞ , then for r ∈ ]0 , 1[ there are constants M ( r ) > 0 , δ ∈ ]0 , 1[ s.t.
[Equation]
Proof.1.
This is directly from [57].2.
This uses Lemma 1 which shows Assumption 1 Q1,Q2 implies the assumptions in
Lemma C.1 from [1].
Then we combine this with Lemma 2.
Both lemmas can be found in
Appendix E.
Throughout the remainder of this paper, β and γ will be ﬁxed and we will adopt the notation
[Equation]
Additionally, references to M ( r ) and δ are meant in the sense of the above proposition.
Next, weintroduce some criteria that ensure that the kernels K, Q are “compatible” in-terms of their driftcriteria from Assumption 1.
Assumption 2 (Compatibility) .
C1 There is r ∗ ∈ ]0 , 1] s.t.
V β ( x ) ≤ U ( x ) r ∗ ∀ x ∈ R
d
with V β beas above.
C2 G satisﬁes the lower bound compatibility criterion with U : for every R >
[Equation]
Assumption 2-C1 is also present in [1], and ensures that V and U are sufﬁciently “comparable”.
Assumption 2-C2 is a novel assumption that we will use to obtain an a priori lower bound on η n ( G ) .
Finally, some straightforward boundedness assumptions on G .
Assumption 3 (Assumptions on G ) .
G1 G is bounded, i.e. (cid:107) G (cid:107) ∞ < ∞ [-.G
-].
G2 G is bounded in the weighted supremum norm for V β ; i.e. (cid:107) G (cid:107) β < ∞ .
E Long-Time Convergence
This is the main long-time convergence result for the mean ﬁeld system.
We will revisit this theoremat the end of this section and provide a full proof.
We will often use the equivalent “distribution ﬂow”interpretation of system (2) deﬁned as
[Equation]
8) for our proofs.
Theorem 2.
Suppose that Assumption 1 and Assumption 2-C1 hold.
Suppose also that J η satisﬁes
[Equation]
(9) and that
[Equation]
(10) for (cid:101) a ∈ ]0 , 1[ , (cid:101) b > 0 and all η ∈ P m G ,M U for suitably chosen constants m G , M U .
Suppose δ is from(7) and set
Case 1: ρ := (1 − ε ) γ if J η ( x, d y ) doesn’t depend on x ; or
Case 2: ρ :=
(1 − ε ) γ + ε (cid:107) J η (cid:63) V (cid:107) V if J η ( x, d y ) does depend on x .
If µ 0 ( V ) , η 0 ( U ) < ∞ then there exists a constant C > 0 s.t.(cid:107)
µ n − π (cid:107) tv,β ≤ ρ n (cid:107) µ 0 − π (cid:107) tv,β + Cn max( ρ, δ ) n .
In particular, if ρ < 1 , we have lim n →∞ µ n = π in V
[Table]
Lemma 1.
Suppose that a Markov kernel P satisﬁes a drift condition
[Equation]
for V : R d → [0 , ∞ [ , V ( x ) → ∞ as (cid:107) x (cid:107) → ∞ , and the minorization condition with (cid:15) ∈ ]0 , 1[ , ν ∈
P ( R d ) , and C :=
{ x | V ( x ) ≤ R } s.t.inf x ∈C P [Equation] x, A ) ≥ (cid:15)ν ( A ) . holds for some R > 2 K/ (1 − a ) .
Then there exists a ∈ ] a, 1[ , b > 0 , ν ∈ P ( R d ) , S ∈ B [Equation] R [Equation]
s.t.
[Equation]
Proof.
This argument merely an elaboration on the remark from the end of [57].
Let a ∈ ] a, 1[ suchthat R > b/ ( a − a ) and S
= { x | V ( x ) ≤ R } .
If x / ∈ S then
[Equation]
If x ∈ S then PV ( x ) ≤ aV ( x ) + b ≤ aV ( x ) + b .
Hence the choices are clear from ﬁxing R asabove.
Lemma 2.
Let P be a Markov kernel with invariant measure π and suppose there are constantsρ ∈ ]0 , 1[ , C > 0 , and r ∈ ]0 , 1] s.t.
[Equation]
V r for any f : R d → R with (cid:107) f (cid:107) V r < ∞ .
Then if µ 0 ( V r ) < ∞ and µ n is the ﬂow of P then there is aconstant C (cid:48) >
0 s.t.
[Equation]
Proof.
Consider
[Equation]
Lemma 3.
Let P, Q be Markov kernels and suppose that P satisﬁes a drift condition
[Equation]
for V : R d → [1 , ∞ [ .
Then
[Equation]
V and
[Equation]
Proof.
For the ﬁrst part:
[Equation]
and
[Equation]
For the second part, let (cid:107) f (cid:107) V ≤ {+
+}
[Equation]
E.2 Proof of Theorem 2
Theorem 2.
Suppose that Assumption 1 and Assumption 2-C1 hold.
Suppose also that J η satisﬁes
[Equation]
(9) and that
[Equation]
(10) for (cid:101) a ∈ ]0 , 1[ , (cid:101) b > 0 and all η ∈ P m G ,M U for suitably chosen constants m G , M U .
Suppose δ is from(7) and set
Case 1: ρ := (1 − ε ) γ if J η ( x, d y ) doesn’t depend on x ; or
Case 2: ρ :=
(1 − ε ) γ + ε (cid:107) J η (cid:63) V (cid:107) V if J η ( x, d y ) does depend on x .
If µ 0 ( V ) , η 0 ( U ) < ∞ then there exists a constant C > 0 s.t.
[Equation]
In particular, if ρ < 1 , we have lim n →∞ µ n = π in V β -total variation.
Proof.
Case 1: For n = 1 , consider
[Equation]
because
[Equation]
(cid:48) (cid:107) tv,β ≤ (cid:107)
η − η (cid:48) (cid:107)
[Equation]
U r ∗ due to the Assumption 2-C1.
Hence assume true for n − 1 .
Then for n :(cid:107)
[Equation]
hence Case 1 is done.
Case 2: We proceed the same way as case 1 except we need to use
[Equation]
using Lemma 3.
Note that since K • satisﬁes a uniform drift criterion for V , using linearity we have
[Equation]
Hence for n = 1 :
[Equation]
Hence assume true for n − 1 .
Then for n :
[Equation]
hence Case 2 is done.
F Large-Particle Convergence
We study the behaviour of the interacting particle system (3) as N → ∞ .
The behaviour of thissystem is probabilistically different from that of the mean ﬁeld system (2) because the particles in (3)are coupled whereas the particles in (2) are independent .
The fact that we recover independence inthe limit of a collection of intercheangeable particles is a remarkable feature of interacting particlesystems [41].
F.1 Result
For a collection of N particles, under suitable assumptions on K • , we will show that any ﬁxed-sizeq -block of particles of { X 1 n , . . .
, X q n } becomes independent as N → ∞ , and moreover this trendtowards independence happens uniformly in time.
This phenomenon is called the uniform propagationof chaos property 10 .
Let us describe the dynamics of the distribution of the IPS (3).
From (3), each X in evolves accordingto
[Equation]
which indicates that, given Y n , X in is sampled independently.
Letting X q,Nn :
=
( X 1 n , . . .
, X qn ) andµ q,Nn :=
Distribution( X q,Nn ) ∈ P (( R d ) q ) , for measurable f : ( R d ) q
→ R one has
[Equation]
where the expectation is taken over the distribution of Y n =
( Y 1 n , . . .
, Y Nn ) , which is η ⊗ N n .
We willuse this decomposition to derive a uniform propagation of chaos result in the following theorem.
Theorem 3.
Let N > 0 , q ∈ { 1 , . . . , N } , and consider the interacting particle system X n :=( X 1 n , . . .
, X Nn )
, Y n :=
( Y 1 n , . . .
, Y Nn ) from (3) .
Let µ q,Nn :=
Distribution( X 1 n , . . .
, X qn ) , and letµ n be the distribution of the (independent) mean ﬁeld system (2) , with X i 0 ∼ µ 0 .
Suppose that∀ x ∈ ( R d ) q , η ∈ P ( R d ) , and f ∈ B b (( R d ) q ) with osc ( f ) ≤ 1 {+
+}
[Equation]
where Y =
{ Y 1 , . . .
, Y N } , Y i ∼ η.
Suppose ﬁnally that µ q,N= µ ⊗ q 0 for any 1 ≤ q ≤ N .
If (Case 1) : J η ( x, • ) doesn’t depend on x , or if (Case 2) :
J η ( x, • ) does depend on x but additionallythat (cid:15) ( K ) < 1 , then there exists a ﬁxed constant C > 0 s.t.
[Equation]
Proof.
Case 1 : We ﬁrst claim that
[Equation]
Let f ∈ B b (( R d ) q ) s.t.
osc ( f ) ≤ 1 .
Consider the following expression for general n :
[Equation]
where we have used the assumption on J .
Thus if n = 1 in the above expression, the base case holds since µ q,N= µ ⊗ q 0 .
Now, if the claimholds for n , then for n + 1 we have
[Equation]
Thus the claim holds for n + 1 .
Since (1 − ε ) (cid:15) ( K ⊗ q ) < 1 , we have the result.
Case 2 :
We ﬁrst claim that
[Equation]
Let f ∈ B b (( R d ) q ) with osc ( f ) ≤ 1 .
We proceed by induction.
Note [-that:| µ q,Nn +
 ( f ) − µ ⊗ qn +1 ( f ) | = | E [ µ q,Nn K ⊗ qm ( Y n ) f ] − µ ⊗ q n K ⊗ q η n [Equation] f-]that:
[Equation]
for the ﬁrst term, we will use the decomposition
[Equation]
Then, using the assumption on J , we have
[Equation]
and also we know we can use
[Equation]
since (cid:15) ( J η n ) ≤ 1 , with osc( h ) ≤ 1 .
Hence putting these together, we see
[Equation]
Now if n = 0 in the above expression, the base case holds since µ q,N= µ ⊗ q 0 .
If it is true for n , thenfor n + 1 we have
[Equation]
K ⊗ q )]
[Equation]
Hence if (cid:15) ( K ⊗ q ) < 1 , i.e. (cid:15) ( K ) < 1 , the result holds.
F.2 Proof of Main Results (Theorem 1 & Corollary 1)
Theorem 1.
[Convergence of Nonlinear MCMC] Under suitable conditions on K η and Q , thereexist ﬁxed constants C 1 , C 2 , C 3 > 0 , a function R : [0 , ∞ [ → [1 , ∞ [ , and ρ > 0 s.t.
[Equation]
Proof.
This follows straightforwardly from the above discussion, the only technicality is convertingthe results from Theorem 2 to the un-weighted total variation.
But note that since {(cid:107) f [Equation] ≤ 1 } ⊂{ [Equation]
f
[Equation]
β ≤ 1 } we have
[Equation]
so we’re done.
Corollary 1.
[Adapted from [41], Theorem 2.2] Suppose that Theorem 1 applies to K η .
Let X n := { X 1 n , . . .
, X Nn } be from (3) .
Then for every n ∈ N and f ∈ B b ( R d ) we have
[Equation]
Proof.
Let f ∈ B b ( R d ) and consider
[Equation]
using interchangeability of the X in s. Since the strong propagation of chaos in Theorem 3 clearlyimplies weak propagation of chaos, i.e. for any bounded f | µ q,Nn ( f ) − µ ⊗ qn ( f ) | → 0 , we see that theexpression above → 0 and we have L 2 -convergence which implies weak convergence.
G Analysis of the Kernels K BGη and K ARηG.1 Analysis of K BGηG.1.1 Statement of Results
A key difﬁculty that arises when working with Ψ G ( η ) =
G d η η ( G ) is obtaining a uniform lower boundthe denominator η ( G ) .
This is important for deriving uniform regularity results and for generallyestablishing conditions under which the measures { Ψ G ( η n ) } ∞ n =0 are well-deﬁned.
An effectiveapproach used in [47] and related works is to assume the uniform lower bound G ( x ) > (cid:15) > [-
-]0 ∀ x ∈ 949 R d for some (cid:15) > 0 .
While this assumption is self-contained in that it works for any choice of η ,it eliminates ubiquitous families of measures such as Gaussians, e.g. G ( x ) = exp( −(cid:107) x (cid:107) 2 / 2) andd η = d x .
We will see below how to relax G ( x ) > ε > 0 using the structure of our problem.
In particular, wecan use the Lyapunov function U for Q to control the probability that the state Y n ∼ η n will venture“far away” from the center of the state space.
This leads to the intuition that, if G is sufﬁciently“compatible” with U , i.e. the function G stays away from zero in the “center” of the state space asdetermined by U , then the expectation η n ( G ) =
E Y n ∼ η n [ G ( Y n )] can be lower-bounded using thesame
Lyapunov function.
This insight is encoded in the compatibility criterion Assumption 2-C2,and we can use it to prove the following a priori lower bound on η n ( G ) .
Lemma 4.
Suppose that Assumption 1-Q1 and Assumption 2-C2 hold, and let η n = η n − 1 Q withη 0 ( U ) < ∞ .
Then we have the lower bound
[Equation]
where R ∗ is ﬁxed and doesn’t depend on n .
This lemma essentially says that G should be bounded away from zero on the level sets of U .
Thisis not a strong condition — if G is bounded away from zero on compact sets and U is continuous,the lemma applies.
As an example, if G = exp( −(cid:107) x (cid:107) 2 / 2) and U ( x ) = c (cid:107) x (cid:107)
+ 1 then this resultapplies.
The constant R ∗ arises since R (cid:55)→ θ ( R ) is nonincreasing and R (cid:55)→ 1 − 1 R is increasing sowe can optimize this bound as a function of R to get R ∗ (which may not be unique, but the value thebound attains will be).
Together with the next lemma, we can obtain the desired convergence from
Theorem 2 for the BG interaction.
Lemma 5.
Let η, η (cid:48) ∈ P
( R d ) , and suppose that Assumption 3 holds, i.e. (cid:107) G (cid:107) ∞ , (cid:107) G (cid:107) β < ∞ .
Then
[Equation]
Ψ G
[Equation]
Clearly, we will use our knowledge that η n ∈ P m G ,M U ( R d ) ∀ n to make the Lipschitz constant in
Lemma 5 uniform over P m G ,M U .
Proposition 2.
Suppose that Assumption 1, 2, and Assumption 3-G1 hold.
Then the drift criterion(10) holds uniformly over all η ∈ P m,M ( R d ) .
Now we can apply Theorem 2 to obtain convergence.
Corollary 2.
Suppose that Assumptions 1, 2, 3 hold.
If η 0 ( G ) > 0 , η 0 ( U ) , µ 0 ( V ) < ∞ , then
Theorem 2 holds for K BG , i.e. the ﬂow µ n converges to π in V β -total variation as long as ρ =(1 − ε ) γ < 1 .
Proof.
Lemma 4 and Lemma 5 imply the regularity condition (9) for constants determined by thoselemmas and by noting that, due to Asssumption 2-C1, η ( V β ) ≤ η ( U r ∗ ) ≤ η ( U ) r ∗ < ∞ by Jensen’sinequality.
Additionally, Proposition 2 implies the condition (10).
Hence, since we are in “Case 1” of
Theorem 2, the result follows.
G.1.2 Proofs
Lemma 4.
Suppose that Assumption 1-Q1 and Assumption 2-C2 hold, and let η n = η n − 1 Q withη 0 ( U ) < ∞ .
Then we have the lower bound
[Equation]
where R ∗ is ﬁxed and doesn’t depend on n .
Proof.
[Equation]
But by Markov’s inequality
[Equation]
so
[Equation]
and hence
[Equation]
Now
[Equation]
and iterating this procedure gives
[Equation]
where we have used the sum of the geometric series to obtain c/ (1 − ξ ) .
Now, θ ( R ) is nonincreasingw.r.t.
R and 1 − 1 /R is increasing w.r.t.
R , so optimizing to get R ∗ and integrating we obtain
[Equation]
Lemma 5.
Let η, η (cid:48) ∈ P
( R d ) , and suppose that Assumption 3 holds, i.e. (cid:107) G (cid:107) ∞ , (cid:107) G (cid:107) β < ∞ .
Then
[Equation]
Proof.
Let (cid:107) f (cid:107) β ≤ 1 .
then
[Equation]
ˆ G ( x ) f ( x ) η ( G ) η (d x ) − ˆ G ( x ) f ( x ) η (cid:48)
( G ) η
(d x ) (cid:12) (cid:12) (cid:12) (cid:12) + (cid:12) (cid:12) (cid:12) (cid:12) ˆ G ( x ) f ( x ) η (cid:48) ( G ) η (d x ) − ˆ G ( x ) f ( x ) η (cid:48) ( G ) η (cid:48) ( d x ) (cid:12) (cid:12) (cid:12) (cid:12) for the ﬁrst term(cid:12)(cid:12)(cid:12)(cid:12) ˆ (cid:18) G ( x ) f ( x ) η ( G ) − G ( x ) f ( x ) η (cid:48) ( G ) (cid:19)
η (d x ) (cid:12)(cid:12)(cid:12)(cid:12) ≤ | η ( G ) − η (cid:48)
( G )
| η ( G ) η (cid:48) ( G ) ˆ G ( x ) | f ( x ) | η (d x ) ≤ (cid:107) G (cid:107) β (cid:107) η − η (cid:48) (cid:107) tv,β η ( G ) η (cid:48)
( G ) ˆ
G ( x ) | f ( x ) | η (d x ) ≤ (cid:107) G (cid:107) β (cid:107) η − η (cid:48) (cid:107) tv,β η ( G ) η (cid:48) ( G ) η ( G (1 + βV )) (cid:107) f (cid:107) β = (cid:18) (cid:107) G (cid:107) β η (cid:48) ( G ) + βη ( V ) (cid:107) G (cid:107)
[Equation]
G
[Equation]
and for the second
[Equation]
since
[Equation]
so putting these together
[Equation]
Using symmetry completes the proof.
Proposition 2.
Suppose that Assumption 1, 2, and Assumption 3-G1 hold.
Then the drift criterion(10) holds uniformly over all η ∈ P m,M ( R d ) .
Proof.
Let η ∈ P m,M ( R d ) and consider
[Equation]
Theorem 4 ([47] Thm 8.7.1 pp.283) .
Suppose that G has bounded oscillations, let N ≥ q ≥ 1 , and
Y =
( Y 1 , . . .
Y N ) , Y i ∼ η .
Then for any f ∈ B b (( R d ) q ) with osc( f ) ≤ 1 , we have
[Equation]
where
[Equation]
In particular, by picking G ≡ 1 , we obtain
[Equation]
Corollary 3.
Suppose that Assumption 3 G1 holds (i.e. G is bounded) the compatibility criterion
Assumption 2-C2 with η n ( G ) ≥ m .
Then K BG • satisﬁes the uniform propagation of chaos in Case 1with
[Equation]
Proof.
This follows from Theorem 3 and Theorem 4 above, noting 1) that (cid:107) G (cid:107) ∞ implies osc( G ) <∞ , and 2) that we can obtain a bound on R G,η n independent of η n by applying Lemma 4 to obtainη n ( G ) ≥ m so
[Equation]
G.2 Analysis of K ARη
G.2.1 Statement of Results
First, the result showing that π is K ARη (cid:63) -invariant.
Proposition 3.
K η (cid:63) is π -invariant.
Proof.
We have
[Equation]
since the integrals with f ( y ) and f ( x ) in the ﬁrst term are equal.
We will establish equvalent results from Section G.1but for K AR • .
This will not require assumption 2C2 since the interaction is well-deﬁned for η ∈ P ( R d ) as α ( x, y ) is in fact bounded .
K AR is themain subject of study for [1], and in fact the regularity and uniform drift conditions were establishedthere.
Note that the statement η ∈ P 0 , ∞ is vaccuous since G ( x ) > 0 =
⇒ η
( G ) > 0 , and the secondsays that η ( V ) < ∞ .
Lemma 6 ([1]) .
Let η, η (cid:48) ∈ P 0 , ∞ ( R d ) .
Then
[Equation]
Lemma 7.
Suppose that Assumption 1, 2-C2, hold.
Then the uniform drift criterion (10) holds forη ∈ P 0 ,M ( R d ) .
Proposition 4.
Let J =
J AR , and Y =
{ Y 1 , . . .
, Y N }
where Y i iid ∼ η .
Then for any f ∈ B b (( R d ) qwith osc( f ) ≤ 1 and x ∈ ( R d ) q , we have
[Equation]
Corollary 4.
Suppose that Assumption 1, 2-C2 hold.
If η 0 ( U ) , µ 0 ( V ) < ∞ , then Theorem 2 holds for
K BG , i.e. the ﬂow µ n converges to π in V β -total variation as long as ρ =
(1 − ε ) γ + ε (cid:107) J
η
V
β
(cid:107) < 1 .
G.2.2 Proofs
Lemma 7.
Suppose that Assumption 1, 2-C2, hold.
Then the uniform drift criterion (10) holds forη ∈ P 0 ,M ( R d ) .
Proof.
Let η ∈ P 0 ,M ( R d ) and consider
[Equation]
Proposition 4.
Let J =
J AR , and Y =
{ Y 1 , . . .
, Y N }
where Y i iid ∼ η .
Then for any f ∈ B b (( R d ) qwith osc( f ) ≤ 1 and x ∈ ( R d ) q , we have
[Equation]
Proof.
Starting with the ﬁrst term, for each ﬁxed x
[Equation]
where
[Equation]
Nowsup
[Equation]
so automatically osc( ϕ f ( x, • )) ≤ 2 .
Hence using Lemma
[Equation]
Corollary 5.
If (cid:15) ( K ) < 1 , then K AR • satisﬁes the uniform propagation of chaos in Case 2 of