Evaluating Posterior Distributions by Selectively Breeding Prior Samples

Using Markov chain Monte Carlo to sample from posterior distributions was the key innovation which made Bayesian data analysis practical. Notoriously, however, MCMC is hard to tune, hard to diagnose, and hard to parallelize. This pedagogical note explores variants on a universal non-Markov-chain Monte Carlo scheme for sampling from posterior distributions. The basic idea is to draw parameter values from the prior distributions, evaluate the likelihood of each draw, and then copy that draw a number of times proportional to its likelihood. The distribution after copying is an approximation to the posterior which becomes exact as the number of initial samples goes to infinity; the convergence of the approximation is easily analyzed, and is uniform over Glivenko-Cantelli classes. While not entirely practical, the schemes are straightforward to implement (a few lines of R), easily parallelized, and require no rejection, burn-in, convergence diagnostics, or tuning of any control settings. I provide references to the prior art which deals with some of the practical obstacles, at some cost in computational and analytical simplicity. Suppose we have a prior probability measure Π0 on Θ, and a likelihood function f(θ, x). (Since x plays little further role in what follows, I suppress it in the notation.) We want to sample from the posterior measure Π1, defined via Bayes’s rule: Π1(A) = Π0(fA) Π0(f) using the de Finetti notation for expectations, so that Π0(fA) = ∫ f(θ)1A(θ)dΠ0(θ). I explain how to obtain a sample from Π1, supposing only that we can draw an IID sample from Π0, and that we can calculate the likelihood. I first develop a weighted sampling scheme, and then consider ways of producing unweighted samples. ∗Departments of Statistics and Data Science, and of Machine Learning, Carnegie Mellon University, Pittsburgh, PA 15213 USA, and the Santa Fe Institute, 1399 Hyde Park Road, Santa Fe, NM 87501 1 ar X iv :2 20 3. 09 07 7v 1 [ st at . C O ] 1 7 M ar 2 02 2 The inspiration for the algorithms below lies in the following observation. If Π0 and Π1 have densities π0 and π1, then Bayes’s rule takes the form π1(θ) = f(θ)π0(θ) Π0(f) and this is exactly the form of the “replicator equation” of evolutionary biology, which describes the effects of natural selection on the gene pool of a population (Shalizi, 2009, Appendix A). In the evolutionary interpretation, θ refers to genotypes rather than parameter values, and f(θ) is not likelihood but “fitness”, the mean number of surviving offspring left by individuals of genotype θ (Hofbauer and Sigmund, 1998). Usually, in biology, the fitness of any one genotype is “frequency-dependent”, i.e., a function of the whole distribution of genotypes, but this is excluded in Bayes’s rule. Bayesian updating describes an evolutionary process in an infinitely large population without competition, cooperation, mutation or sex. The algorithms below are all variants on taking this literally with a finite population. Very similar algorithms have of course been published before; §4.2 provides references, and §§4.3–4.4 discusses some of their pros and cons. 1 Fitness-Weighted Samples The fitness-weighted algorithm is very simple. Algorithm 1: Likelihood Importance Prior Sampling (LIPS) 1. Draw n samples θ1, θ2, . θn iidly from Π0. 2. For each θi, calculate the likelihood f(θi). 3. Approximate Π1 by Π̂1 = ∑n i=1 f(θi)δθi ∑n j=1 f(θj) (1) In particular, the posterior probability of any measurable set A ⊂ Θ is approximated by Π̂1(A) = ∑n i=1 f(θi)1A(θi) ∑n j=1 f(θj) (2) Clearly, Π̂1 is a probability measure. It converges on Π1 as n grows: Proposition 1 Suppose Π1(f) > 0. Then as n → ∞, Π̂1(A) → Π1(A) almost surely (Π0). Proof: Divide both the numerator and denominator of Eq. 2 by n: Π̂1(A) = n−1 ∑n i=1 f(θi)1A(θi) n−1 ∑n j=1 f(θj)