variable transformation to obtain geometric ergodicity in the random walk metropolis algorithm a random walk metropolis sampler is geometrically ergodic if its equilibrium density is super exponentially light and satisfies a curvature condition stochastic process appl 85 2000 341 361 many applications including bayesian analysis with conjugate priors of logistic and poisson regression and of log linear models for categorical data result in posterior distributions that are not super exponentially light we show how to apply the change of variable formula for diffeomorphisms to obtain new densities that do satisfy the conditions for geometric ergodicity sampling the new variable and mapping the results back to the old gives a geometrically ergodic sampler for the original variable this method of obtaining geometric ergodicity has very wide applicability

bayesian isotonic regression and trend analysis in many applications the mean of a response variable can be assumed to be a nondecreasing function of a continuous predictor controlling for covariates in such cases interest often focuses on estimating the regression function while also assessing evidence of an association this article proposes a new framework for bayesian isotonic regression and order restricted inference approximating the regression function with a high dimensional piecewise linear model the nondecreasing constraint is incorporated through a prior distribution for the slopes consisting of a product mixture of point masses accounting for flat regions and truncated normal densities to borrow information across the intervals and smooth the curve the prior is formulated as a latent autoregressive normal process this structure facilitates efficient posterior computation since the full conditional distributions of the parameters have simple conjugate forms point and interval estimates of the regression function and posterior probabilities of an association for different regions of the predictor can be estimated from a single mcmc run generalizations to categorical outcomes and multiple predictors are described and the approach is applied to an epidemiology application

hidden markov models for longitudinal comparisons medical researchers interested in temporal multivariate measurements of complex diseases have recently begun developing health state models which divide the space of patient characteristics into medically distinct clusters the current state of the art in health services research uses k means clustering to form the health states and a first order markov chain to describe transitions between the states this fitting procedure ignores information from temporally adjacent observations and prevents uncertainty from parameter estimation and cluster assignments from being incorporated into the analysis a natural way to address these issues is to combine clustering and longitudinal analyses using a hidden markov model we fit hidden markov models to longitudinal data using bayesian methods that account for all of the uncertainty in the parameters conditional only on the underlying correctness of the model potential lack of time homogeneity in the markov chain is accounted for by embedding transition probabi

maximum likelihood estimation of a multi dimensional log concave density fn is shown to have smaller mean integrated squared error compared with kernel based methods even when we allow the use of a theoret ical optimal fixed bandwidth for the kernel estimator that would not be available in practice we also present a real data clustering example which shows that our methodology can be used in conjunction with the expectation maximization algorithm to fit finite mixtures of log concave densities

bayesian computation via the gibbs sampler and related markov chain monte carlo methods the use of the gibbs sampler for bayesian computation is reviewed and illustrated in the context of some canonical examples other markov chain monte carlo simulation methods are also briefly described and comments are made on the advantages of sample based approaches for bayesian inference summaries

markov chain monte carlo 10 years and still running although markov chain monte carlo mcmc techniques can be traced back to the early 1950s and are almost contemporary with independent monte carlo methods the use of these techniques has only really taken off in the past 10 years at least in mainstream statistics with the work of gelfand and smith 1990 and the rediscovery of metropolis s and hastings s papers the impact on the discipline is deep and durable because these methods have opened new horizons in the scale of the problems that one can deal with thus enhancing the position of statistics in most applied fields the mcmc revolution has in particular boosted bayesian statistics to new heights by providing a virtually universal tool for dealing with integration and optimization problems this can be seen in for instance the explosion of papers dealing with complex models hierarchical modelings nonparametric bayesian estimation and spatial statistics this trend has also created new synergies with mathematicians and probabilists as well as econometricians engineers ecologists astronomers and others for theoretical requests and practical implications of mcmc techniques it is impossible to provide here a complete perspective on mcmc methods and applications for this as well as for other references we refer the reader to the books by gilks richardson and spiegelhalter 1995 and robert and casella 1999 here we focus on four particular aspects of mcmc algorithms theoretical foundations practical implementation bayesian inference and prospects the web provides many relevant sites including the mcmc preprint server http www statslab cam ac uk r mcmc which offers a most useful window to the current research on mcmc wilson s site on perfect simulation http dimacs rutgers edu dbwilson exact h which keeps track of papers and applets on this topic and a site on convergence control techniques http www ensae fr crest statistique robert mcdiag even more recent developments allow for on line computations through applets like breyer s http www maths lancs ac uk breyer java html which impressively illustrates the behavior of different samplers for simple problems note also that the bugs and coda software are freely available at http www mrc bsu cam ac uk bu

on the geometric convergence of the gibbs sampler summary the rate of convergence of the gibbs sampler is discussed the gibbs sampler is a monte carlo simulation method with extensive application to computational issues in the bayesian paradigm conditions for the geometric rate of convergence of the algorithm for discrete and continuous parameter spaces are derived and an illustrative exponential family example is given this paper investigates conditions under which the gibbs sampler gelfand and smith 1990 tanner and wong 1987 geman and geman 1984 converges at a geometric rate the main results appear in sections 2 and 3 where geometric convergence results are established with respect to total variation and supremum norms under fairly natural conditions on the underlying distribution for ease of exposition we shall concentrate on the two most commonly encountered situations where the state space is finite or continuous all our results will establish uniform convergence a strong form of geometric convergence under appropriate regularity conditions uniform convergence is a useful property in its own right but also happens to be a sufficient condition for certain ergodic central limit theorems such results are important for estimation in markov chain simulation but will not be considered in detail here our approach is to apply the theory of markov chains to the specific gibbs sampler case in the finite state space case uniform ergodicity is automatic however the situation is more complicated for continuous state spaces where even well behaved underlying distributions can give rise to markov chains which converge slowly or have unbounded kernels we give two results in this context corollary 2 and corollary 3 which establish uniform convergence under different sets of conditions on the underlying density finally we apply these results to an example of a bayesian hierarchical model where regularity conditions for geometric convergence are naturally satisfied here the hierarchical structure of the model is crucial in permitting application of corollary 3

central limit theorem for triangular arrays of non homogeneous markov chains in this paper we obtain the central limit theorem for triangular arrays of non homogeneous markov chains under a condition imposed to the maximal coefficient of correlation the proofs are based on martingale techniques and a sharp lower bound estimate for the variance of partial sums the results complement an important central limit theorem of dobrushin based on the contraction coefficient

bounding the convergence time of the gibbs sampler in bayesian image restoration agibbs mathstat yorku ca summary this paper shows how coupling methodology can be used to give precise a priori bounds on the convergence time of markov chain monte carlo algorithms for which a partial order exists on the state space which is preserved by the markov chain transitions this methodology is applied to give a bound on the convergence time of the random scan gibbs sampler used in the bayesian restoration of an image of n pixels for our algorithm in which only one pixel is updated at each iteration the bound is a constant times n2 the proportionality constant is given and is easily calculated these bounds also give an indication of the running time of coupling from the past algorithms

markov chains for exploring posterior distributions several markov chain methods are available for sampling from a posterior distribution two important examples are the gibbs sampler and the metropolis algorithm in addition several strategies are available for constructing hybrid algorithms this paper outlines some of the basic methods and strategies and discusses some related theoretical and practical issues on the theoretical side results from the theory of general state space markov chains can be used to obtain convergence rates laws of large numbers and central limit theorems for estimates obtained from markov chain methods these theoretical results can be used to guide the construction of more efficient algorithms for the practical use of markov chain methods standard simulation methodology provides several variance reduction techniques and also give guidance on the choice of sample size and allocation

convergence of adaptive mixtures of importance sampling schemes in the design of ecient simulation algorithms one is often beset with a poor choice of proposal distributions although the performances of a given kernel can clarify how adequate it is for the problem at hand a permanent on line modification of kernels causes concerns about the validity of the resulting algorithm while the issue is quite complex and most often intractable for mcmc algorithms the equivalent version for importance sampling algorithms can be validated quite precisely we derive sucient convergence conditions for a wide class of population monte carlo algorithms and show that rao blackwellized versions asymptotically achieve an optimum in terms of a kullback divergence criterion while more rudimentary versions simply do not benefit from repeated updating

stochastic optimization on continuous domains with finite time guarantees by markov chain monte carlo methods we introduce bounds on the finite time performance of markov chain monte carlo mcmc algorithms in solving global stochastic optimization problems defined over continuous domains it is shown that mcmc algorithms with finite time guarantees can be developed with a proper choice of the target distribution and by studying their convergence in total variation norm this work is inspired by the concept of finite time learning with known accuracy and confidence developed in statistical learning theory

geometric convergence rates for stochastically ordered markov chains let n be a markov chain on the state space 0 that is stochastically ordered in its initial state that is a stochastically larger initial state produces a stochastically larger chain at all other times examples of such chains include random walks the number of customers in various queueing systems and a plethora of storage processes a large body of recent literature concentrates on establishing geometric ergodicity of n in total variation that is proving the existence of a limiting probability measure and a number r 1 such that lim n to infty r n sup a in cal b 0 infty vert p x phi n in a pia vert 0 for every deterministic initial state 0 x we seek to identity the largest r that satisfies this relationship a dependent sample path coupling and a foster lyapunov drift inequality are used to derive convergence rate bounds we then show that the bounds obtained are frequently the best possible application of the methods to queues and random walks are included

fixed width output analysis for markov chain monte carlo markov chain monte carlo is a method of producing a correlated sample to estimate features of a target distribution through ergodic averages a fundamental question is when sampling should stop that is at what point the ergodic averages are good estimates of the desired quantities we consider a method that stops the simulation when the width of a confidence interval based on an ergodic average is less than a user specified value hence calculating a monte carlo standard error is a critical step in assessing the simulation output we consider the regenerative simulation and batch means methods of estimating the variance of the asymptotic normal distribution we give sufficient conditions for the strong consistency of both methods and investigate their finite sample properties in various examples

sequentially interacting markov chain monte carlo methods we introduce a novel methodology for sampling from a sequence of probability distributions of increasing dimension and estimating their normalizing constants these problems are usually addressed using sequential monte carlo smc methods the alternative sequentially interacting markov chain monte carlo simcmc scheme proposed here works by generating interacting non markovian sequences which behave asymptotically like independent metropolis hastings mh markov chains with the desired limiting distributions contrary to smc methods this scheme allows us to iteratively improve our estimates in an mcmc like fashion we establish convergence of the algorithm under realistic verifiable assumptions and demonstrate its performance on several examples arising in bayesian time series analysis

particle markov chain monte carlo methods summary markov chain monte carlo and sequential monte carlo methods have emerged as the two main tools to sample from high dimensional probability distributions although asymptotic convergence of markov chain monte carlo algorithms is ensured under weak assumptions the performance of these algorithms is unreliable when the proposal distributions that are used to explore the space are poorly chosen and or if highly correlated variables are updated independently we show here how it is possible to build efficient high dimensional proposal distributions by using sequential monte carlo methods this allows us not only to improve over standard markov chain monte carlo schemes but also to make bayesian inference feasible for a large class of statistical models where this was not previously so we demonstrate these algorithms on a non linear state space model and a levy driven stochastic volatility model

on the effectiveness of monte carlo for initial uncertainty forecasting in nonlinear dynamical systems abstract this paper considers the problem of initial uncertainty forecasting in deterministic nonlinear continuous time dynamical systems via particle ensembles the popular monte carlo method which while simple to implement faces fundamental issues in particular it is not clear how well the propagated particles continue to represent the true state uncertainty at future times this paper evaluates the performance of monte carlo forecasting by analyzing it in the context of markov chain monte carlo mcmc theory the propagated ensemble is viewed as the realization of a markov chain at each time instant generated by an associated instantaneous transition kernel it is shown that for a special class of nonlinear systems that have zero divergence the propagated kernel is in detailed balance with the true state probability density function this guarantees statistical consistency of the monte carlo ensemble with the truth at all times for such systems on the other hand no such guarantee is possible for systems with non zero divergence

estimates and standard errors for ratios of normalizing constants from multiple markov chains via regeneration in the classical biased sampling problem we have k densities 1 k each known up to a normalizing constant i e for l 1 k l l ml where l is a known function and ml is an unknown constant for each l we have an iid sample from l and the problem is to estimate the ratios ml ms for all l and all s this problem arises frequently in several situations in both frequentist and bayesian inference an estimate of the ratios was developed and studied by vardi and his co workers over two decades ago and there has been much subsequent work on this problem from many different perspectives in spite of this there are no rigorous results in the literature on how to estimate the standard error of the estimate we present a class of estimates of the ratios of normalizing constants that are appropriate for the case where the samples from the l s are not necessarily iid sequences but are markov chains we also develop an approach based on regenerative simulation for obtaining standard errors for the estimates of ratios of normalizing constants these standard error estimates are valid for both the iid case and the markov chain case

geometric convergence and central limit theorems for multidimensional hastings and metropolis algorithms we develop results on geometric ergodicity of markov chains and apply these and other recent results in markov chain theory to multidimensional hastings and metropolis algorithms for those based on random walk candidate distributions we find sufficient conditions for moments and moment generating functions to converge at a geometric rate to a prescribed distribution by phrasing the conditions in terms of the curvature of the densities we show that the results apply to all distributions with positive densities in a large class which encompasses many commonly used statistical forms from these results we develop central limit theorems for the metropolis algorithm converse results showing non geometric convergence rates for chains where the rejection rate is not bounded away from unity are also given these show that the negative definiteness property is not redundant

minorization conditions and convergence rates for markov chain monte carlo abstract general methods are provided for analyzing the convergence of discrete time general state space markov chains such as those used in stochastic simulation algorithms including the gibbs sampler the methods provide rigorous a priori bounds on how long these simulations should be run to give satisfactory results results are applied to two models of the gibbs sampler a bivariate normal model and a hierarchical poisson model with gamma conditionals the methods use the notion of minorization conditions for markov chains

a shuffled complex evolution metropolis algorithm for optimization and uncertainty assessment of hydrologic model parameters author s vrugt ja gupta hv bouten w sorooshian s abstract markov chain monte carlo mcmc methods have become increasingly popular for estimating the posterior probability distribution of parameters in hydrologic models however mcmc methods require the a priori definition of a proposal or sampling distribution which determines the explorative capabilities and efficiency of the sampler and therefore the statistical properties of the markov chain and its rate of convergence in this paper we present an mcmc sampler entitled the shuffled complex evolution metropolis algorithm scem ua which is well suited to infer the posterior distribution of hydrologic model parameters the scem ua algorithm is a modified version of the original sce ua global optimization algorithm developed by duan et al 1992 the scem ua algorithm operates by merging the strengths of the metropolis algorithm controlled random search competitive evolution and complex shuffling in order to continuously update the proposal distribution and evolve the sampler to the posterior target distribution three case studies demonstrate that the adaptive capability of the scem ua algorithm significantly reduces the number of model simulations needed to infer the posterior distribution of the parameters when compared with the traditional metropolis hastings samplers

attaining the optimal gaussian diffusion acceleration sampling from probability distributions in high dimensional spaces is generally impractical diffusion processes with invariant equilibrium distributions can be used as a means to generate approximations an important task in such an endeavor is to design an equilibrium preserving drift to accelerate the convergence starting from a reversible diffusion it is desirable to depart for non reversible dynamics via a perturbed drift so that the convergence rate is maximized with the common equilibrium in the gaussian diffusion acceleration this problem can be cast as perturbing the inverse of a given covariance matrix by skew symmetric matrices so that all resulting eigenvalues have identical real part this paper describes two approaches to obtain the optimal rate of gaussian diffusion the asymptotical approach works universally for arbitrary ornstein uhlenbeck processes whereas the direct approach can be implemented as a fast divide and conquer algorithm a comparison with recently proposed lelievre nier pavliotis algorithm is made

approximating hidden gaussian markov random fields summary gaussian markov random field gmrf models are frequently used in a wide variety of applications in most cases parts of the gmrf are observed through mutually independent data hence the full conditional of the gmrf a hidden gmrf hgmrf is of interest we are concerned with the case where the likelihood is non gaussian leading to non gaussian hgmrf models several researchers have constructed block sampling markov chain monte carlo schemes based on approximations of the hgmrf by a gmrf using a second order expansion of the log density at or near the mode this is possible as the gmrf approximation can be sampled exactly with a known normalizing constant the markov property of the gmrf approximation yields computational efficiency the main contribution in the paper is to go beyond the gmrf approximation and to construct a class of non gaussian approximations which adapt automatically to the particular hgmrf that is under study the accuracy can be tuned by intuitive parameters to nearly any precision these non gaussian approximations share the same computational complexity as those which are based on gmrfs and can be sampled exactly with computable normalizing constants we apply our approximations in spatial disease mapping and model based geostatistical models with different likelihoods obtain procedures for block updating and construct metropolized independence samplers

a remark on the characteristic cauchy problem abstract the cauchy problem with data of finite energy is solved for a general class of non timelike surfaces in the characteristic case this means that the restriction to the surface is given in the sobolev space h 1

on the use of local optimizations within metropolis hastings updates summary we propose new metropolis hastings algorithms for sampling from multimodal dis tributions on n tjelmeland and hegstad have obtained direct mode jumping proposals by optimization within metropolis hastings updates and different proposals for forward and backward steps we generalize their scheme by allowing the probability distribution for forward and backward kernels to depend on the current state we use the new setting to combine mode jumping proposals and proposals from a prior approximation we obtain that the frequency of proposals from the different proposal kernels is automatically adjusted to their quality mode jumping proposals include local optimizations when combining this with a prior approximation it is tempting to use local optimization results not only for mode jumping proposals but also to improve the prior approximation we show how this idea can be implemented the resulting algorithm is adaptive but has a markov structure we evaluate the effectiveness of the proposed algorithms in two simulation examples

likelihood subgradient densities we introduce likelihood subgradient densities and explore their basic properties using mixtures of likelihood subgradient densities we propose an approach for constructing tight enveloping functions in the bayesian context in the case of normal priors with normal data the area underneath the resulting enveloping function is bounded above by the approach is extended to k dimensional models where the corresponding bound is more generally our approach should also yield tight enveloping functions for other models in which the data are close to normal such models include generalized linear models e g bayesian poisson regression and the bayesian logit model simulations based on the approach are performed for two separate models using accept reject methods

rates of convergence of the hastings and metropolis algorithms we apply recent results in markov chain theory to hastings and metropolis algorithms with either independent or symmetric candidate distributions and provide necessary and sufficient conditions for the algorithms to converge at a geometric rate to a prescribed distribution pi in the independence case in mathbb r k these indicate that geometric convergence essentially occurs if and only if the candidate density is bounded below by a multiple of pi in the symmetric case in mathbb r only we show geometric convergence essentially occurs if and only if pi has geometric tails we also evaluate recently developed computable bounds on the rates of convergence in this context examples show that these theoretical bounds can be inherently extremely conservative although when the chain is stochastically monotone the bounds may well be effective

asymptotic behavior of the gibbs sampler mild sufficient conditions for the ergodicity of the gibbs sampler are obtained the geometric ergodicity of the gibbs sampler is then discussed where the main tool is the drift criterion this criterion requires the construction of a generalized energy function so that the chain is dissipating energy at a suitable rate whenever it lies outside the center of the state space for the two variable case a useful recipe is provided for constructing appropriate energy functions these results are illustrated with four examples

numerical simulation of polynomial speed convergence phenomenon we provide a hybrid method that captures the polynomial speed of convergence and polynomial speed of mixing for markov processes the hybrid method that we introduce is based on the coupling technique and renewal theory we propose to replace some estimates in classical results about the ergodicity of markov processes by numerical simulations when the corresponding analytical proof is difficult after that all remaining conclusions can be derived from rigorous analysis then we apply our results to seek numerical justification for the ergodicity of two 1d microscopic heat conduction models the mixing rate of these two models are expected to be polynomial but very difficult to prove in both examples our numerical results match the expected polynomial mixing rate well

convergence properties of the gibbs sampler for perturbations of gaussians 1993 and smith and roberts 1993 a host of other applications of these methods in the context of hierarchical bayesian models is described in all these examples the question arises as to the rate of convergence of the markov chain there is no unified theory to answer this question especially in the case of continuous densities on r n and currently many different approaches are being attempted for the different types of simulation methods one direction of research uses doeblin type bounds on bounded sets together with geometric drift conditions or some form of lyapounov functions to obtain rather general results on exponential convergence for continuous n densities on r see mengersen and tweedie 1996 for the metropolis algorithm and rosenthal 1995 for the gibbs sampler these results are then applied to certain special cases and yield bounds on the rate of convergence in variational norm distance another approach analyzes a discretized version of the problem using general discrete state space markov chain techniques see frieze kannan and polson 1994 finally there are attempts to use spectral analysis to obtain rates of convergence in the l sense where the markov chain operator is considered 2 as a bounded operator in the hilbert space of square integrable functions

population based reversible jump markov chain monte carlo we present an extension of population based markov chain monte carlo to the transdimensional case a major challenge is that of simulating from high and transdimensional target measures in such cases markov chain monte carlo methods may not adequately traverse the support of the target the simulation results will be unreliable we develop population methods to deal with such problems and give a result proving the uniform ergodicity of these population algorithms under mild assumptions this result is used to demonstrate the superiority in terms of convergence rate of a population transition kernel over a reversible jump sampler for a bayesian variable selection problem we also give an example of a population algorithm for a bayesian multivariate mixture model with an unknown number of components this is applied to gene expression data of 1000 data points in six dimensions and it is demonstrated that our algorithm outperforms some competing markov chain samplers in this example we show how to combine the methods of parallel chains geyer 1991 tempering geyer thompson 1995 snooker algorithms gilks et al 1994 constrained sampling and delayed rejection green mira 2001

mcmc convergence diagnosis via multivariate bounds on log concave densities we begin by showing how piecewise linear bounds may be devised which bound both above and below any concave log density in general dimensions we then show how these bounds may be used to gain an upper bound to the volume in the tails outside the convex hull of the sample path in order to assess how well the sampler has explored the target distribution this method can be used as a stand alone diagnostic to determine when the sampler output provides a reliable basis for inference on the stationary density or in conjunction with existing convergence diagnostics to ensure that they are based upon good sampler output we provide an example and briefly discuss possible extensions to the method and alternative applications of the bounds 1 introduction the use of markov chain monte carlo mcmc methods has increased dramatically within the statistical community since their

pathwise accuracy and ergodicity of metropolized integrators for sdes metropolized integrators for ergodic stochastic differential equations sdes are proposed that 1 are ergodic with respect to the known equilibrium distribution of the sdes and 2 approximate pathwise the solutions of the sdes on finitetime intervals both these properties are demonstrated in the paper and precise strong error estimates are obtained it is also shown that the metropolized integrator retains these properties even in situations where the drift in the sde is nonglobally lipschitz and vanilla explicit integrators for sdes typically become unstable and fail to be ergodic 2009 wiley periodicals inc

tensor decompositions and sparse log linear models contingency table analysis routinely relies on log linear models with latent structure analysis providing a common alternative latent structure models lead to a reduced rank tensor factorization of the probability mass function for multivariate categorical data while log linear models achieve dimensionality reduction through sparsity little is known about the relationship between these notions of dimensionality reduction in the two paradigms we derive several results relating the support of a log linear model to nonnegative ranks of the associated probability tensor motivated by these findings we propose a new collapsed tucker class of tensor decompositions which bridge existing parafac and tucker decompositions providing a more flexible framework for parsimoniously characterizing multivariate categorical data taking a bayesian approach to inference we illustrate empirical advantages of the new decompositions