multiscale inference and long run variance estimation in non parametric regression with time series errors we develop new multiscale methods to test qualitative hypotheses about the function m in the non parametric regression model yt t m t t t with time series errors t in time series applications m represents a non parametric time trend practitioners are often interested in whether the trend m has certain shape properties for example they would like to know whether m is constant or whether it is increasing or decreasing in certain time intervals our multiscale methods enable us to test for such shape properties of the trend m to perform the methods we require an estimator of the long run error variance 2 l cov e0 el we propose a new difference based estimator of 2 for the case that t belongs to the class of auto regressive ar processes in the technical part of the paper we derive asymptotic theory for the proposed multiscale test and the estimator of the long run error variance the theory is complemented by a simulation study and an empirical application to climate data

bayesian reconstructions from emission tomography data using a modified em algorithm a novel method of reconstruction from single photon emission computerized tomography data is proposed this method builds on the expectation maximization em approach to maximum likelihood reconstruction from emission tomography data but aims instead at maximum posterior probability estimation which takes account of prior belief about smoothness in the isotope concentration a novel modification to the em algorithm yields a practical method the method is illustrated by an application to data from brain scans

optimal strategies for adaptive zero sum average markov games abstract we consider a class of discrete time two person zero sum markov games with borel state and action spaces and possibly unbounded payoffs the game evolves according to the recursive equation x n 1 f x n a n b n n n 0 1 where the disturbance process n is formed by independent and identically distributed r k valued random vectors which are observable but their common density is unknown for both players combining suitable methods of statistical estimation of with optimization procedures we construct a pair of average optimal strategies

structural image restoration through deformable templates abstract prior knowledge on the space of possible images is given in the form of a function or template in some domain the set of all possible true images is assumed to be formed by a composition of that function with continuous mappings of the domain into itself a prior gaussian distribution is given on the set of continuous mappings the observed image is assumed to be a degradation of the true image with additive noise given the observed image a posterior distribution is then obtained and has the form of a nonlinear perturbation of the gaussian measure on the space of mappings we present simulations of the posterior distribution that lead to structural reconstructions of the true image in the sense that it enables us to determine landmarks and other characteristic features of the image as well as to spot pathologies in it moreover we show that the reconstruction algorithm is relatively robust when the images are degraded by noise that is not necessarily additive

marginal likelihood estimation via power posteriors model choice plays an increasingly important role in statistics from a bayesian perspective a crucial goal is to compute the marginal likelihood of the data for a given model however this is typically a difficult task since it amounts to integrating over all model parameters the aim of the paper is to illustrate how this may be achieved by using ideas from thermodynamic integration or path sampling we show how the marginal likelihood can be computed via markov chain monte carlo methods on modified posterior distributions for each model this then allows bayes factors or posterior model probabilities to be calculated we show that this approach requires very little tuning and is straightforward to implement the new method is illustrated in a variety of challenging statistical settings

generalized monte carlo significance tests summary simple monte carlo significance testing has many applications particularly in the preliminary analysis of spatial data the method requires the value of the test statistic to be ranked among a random sample of values generated according to the null hypothesis however there are situations in which a sample of values can only be conveniently generated using a markov chain initiated by the observed data so that independence is violated this paper describes two methods that overcome the problem of dependence and allow exact tests to be carried out the methods are applied to the rasch model to the finite lattice ising model and to the testing of association between spatial processes power is discussed in a simple case

bayesian computation via the gibbs sampler and related markov chain monte carlo methods the use of the gibbs sampler for bayesian computation is reviewed and illustrated in the context of some canonical examples other markov chain monte carlo simulation methods are also briefly described and comments are made on the advantages of sample based approaches for bayesian inference summaries

action time sharing policies for ergodic control of markov chains ergodic control for discrete time controlled markov chains with a locally compact state space and a compact action space is considered under suitable stability irreducibility and feller continuity conditions a flexible family of controls called action time sharing ats policies associated with a given continuous stationary markov control is introduced it is shown that the long term average cost for such a control policy for a broad range of one stage cost functions is the same as that for the associated stationary markov policy in addition ats policies are well suited for a range of estimation information collection and adaptive control goals to illustrate the possibilities we present two examples the first demonstrates a construction of an ats policy that leads to consistent estimators for unknown model parameters while producing the desired long term average cost value the second example considers a setting where the target stationary markov control q is not known but there are sampling schemes available that allow for consistent estimation of q we construct an ats policy which uses dynamic estimators for q for control decisions and show that the associated cost coincides with that for the unknown markov control q

adaptive importance sampling and control variates abstract we construct and investigate an adaptive variance reduction framework in which both importance sampling and control variates are employed the three lines monte carlo averaging and two variance reduction parameter search lines run in parallel on a common sequence of uniform random vectors on the unit hypercube given that these two variance reduction techniques are effective often in a complementary way their combined application is well expected to widen the applicability of adaptive variance reduction we derive convergence rates of the theoretical estimator variance towards its minimum as a fixed computing budget increases when stochastic approximation runs with optimal constant learning rates we derive sufficient conditions for the proposed algorithm to attain the minimal estimator variance in the limit by stochastic approximation with decreasing learning rates or by sample average approximation when computing budget is unlimitedly available numerical results support our theoretical findings and illustrate the effectiveness of the proposed framework

a vanilla rao blackwellization of metropolis hastings algorithms casella and robert biometrika 83 1996 81 94 presented a general rao blackwellization principle for accept reject and metropolis hastings schemes that leads to significant decreases in the variance of the resulting estimators but at a high cost in computation and storage adopting a completely different perspective we introduce instead a universal scheme that guarantees variance reductions in all metropolis hastings based estimators while keeping the computation cost under control we establish a central limit theorem for the improved estimators and illustrate their performances on toy examples and on a probit model estimation

rao blackwellization of particle markov chain monte carlo methods using forward filtering backward sampling smoothing in state space models amounts to computing the conditional distribution of the latent state trajectory given observations or expectations of functionals of the state trajectory with respect to this distribution in recent years there has been an increased interest in monte carlo based methods often involving particle filters for approximate smoothing in nonlinear and or non gaussian state space models one such method is to approximate filter distributions using a particle filter and then to simulate using backward kernels a state trajectory backwards on the set of particles we show that by simulating multiple realizations of the particle filter and adding a metropolis hastings step one obtains a markov chain monte carlo scheme whose stationary distribution is the exact smoothing distribution this procedure expands upon a similar one recently proposed by andrieu doucet holenstein and whiteley we also show that simulating multiple trajectories from each realization of the particle filter can be beneficial from a perspective of variance versus computation time and illustrate this idea using two examples

statistical analysis of field experiments using neighbouring plots the paper describes and provides examples of four different applications of the use of neighbouring plot values in the analysis of agricultural field experiments the topics include the use of check plots to control environmental variation in large unreplicated variety trials spatial models based on first differences to accommodate fertility effects in trials that have some degree of replication adjustment for interplot competition using a simultaneous equations formulation and the analysis of trials in which plot values are affected by the particular treatments on neighbouring plots

a nonparametric bayesian modeling approach for cytogenetic dosimetry summary in cytogenetic dosimetry samples of cell cultures are exposed to a range of doses of a given agent in each sample at each dose level some measure of cell disability is recorded the objective is to develop models that explain cell response to dose such models can be used to predict response at unobserved doses more important such models can provide inference for unknown exposure doses given the observed responses typically cell disability is viewed as a poisson count but in the present work a more appropriate response is a categorical classification in the literature modeling in this case is very limited what exists is purely parametric we propose a fully bayesian nonparametric approach to this problem we offer comparison with a parametric model through a simulation study and the analysis of a real dataset modeling blood cultures exposed to radiation where classification is with regard to number of micronuclei per cell

absence of mass gap for a class of stochastic contour models we study a class of markovian stochastic processes in which the state space is a space of lattice contours and the elementary motions are local deformations we show under suitable hypotheses on the jump rates that the infinitesimal generator has zero mass gap this result covers among others the bfacf dynamics for fixed endpoint self avoiding walks and the sterling greensite dynamics for fixed boundary self avoiding surfaces our models also mimic the glauber dynamics for the low temperature ising model the proofs are based on two new general principles the minimum hitting time argument and the mean or mean exponential hitting time argument

asymptotic theory of cepstral random fields random fields play a central role in the analysis of spatially correlated data and as a result have a significant impact on a broad array of scientific applications given the importance of this topic there has been a substantial amount of research devoted to this area however the cepstral random field model remains largely underdeveloped outside the engineering literature we provide a comprehensive treatment of the asymptotic theory for two dimensional random field models in particular we provide recursive formulas that connect the spatial cepstral coefficients to an equivalent moving average random field which facilitates easy computation of the necessary autocovariance matrix additionally we establish asymptotic consistency results for bayesian maximum likelihood and quasi maximum likelihood estimation of random field parameters and regression parameters further in both the maximum and quasi maximum likelihood frameworks we derive the asymptotic distribution of our estimator the theoretical results are presented generally and are of independent interest pertaining to a wide class of random field models the results for the cepstral model facilitate model building because the cepstral coefficients are unconstrained in practice numerical optimization is greatly simplified and we are always guaranteed a positive definite covariance matrix we show that inference for individual coefficients is possible and one can refine models in a disciplined manner finally our results are illustrated through simulation and the analysis of straw yield data in an agricultural field experiment

a coupling regeneration scheme for diagnosing convergence in markov chain monte carlo algorithms abstract here i propose a convergence diagnostic for markov chain monte carlo mcmc algorithms based on couplings of a markov chain with an auxiliary chain that is periodically restarted from a fixed parameter value the diagnostic provides a mechanism for estimating the specific constants governing the rate of convergence of geometrically and uniformly ergodic chains and provides a lower bound on the effective sample size of a mcmc run it also provides a simple procedure for obtaining what is with high probability an independent sample from the stationary distribution

multiscale clustering of nonparametric regression curves abstract in a wide range of modern applications one observes a large number of time series rather than only a single one it is often natural to suppose that there is some group structure in the observed time series when each time series is modeled by a nonparametric regression equation one may in particular assume that the observed time series can be partitioned into a small number of groups whose members share the same nonparametric regression function we develop a bandwidth free clustering method to estimate the unknown group structure from the data more precisely speaking we construct multiscale estimators of the unknown groups and their unknown number which are free of classical bandwidth or smoothing parameters in the theoretical part of the paper we analyze the statistical properties of our estimators our theoretical results are derived under general conditions which allow the data to be dependent both in time series direction and across different time series the technical analysis of the paper is complemented by simulated and real data examples

iterative simulation methods abstract the standard methods of generating sample from univariate distributions often become hopelessly inefficient when applied to realizations of stochastic processes iterative methods are not widely known amongst statisticians but some are standard practice in statistical physics and chemistry the methods are surveyed and compared with particular reference to their convergence properties

the empirical efficiency and validity of two neighbour models the validity and efficiency of two neighbour models the first difference model of besag and kempton 1986 biometrics 42 231 251 and their extended error in variables model were studied for a variety of simulated yield models the extended error in variables model is equivalent to the linear variance neighbour model of williams 1986 biometrika 73 279 287 the parameters of the linear variance model were estimated using the method of restricted maximum likelihood estimation since the method of maximum likelihood used by besag and kempton was found to be biased the efficiencies of the two neighbour models were compared with those of two classical analyses a randomized block analysis and an incomplete block analysis in general the neighbour analyses were more efficient than the classical analyses where the simulated yield model contained a trend component the exceptions were when the trend was effectively eliminated by replicate blocks where the incomplete block analysis made gains over the randomized block analysis the neighbour models made proportionately greater gains the linear variance analysis was the best overall in efficiency of those examined being in all cases either the most efficient analysis or close in efficiency to the best analysis although the first difference model is a special case of the linear variance model besag and kempton recommended a different estimator of treatment precision for the first difference model from that recommended for the linear variance model for both neighbour models the recommended estimators of treatment precision were judged to be empirically valid for all simulated yield models

boundary detection by constrained optimization a statistical framework is used for finding boundaries and for partitioning scenes into homogeneous regions the model is a joint probability distribution for the array of pixel gray levels and an array of labels in boundary finding the labels are binary zero or one representing the absence or presence of boundary elements in partitioning the label values are generic two labels are the same when the corresponding scene locations are considered to belong to the same region the distribution incorporates a measure of disparity between certain spatial features of block pairs of pixel gray levels using the kolmogorov smirnov nonparametric measures of difference between the distributions of these features the number of model parameters is minimized by forbidding label configurations which are assigned probability zero the maximum a posteriori estimator of boundary placements and partitionings is examined the forbidden states introduce constraints into the calculation of these configurations stochastic relaxation methods are extended to accommodate constrained optimization

physiological pharmacokinetic analysis using population modeling and informative prior distributions abstract we describe a general approach using bayesian analysis for the estimation of parameters in physiological pharmacokinetic models the chief statistical difficulty in estimation with these models is that any physiological model that is even approximately realistic will have a large number of parameters often comparable to the number of observations in a typical pharmacokinetic experiment e g 28 measurements and 15 parameters for each subject in addition the parameters are generally poorly identified akin to the well known ill conditioned problem of estimating a mixture of declining exponentials our modeling includes a hierarchical population modeling which allows partial pooling of information among different experimental subjects b a pharmacokinetic model including compartments for well perfused tissues poorly perfused tissues fat and the liver and c informative prior distributions for population parameters which is possible because the parameters represent real physiological

efficient sequential monte carlo with multiple proposals and control variates sequential monte carlo is a useful simulation based method for online filtering of state space models for certain complex state space models a single proposal distribution is usually not satisfactory and using multiple proposal distributions is a general approach to address various aspects of the filtering problem this article proposes an efficient method of using multiple proposals in combination with control variates the likelihood approach of tan 2004 is used in both resampling and estimation the new algorithm is shown to be asymptotically more efficient than the direct use of multiple proposals and control variates the guidance for selecting multiple proposals and control variates is also given numerical examples are used to demonstrate that the new algorithm can significantly improve over the bootstrap filter and auxiliary particle filter

asymptotically efficient adaptive allocation schemes for controlled i i d processes finite parameter space the authors consider a controlled i i d independently identically distributed process whose distribution is parametrized by an unknown parameter theta belonging to some known parameter space theta and a one step reward associated with each pair of control and the following state of the process the objective is to maximize the expected value of the sum of one step rewards over an infinite horizon by introducing the loss associated with a control scheme it is shown that the problem is equivalent to minimizing this loss uniformly good adaptive control schemes are defined and emphasized a lower bound on the loss associated with any uniformly good control scheme is developed finally an adaptive control scheme is constructed whose loss equals the lower bound and is therefore asymptotically efficient

stochastic relaxation gibbs distributions and the bayesian restoration of images we make an analogy between images and statistical mechanics systems pixel gray levels and the presence and orientation of edges are viewed as states of atoms or molecules in a lattice like physical system the assignment of an energy function in the physical system determines its gibbs distribution because of the gibbs distribution markov random field mrf equivalence this assignment also determines an mrf image model the energy function is a more convenient and natural mechanism for embodying picture attributes than are the local characteristics of the mrf for a range of degradation mechanisms including blurring nonlinear deformations and multiplicative or additive noise the posterior distribution is an mrf with a structure akin to the image model by the analogy the posterior distribution defines another imaginary physical system gradual temperature reduction in the physical system isolates low energy states annealing or what is the same thing the most probable states under the gibbs distribution the analogous operation under the posterior distribution yields the maximum a posteriori map estimate of the image given the degraded observations the result is a highly parallel relaxation algorithm for map estimation we establish convergence properties of the algorithm and we experiment with some simple pictures for which good restorations are obtained at low signal to noise ratios

control variates for estimation based on reversible markov chain monte carlo samplers summary a general methodology is introduced for the construction and effective application of control variates to estimation problems involving data from reversible markov chain monte carlo samplers we propose the use of a specific class of functions as control variates and we introduce a new consistent estimator for the values of the coefficients of the optimal linear combination of these functions for a specific markov chain monte carlo scenario the form and proposed construction of the control variates is shown to provide an exact solution of the associated poisson equation this implies that the estimation variance in this case in the central limit theorem regime is exactly zero the new estimator is derived from a novel finite dimensional explicit representation for the optimal coefficients the resulting variance reduction methodology is primarily though certainly not exclusively applicable when the simulated data are generated by a random scan gibbs sampler markov chain monte carlo examples of bayesian inference problems demonstrate that the corresponding reduction in the estimation variance is significant and that in some cases it can be quite dramatic extensions of this methodology are discussed and simulation examples are presented illustrating the utility of the methods proposed all methodological and asymptotic arguments are rigorously justified under essentially minimal conditions

asymptotically efficient adaptive strategies in repeated games part i certainty equivalence strategies this paper addresses the problem of dynamic decision making in an uncertain and competitive environment a decision maker player 1 faces a system about which he has some parametric uncertainty and which is affected also by the actions of other agents we focus on a worst case analysts from the viewpoint of player 1 using the simplified model of a repeated matrix game with lack of information on one side where single stage rewards are random but announced and perfect observations are assumed certain ideas from the field of stochastic adaptive control are used to formulate performance criteria in a non bayesian setting and to devise appropriate control strategies the basic performance measure is the total reward accumulated by player 1 over all stages played the purpose of player 1 is to guarantee that his expected total reward will be close to what he could guarantee under complete information the present paper considers adaptive decision strategies of the certainty equivalence type based on a modified maximum likelihood estimator and studies their asymptotic long term performance a sequel paper will be devoted to asymptotically optimal strategies

efficiency of multivariate control variates in monte carlo simulation this paper considers some statistical aspects of applying control variates to achieve variance reduction in the estimation of a vector of response variables in monte carlo simulation it gives a result that quantifies the loss in variance reduction caused by the estimation of the optimal control matrix for the one dimensional case we derive analytically the optimal size of the vector of control variates under specific assumptions on the covariance matrix for the multidimensional case our numerical results show that good variance reduction is achieved when the number of control variates is relatively small approximately of the same order as the number of unknown parameters finally we give some recommendations for future research

asymptotically efficient adaptive strategies in repeated games part ii asymptotic optimality this paper continues the analysis of a dynamic decision problem modeled as a two person repeated game with random rewards perfect observations and incomplete information on one side the emphasis is on strategies of player 1 the uninformed player which maximize his worst case total reward in a strong non bayesian sense namely for all possible states of nature an asymptotic bound on performance is first established followed by the construction of strategies which achieve this bound the analysis highlights the efficient acquisition of statistical information under conflict conditions and especially the relations between information and payoff which are inherent in this problem

zero variance principle for monte carlo algorithms we present a general approach to greatly increase at little cost the efficiency of monte carlo algorithms to each observable to be computed we associate a renormalized observable improved estimator having the same average but a different variance by writing down the zero variance condition a fundamental equation determining the optimal choice for the renormalized observable is derived zero variance principle for each observable separately we show with several examples including classical and quantum monte carlo calculations that the method can be very powerful

analysis of field experiments by least squares smoothing summary assuming a smooth trend plus independent error model for the environmental effects in the yields of a field plot experiment least squares smoothing methods are developed to estimate both the treatment effects and the unknown trend treatment estimates are closely related to those resulting from a generalized least squares analysis in which the covariance structure for the environmental effects has a particular form however the main emphases are on the accuracy of treatment estimates under a fixed smooth trend plus error model and the exploratory power of the basic method to isolate trend effects of unknown form although the detailed development is for the one dimensional case generalizations of the smoothness concept and extensions to two dimensions are also discussed application of the basic method is illustrated on three data sets and the results compared with other analyses

exponential convergence to equilibrium for a class of random walk models we prove exponential convergence to equilibrium l 2 geometric ergodicity for a random walk with inward drift on a sub cayley rooted tree this randomwalk model generalizes a monte carlo algorithm for the self avoiding walk proposed by berretti and sokal if the number of vertices of leveln in the tree grows asc n n n 1 we prove that the autocorrelation time satisfies n 2 n 1

information rates of nonparametric gaussian process methods we consider the quality of learning a response function by a nonparametric bayesian approach using a gaussian process gp prior on the response function we upper bound the quadratic risk of the learning procedure which in turn is an upper bound on the kullback leibler information between the predictive and true data distribution the upper bound is expressed in small ball probabilities and concentration measures of the gp prior we illustrate the computation of the upper bound for the matern and squared exponential kernels for these priors the risk and hence the information criterion tends to zero for all continuous response functions however the rate at which this happens depends on the combination of true response function and gaussian prior and is expressible in a certain concentration function in particular the results show that for good performance the regularity of the gp prior should match the regularity of the unknown response function

positron emission tomography by markov chain monte carlo with auxiliary variables we propose a new algorithm for positron emission tomography pet image reconstruction the algorithm belongs to the family of markov chain monte carlo methods with auxiliary variables the idea is to iteratively generate hidden variables at one step and use them for image restoration at another step the well known model of vardi et al j amer statist assoc 80 1985 8 for pet is combined with the bayesian model of lasota and niemiro pattern recognition 36 2003 931 for the underlying images this latter model takes advantage of the fact that medical images often consist of relatively few grey levels of unknown intensity the algorithm of lasota and niemiro pattern recognition 36 2003 931 is used in the image restoration part of the pet algorithm essentially as a noise filtering and smoothing device it is now equipped with an additional data reconstruction step we include simulation results which suggest that the method is truly reliable we also describe a version of the basic algorithm in which a random simulation step is replaced by computation of expected value similarly as in the em algorithm

rao blackwellization for bayesian variable selection and model averaging in linear and binary regression a novel data augmentation approach choosing the subset of covariates to use in regression or generalized linear models is a ubiquitous problem the bayesian paradigm addresses the problem of model uncertainty by considering models corresponding to all possible subsets of the covariates where the posterior distribution over models is used to select models or combine them via bayesian model averaging bma although conceptually straightforward bma is often difficult to implement in practice since either the number of covariates is too large for enumeration of all subsets calculations cannot be done analytically or both for orthogonal designs with the appropriate choice of prior the posterior probability of any model can be calculated without having to enumerate the entire model space and scales linearly with the number of predictors p in this article we extend this idea to a much broader class of nonorthogonal design matrices we propose a novel method which augments the observed nonorthogonal design by at most p new rows to obtain

convergence properties of the gibbs sampler for perturbations of gaussians 1993 and smith and roberts 1993 a host of other applications of these methods in the context of hierarchical bayesian models is described in all these examples the question arises as to the rate of convergence of the markov chain there is no unified theory to answer this question especially in the case of continuous densities on r n and currently many different approaches are being attempted for the different types of simulation methods one direction of research uses doeblin type bounds on bounded sets together with geometric drift conditions or some form of lyapounov functions to obtain rather general results on exponential convergence for continuous n densities on r see mengersen and tweedie 1996 for the metropolis algorithm and rosenthal 1995 for the gibbs sampler these results are then applied to certain special cases and yield bounds on the rate of convergence in variational norm distance another approach analyzes a discretized version of the problem using general discrete state space markov chain techniques see frieze kannan and polson 1994 finally there are attempts to use spectral analysis to obtain rates of convergence in the l sense where the markov chain operator is considered 2 as a bounded operator in the hilbert space of square integrable functions

bayesian inference for the uncertainty distribution of computer model outputs we consider a problem of inference for the output of a computationally expensive computer model we suppose that the model is to be used in a context where the values of one or more inputs are uncertain so that the input configuration is a random variable we require to make inference about the induced distribution of the output this distribution is called the uncertainty distribution and the general problem is known to users of computer models as uncertainty analysis to be specific we develop bayesian inference for the distribution and density functions of the model output modelling the output as a function of its inputs as a gaussian process we derive expressions for the posterior mean and variance of the distribution and density functions based on data comprising observed outputs at a sample of input configurations we show that direct computation of these expressions may encounter numerical difficulties we develop an alternative approach based on simulating approximate realisations from the posterior distribution of the output function two examples are given to illustrate our methods

riemann manifold langevin and hamiltonian monte carlo methods the paper proposes metropolis adjusted langevin and hamiltonian monte carlo sampling methods defined on the riemann manifold to resolve the shortcomings of existing monte carlo algorithms when sampling from target densities that may be high dimensional and exhibit strong correlations the methods provide fully automated adaptation mechanisms that circumvent the costly pilot runs that are required to tune proposal densities for metropolis hastings or indeed hamiltonian monte carlo and metropolis adjusted langevin algorithms this allows for highly efficient sampling even in very high dimensions where different scalings may be required for the transient and stationary phases of the markov chain the methodology proposed exploits the riemann geometry of the parameter space of statistical models and thus automatically adapts to the local structure when simulating paths across this manifold providing highly efficient convergence and exploration of the target density the performance of these riemann manifold monte carlo methods is rigorously assessed by performing inference on logistic regression models log gaussian cox point processes stochastic volatility models and bayesian estimation of dynamic systems described by non linear differential equations substantial improvements in the time normalized effective sample size are reported when compared with alternative sampling approaches matlab code that is available from http www ucl ac uk statistics research rmhmc allows replication of all the results reported

control functionals for monte carlo integration summary a non parametric extension of control variates is presented these leverage gradient information on the sampling density to achieve substantial variance reduction it is not required that the sampling density be normalized the novel contribution of this work is based on two important insights a trade off between random sampling and deterministic approximation and a new gradient based function space derived from stein s identity unlike classical control variates our estimators improve rates of convergence often requiring orders of magnitude fewer simulations to achieve a fixed level of precision theoretical and empirical results are presented the latter focusing on integration problems arising in hierarchical models and models based on non linear ordinary differential equations

on the irreducibility of a markov chain defined on a space of genotype configurations by a sampling scheme techniques from image processing are now being considered for use in pedigree analysis although the method that we develop here was motivated by considering it as an analogue to the gibbs sampler we justify it independently using the ergodic theorem for aperiodic irreducible finite markov chains while it is trivial to show that aperiodicity holds irreducibility is a more interesting condition proofs of irreducibility are provided for some special cases and counterexamples are provided for others where irreducibility does not hold an alternative method using relaxed genetic parameters and rejection is recommended and justified

two stage importance sampling with mixture proposals for importance sampling is multiple proposals can be combined to address different aspects of a target distribution there are various methods for is with multiple proposals including hesterberg s stratified is estimator owen and zhou s regression estimator and tan s maximum likelihood estimator for the problem of efficiently allocating samples to different proposals it is natural to use a pilot sample to select the mixture proportions before the actual sampling and estimation however most current discussions are in an empirical sense for such a two stage procedure in this article we establish a theoretical framework of applying the two stage procedure for various methods including the asymptotic properties and the choice of the pilot sample size by our simulation studies these two stage estimators can outperform estimators with naive choices of mixture proportions furthermore while owen and zhou s and tan s estimators are designed for estimating normalizing constants we extend their usa

constrained monte carlo maximum likelihood for dependent data maximum likelihood estimates mles in autologistic models and other exponential family models for dependent data can be calculated with markov chain monte carlo methods the metropolis algorithm or the gibbs sampler which simulate ergodic markov chains having equilibrium distributions in the model from one realization of such a markov chain a monte carlo approximant to the whole likelihood function can be constructed the parameter value if any maximizing this function approximates the mle

sequential monte carlo p values summary the assessment of statistical significance by monte carlo simulation may be costly in computer time this paper looks at a number of ways of calculating exact monte carlo p values by sequential sampling such p values are shown to have properties similar to those obtained by sampling with a fixed sample size both standard and generalized monte carlo procedures are discussed and in particular a sequential method is proposed for dealing with situations in which values can only be conveniently generated using a markov chain conditioned to pass through the observed data

spatial statistics and bayesian computation on wednesday may 6th 1992 professor b w silverman in the chair summary markov chain monte carlo mcmc algorithms such as the gibbs sampler have provided a bayesian inference machine in image analysis and in other areas of spatial statistics for several years founded on the pioneering ideas of ulf grenander more recently the observation that hyperparameters can be included as part of the updating schedule and the fact that almost any multivariate distribution is equivalently a markov random field has opened the way to the use of mcmc in general bayesian computation in this paper we trace the early development of mcmc in bayesian inference review some recent computational progress in statistical physics based on the introduction of auxiliary variables and discuss its current and future relevance in bayesian applications we briefly describe a simple mcmc implementation for the bayesian analysis of agricultural field experiments with which we have some practical experience

optimal adaptive policies for markov decision processes in this paper we consider the problem of adaptive control for markov decision processes we give the explicit form for a class of adaptive policies that possess optimal increase rate properties for the total expected finite horizon reward under sufficient assumptions of finite state action spaces and irreducibility of the transition law a main feature of the proposed policies is that the choice of actions at each state and time period is based on indices that are inflations of the right hand side of the estimated average reward optimality equations

bayesian adaptive stochastic process termination this paper considers the problem of optimally terminating a number of stochastic processes when the time varying random rewards have distributions belonging to an exponential family with an unknown parameter the problem is formulated as a bayesian adaptive control model with the objective of minimizing the difference between the expected reward and the optimal reward when the parameter is known the paper establishes an asymptotic lower bound on this difference and constructs policies based on a kullback leibler index that obtain this lower bound the results are applied to models of tree harvesting and destructive testing a simulation study shows that these policies are efficient when the number of processes is large