hierarchical generalised linear models a synthesis of generalised linear models random effect models and structured dispersions summary hierarchical generalised linear models are developed as a synthesis of generalised linear models mixed linear models and structured dispersions we generalise the restricted maximum likelihood method for the estimation of dispersion to the wider class and show how the joint fitting of models for mean and dispersion can be expressed by two interconnected generalised linear models the method allows models with i any combination of a generalised linear model distribution for the response with any conjugate distribution for the random effects ii structured dispersion components iii different link and variance functions for the fixed and random effects and iv the use of quasilikelihoods in place of likelihoods for either or both of the mean and dispersion models inferences can be made by applying standard procedures in particular those for model checking to components of either generalised linear model we also show by numerical studies that the new method gives an efficient estimation procedure for substantial class of models of practical importance likelihood type inference is extended to this wide class of models in a unified way

approximate likelihood for large irregularly spaced spatial data likelihood approaches for large irregularly spaced spatial datasets are often very difficult if not infeasible to implement due to computational limitations even when we can assume normality exact calculations of the likelihood for a gaussian spatial process observed at n locations requires o n3 operations we present a version of whittle s approximation to the gaussian log likelihood for spatial regular lattices with missing values and for irregularly spaced datasets this method requires o nlog2n operations and does not involve calculating determinants we present simulations and theoretical results to show the benefits and the performance of the spatial likelihood approximation method presented here for spatial irregularly spaced datasets and lattices with missing values we apply these methods to estimate the spatial structure of sea surface temperatures using satellite data with missing values

partial non gaussian state space summary in this paper we suggest the use of simulation techniques to extend the applicability of the usual gaussian state space filtering and smoothing techniques to a class of nongaussian time series models this allows a fully bayesian or maximum likelihood analysis of some interesting models including outlier models discrete markov chain components multiplicative models and stochastic variance models finally we discuss at some length the use of a non gaussian model to seasonally adjust the published money supply figures

ergodicity of foliations with singularities abstract this paper deals with the question of ergodicity of foliations defined by smooth closed one forms on a compact manifold without boundary

likelihood analysis of non gaussian measurement time series summary in this paper we provide methods for estimating non gaussian time series models these techniques rely on markov chain monte carlo to carry out simulation smoothing and bayesian posterior analysis of parameters and on importance sampling to estimate the likelihood function for classical inference the time series structure of the models is used to ensure that our simulation algorithms are efficient

an estimating function approach to inference for inhomogeneous neyman scott processes this article is concerned with inference for a certain class of inhomogeneous neyman scott point processes depending on spatial covariates regression parameter estimates obtained from a simple estimating function are shown to be asymptotically normal when the mother intensity for the neyman scott process tends to infinity clustering parameter estimates are obtained using minimum contrast estimation based on the k function the approach is motivated and illustrated by applications to point pattern data from a tropical rain forest plot

gauss markov measure field models for low level vision we present a class of models derived from classical discrete markov random fields that may be used for the solution of ill posed problems in image processing and in computational vision they lead to reconstruction algorithms that are flexible computationally efficient and biologically plausible to illustrate their use we present their application to the reconstruction of the dominant orientation and direction fields to the classification of multiband images and to image quantization and filtering

the computation of visible surface representations a computational theory of visible surface representations is developed the visible surface reconstruction process that computes these quantitative representations unifies formal solutions to the key problems of 1 integrating multiscale constraints on surface depth and orientation from multiple visual sources 2 interpolating dense piecewise smooth surfaces from these constraints 3 detecting surface depth and orientation discontinuities to apply boundary conditions on interpolation and 4 structuring large scale distributed surface representations to achieve computational efficiency visible surface reconstruction is an inverse problem a well posed variational formulation results from the use of a controlled continuity surface model discontinuity detection amounts to the identification of this generic model s distributed parameters from the data finite element shape primitives yield a local discretization of the variational principle the result is an efficient algorithm for visible surface reconstruction

bayesian measures of model complexity and fit summary we consider the problem of comparing complex hierarchical models in which the number of parameters is not clearly defined using an information theoretic argument we derive a measure pd for the effective number of parameters in a model as the difference between the posterior mean of the deviance and the deviance at the posterior means of the parameters of interest in general pd approximately corresponds to the trace of the product of fisher s information and the posterior covariance which in normal models is the trace of the hat matrix projecting observations onto fitted values its properties in exponential families are explored the posterior mean deviance is suggested as a bayesian measure of fit or adequacy and the contributions of individual observations to the fit and complexity can give rise to a diagnostic plot of deviance residuals against leverages adding pd to the posterior mean deviance gives a deviance information criterion for comparing models which is related to other information criteria and has an approximate decision theoretic justification the procedure is illustrated in some examples and comparisons are drawn with alternative bayesian and classical proposals throughout it is emphasized that the quantities required are trivial to compute in a markov chain monte carlo analysis

expectation propagation for likelihood free inference many models of interest in the natural and social sciences have no closed form likelihood function which means that they cannot be treated using the usual techniques of statistical inference in the case where such models can be efficiently simulated bayesian inference is still possible thanks to the approximate bayesian computation abc algorithm although many refinements have been suggested abc inference is still far from routine abc is often excruciatingly slow due to very low acceptance rates in addition abc requires introducing a vector of summary statistics s y the choice of which is relatively arbitrary and often require some trial and error making the whole process laborious for the user we introduce in this work the ep abc algorithm which is an adaptation to the likelihood free context of the variational approximation algorithm known as expectation propagation the main advantage of ep abc is that it is faster by a few orders of magnitude than standard algorithms while producing

efficient recursions for general factorisable models let n s valued categorical variables be jointly distributed according to a distribution known only up to an unknown normalising constant for an unnormalised joint likelihood expressible as a product of factors we give an algebraic recursion which can be used for computing the normalising constant and other summations a saving in computation is achieved when each factor contains a lagged subset of the components combining in the joint distribution with maximum computational efficiency as the subsets attain their minimum size if each subset contains at most r 1 of the n components in the joint distribution we term this a lag r model whose normalising constant can be computed using a forward recursion in o sr 1 computations as opposed to o sn for the direct computation we show how a lag r model represents a markov random field and allows a neighbourhood structure to be related to the unnormalised joint likelihood we illustrate the method by showing how the normalising constant of the ising or autologistic model can be computed

recursive computing and simulation free inference for general factorizable models summary we illustrate how the recursive algorithm of reeves pettitt 2004 for general factorizable models can be extended to allow exact sampling maximization of distributions and computation of marginal distributions all of the methods we describe apply to discrete valued markov random fields with nearest neighbour integrations defined on regular lattices in particular we illustrate that exact inference can be performed for hidden autologistic models defined on moderately sized lattices in this context we offer an extension of this methodology which allows approximate inference to be carried out for larger lattices without resorting to simulation techniques such as markov chain monte carlo in particular our work offers the basis for an automatic inference machine for such models

on gibbs sampling for state space models summary we show how to use the gibbs sampler to carry out bayesian inference on a linear state space model with errors that are a mixture of normals and coefficients that can switch over time our approach simultaneously generates the whole of the state vector given the mixture and coefficient indicator variables and simultaneously generates all the indicator variables conditional on the state vectors the states are generated efficiently using the kalman filter we illustrate our approach by several examples and empirically compare its performance to another gibbs sampler where the states are generated one at a time the empirical results suggest that our approach is both practical to implement and dominates the gibbs sampler that generates the states one at a time

nonseparable stationary covariance functions for space time data geostatistical approaches to spatiotemporal prediction in environmental science climatology meteorology and related fields rely on appropriate covariance models this article proposes general classes of nonseparable stationary covariance functions for spatiotemporal random processes the constructions are directly in the space time domain and do not depend on closed form fourier inversions the model parameters can be associated with the data s spatial and temporal structures respectively and a covariance model with a readily interpretable space time interaction parameter is fitted to wind data from ireland

predictive model assessment for count data we discuss tools for the evaluation of probabilistic forecasts and the critique of statistical models for count data our proposals include a nonrandomized version of the probability integral transform marginal calibration diagrams and proper scoring rules such as the predictive deviance in case studies we critique count regression models for patent data and assess the predictive performance of bayesian age period cohort models for larynx cancer counts in germany the toolbox applies in bayesian or classical and parametric or nonparametric settings and to any type of ordered discrete outcomes

hierarchical generalized linear models we consider hierarchical generalized linear models which allow extra error components in the linear predictors of generalized linear models the distribution of these components is not restricted to be normal this allows a broader class of models which includes generalized linear mixed models we use a generalization of henderson s joint likelihood called a hierarchical or h likelihood for inferences from hierarchical generalized linear models this avoids the integration that is necessary when marginal likelihood is used under appropriate conditions maximizing the h likelihood gives fixed effect estimators that are asymptotically equivalent to those obtained from the use of marginal likelihood at the same time we obtain the random effect estimates that are asymptotically best unbiased predictors an adjusted profile h likelihood is shown to give the required generalization of restricted maximum likelihood for the estimation of dispersion components a scaled deviance test for the goodness of fit a model selection criterion for choosing between various dispersion models and a graphical method for checking the distributional assumption of random effects are proposed the ideas of quasi likelihood and extended quasi likelihood are generalized to the new class we give examples of the poisson gamma binomial beta and gamma inverse gamma hierarchical generalized linear models a resolution is proposed for the apparent difference between population averaged and subject specific models a unified framework is provided for viewing and extending many existing methods

time series analysis of non gaussian observations based on state space models from both classical and bayesian perspectives the analysis of non gaussian time series by using state space models is considered from both classical and bayesian perspectives the treatment in both cases is based on simulation using importance sampling and antithetic variables markov chain monte carlo methods are not employed non gaussian disturbances for the state equation as well as for the observation equation are considered methods for estimating conditional and posterior means of functions of the state vector given the observations and the mean square errors of their estimates are developed these methods are extended to cover the estimation of conditional and posterior densities and distribution functions the choice of importance sampling densities and antithetic variables is discussed the techniques work well in practice and are computationally efficient their use is illustrated by applying them to a univariate discrete time series a series with outliers and a volatility series

spatiotemporal prediction for log gaussian cox processes space time point pattern data have become more widely available as a result of technological developments in areas such as geographic information systems we describe a flexible class of space time point processes our models are cox processes whose stochastic intensity is a space time ornstein uhlenbeck process we develop moment based methods of parameter estimation show how to predict the underlying intensity by using a markov chain monte carlo approach and illustrate the performance of our methods on a synthetic data set

auxiliary mixture sampling for parameter driven models of time series of counts with applications to state space modelling we consider parameter driven models of time series of counts where the observations are assumed to arise from a poisson distribution with a mean changing over time according to a latent process estimation of these models is carried out within a bayesian framework using data augmentation and markov chain monte carlo methods we suggest a new auxiliary mixture sampler which possesses a gibbsian transition kernel where we draw from full conditional distributions belonging to standard distribution families only emphasis lies on application to state space modelling of time series of counts but we show that auxiliary mixture sampling may be applied to a wider range of parameter driven models including random effects models and panel data models based on the poisson distribution

sizer for exploration of structures in curves abstract in the use of smoothing methods in data analysis an important question is which observed features are really there as opposed to being spurious sampling artifacts an approach is described based on scale space ideas originally developed in the computer vision literature assessment of significant zero crossings of derivatives results in the sizer map a graphical device for display of significance of features with respect to both location and scale here scale means level of resolution that is bandwidth

an assessment of estimation methods for generalized linear mixed models with binary outcomes two main classes of methodology have been developed for addressing the analytical intractability of generalized linear mixed models likelihood based methods and bayesian methods likelihood based methods such as the penalized quasi likelihood approach have been shown to produce biased estimates especially for binary clustered data with small clusters sizes more recent methods using adaptive gaussian quadrature perform well but can be overwhelmed by problems with large numbers of random effects and efficient algorithms to better handle these situations have not yet been integrated in standard statistical packages bayesian methods although they have good frequentist properties when the model is correct are known to be computationally intensive and also require specialized code limiting their use in practice in this article we introduce a modification of the hybrid approach of capanu and begg 2011 biometrics 67 371 380 as a bridge between the likelihood based and bayesian approaches by employing bayesian estimation for the variance components followed by laplacian estimation for the regression coefficients we investigate its performance as well as that of several likelihood based methods in the setting of generalized linear mixed models with binary outcomes we apply the methods to three datasets and conduct simulations to illustrate their properties simulation results indicate that for moderate to large numbers of observations per random effect adaptive gaussian quadrature and the laplacian approximation are very accurate with adaptive gaussian quadrature preferable as the number of observations per random effect increases the hybrid approach is overall similar to the laplace method and it can be superior for data with very sparse random effects copyright 2013 john wiley sons ltd

structured additive regression for categorical space time data a mixed model approach motivated by a space time study on forest health with damage state of trees as the response we propose a general class of structured additive regression models for categorical responses allowing for a flexible semiparametric predictor nonlinear effects of continuous covariates time trends and interactions between continuous covariates are modeled by penalized splines spatial effects can be estimated based on markov random fields gaussian random fields or two dimensional penalized splines we present our approach from a bayesian perspective with inference based on a categorical linear mixed model representation the resulting empirical bayes method is closely related to penalized likelihood estimation in a frequentist setting variance components corresponding to inverse smoothing parameters are estimated using approximate restricted maximum likelihood in simulation studies we investigate the performance of different choices for the spatial effect compare the empirical bayes approach to competing methodology and study the bias of mixed model estimates as an application we analyze data from the forest health survey

monte carlo maximum likelihood estimation for non gaussian state space models state space models are considered for observations which have non gaussian distributions we obtain accurate approximations to the loglikelihood for such models by monte carlo simulation devices are introduced which improve the accuracy of the approximations and which increase computational efficiency the loglikelihood function is maximised numerically to obtain estimates of the unknown hyperparameters standard errors of the estimates due to simulation are calculated details are given for the important special cases where the observations come from an exponential family distribution and where the observation equation is linear but the observation errors are non gaussian the techniques are illustrated with a series for which the observations have a poisson distribution and a series for which the observation errors have a t distribution

geometric convergence and central limit theorems for multidimensional hastings and metropolis algorithms we develop results on geometric ergodicity of markov chains and apply these and other recent results in markov chain theory to multidimensional hastings and metropolis algorithms for those based on random walk candidate distributions we find sufficient conditions for moments and moment generating functions to converge at a geometric rate to a prescribed distribution by phrasing the conditions in terms of the curvature of the densities we show that the results apply to all distributions with positive densities in a large class which encompasses many commonly used statistical forms from these results we develop central limit theorems for the metropolis algorithm converse results showing non geometric convergence rates for chains where the rejection rate is not bounded away from unity are also given these show that the negative definiteness property is not redundant

statistical applications of the multivariate skew normal distribution azzalini and dalla valle have recently discussed the multivariate skew normal distribution which extends the class of normal distributions by the addition of a shape parameter the first part of the present paper examines further probabilistic properties of the distribution with special emphasis on aspects of statistical relevance inferential and other statistical issues are discussed in the following part with applications to some multivariate statistics problems illustrated by numerical examples finally a further extension is described which introduces a skewing factor of an elliptical density

nonparametric estimation of nonstationary spatial covariance structure abstract estimation of the covariance structure of spatial processes is a fundamental prerequisite for problems of spatial interpolation and the design of monitoring networks we introduce a nonparametric approach to global estimation of the spatial covariance structure of a random function z x t observed repeatedly at times ti i 1 t at a finite number of sampling stations xi i 1 2 n in the plane our analyses assume temporal stationarity but do not assume spatial stationarity or isotropy we analyze the spatial dispersions var z xi t z xj t as a natural metric for the spatial covariance structure and model these as a general smooth function of the geographic coordinates of station pairs xi xj the model is constructed in two steps first using nonmetric multidimensional scaling mds we compute a two dimensional representation of the sampling stations for which a monotone function of interpoint distances ij approximates the spatial dispersions mds transforms the problem

markov chain monte carlo for dynamic generalised linear models summary this paper presents a new methodological approach for carrying out bayesian inference about dynamic models for exponential family observations the approach is simulationbased and involves the use of markov chain monte carlo techniques a metropolishastings algorithm is combined with the gibbs sampler in repeated use of an adjusted version of normal dynamic linear models different alternative schemes based on sampling from the system disturbances and state parameters separately and in a block are derived and compared the approach is fully bayesian in obtaining posterior samples with state parameters and unknown hyperparameters illustrations with real datasets with sparse counts and missing values are presented extensions to accommodate more general evolution forms and distributions for observations and disturbances are outlined

approximating likelihoods for large spatial data sets summary likelihood methods are often difficult to use with large irregularly sited spatial data sets owing to the computational burden even for gaussian models exact calculations of the likelihood for n observations require o n3 operations since any joint density can be written as a product of conditional densities based on some ordering of the observations one way to lessen the computations is to condition on only some of the past observations when computing the conditional densities we show how this approach can be adapted to approximate the restricted likelihood and we demonstrate how an estimating equations approach allows us to judge the efficacy of the resulting approximation previous work has suggested conditioning on those past observations that are closest to the observation whose conditional density we are approximating through theoretical numerical and practical examples we show that there can often be considerable benefit in conditioning on some distant observations as well

fixed rank kriging for very large spatial data sets summary spatial statistics for very large spatial data sets is challenging the size of the data set n causes problems in computing optimal spatial predictors such as kriging since its computa tional cost is of order a73 in addition a large data set is often defined on a large spatial domain so the spatial process of interest typically exhibits non stationary behaviour over that domain a flexible family of non stationary covariance functions is defined by using a set of basis functions that is fixed in number which leads to a spatial prediction method that we call fixed rank kriging specifically fixed rank kriging is kriging within this class of non stationary covariance functions it relies on computational simplifications when n is very large for obtaining the spatial best linear unbiased predictor and its mean squared prediction error for a hidden spatial process a method based on minimizing a weighted frobenius norm yields best estimators of the covari ance function parameters which are then substituted into the fixed rank kriging equations the new methodology is applied to a very large data set of total column ozone data observed over the entire globe where n is of the order of hundreds of thousands

gaussian predictive process models for large spatial data sets with scientific data available at geocoded locations investigators are increasingly turning to spatial process models for carrying out statistical inference over the last decade hierarchical models implemented through markov chain monte carlo methods have become especially popular for spatial modelling given their flexibility and power to fit models that would be infeasible with classical methods as well as their avoidance of possibly inappropriate asymptotics however fitting hierarchical spatial models often involves expensive matrix decompositions whose computational complexity increases in cubic order with the number of spatial locations rendering such models infeasible for large spatial data sets this computational burden is exacerbated in multivariate settings with several spatially dependent response variables it is also aggravated when data are collected at frequent time points and spatiotemporal process models are used with regard to this challenge our contribution is to work with what we call predictive process models for spatial and spatiotemporal data every spatial or spatiotemporal process induces a predictive process model in fact arbitrarily many of them the latter models project process realizations of the former to a lower dimensional subspace thereby reducing the computational burden hence we achieve the flexibility to accommodate non stationary non gaussian possibly multivariate possibly spatiotemporal processes in the context of large data sets we discuss attractive theoretical properties of these predictive processes we also provide a computational template encompassing these diverse settings finally we illustrate the approach with simulated and real data sets

advanced distribution theory for sizer sizer is a powerful method for exploratory data analysis in this article approximations to the distributions underlying the simultaneous statistical inference are investigated and large improvements are made in the approximation using extreme value theory this results in improved size and also in an improved global inference version of sizer the main points are illustrated with real data and simulated examples

markov chain monte carlo methods for stochastic volatility models this paper is concerned with simulation based inference in generalized models of stochastic volatility defined by heavy tailed student t distributions with unknown degrees of freedom and exogenous variables in the observation and volatility equations and a jump component in the observation equation by building on the work of kim shephard and chib rev econom stud 65 1998 361 we develop efficient markov chain monte carlo algorithms for estimating these models the paper also discusses how the likelihood function of these models can be computed by appropriate particle filter methods computation of the marginal likelihood by the method of chib j amer statist assoc 90 1995 1313 is also considered the methodology is extensively tested and validated on simulated data and then applied in detail to daily returns data on the s p 500 index where several stochastic volatility models are formally compared under different priors on the parameters

approximating hidden gaussian markov random fields summary gaussian markov random field gmrf models are frequently used in a wide variety of applications in most cases parts of the gmrf are observed through mutually independent data hence the full conditional of the gmrf a hidden gmrf hgmrf is of interest we are concerned with the case where the likelihood is non gaussian leading to non gaussian hgmrf models several researchers have constructed block sampling markov chain monte carlo schemes based on approximations of the hgmrf by a gmrf using a second order expansion of the log density at or near the mode this is possible as the gmrf approximation can be sampled exactly with a known normalizing constant the markov property of the gmrf approximation yields computational efficiency the main contribution in the paper is to go beyond the gmrf approximation and to construct a class of non gaussian approximations which adapt automatically to the particular hgmrf that is under study the accuracy can be tuned by intuitive parameters to nearly any precision these non gaussian approximations share the same computational complexity as those which are based on gmrfs and can be sampled exactly with computable normalizing constants we apply our approximations in spatial disease mapping and model based geostatistical models with different likelihoods obtain procedures for block updating and construct metropolized independence samplers

robust modeling for inference from generalized linear model classes generalized linear models glms are widely used for data analysis however their maximum likelihood estimators can be sensitive to outliers we propose new statistical models that allow robust inferences from the glm class of models including poisson and binomial glms and their extension to generalized linear mixed models the likelihood score equations from the new models give estimators with bounded influence so that the resulting estimators are robust against outliers while maintaining high efficiency in the absence of outliers

posterior mode estimation by extended kalman filtering for multivariate dynamic generalized linear models abstract a family of multivariate dynamic generalized linear models is introduced as a general framework for the analysis of time series with observations from the exponential family besides common conditionally gaussian models this article deals with univariate models for counted and binary data and as the most interesting multivariate case models for nonstationary multicategorical time series for univariate responses a related yet different class of models has been introduced in a bayesian setting by west harrison and migon assuming conjugate prior posterior distributions for the natural parameter of the exponential family they derive an approximate filter for estimation of time varying states or parameters however their method raises some problems in particular in extending it to the multivariate case a different approach to filtering and smoothing is chosen in this article to avoid a full bayesian analysis based on numerical integration which becomes computationally critical for higher

posterior bimodality in the balanced one way random effects model summary although some researchers have examined posterior multimodality for specific richly parameterized models multimodality is not well characterized for any such model the paper characterizes bimodality of the joint and marginal posteriors for a conjugate analysis of the balanced one way random effects model with a flat prior on the mean this apparently simple model has surprisingly complex and even bizarre mode behaviour bimodality usually arises when the data indicate a much larger between groups variance than does the prior we examine an example in detail present a graphical display for describing bimodality and use real data sets from a statistical practice to shed light on the practical relevance of bimodality for these models

maximization by parts in likelihood inference this article presents and examines a new algorithm for solving a score equation for the maximum likelihood estimate in certain problems of practical interest the method circumvents the need to compute second order derivatives of the full likelihood function it exploits the structure of certain models that yield a natural decomposition of a very complicated likelihood function in this decomposition the first part is a log likelihood from a simply analyzed model and the second part is used to update estimates from the first part convergence properties of this iterative fixed point algorithm are examined and asymptotics are derived for estimators obtained using only a finite number of iterations illustrative examples considered in the article include multivariate gaussian copula models nonnormal random effects models generalized linear mixed models and state space models properties of the algorithm and of estimators are evaluated in simulation studies on a bivariate copula model and a nonnormal

spatial analyses of periodontal data using conditionally autoregressive priors having two classes of neighbor relations attachment loss the extent of a tooth s root in millimeters that is no longer attached to surrounding bone by periodontal ligament is often used to measure the current state of a patient s periodontal disease and monitor disease progression attachment loss data can be analyzed using a conditionally autoregressive car prior distribution that smooths fitted values toward neighboring values however it may be desirable to have more than one class of neighbor relation in the spatial structure so the different classes of neighbor relations can induce different degrees of smoothing for example we may wish to allow smoothing of neighbor pairs bridging the gap between teeth to differ from smoothing of pairs that do not bridge such gaps adequately modeling the spatial structure may improve the monitoring of periodontal disease progression this article develops a two neighbor relation car model to handle this situation and presents associated theory to help explain the sometimes unusual posterior distr

estimation and model identification for continuous spatial processes summary formal parameter estimation and model identification procedures for continuous domain spatial processes are introduced the processes are assumed to be adequately described by a linear model with residuals that follow a second order stationary gaussian random field and data are assumed to consist of noisy observations of the process at arbitrary sampling locations a general class of two dimensional rational spectral density functions with elliptic contours is used to model the spatial covariance function an iterative estimation procedure alleviates many of the computational difficulties of conventional maximum likelihood estimation for non lattice data the procedure is applied to several generated data sets and to an actual ground water data set

approximate inference in generalized linear mixed models statistical approaches to overdispersion correlated errors shrinkage estimation and smoothing of regression relationships may be encompassed within the framework of the generalized linear mixed model glmm given an unobserved vector of random effects observations are assumed to be conditionally independent with means that depend on the linear predictor through a specified link function and conditional variances that are specified by a variance function known prior weights and a scale factor the random effects are assumed to be normally distributed with mean zero and dispersion matrix depending on unknown variance components for problems involving time series spatial aggregation and smoothing the dispersion may be specified in terms of a rank deficient inverse covariance matrix approximation of the marginal quasi likelihood using laplace s method leads eventually to estimating equations based on penalized quasilikelihood or pql for the mean parameters and pseudo likelihood for the variances im

bayesian analysis of binary and polychotomous response data abstract a vast literature in statistics biometrics and econometrics is concerned with the analysis of binary and polychotomous response data the classical approach fits a categorical response regression model using maximum likelihood and inferences about the model are based on the associated asymptotic theory the accuracy of classical confidence statements is questionable for small sample sizes in this article exact bayesian methods for modeling categorical response data are developed using the idea of data augmentation the general approach can be summarized as follows the probit regression model for binary outcomes is seen to have an underlying normal regression structure on latent continuous data values of the latent data can be simulated from suitable truncated normal distributions if the latent data are known then the posterior distribution of the parameters can be computed using standard results for normal linear models draws from this posterior are used to sample new latent data and t

a bayesian model averaging approach for estimating the relative risk of mortality associated with heat waves in 105 u s cities estimating the risks heat waves pose to human health is a critical part of assessing the future impact of climate change in this article we propose a flexible class of time series models to estimate the relative risk of mortality associated with heat waves and conduct bayesian model averaging bma to account for the multiplicity of potential models applying these methods to data from 105 u s cities for the period 1987 2005 we identify those cities having a high posterior probability of increased mortality risk during heat waves examine the heterogeneity of the posterior distributions of mortality risk across cities assess sensitivity of the results to the selection of prior distributions and compare our bma results to a model selection approach our results show that no single model best predicts risk across the majority of cities and that for some cities heat wave risk estimation is sensitive to model choice although model averaging leads to posterior distributions with increased variance as compared to statistical inference conditional on a model obtained through model selection we find that the posterior mean of heat wave mortality risk is robust to accounting for model uncertainty over a broad class of models

variational bayes inference of spatial mixture models for segmentation mixture models are commonly used in the statistical segmentation of images for example they can be used for the segmentation of structural medical images into different matter types or of statistical parametric maps into activating and nonactivating brain regions in functional imaging spatial mixture models have been developed to augment histogram information with spatial regularization using markov random fields mrfs in previous work an approximate model was developed to allow adaptive determination of the parameter controlling the strength of spatial regularization inference was performed using markov chain monte carlo mcmc sampling however this approach is prohibitively slow for large datasets in this work a more efficient inference approach is presented this combines a variational bayes approximation with a second order taylor expansion of the components of the posterior distribution which would otherwise be intractable to variational bayes this provides inference on fully adaptive spatial mixture models an order of magnitude faster than mcmc we examine the behavior of this approach when applied to artificial data with different spatial characteristics and to functional magnetic resonance imaging statistical parametric maps

some covariance models for longitudinal count data with overdispersion a family of covariance models for longitudinal counts with predictive covariates is presented these models account for overdispersion heteroscedasticity and dependence among repeated observations the approach is a quasi likelihood regression similar to the formulation given by liang and zeger 1986 biometrika 73 13 22 generalized estimating equations for both the covariate parameters and the variance covariance parameters are presented large sample properties of the parameter estimates are derived the proposed methods are illustrated by an analysis of epileptic seizure count data arising from a study of progabide as an adjuvant therapy for partial seizures

gaussian processes for ordinal regression we present a probabilistic kernel approach to ordinal regression based on gaussian processes a threshold model that generalizes the probit function is used as the likelihood function for ordinal variables two inference techniques based on the laplace approximation and the expectation propagation algorithm respectively are derived for hyperparameter learning and model selection we compare these two gaussian process approaches with a previous ordinal regression method based on support vector machines on some benchmark and real world data sets including applications of ordinal regression to collaborative filtering and gene expression analysis experimental results on these data sets verify the usefulness of our approach

asymptotic normality and valid inference for gaussian variational approximation we derive the precise asymptotic distributional behavior of gaussian variational approximate estimators of the parameters in a single predictor poisson mixed model these results are the deepest yet obtained concerning the statistical properties of a variational approximation method moreover they give rise to asymptotically valid statistical inference a simulation study demonstrates that gaussian variational approximate confidence intervals possess good to excellent coverage properties and have a similar precision to their exact likelihood counterparts 1 introduction variational approximation methods are enjoying an increasing amount of development and use in statistical problems this raises questions regarding their statistical properties such as consistency of point estimators and validity of statistical inference we make significant inroads into answering such questions via thorough theoretical treatment of one of the simplest nontrivial settings for which variational approximation is beneficial the poisson mixed model with a single predictor variable and random intercept we call this the simple poisson mixed model the model treated here is also treated in 7 but there attention is confined to bounds and rates of convergence we improve upon their results by obtaining the asymptotic distributions of the estimators the results reveal that the estimators are asymptotically normal have negligible bias and that their variances decay at least as fast as m 1 w herem is the number of groups for the slope parameter the faster mn 1 rate is obtained where n is the number of repeated measures an important practical ramification of our theory is asymptotically valid statistical inference for the model parameters in particular a form of studentization leads to theoretically justifiable confidence intervals for all model parameters unlike those based on the exact likelihood all gaussian variational approximate point estimates and confidence intervals can be computed without the need for numer

gaussian processes for classification mean field algorithms we derive a mean field algorithm for binary classification with gaussian processes that is based on the tap approach originally proposed in statistical physics of disordered systems the theory also yields an approximate leave one out estimator for the generalization error which is computed with no extra computational cost we show that from the tap approach it is possible to derive both a simpler naive mean field theory and support vector machines svms as limiting cases for both mean field algorithms and support vector machines simulation results for three small benchmark data sets are presented they show that one may get state of the art performance by using the leave one out estimator for model selection and the built in leave one out estimators are extremely precise when compared to the exact leave one out estimate the second result is taken as strong support for the internal consistency of the mean field approach

fast sampling of gaussian markov random fields this paper demonstrates how gaussian markov random fields conditional autoregressions can be sampled quickly by using numerical techniques for sparse matrices the algorithm is general and efficient and expands easily to various forms for conditional simulation and evaluation of normalization constants we demonstrate its use by constructing efficient block updates in markov chain monte carlo algorithms for disease mapping

generalized autoregressive conditional heteroskedasticity a natural generalization of the arch autoregressive conditional heteroskedastic process introduced in engle 1982 to allow for past conditional variances in the current conditional variance equation is proposed stationarity conditions and autocorrelation structure for this new class of parametric models are derived maximum likelihood estimation and testing are also considered finally an empirical example relating to the uncertainty of the inflation rate is presented

fully exponential laplace approximations to expectations and variances of nonpositive functions abstract tierney and kadane 1986 presented a simple second order approximation for posterior expectations of positive functions they used laplace s method for asymptotic evaluation of integrals in which the integrand is written as f exp nh and the function h is approximated by a quadratic the form in which they applied laplace s method however was fully exponential the integrand was written instead as exp nh log f this allowed first order approximations to be used in the numerator and denominator of a ratio of integrals to produce a second order expansion for the ratio other second order expansions hartigan 1965 johnson 1970 lindley 1961 1980 mosteller and wallace 1964 require computation of more derivatives of the log likelihood function in this article we extend the fully exponential method to apply to expectations and variances of nonpositive functions to obtain a second order approximation to an expectation e g we use the fully exponential method to approximate

laplace approximation of high dimensional integrals summary it is shown that the usual laplace approximation is not a valid asymptotic approximation when the dimension of the integral is comparable with the limiting parameter n the formal laplace expansion for multidimensional integrals is given and used to construct asymptotic approximations for high dimensional integrals one example is considered in which the dimension of the integral is o n 1 2 and the relative error of the unmodified laplace approximation is 0 1 nevertheless it is possible to construct a valid asymptotic expansion by regrouping terms in the formal expansion according to asymptotic order in n

approximate bayesian inference for latent gaussian models by using integrated nested laplace approximations summary structured additive regression models are perhaps the most commonly used class of models in statistical applications it includes among others generalized linear models generalized additive models smoothing spline models state space models semiparametric regression spatial and spatiotemporal models log gaussian cox processes and geostatistical and geoadditive models we consider approximate bayesian inference in a popular subset of structured additive regression models latent gaussian models where the latent field is gaussian controlled by a few hyperparameters and with non gaussian response variables the posterior marginals are not available in closed form owing to the non gaussian response variables for such models markov chain monte carlo methods can be implemented but they are not without problems in terms of both convergence and computational time in some practical applications the extent of these problems is such that markov chain monte carlo sampling is simply not an appropriate tool for routine analysis we show that by using an integrated nested laplace approximation and its simplified version we can directly compute very accurate approximations to the posterior marginals the main benefit of these approximations is computational where markov chain monte carlo algorithms need hours or days to run our approximations provide more precise estimates in seconds or minutes another advantage with our approach is its generality which makes it possible to perform bayesian analysis in an automatic streamlined way and to compute model comparison criteria and various predictive measures so that models can be compared and the model under study can be challenged

parameter orthogonality and approximate conditional inference on propose une statistique du rapport de vraisemblance construite a partir de la distribution conditionnelle des observations etant donne les estimateurs du maximum de vraisemblance pour les parametres de nuisance

blow up of solutions to critical semilinear wave equations with variable coefficients abstract we verify the critical case p p 0 n of strauss conjecture 30 concerning the blow up of solutions to semilinear wave equations with variable coefficients in r n where n 2 the perturbations of laplace operator are assumed to be smooth and decay exponentially fast at infinity we also obtain a sharp lifespan upper bound for solutions with compactly supported data when p p 0 n the unified approach to blow up problems in all dimensions combines several classical ideas in order to generalize and simplify the method of zhou 43 and zhou han 45 exponential eigenfunctions of the laplacian 37 are used to construct the test function q for linear wave equation with variable coefficients and john s method of iterations 13 is augmented with the slicing method of agemi kurokawa and takamura 1 for lower bounds in the critical case

robust estimation in mixed linear models with non monotone missingness we introduce a model to account for abrupt changes among repeated measures with non monotone missingness development of likelihood inferences for such models is hard because it involves intractable integration to obtain the marginal likelihood we use hierarchical likelihood to overcome such difficulty abrupt changes among repeated measures can be well described by introducing random effects in the dispersion a simulation study shows that the resulting estimator is efficient robust against misspecification of fatness of tails for illustration we use a schizophrenic behaviour data presented by rubin and wu

disease mapping of stage specific cancer incidence data we propose two approaches for the spatial analysis of cancer incidence data with additional information on the stage of the disease at time of diagnosis the two formulations are extensions of commonly used models for multicategorical response data on an ordinal scale we include spatial and age group effects in both formulations which we estimate in a nonparametric smooth way more specifically we adopt a fully bayesian approach based on gaussian pairwise difference priors where additional smoothing parameters are treated as unknown as well we apply our methods to data on cervical cancer in the former german democratic republic the results suggest that there are large spatial differences in the stage proportions which indicates spatial variability with respect to the introduction and effectiveness of screening programs

practical filtering with sequential parameter learning summary the paper develops a simulation based approach to sequential parameter learning and filtering in general state space models our approach is based on approximating the target posterior by a mixture of fixed lag smoothing distributions parameter inference exploits a sufficient statistic structure and the methodology can be easily implemented by modifying state space smoothing algorithms we avoid reweighting particles and hence sample degeneracy problems that plague particle filters that use sequential importance sampling the method is illustrated by using two examples a benchmark auto regressive model with observation error and a high dimensional dynamic spatiotemporal model we show that the method provides accurate inference in the presence of outliers model misspecification and high dimensionality

approximate bayesian inference in conditionally independent hierarchical models parametric empirical bayes models abstract we consider two stage models of the kind used in parametric empirical bayes peb methodology calling them conditionally independent hierarchical models we suppose that there are k units which may be experimental subjects cities study centers etcetera at the first stage the observation vectors yi for units i 1 k are independently distributed with densities p yi i or more generally p yi i at the second stage the unit specific parameter vectors i are iid with densities p i the peb approach proceeds by regarding the second stage distribution as a prior and noting that if were known inference about could be based on its posterior since is not known the simplest peb methods estimate the parameter by maximum likelihood or some variant and then treat as if it were known to be equal to this estimate although this procedure is sometimes satisfactory a well known defect is that it neglects the uncertainty due to the estimation of in this article w

the conditional predictive ordinate for the normal distribution summary the conditional predictive ordinate cpo is a bayesian diagnostic which detects surprising observations it has been used in a variety of situations such as univariate samples the multivariate normal distribution and regression models results are presented about the most surprising observation which has minimum cpo for the multivariate normal distribution it is shown that the most surprising observation must lie at one of the vertices of the convex hull it is also shown that the observation with maximum mahalanobis distance from the sample mean must lie on the convex hull results are given for the expected number of vertices on the convex hull when the sample is contaminated an alternative closely related diagnostic the ratio ordinate measure is presented a numerical comparison of the two measures is given

the signal extraction approach to nonlinear regression and spline smoothing abstract this article shows how to fit a smooth curve polynomial spline to pairs of data values yi xi prior specification of a parametric functional form for the curve is not required the resulting curve can be used to describe the pattern of the data and to predict unknown values of y given x both point and interval estimates are produced the method is easy to use and the computational requirements are modest even for large sample sizes our method is based on maximum likelihood estimation of a signal in noise model of the data we use the kalman filter to evaluate the likelihood function and achieve significant computational advantages over previous approaches to this problem

accurate approximations for posterior moments and marginal densities abstract this article describes approximations to the posterior means and variances of positive functions of a real or vector valued parameter and to the marginal posterior densities of arbitrary i e not necessarily positive parameters these approximations can also be used to compute approximate predictive densities to apply the proposed method one only needs to be able to maximize slightly modified likelihood functions and to evaluate the observed information at the maxima nevertheless the resulting approximations are generally as accurate and in some cases more accurate than approximations based on third order expansions of the likelihood and requiring the evaluation of third derivatives the approximate marginal posterior densities behave very much like saddle point approximations for sampling distributions the principal regularity condition required is that the likelihood times prior be unimodal

approximations for binary gaussian process classification we provide a comprehensive overview of many recent algorithms for approximate inference in gaussian process models for probabilistic binary classification the relationships between several approaches are elucidated theoretically and the properties of the different algorithms are corroborated by experimental results we examine both 1 the quality of the predictive distributions and 2 the suitability of the different marginal likelihood approximations for model selection selecting hyperparameters and compare to a gold standard based on mcmc interestingly some methods produce good predictive distributions although their marginal likelihood approximations are poor strong conclusions are drawn about the methods the expectation propagation algorithm is almost always the method of choice unless the computational budget is very tight we also extend existing methods in various ways and provide unifying code implementing all approaches

strictly proper scoring rules prediction and estimation scoring rules assess the quality of probabilistic forecasts by assigning a numerical score based on the predictive distribution and on the event or value that materializes a scoring rule is proper if the forecaster maximizes the expected score for an observation drawn from the distributionf if he or she issues the probabilistic forecast f rather than g f it is strictly proper if the maximum is unique in prediction problems proper scoring rules encourage the forecaster to make careful assessments and to be honest in estimation problems strictly proper scoring rules provide attractive loss and utility functions that can be tailored to the problem at hand this article reviews and develops the theory of proper scoring rules on general probability spaces and proposes and discusses examples thereof proper scoring rules derive from convex functions and relate to information measures entropy functions and bregman divergences in the case of categorical variables we prove a rigorous version of the

monte carlo estimation for nonlinear non gaussian state space models we develop a proposal or importance density for state space models with a nonlinear non gaussian observation vector y p y and an unobserved linear gaussian signal vector p the proposal density is obtained from the laplace approximation of the smoothing density p y we present efficient algorithms to calculate the mode of p y and to sample from the proposal density the samples can be used for importance sampling and markov chain monte carlo methods the new results allow the application of these methods to state space models where the observation density p y is not log concave additional results are presented that lead to computationally efficient implementations we illustrate the methods for the stochastic volatility model with leverage

perturbative corrections for approximate inference in gaussian latent variable models expectation propagation ep provides a framework for approximate inference when the model under consideration is over a latent gaussian field with the approximation being gaussian we show how these approximations can systematically be corrected a perturbative expansion is made of the exact but intractable correction and can be applied to the model s partition function and other moments of interest the correction is expressed over the higher order cumulants which are neglected by ep s local matching of moments through the expansion we see that ep is correct to first order by considering higher orders corrections of increasing polynomial complexity can be applied to the approximation the second order provides a correction in quadratic time which we apply to an array of gaussian process and ising models the corrections generalize to arbitrarily complex approximating families which we illustrate on tree structured ising model approximations furthermore they provide a polynomial time assessment of the approximation error we also provide both theoretical and practical insights on the exactness of the ep solution

constrained monte carlo maximum likelihood for dependent data maximum likelihood estimates mles in autologistic models and other exponential family models for dependent data can be calculated with markov chain monte carlo methods the metropolis algorithm or the gibbs sampler which simulate ergodic markov chains having equilibrium distributions in the model from one realization of such a markov chain a monte carlo approximant to the whole likelihood function can be constructed the parameter value if any maximizing this function approximates the mle

bayesian classification with gaussian processes we consider the problem of assigning an input vector to one of m classes by predicting p c x for c 1 m for a two class problem the probability of class one given x is estimated by spl sigma y x where spl sigma y 1 1 e sup y a gaussian process prior is placed on y x and is combined with the training data to obtain predictions for new x points we provide a bayesian treatment integrating over uncertainty in y and in the parameters that control the gaussian process prior the necessary integration over y is carried out using laplace s approximation the method is generalized to multiclass problems m 2 using the softmax function we demonstrate the effectiveness of the method on a number of datasets

scaling limits for the transient phase of local metropolis hastings algorithms summary the paper considers high dimensional metropolis and langevin algorithms in their initial transient phase in stationarity these algorithms are well understood and it is now well known how to scale their proposal distribution variances for the random walk metropolis algorithm convergence during the transient phase is extremely regular to the extent that the algo rithm s sample path actually resembles a deterministic trajectory in contrast the langevin algorithm with variance scaled to be optimal for stationarity performs rather erratically we give weak convergence results which explain both of these types of behaviour and practical guidance on implementation based on our theory

a recursive algorithm for markov random fields summary we propose a recursive algorithm as a more useful alternative to the brook expansion for the joint distribution of a vector of random variables when the original formulation is in terms of the corresponding full conditional distributions as occurs for markov random fields usually in practical applications the computational load will still be excessive but then the algorithm can be used to obtain the componentwise full conditionals of a system after marginalising over some variables or the joint distribution of subsets of the variables conditioned on values of the remainder which is required for block gibbs sampling as an illustrative example we apply the algorithm in the simplest nontrivial setting of hidden markov chains more important we demonstrate how it applies to markov random fields on regular lattices and to perfect block gibbs sampling for binary systems