modeling of the fading statistics of wireless sensor network channels in industrial environments this paper presents an investigation of how to model the statistical properties of radio channels arising in industrial environments over long time horizons e g hours and days based on extensive measurement campaigns conducted at three different factory buildings it is shown that for mobile transceivers the fading characteristics are rayleigh or close to rayleigh however for transceivers mounted at fixed locations the use of conventional single fading distributions is not sufficient it is shown that a suitable model structure for describing the fading properties of the radio channels as measured by power is a mixture of gamma and compound gamma lognormal distributions furthermore the complexity of the model generally increases with the observation interval a model selection approach based on a connection between kullback s mean discrimination information and the log likelihood provides a robust choice of model structure we show that while a semi markov chain constitute a suitable model for the channel dynamics the time dependence of the data can be neglected in the estimation of the parameters of the mixture distributions neglecting the time dependence in the data leads to a more efficient parametrization moreover it is shown that the considered class of mixture distributions is identifiable for both continuous and quantized data under certain conditions and under those conditions a maximum likelihood under independence assumption estimator is shown to give consistent parameters also for data which are not independent the parameter estimates are obtained by maximizing the log likelihood using a genetic and a local interior point algorithm

expectation consistent approximate inference we propose a novel framework for approximations to intractable probabilistic models which is based on a free energy formulation the approximation can be understood as replacing an average over the original intractable distribution with a tractable one it requires two tractable probability distributions which are made consistent on a set of moments and encode different features of the original intractable distribution in this way we are able to use gaussian approximations for models with discrete or bounded variables which allow us to include non trivial correlations these are neglected in many other methods we test the framework on toy benchmark problems for binary variables on fully connected graphs and 2d grids and compare with other methods such as loopy belief propagation good performance is already achieved by using single nodes as tractable substructures significant improvements are obtained when a spanning tree is used instead

a nonparametric multidimensional latent class irt model in a bayesian framework we propose a nonparametric item response theory model for dichotomously scored items in a bayesian framework the model is based on a latent class lc formulation and it is multidimensional with dimensions corresponding to a partition of the items in homogenous groups that are specified on the basis of inequality constraints among the conditional success probabilities given the latent class moreover an innovative system of prior distributions is proposed following the encompassing approach in which the largest model is the unconstrained lc model a reversible jump type algorithm is described for sampling from the joint posterior distribution of the model parameters of the encompassing model by suitably post processing its output we then make inference on the number of dimensions i e number of groups of items measuring the same latent trait and we cluster items according to the dimensions when unidimensionality is violated the approach is illustrated by two examples on simulated data and two applications based on educational and quality of life data

semidirect products and reduction in mechanics this paper shows how to reduce a hamiltonian system on the cotangent bundle of a lie group to a hamiltonian system in the dual of the lie algebra of a semidirect product the procedure simplifies unifies and extends work of greene guillemin holm holmes kupershmidt marsden morrison ratiu sternberg and others the heavy top compressible fluids magnetohydrodynamics elasticity the maxwell vlasov equations and multifluid plasmas are presented as examples starting with lagrangian variables our method explains in a direct way why semidirect products occur so frequently in examples it also provides a framework for the systematic introduction of clebsch or canonical variables

predictive updating methods with application to bayesian classification we propose algorithms based on random draws from predictive distributions of unknown quantities missing values for instance this procedure can either be iterative which is a special variation of the gibbs sampler or be sequential which is a variation of sequential imputation in the latter case one can update the posterior distribution with new observations easily the methods proposed have intuitive statistical implications and can be generalized to accommodate other bayesian like procedures we display some applications of the method in connection with the bayesian bootstrap classification hierarchical models and selection of variables in particular as an application of the method we present a unified treatment of switching regression models driven by a general binary process and we develop a bayesian testing procedure some simulations and a real example are used to illustrate the methods proposed

extended object tracking using monte carlo methods this correspondence addresses the problem of tracking extended objects such as ships or a convoy of vehicles moving in urban environment two monte carlo techniques for extended object tracking are proposed an interacting multiple model data augmentation imm da algorithm and a modified version of the mixture kalman filter mkf of chen and liu called the mixture kalman filter modified mkfm the data augmentation da technique with finite mixtures estimates the object extent parameters whereas an interacting multiple model imm filter estimates the kinematic states position and speed of the manoeuvring object next the system model is formulated in a partially conditional dynamic linear pcdl form this affords us to propose two latent indicator variables characterizing respectively the motion mode and object size then an mkfm is developed with the pcdl model the imm da and the mkfm performance is compared with a combined imm particle filter imm pf algorithm with respect to accuracy and computational complexity the most accurate parameter estimates are obtained by the da algorithm followed by the mkfm and pf

modelling spatially correlated data via mixtures a bayesian approach the paper develops mixture models for spatially indexed data we confine attention to the case of finite typically irregular patterns of points or regions with prescribed spatial relationships and to problems where it is only the weights in the mixture that vary from one location to another our specific focus is on poisson distributed data and applications in disease mapping we work in a bayesian framework with the poisson parameters drawn from gamma priors and an unknown number of components we propose two alternative models for spatially dependent weights based on transformations of autoregressive gaussian processes in one the logistic normal model the mixture component labels are exchangeable in the other the grouped continuous model they are ordered reversible jump markov chain monte carlo algorithms for posterior inference are developed finally the performances of both of these formulations are examined on synthetic data and real data on mortality from a rare disease

bayesian random threshold estimation in a cox proportional hazards cure model in this paper we develop a bayesian approach to estimate a cox proportional hazards model that allows a threshold in the regression coefficient based on a threshold in a covariate when some fraction of subjects are not susceptible to the event of interest a data augmentation scheme with latent binary cure indicators is adopted to simplify the markov chain monte carlo implementation given the binary cure indicators the cox cure model reduces to a standard cox model and a logistic regression model furthermore the threshold detection problem reverts to a threshold problem in a regular cox model the baseline cumulative hazard for the cox model is formulated non parametrically using counting processes with a gamma process prior simulation studies demonstrate that the method provides accurate point and interval estimates application to a data set of oropharynx cancer patients suggests a significant threshold in age at diagnosis such that the effect of gender on disease specific survival changes after the threshold

improving variational methods via pairwise linear response identities inference methods are often formulated as variational approximations these approximations allow easy evaluation of statistics by marginalization or linear response but these estimates can be inconsistent we show that by introducing constraints on covariance one can ensure consistency of linear response with the variational parameters and in so doing inference of marginal probability distributions is improved for the bethe approximation and its generalizations improvements are achieved with simple choices of the constraints the approximations are presented as variational frameworks iterative procedures related to message passing are provided for finding the minima

bayesian computation via the gibbs sampler and related markov chain monte carlo methods the use of the gibbs sampler for bayesian computation is reviewed and illustrated in the context of some canonical examples other markov chain monte carlo simulation methods are also briefly described and comments are made on the advantages of sample based approaches for bayesian inference summaries

a new heterogeneous multidimensional unfolding procedure a variety of joint space multidimensional scaling mds methods have been utilized for the spatial analysis of two or three way dominance data involving subjects preferences choices considerations intentions etc so as to provide a parsimonious spatial depiction of the underlying relevant dimensions attributes stimuli and or subjects utility structures in the same joint space representation we demonstrate that care must be taken with respect to a key assumption in existent joint space mds models such that all estimated dimensions are utilized by each and every subject in the sample as this assumption can lead to serious distortions with respect to the derived joint spaces we develop a new bayesian dimension selection methodology for the multidimensional unfolding model which accommodates heterogeneity with respect to such dimensional utilization at the individual subject level for the analysis of two or three way dominance data a consumer psychology application regarding the preference for over the counter otc analgesics is provided we conclude by discussing the practical implications of the results as well as directions for future research

dealing with label switching in mixture models summary in a bayesian analysis of finite mixture models parameter estimation and clustering are sometimes less straightforward than might be expected in particular the common practice of estimating parameters by their posterior mean and summarizing joint posterior distributions by marginal distributions often leads to nonsensical answers this is due to the so called label switching problem which is caused by symmetry in the likelihood of the model parameters a frequent response to this problem is to remove the symmetry by using artificial identifiability constraints we demonstrate that this fails in general to solve the problem and we describe an alternative class of approaches relabelling algorithms which arise from attempting to minimize the posterior expected loss under a class of loss functions we describe in detail one particularly simple and general relabelling algorithm and illustrate its success in dealing with the label switching problem on two examples

detecting features in spatial point processes with clutter via model based clustering abstract we consider the problem of detecting features such as minefields or seismic faults in spatial point processes when there is substantial clutter we use model based clustering based on a mixture model for the process in which features are assumed to generate points according to highly linear multivariate normal densities and the clutter arises according to a spatial poisson process nonlinear features are represented by several densities giving a piecewise linear representation hierarchical model based clustering provides a first estimate of the features and this is then refined using the em algorithm the number of features is estimated from an approximation to its posterior distribution the method gives good results for the minefield and seismic fault problems software to implement it is available on the world wide web

joint bayesian analysis of birthweight and censored gestational age using finite mixture models birthweight and gestational age are closely related and represent important indicators of a healthy pregnancy customary modeling for birthweight is conditional on gestational age however joint modeling directly addresses the relationship between gestational age and birthweight and provides increased flexibility and interpreta tion as well as a strategy to avoid using gestational age as an intermediate variable previous proposals have utilized finite mixtures of bivariate regression models to incorporate well established risk factors into analysis e g sex and birth order of the baby maternal age race and tobacco use while examining the non gaussian shape of the joint birthweight and gestational age distribution we build on this approach by demonstrating the inferential prognostic benefits of joint modeling e g investigation of age inappropriate outcomes like small for gestational age and hence re emphasize the importance of capturing the non gaussian distributional shapes we additionally extend current models through a latent specification which admits interval censored gestational age we work within a bayesian framework which enables inference beyond customary parameter estimation and prediction as well as exact uncertainty assessment the model is applied to a portion of the 2003 2006 north carolina detailed birth record data n 336129 available through the children s environmental health initiative and is fitted using the bayesian methodology and markov chain monte carlo approaches

markov chains for exploring posterior distributions several markov chain methods are available for sampling from a posterior distribution two important examples are the gibbs sampler and the metropolis algorithm in addition several strategies are available for constructing hybrid algorithms this paper outlines some of the basic methods and strategies and discusses some related theoretical and practical issues on the theoretical side results from the theory of general state space markov chains can be used to obtain convergence rates laws of large numbers and central limit theorems for estimates obtained from markov chain methods these theoretical results can be used to guide the construction of more efficient algorithms for the practical use of markov chain methods standard simulation methodology provides several variance reduction techniques and also give guidance on the choice of sample size and allocation

expectation propagation for likelihood free inference many models of interest in the natural and social sciences have no closed form likelihood function which means that they cannot be treated using the usual techniques of statistical inference in the case where such models can be efficiently simulated bayesian inference is still possible thanks to the approximate bayesian computation abc algorithm although many refinements have been suggested abc inference is still far from routine abc is often excruciatingly slow due to very low acceptance rates in addition abc requires introducing a vector of summary statistics s y the choice of which is relatively arbitrary and often require some trial and error making the whole process laborious for the user we introduce in this work the ep abc algorithm which is an adaptation to the likelihood free context of the variational approximation algorithm known as expectation propagation the main advantage of ep abc is that it is faster by a few orders of magnitude than standard algorithms while producing

bayesian inference and state number determination for hidden markov models an application to the information content of the yield curve about inflation this paper is concerned with bayesian inference in hidden markov models focusing on switching regression models we propose a new methodology that delivers a joint estimation of the parameters and the number of regimes that have actually appeared in the studied sample the only prior information that is required on the latter quantity is an upper bound we implement a particle filter algorithm to compute the corresponding estimates applying this methodology to the information content of the yield curve regarding future inflation in four oecd countries we show that the predictive content for given country and combination of maturities is subject to regime switching

approximate inference for disease mapping with sparse gaussian processes gaussian process gp models are widely used in disease mapping as they provide a natural framework for modeling spatial correlations their challenges however lie in computational burden and memory requirements in disease mapping models the other difficulty is inference which is analytically intractable due to the non gaussian observation model in this paper we address both these challenges we show how to efficiently build fully and partially independent conditional fic pic sparse approximations for the gp in two dimensional surface and how to conduct approximate inference using expectation propagation ep algorithm and laplace approximation la we also propose to combine fic with a compactly supported covariance function to construct a computationally efficient additive model that can model long and short length scale spatial correlations simultaneously the benefit of these approximations is computational the sparse gps speed up the computations and reduce the memory requirements the posterior inference via ep and laplace approximation is much faster and is practically as accurate as via markov chain monte carlo

discriminant analysis using a multivariate linear mixed model with a normal mixture in the random effects distribution we have developed a method to longitudinally classify subjects into two or more prognostic groups using longitudinally observed values of markers related to the prognosis we assume the availability of a training data set where the subjects allocation into the prognostic group is known the proposed method proceeds in two steps as described earlier in the literature first multivariate linear mixed models are fitted in each prognostic group from the training data set to model the dependence of markers on time and on possibly other covariates second fitted mixed models are used to develop a discrimination rule for future subjects our method improves upon existing approaches by relaxing the normality assumption of random effects in the underlying mixed models namely we assume a heteroscedastic multivariate normal mixture for random effects inference is performed in the bayesian framework using the markov chain monte carlo methodology software has been written for the proposed method and it is freely available the methodology is applied to data from the dutch primary biliary cirrhosis study

sampling based approaches to calculating marginal densities abstract stochastic substitution the gibbs sampler and the sampling importance resampling algorithm can be viewed as three alternative sampling or monte carlo based approaches to the calculation of numerical estimates of marginal probability distributions the three approaches will be reviewed compared and contrasted in relation to various joint probability structures frequently encountered in applications in particular the relevance of the approaches to calculating bayesian posterior densities for a variety of structured models will be discussed and illustrated

assessing a mixture model for clustering with the integrated completed likelihood we propose an assessing method of mixture model in a cluster analysis setting with integrated completed likelihood for this purpose the observed data are assigned to unknown clusters using a maximum a posteriori operator then the integrated completed likelihood icl is approximated using the bayesian information criterion bic numerical experiments on simulated and real data of the resulting icl criterion show that it performs well both for choosing a mixture model and a relevant number of clusters in particular icl appears to be more robust than bic to violation of some of the mixture model assumptions and it can select a number of dusters leading to a sensible partitioning of the data

perturbation corrections in approximate inference mixture modelling applications bayesian inference is intractable for many interesting models making deterministic algorithms for approximate inference highly desirable unlike stochastic methods which are exact in the limit the accuracy of these approaches cannot be reasonably judged in this paper we show how low order perturbation corrections to an expectation consistent ec approximation can provide the necessary tools to ameliorate inference accuracy and to give an indication of the quality of approximation without having to resort to monte carlo methods further comparisons are given with variational bayes and parallel tempering pt combined with thermodynamic integration on a gaussian mixture model to obtain practical results we further generalize pt to temper from arbitrary distributions rather than a prior in bayesian inference

an alternate approach to constructing distortion product otoacoustic emission dpoae suppression tuning curves distortion product otoacoustic emission dpoae suppression tuning curves stcs typically are constructed using suppression criteria of 3 or 6 db this paper describes a technique for generating dpoae stcs using criteria ranging from 3 db to complete suppression the criterion effect was examined at various primary levels f2 4 khz in normal hearing ears as expected qerb and tip to tail differences decreased as probe level l2 increased qerb values were not systematically dependent on criterion for low l2 s tip to tail differences decreased as criterion increased given similarities in methodology dpoae stcs constructed from completely suppressed responses might be most appropriate for comparison to psychophysical tuning curves

estimating output targets to evaluate output specific efficiencies a statistical framework this paper presents a statistical framework for estimating output specific efficiencies for the 2 output case based upon a dea frontier estimate the key to this approach are estimated output targets which are used to identify proper reference points for efficiency comparisons being usually unobserved output targets of firms are modelled as missing data and estimated jointly with output specific efficiencies the involved likelihood function is analytically intractable so a data augmented bayesian approach is proposed for estimation and adapted to the present purpose some implementation issues are discussed leading to an empirical bayes setup with data informed priors a prove of scale invariance is provided

expectation propagation with factorizing distributions a gaussian approximation and performance results for simple models we discuss the expectation propagation ep algorithm for approximate bayesian inference using a factorizing posterior approximation for neural network models we use a central limit theorem argument to make ep tractable when the number of parameters is large for two types of models we show that ep can achieve optimal generalization performance when data are drawn from a simple distribution

stochastic relaxation gibbs distributions and the bayesian restoration of images we make an analogy between images and statistical mechanics systems pixel gray levels and the presence and orientation of edges are viewed as states of atoms or molecules in a lattice like physical system the assignment of an energy function in the physical system determines its gibbs distribution because of the gibbs distribution markov random field mrf equivalence this assignment also determines an mrf image model the energy function is a more convenient and natural mechanism for embodying picture attributes than are the local characteristics of the mrf for a range of degradation mechanisms including blurring nonlinear deformations and multiplicative or additive noise the posterior distribution is an mrf with a structure akin to the image model by the analogy the posterior distribution defines another imaginary physical system gradual temperature reduction in the physical system isolates low energy states annealing or what is the same thing the most probable states under the gibbs distribution the analogous operation under the posterior distribution yields the maximum a posteriori map estimate of the image given the degraded observations the result is a highly parallel relaxation algorithm for map estimation we establish convergence properties of the algorithm and we experiment with some simple pictures for which good restorations are obtained at low signal to noise ratios

bayesian estimation of movement and survival probabilities from capture recapture data summary the arnason schwarz model is usually used for estimating survival and movement probabilities of animal populations from capture recapture data the missing data structure of this capture recapture model is exhibited and summarised via a directed graph representation taking advantage of this structure we implement a gibbs sampling algorithm from which bayesian estimates and credible intervals for survival and movement probabilities are derived convergence of the algorithm is proved using a duality principle we illustrate our approach through a real example

monte carlo methods for adaptive sparse approximations of time series this paper deals with adaptive sparse approximations of time series the work is based on a bayesian specification of the shift invariant sparse coding model to learn approximations for a particular class of signals two different learning strategies are discussed the first method uses a gradient optimization technique commonly employed in sparse coding problems the other method is novel in this context and is based on a sampling estimate to approximate the gradient in the first approach we compare two monte carlo estimation techniques gibbs sampling and a novel importance sampling method the second approach is based on a direct sample estimate and uses an extension of the gibbs sampler used with the first approach both approaches allow the specification of different prior distributions and we here introduce a novel mixture prior based on a modified rayleigh distribution experiments demonstrate that all gibbs sampler based methods show comparable performance the importance sampler was found to work nearly as well as the gibbs sampler on smaller problems in terms of estimating the model parameters however the method performed substantially worse on estimating the sparse coefficients for large problems we found that the combination of a subset selection heuristic with the gibbs sampling approaches can outperform previous suggested methods in addition the methods studied here are flexible and allow the incorporation of additional prior knowledge such as the nonnegativity of the approximation coefficients which was found to offer additional benefits where applicable

stochastic sampling algorithms for state estimation of jump markov linear systems jump markov linear systems are linear systems whose parameters evolve with time according to a finite state markov chain given a set of observations our aim is to estimate the states of the finite state markov chain and the continuous in space states of the linear system the computational cost in computing conditional mean or maximum a posteriori map state estimates of the markov chain or the state of the jump markov linear system grows exponentially in the number of observations we present three globally convergent algorithms based on stochastic sampling methods for state estimation of jump markov linear systems the cost per iteration is linear in the data length the first proposed algorithm is a data augmentation da scheme that yields conditional mean state estimates the second proposed scheme is a stochastic annealing sa version of da that computes the joint map sequence estimate of the finite and continuous states finally a metropolis hastings da scheme based on sa is designed to yield the map estimate of the finite state markov chain convergence results of the three above mentioned stochastic algorithms are obtained computer simulations are carried out to evaluate the performances of the proposed algorithms the problem of estimating a sparse signal developing from a neutron sensor based on a set of noisy data from a neutron sensor and the problem of narrow band interference suppression in spread spectrum code division multiple access cdma systems are considered

bayesian analysis of mixtures of factor analyzers for bayesian inference on the mixture of factor analyzers natural conjugate priors on the parameters are introduced and then a gibbs sampler that generates parameter samples following the posterior is constructed in addition a deterministic estimation algorithm is derived by taking modes instead of samples from the conditional posteriors used in the gibbs sampler this is regarded as a maximum a posteriori estimation algorithm with hyperparameter search the behaviors of the gibbs sampler and the deterministic algorithm are compared on a simulation experiment

locus of generation for the 2 f1 f2 vs 2 f2 f1 distortion product otoacoustic emissions in normal hearing humans revealed by suppression tuning onset latencies and amplitude correlations the present study used distortion product otoacoustic emission dpoae suppression tuning curves stcs dpoae onset latencies ols and dpoae amplitude correlations to investigate the locus of generation of the 2 f1 f2 dpoae versus the 2 f2 f1 dpoae in humans the results of the tuning study revealed that for the 2 f1 f2 dpoae the tips of the stcs tuned consistently below the geometric mean gm frequency of the primary tones in contrast for the 2 f2 f1 dpoae stcs tuned above the gm of the primaries with 50 of the tip frequencies at or above the 2 f2 f1 frequency place when the average ratio of the 2 f2 f1 to the 2 f1 f2 tip frequencies was computed a factor of 1 44 provided an estimate of the frequency shift needed to align the two dpoae generation sites other results showed that ols for the 2 f2 f1 dpoae were uniformly shorter than those for the 2 f1 f2 with differences at the low frequencies amounting to as much as 6 7 ms further for both dpoaes curves describing latency decreases as

hidden markov processes an overview of statistical and information theoretic aspects of hidden markov processes hmps is presented an hmp is a discrete time finite state homogeneous markov chain observed through a discrete time memoryless invariant channel in recent years the work of baum and petrie 1966 on finite state finite alphabet hmps was expanded to hmps with finite as well as continuous state spaces and a general alphabet in particular statistical properties and ergodic theorems for relative entropy densities of hmps were developed consistency and asymptotic normality of the maximum likelihood ml parameter estimator were proved under some mild conditions similar results were established for switching autoregressive processes these processes generalize hmps new algorithms were developed for estimating the state parameter and order of an hmp for universal coding and classification of hmps and for universal decoding of hidden markov channels these and other related topics are reviewed

semiparametric estimation of a two component mixture model suppose that univariate data are drawn from a mixture of two distributions that are equal up to a shift parameter such a model is known to be nonidentifiable from a nonparametric viewpoint however if we assume that the unknown mixed distribution is symmetric we obtain the identifiability of this model which is then defined by four unknown parameters the mixing proportion two location parameters and the cumulative distribution function of the symmetric mixed distribution we propose estimators for these four parameters when no training data is available our estimators are shown to be strongly consistent under mild regularity assumptions and their convergence rates are studied their finite sample properties are illustrated by a monte carlo study and our method is applied to real data

truncating the loop series expansion for belief propagation recently chertkov and chernyak 2006b derived an exact expression for the partition sum normalization constant corresponding to a graphical model which is an expansion around the belief propagation bp solution by adding correction terms to the bp free energy one for each generalized loop in the factor graph the exact partition sum is obtained however the usually enormous number of generalized loops generally prohibits summation over all correction terms in this article we introduce truncated loop series bp tlsbp a particular way of truncating the loop series of chertkov chernyak by considering generalized loops as compositions of simple loops we analyze the performance of tlsbp in different scenarios including the ising model on square grids and regular random graphs and on promedas a large probabilistic medical diagnostic system we show that tlsbp often improves upon the accuracy of the bp solution at the expense of increased computation time we also show that the performance of tlsbp strongly depends on the degree of interaction between the variables for weak interactions truncating the series leads to significant improvements whereas for strong interactions it can be ineffective even if a high number of terms is considered

the use of distortion product otoacoustic emission suppression as an estimate of response growth distortion product otoacoustic emission dpoae levels in response to primary pairs f2 2 or 4 khz l2 ranging from 20 to 60 db spl l1 0 4l2 39 db were measured with and without suppressor tones f3 which varied from 1 octave below to 12 octave above f2 in normal hearing subjects suppressor level l3 varied from 5 to 85 db spl dpoae levels were converted into decrements by subtracting the level in the presence of the suppressor from the level in the absence of a suppressor dpoae decrement vs l3 functions showed steeper slopes when f3 f2 this pattern is similar to other measurements of response growth such as direct measures of basilar membrane motion single unit rate level functions suppression of basilar membrane motion and discharge rate suppression from lower animals as l2 increased the l3 necessary to maintain 3 db of suppression increased at a rate of about 1 db db when f3 was approximately equal to f2 but increased more slowly when f3 f2 functions r

a note on density model size testing let f sub k sub k spl ges 1 be a nested family of parametric classes of densities with finite vapnik chervonenkis dimension let f be a probability density belonging to f sub k sup where k sup is the unknown smallest integer such that f spl isin f sub k given a random sample x sub 1 x sub n drawn from f an integer k sub 0 spl ges 1 and a real number spl alpha spl isin 0 1 we introduce a new simple explicit spl alpha level consistent testing procedure of the null hypothesis h sub 0 k sup k sub 0 versus the alternative h sub 1 k sup spl ne k sub 0 our method is inspired by the combinatorial tools developed in devroye and lugosi and it includes a wide range of density models such as mixture models neural networks or exponential families

regression density estimation using smooth adaptive gaussian mixtures we model a regression density flexibly so that at each value of the covariates the density is a mixture of normals with the means variances and mixture probabilities of the components changing smo

information geometry of turbo and low density parity check codes since the proposal of turbo codes in 1993 many studies have appeared on this simple and new type of codes which give a powerful and practical performance of error correction although experimental results strongly support the efficacy of turbo codes further theoretical analysis is necessary which is not straightforward it is pointed out that the iterative decoding algorithm of turbo codes shares essentially similar ideas with low density parity check ldpc codes with pearl s belief propagation algorithm applied to a cyclic belief diagram and with the bethe approximation in statistical physics therefore the analysis of the turbo decoding algorithm will reveal the mystery of those similar iterative methods in this paper we recapture and extend the geometrical framework initiated by richardson to the information geometrical framework of dual affine connections focusing on both of the turbo and ldpc decoding algorithms the framework helps our intuitive understanding of the algorithms and opens a new prospect of further analysis we reveal some properties of these codes in the proposed framework including the stability and error analysis based on the error analysis we finally propose a correction term for improving the approximation

gaussian processes for classification mean field algorithms we derive a mean field algorithm for binary classification with gaussian processes that is based on the tap approach originally proposed in statistical physics of disordered systems the theory also yields an approximate leave one out estimator for the generalization error which is computed with no extra computational cost we show that from the tap approach it is possible to derive both a simpler naive mean field theory and support vector machines svms as limiting cases for both mean field algorithms and support vector machines simulation results for three small benchmark data sets are presented they show that one may get state of the art performance by using the leave one out estimator for model selection and the built in leave one out estimators are extremely precise when compared to the exact leave one out estimate the second result is taken as strong support for the internal consistency of the mean field approach

bayesian model selection analysis of a survival model with a surviving fraction we describe a methodology for model comparison in a bayesian framework as applied to survival with a surviving fraction this is illustrated using a case study of a randomized and controlled clinical trial investigating time until recurrence of depression posterior distributions are simulated using metropolis within gibbs markov chain methods models reflecting the effects of covariates on the log odds of being in the surviving fraction the log of the hazard rate as well as both and neither are compared bayes factors for comparing the models are obtained by using the bridge sampling method of calculating normalizing constants

a bayesian approach to finite mixture models in bioassay via data augmentation and gibbs sampling and its application to insecticide resistance after continued treatment with an insecticide within the population of the susceptible insects resistant strains will occur it is important to know whether there are any resistant strains what the proportions are and what the median lethal doses are for the insecticide lwin and martin 1989 biometrics 45 721 732 propose a probit mixture model and use the em algorithm to obtain the maximum likelihood estimates for the parameters this approach has difficulties in estimating the confidence intervals and in testing the number of components we propose a bayesian approach to obtaining the credible intervals for the location and scale of the tolerances in each component and for the mixture proportions by using data augmentation and gibbs sampler we use bayes factor for model selection and determining the number of components we illustrate the method with data published in lwin and martin 1989

markov chain monte carlo estimation of classical and dynamic switching and mixture models bayesian estimation of a very general model class where the distribution of the observations depends on a latent process taking values in a discrete state space is discussed in this article this model class covers finite mixture modeling markov switching autoregressive modeling and dynamic linear models with switching the consequences the unidentifiability of this type of model has on markov chain monte carlo mcmc estimation are explicitly dealt with joint bayesian estimation of all latent variables model parameters and parameters that determine the probability law of the latent process is carried out by a new mcmc method called permutation sampling the permutation sampler first samples from the unconstrained posterior which often can be done in a convenient multimove manner and then applies a permutation of the current labeling of the states of the latent process in a first run the random permutation sampler used selected the permutation randomly the mcmc output of the random permutation s

bayesian inference for non gaussian ornstein uhlenbeck stochastic volatility processes we develop markov chain monte carlo methodology for bayesian inference for non gaussian ornstein uhlenbeck stochastic volatility processes the approach introduced involves expressing the unobserved stochastic volatility process in terms of a suitable marked poisson process we introduce two specific classes of metropolis hastings algorithms which correspond to different ways of jointly parameterizing the marked point process and the model parameters the performance of the methods is investigated for different types of simulated data the approach is extended to consider the case where the volatility process is expressed as a superposition of ornstein uhlenbeck processes we apply our methodology to the us dollar deutschmark exchange rate

sufficient conditions for a nondegenerate hopf bifurcation in a generalized lagrange poisson problem in this paper we provide sufficient conditions for the existence of a nondegenerate hamiltonian hopf bifurcation at the equilibria corresponding to the rotation around the vertical axis of a symmetric gyrostat with a fixed point under the effect of an axially symmetric potential

practical bayesian density estimation using mixtures of normals abstract mixtures of normals provide a flexible model for estimating densities in a bayesian framework there are some difficulties with this model however first standard reference priors yield improper posteriors second the posterior for the number of components in the mixture is not well defined if the reference prior is used third posterior simulation does not provide a direct estimate of the posterior for the number of components we present some practical methods for coping with these problems finally we give some results on the consistency of the method when the maximum number of components is allowed to grow with the sample size

a statistical framework for expression based molecular classification in cancer summary genome wide measurement of gene expression is a promising approach to the identification of subclasses of cancer that are currently not differentiable but potentially biologically heterogeneous this type of molecular classification gives hope for highly individualized and more effective prognosis and treatment of cancer statistically the analysis of gene expression data from unclassified tumours is a complex hypothesis generating activity involving data exploration modelling and expert elicitation we propose a modelling framework that can be used to inform and organize the development of exploratory tools for classification our framework uses latent categories to provide both a statistical definition of differential expression and a precise experiment independent definition of a molecular profile it also generates natural similarity measures for traditional clustering and gives probabilistic statements about the assignment of tumours to molecular profiles

a semi parametric bayesian approach to average bioequivalence bioequivalence assessment is an issue of great interest development of statistical methods for assessing bioequivalence is an important area of research for statisticians bioequivalence is usually determined based on the normal distribution we relax this assumption and develop a semi parametric mixed model for bioequivalence data the proposed method is quite flexible and practically meaningful our proposed method is based on a mixture normal distribution and a non parametric bayesian approach using a dirichlet process mixture prior a numerical example illustrates the use of our procedure

expectation propagation in the large data limit summary expectation propagation ep is a widely successful algorithm for variational inference ep is an iterative algorithm used to approximate complicated distributions typically to find a gaussian approximation of posterior distributions in many applications of this type ep performs extremely well surprisingly despite its widespread use there are very few theoretical guarantees on gaussian ep and it is quite poorly understood to analyse ep we first introduce a variant of ep averaged ep which operates on a smaller parameter space we then consider averaged ep and ep in the limit of infinite data where the overall contribution of each likelihood term is small and where posteriors are almost gaussian in this limit we prove that the iterations of both averaged ep and ep are simple they behave like iterations of newton s algorithm for finding the mode of a function we use this limit behaviour to prove that ep is asymptotically exact and to obtain other insights into the dynamic behaviour of ep e g that it may diverge under poor initialization exactly like newton s method ep is a simple algorithm to state but a difficult one to study our results should facilitate further research into the theoretical properties of this important method

simulated annealing for maximum a posteriori parameter estimation of hidden markov models hidden markov models are mixture models in which the populations from one observation to the next are selected according to an unobserved finite state space markov chain given a realization of the observation process our aim is to estimate both the parameters of the markov chain and of the mixture model in a bayesian framework we present an original simulated annealing algorithm which in the same way as the em expectation maximization algorithm relies on data augmentation and is based on stochastic simulation of the hidden markov chain this algorithm is shown to converge toward the set of maximum a posteriori map parameters under suitable regularity conditions

bayesian analysis of cross section and clustered data treatment models this paper is concerned with the problem of determining the effect of a categorical treatment variable on a response given that the treatment is non randomly assigned and the response on any given subject is observed for one setting of the treatment we consider classes of models that are designed for such problems these models are subjected to a fully bayesian analysis based on markov chain monte carlo methods the analysis of the treatment effect is then based on amongst other things the posterior distribution of the potential outcomes counter factuals at the subject level which is obtained as a by product of the mcmc simulation procedure the analysis is extended to models with categorical treatments and binary and clustered outcomes the problem of model comparisons is also considered different aspects of the methodology are illustrated through two data examples

perturbative corrections for approximate inference in gaussian latent variable models expectation propagation ep provides a framework for approximate inference when the model under consideration is over a latent gaussian field with the approximation being gaussian we show how these approximations can systematically be corrected a perturbative expansion is made of the exact but intractable correction and can be applied to the model s partition function and other moments of interest the correction is expressed over the higher order cumulants which are neglected by ep s local matching of moments through the expansion we see that ep is correct to first order by considering higher orders corrections of increasing polynomial complexity can be applied to the approximation the second order provides a correction in quadratic time which we apply to an array of gaussian process and ising models the corrections generalize to arbitrarily complex approximating families which we illustrate on tree structured ising model approximations furthermore they provide a polynomial time assessment of the approximation error we also provide both theoretical and practical insights on the exactness of the ep solution

inference and model choice for sequentially ordered hidden markov models summary the system equation of a hidden markov model is rewritten to label the components by order of appearance and to make explicit the random behaviour of the number of components mt we argue that this reformulation is often a good way to achieve identifiability as it facilitates the interpretation of the posterior density and the estimation of the number of components that have appeared in a given sample we develop a sequential monte carlo algorithm for estimating the reformulated model which relies on particle filtering and gibbs sampling our algorithm has a computational cost that is similar to that of a markov chain monte carlo sampler and is much less likely to be affected by label switching i e the possibility of becoming trapped in a local mode of the posterior density the extension to transdimensional priors is also considered the approach is illustrated by two real data examples

bayesian methods for hidden markov models markov chain monte carlo mcmc sampling strategies can be used to simulate hidden markov model hmm parameters from their posterior distribution given observed data some mcmc methods used in practice for computing likelihood conditional probabilities of hidden states and the most likely sequence of states can be improved by incorporating established recursive algorithms the most important of these is a set of forward backward recursions calculating conditional distributions of the hidden states given observed data and model parameters i show how to use the recursive algorithms in an mcmc context and demonstrate mathematical and empirical results showing a gibbs sampler using the forward backward recursions mixes more rapidly than another sampler often used for hmms iintroduce an augmented variables technique for obtaining unique state labels in hmms and finite mixture models i show how recursive computing allows the statistically efficient use of mcmc output when estimating the hidden states i

lagrangian relative equilibria for a gyrostat in the three body problem bifurcations and stability in this paper we consider the non canonical hamiltonian dynamics of a gyrostat in the frame of the three body problem using geometric mechanic methods we study the approximate dynamics of the truncated legendre series representation of the potential of an arbitrary order working in the reduced problem we study the existence of relative equilibria that we refer to as lagrange type following the analogy with the standard techniques we provide necessary and sufficient conditions for the linear stability of lagrangian relative equilibria if the gyrostat morphology form is close to a sphere thus we generalize the classical results on equilibria of the three body problem and many results on them obtained by the classic approach for the case of rigid bodies

bayesian model selection in finite mixtures by marginal density decompositions we consider the problem of estimating the number of components d and the unknown mixing distribution in a finite mixture model in which d is bounded by some fixed finite number n our approach relies on the use of a prior over the space of mixing distributions with at most n components by decomposing the resulting marginal density under this prior we discover a weighted bayes factor method for consistently estimating d that can be implemented by an iid generalized weighted chinese restaurant gwcr monte carlo algorithm we also discuss a gibbs sampling method the blocked gibbs sampler for estimating d and also the mixing distribution we show that our resulting posterior is consistent and achieves the frequentist optimal op n 1 4 rate of estimation we compare the performance of the new gwcr model selection procedure with that of the akaike information criterion and the bayes information criterion implemented through an em algorithm applications of our methods to five real datasets and simulation

estimation of finite mixture distributions through bayesian sampling summary a formal bayesian analysis of a mixture model usually leads to intractable calculations since the posterior distribution takes into account all the partitions of the sample we present approximation methods which evaluate the posterior distribution and bayes estimators by gibbs sampling relying on the missing data structure of the mixture model the data augmentation method is shown to converge geometrically since a duality principle transfers properties from the discrete missing data chain to the parameters the fully conditional gibbs alternative is shown to be ergodic and geometric convergence is established in the normal case we also consider non informative approximations associated with improper priors assuming that the sample corresponds exactly to a k component mixture

spatial statistics and bayesian computation on wednesday may 6th 1992 professor b w silverman in the chair summary markov chain monte carlo mcmc algorithms such as the gibbs sampler have provided a bayesian inference machine in image analysis and in other areas of spatial statistics for several years founded on the pioneering ideas of ulf grenander more recently the observation that hyperparameters can be included as part of the updating schedule and the fact that almost any multivariate distribution is equivalently a markov random field has opened the way to the use of mcmc in general bayesian computation in this paper we trace the early development of mcmc in bayesian inference review some recent computational progress in statistical physics based on the introduction of auxiliary variables and discuss its current and future relevance in bayesian applications we briefly describe a simple mcmc implementation for the bayesian analysis of agricultural field experiments with which we have some practical experience

calculating posterior distributions and modal estimates in markov mixture models abstract this paper is concerned with finite mixture models in which the populations from one observation to the next are selected according to an unobserved markov process a new full bayesian approach based on the method of gibbs sampling is developed calculations are simplified by data augmentation achieved by introducing a population index variable into the list of unknown parameters it is shown that the latent variables one for each observation can be simulated from their joint distribution given the data and the remaining parameters this result serves to accelerate the convergence of the gibbs sample modal estimates are also computed by stochastic versions of the em algorithm these provide an alternative to a full bayesian approach and to existing methods of locating the maximum likelihood estimate the ideas are applied in detail to poisson data mixtures of multivariate normal distributions and autoregressive time series

on fast supervised learning for normal mixture models with missing information it is an important research issue to deal with mixture models when missing values occur in the data in this paper computational strategies using auxiliary indicator matrices are introduced for efficiently handling mixtures of multivariate normal distributions when the data are missing at random and have an arbitrary missing data pattern meaning that missing data can occur anywhere we develop a novel em algorithm that can dramatically save computation time and be exploited in many applications such as density estimation supervised clustering and prediction of missing values in the aspect of multiple imputations for missing data we also offer a data augmentation scheme using the gibbs sampler our proposed methodologies are illustrated through some real data sets with varying proportions of missing values

a bayesian semiparametric item response model with dirichlet process priors in item response theory irt item characteristic curves iccs are illustrated through logistic models or normal ogive models and the probability that examinees give the correct answer is usually a monotonically increasing function of their ability parameters however since only limited patterns of shapes can be obtained from logistic models or normal ogive models there is a possibility that the model applied does not fit the data as a result the existing method can be rejected because it cannot deal with various item response patterns to overcome these problems we propose a new semiparametric irt model using a dirichlet process mixture logistic distribution our method does not rely on assumptions but only requires that the iccs be a monotonically nondecreasing function that is our method can deal with more types of item response patterns than the existing methods such as the one parameter normal ogive models or the two or three parameter logistic models we conducted two simulation studies whose results indicate that the proposed method can express more patterns of shapes for iccs and can estimate the ability parameters more accurately than the existing parametric and nonparametric methods the proposed method has also been applied to facial expression recognition data with noteworthy results

the calculation of posterior distributions by data augmentation abstract the idea of data augmentation arises naturally in missing value problems as exemplified by the standard ways of filling in missing cells in balanced two way tables thus data augmentation refers to a scheme of augmenting the observed data so as to make it more easy to analyze this device is used to great advantage by the em algorithm dempster laird and rubin 1977 in solving maximum likelihood problems in situations when the likelihood cannot be approximated closely by the normal likelihood maximum likelihood estimates and the associated standard errors cannot be relied upon to make valid inferential statements from the bayesian point of view one must now calculate the posterior distribution of parameters of interest if data augmentation can be used in the calculation of the maximum likelihood estimate then in the same cases one ought to be able to use it in the computation of the posterior distribution it is the purpose of this article to explain how this can be done the basic idea