bayesian analysis of factorial experiments by mixture modelling a bayesian analysis for factorial experiments is presented using finite mixture distributions to model the main effects and interactions this allows both estimation and an analogue of hypothesis testing in a posterior analysis using a single prior specification a detailed formulation based on this approach is provided for the case of the two way model with replication allowing interactions issues in formulating a suitable prior are discussed in detail and in the context of two illustrative applications we discuss implementation presentation of posterior distributions sensitivity and performance of the markov chain monte carlo methods that are used

bayesian analysis of mixture models with an unknown number of components an alternative to reversible jump methods richardson and green present a method of performing a bayesian analysis of data from a finite mixture distribution with an unknown number of components their method is a markov chain monte carlo mcmc approach which makes use of the reversible jump methodology described by green we describe an alternative mcmc method which views the parameters of the model as a marked point process extending methods suggested by ripley to create a markov birth death process with an appropriate stationary distribution our method is easy to implement even in the case of data in more than one dimension and we illustrate it on both univariate and bivariate data there appears to be considerable potential for applying these ideas to other contexts as an alternative to more general reversible jump methods and we conclude with a brief discussion of how this might be achieved

rotation to simple loadings using component loss functions the oblique case component loss functions clfs similar to those used in orthogonal rotation are introduced to define criteria for oblique rotation in factor analysis it is shown how the shape of the clf affects the performance of the criterion it defines for example it is shown that monotone concave clfs give criteria that are minimized by loadings with perfect simple structure when such loadings exist moreover if the clfs are strictly concave minimizing must produce perfect simple structure whenever it exists examples show that methods defined by concave clfs perform well much more generally while it appears important to use a concave clf the specific clf used is less important for example the very simple linear clf gives a rotation method that can easily outperform the most popular oblique rotation methods promax and quartimin and is competitive with the more complex simplimax and geomin methods

bayesian estimation of sparse dynamic factor models with order independent and ex post mode identification abstract common variation in n series is extracted into k n dynamic factors we induce sparsity by using a zero point mass normal mixture prior distribution on the loadings estimation and rotational identification are independent of variable ordering sparsity helps identifying the factor space and the factors rotational identification including factor order and sign is obtained by processing the posterior output and based on factor draws rather than factor loading draws simulating data we document sampler and estimation efficiency to illustrate we estimate the model for a large panel of swiss macroeconomic and detailed price data we identify 16 factors with a clear economic interpretation

generic global indentification in factor analysis abstract the paper proves global indentification of the unique variances in factor analysis for almost all parameter points if the number of factors is smaller than the ledermann bound it also shows how to compute using a closed form representation the second solution in the case of six variables and three factors

stochastic approximation em for large scale exploratory irt factor analysis a stochastic approximation em algorithm saem is described for exploratory factor analysis of dichotomous or ordinal variables the factor structure is obtained from sufficient statistics that are updated during iterations with the robbins monro procedure two large scale simulations are reported that compare accuracy and cpu time of the proposed saem algorithm to the metropolis hasting robbins monro procedure and to a generalized least squares analysis of the polychoric correlation matrix a smaller scale application to real data is also reported including a method for obtaining standard errors of rotated factor loadings a simulation study based on the real data analysis is conducted to study bias and error estimates the saem factor algorithm requires minimal lines of code no derivatives and no large matrix inversion it is programmed entirely in r

rotation to simple loadings using component loss functions the orthogonal case component loss functions clfs are used to generalize the quartimax criterion for orthogonal rotation in factor analysis these replace the fourth powers of the factor loadings by an arbitrary function of the second powers criteria of this form were introduced by a number of authors primarily katz and rohlf 1974 and rozeboom 1991 but there has been essentially no follow up to this work a method so simple natural and general deserves to be investigated more completely a number of theoretical results are derived including the fact that any method using a concave clf will recover perfect simple structure whenever it exists and there are methods that will recover thurstone simple structure whenever it exists specific clfs are identified and it is shown how to compare these using standardized plots numerical examples are used to illustrate and compare clf and other methods sorted absolute loading plots are introduced to aid in comparing results and setting parameters for methods that require them

bayesian analysis of static and dynamic factor models an ex post approach towards the rotation problem due to their indeterminacies static and dynamic factor models require identifying assumptions to guarantee uniqueness of the parameter estimator the indeterminacy of the parameter estimator with respect to an orthogonal transformation is known as the rotation problem the typical strategy in bayesian factor analysis to solve the rotation problem is to introduce ex ante constraints on certain model parameters via degenerate and truncated prior distributions this strategy however results in posterior distributions whose shapes depend on the ordering of the variables in the data set we propose an alternative approach where the rotation problem is solved ex post using procrustean postprocessing the resulting order invariance of the posterior estimator is illustrated in a simulation study and an empirical application using an established data set containing 120 macroeconomic time series favorable properties of the ex post approach with respect to convergence statistical and numerical accuracy are revealed

seeking efficient data augmentation schemes via conditional and marginal augmentation data augmentation sometimes known as the method of auxiliary variables is a powerful tool for constructing optimisation and simulation algorithms in the context of optimisation meng van dyk 1997 1998 reported several successes of the working parameter approach for constructing efficient data augmentation schemes for fast and simple em type algorithms this paper investigates the use of working parameters in the context of markov chain monte carlo in particular in the context of tanner wong s 1987 data augmentation algorithm via a theoretical study of two working parameter approaches the conditional augmentation approach and the marginal augmentation approach posterior sampling under the univariate t model is used as a running example which particularly illustrates how the marginal augmentation approach obtains a fast mixing positive recurrent markov chain by first constructing a nonpositive recurrent markov chain in a larger space

an efficient tvl1 algorithm for deblurring multichannel images corrupted by impulsive noise we extend the alternating minimization algorithm recently proposed in y wang j yang w yin and y zhang siam j imag sci 1 2008 pp 248 272 j yang w yin y zhang and y wang siam j imag sci 2 2009 pp 569 592 to the case of recovering blurry multichannel color images corrupted by impulsive rather than gaussian noise the algorithm minimizes the sum of a multichannel extension of total variation and a data fidelity term measured in the ell 1 norm and is applicable to both salt and pepper and random valued impulsive noise we derive the algorithm by applying the well known quadratic penalty function technique and prove attractive convergence properties including finite convergence for some variables and q linear convergence rate under periodic boundary conditions the main computational requirements of the algorithm are fast fourier transforms and a low complexity gaussian elimination procedure numerical results on images with different blurs and impulsive noise are presented to demonstrate the efficiency of the algorithm in addition it is numerically compared to the least absolute deviation method h y fu m k ng m nikolova and j l barlow siam j sci comput 27 2006 pp 1881 1902 and the two phase method j f cai r chan and m nikolova aims j inverse problems and imaging 2 2008 pp 187 204 for recovering grayscale images we also present results of recovering multichannel images

a bayesian subgroup analysis with a zero enriched polya urn scheme we introduce a new approach to inference for subgroups in clinical trials we use bayesian model selection and a threshold on posterior model probabilities to identify subgroup effects for reporting for each covariate of interest we define a separate class of models and use the posterior probability associated with each model and the threshold to determine the existence of a subgroup effect as usual in bayesian clinical trial design we compute frequentist operating characteristics and achieve the desired error probabilities by choosing an appropriate threshold s for the posterior probabilities copyright 2010 john wiley sons ltd

parameter expansion for data augmentation abstract viewing the observed data of a statistical model as incomplete and augmenting its missing parts are useful for clarifying concepts and central to the invention of two well known statistical algorithms expectation maximization em and data augmentation recently liu rubin and wu demonstrated that expanding the parameter space along with augmenting the missing data is useful for accelerating iterative computation in an em algorithm the main purpose of this article is to rigorously define a parameter expanded data augmentation px da algorithm and to study its theoretical properties the px da is a special way of using auxiliary variables to accelerate gibbs sampling algorithms and is closely related to reparameterization techniques we obtain theoretical results concerning the convergence rate of the px da algorithm and the choice of prior for the expansion parameter to understand the role of the expansion parameter we establish a new theory for iterative conditional sampling under the tra

bayesian exploratory factor analysis this paper develops and applies a bayesian approach to exploratory factor analysis that improves on ad hoc classical approaches our framework relies on dedicated factor models and simultaneously determines the number of factors the allocation of each measurement to a unique factor and the corresponding factor loadings classical identification criteria are applied and integrated into our bayesian procedure to generate models that are stable and clearly interpretable a monte carlo study confirms the validity of the approach the method is used to produce interpretable low dimensional aggregates from a high dimensional set of psychological measurements

gcsdecolor gradient correlation similarity for efficient contrast preserving decolorization this paper presents a novel gradient correlation similarity gcs measure based decolorization model for faithfully preserving the appearance of the original color image contrary to the conventional data fidelity term consisting of gradient error norm based measures the newly defined gcs measure calculates the summation of the gradient correlation between each channel of the color image and the transformed grayscale image two efficient algorithms are developed to solve the proposed model on one hand due to the highly nonlinear nature of gcs measure a solver consisting of the augmented lagrangian and alternating direction method is adopted to deal with its approximated linear parametric model the presented algorithm exhibits excellent iterative convergence and attains superior performance on the other hand a discrete searching solver is proposed by determining the solution with the minimum function value from the linear parametric model induced candidate images the non iterative solver has advantages in simplicity and speed with only several simple arithmetic operations leading to real time computational speed in addition it is very robust with respect to the parameter and candidates extensive experiments under a variety of test images and a comprehensive evaluation against existing state of the art methods consistently demonstrate the potential of the proposed model and algorithms

high dimensional sparse factor modeling applications in gene expression genomics we describe studies in molecular profiling and biological pathway analysis that use sparse latent factor and regression models for microarray gene expression data we discuss breast cancer applications and key aspects of the modeling and computational methodology our case studies aim to investigate and characterize heterogeneity of structure related to specific oncogenic pathways as well as links between aggregate patterns in gene expression profiles and clinical biomarkers based on the metaphor of statistically derived factors as representing biological subpathway structure we explore the decomposition of fitted sparse factor models into pathway subcomponents and investigate how these components overlay multiple aspects of known biological activity our methodology is based on sparsity modeling of multivariate regression anova and latent factor models as well as a class of models that combines all components hierarchical sparsity priors address questions of dimension reduction and multiple co

a bayesian analysis of the multinomial probit model using marginal data augmentation we introduce a set of new markov chain monte carlo algorithms for bayesian analysis of the multinomial probit model our bayesian representation of the model places a new and possibly improper prior distribution directlyon the identi0able parameters and thus is relativelyeasyto interpret and use our algorithms which are based on the method of marginal data augmentation involve onlydraws from standard distributions and dominate other available bay esian methods in that theyare as quick to converge as the fastest methods but with a more attractive prior speci0cation c code along with an r interface for our algorithms is publiclyavailable 1

a simple general method for oblique rotation a simple and very general algorithm for oblique rotation is identified while motivated by the rotation problem in factor analysis it may be used to minimize almost any function of a not necessarily square matrix whose columns are restricted to have unit length the algorithm has two steps the first is to compute the gradient of the rotation criterion and the second is to project this onto a manifold of matrices with unit length columns for this reason it is called a gradient projection algorithm because the projection step is very simple implementation of the algorithm involves little more than computing the gradient of the rotation criterion which for many applications is very simple it is proven that the algorithm is strictly monotone that is as long as it is not already at a stationary point each step will decrease the value of the criterion examples from a variety of areas are used to demonstrate the algorithm including oblimin rotation target rotation simplimax rotation and rotation to similarity and simplicity simultaneously while it may be the algorithm is not intended for use as a standard algorithm for well established problems but rather as a tool for investigating new methods where its generality and simplicity may save an investigator substantial effort

a simple general procedure for orthogonal rotation a very general algorithm for orthogonal rotation is identified it is shown that when an algorithm parameter is sufficiently large the algorithm converges monotonically to a stationary point of the rotation criterion from any starting value because a sufficiently large is in general hard to find a modification that does not require it is introduced without this requirement the modified algorithm is not only very general but also very simple its implementation involves little more than computing the gradient of the rotation criterion while the modified algorithm converges monotonically from any starting value it is not guaranteed to converge to a stationary point it however does so in all of our examples while motivated by the rotation problem in factor analysis the algorithms discussed may be used to optimize almost any function of a not necessarily square column wise orthonormal matrix a number of these more general applications are considered empirical examples show that the modified algorithm can be reasonably fast but its purpose is to save an investigator s effort rather than that of his or her computer this makes it more appropriate as a research tool than as an algorithm for established methods

log euclidean metrics for contrast preserving decolorization this paper presents a novel log euclidean metric inspired color to gray conversion model for faithfully preserving the contrast details of color image which differs from the traditional euclidean metric approaches in the proposed model motivated by the fact that log euclidean metric has promising invariance properties such as inversion invariant and similarity invariant we present a log euclidean metric based maximum function to model the decolorization procedure the gaussian like penalty function consisting of the log euclidean metric between gradients of the input color image and transformed grayscale image is incorporated to better reflect the degree of preserving feature discriminability and color ordering in color to gray conversion a discrete searching algorithm is employed to solve the proposed model with linear parametric and non negative constraints extensive evaluation experiments show that the proposed method outperforms the state of the art methods both quantitatively and qualitatively

sparse bayesian infinite factor models we focus on sparse modelling of high dimensional covariance matrices using bayesian latent factor models we propose a multiplicative gamma process shrinkage prior on the factor loadings which allows introduction of infinitely many factors with the loadings increasingly shrunk towards zero as the column index increases we use our prior on a parameter expanded loading matrix to avoid the order dependence typical in factor analysis models and develop an efficient gibbs sampler that scales well as data dimensionality increases the gain in efficiency is achieved by the joint conjugacy property of the proposed prior which allows block updating of the loadings matrix we propose an adaptive gibbs sampler for automatically truncating the infinite loading matrix through selection of the number of important factors theoretical results are provided on the support of the prior and truncation approximation bounds a fast algorithm is proposed to produce approximate bayes estimates latent factor regression methods are developed for prediction and variable selection in applications with high dimensional correlated predictors operating characteristics are assessed through simulation studies and the approach is applied to predict survival times from gene expression data

bayesian inference in econometric models using monte carlo integration methods for the systematic application of monte carlo integration with importance sampling to bayesian inference in econometric models are developed conditions under which the numerical approximation of a posterior moment converges almost surely to the true value as the number of monte carlo replications increases and the numerical accuracy of this approximation may be assessed reliably are set forth methods for the analytical verification of these conditions are discussed

smoothing balanced single error term analysis of variance we present an approach to smoothing balanced single error term analysis of variance anova descended from smith that also allows spatial temporal or spatiotemporal smoothing the approach addresses unreplicated designs masked contrasts in effects with many degrees of freedom and subgroup analysis demonstrated using a study of denture lining materials our approach is bayesian but can be viewed as a way to generate frequentist procedures a simulation experiment compares four priors unsmoothed anova and dropping nonsignificant interactions three priors have advantages when some interactions are absent dropping nonsignificant interactions has serious flaws we contrast our approach with the approaches of nobile green and gelman

reversible jump markov chain monte carlo computation and bayesian model determination markov chain monte carlo methods for bayesian computation have until recently been restricted to problems where the joint distribution of all variables has a density with respect to some fixed standard underlying measure they have therefore not been available for application to bayesian model determination where the dimensionality of the parameter vector is typically not fixed this paper proposes a new framework for the construction of reversible markov chain samplers that jump between parameter subspaces of differing dimensionality which is flexible and entirely constructive it should therefore have wide applicability in model determination problems the methodology is illustrated with applications to multiple change point analysis in one and two dimensions and to a bayesian comparison of binomial experiments

on bayesian analysis of mixtures with an unknown number of components with discussion new methodology for fully bayesian mixture analysis is developed making use of reversible jump markov chain monte carlo methods that are capable of jumping between the parameter subspaces corresponding to different numbers of components in the mixture a sample from the full joint distribution of all unknown variables is thereby generated and this can be used as a basis for a thorough presentation of many aspects of the posterior distribution the methodology is applied here to the analysis of univariate normal mixtures using a hierarchical prior model that offers an approach to dealing with weak prior information while avoiding the mathematical pitfalls of using improper priors in the mixture context

covariance structure of the gibbs sampler with applications to the comparisons of estimators and augmentation schemes summary we study the covariance structure of a markov chain generated by the gibbs sampler with emphasis on data augmentation when applied to a bayesian missing data problem the gibbs sampler produces two natural approximations for the posterior distribution of the parameter vector the empirical distribution based on the sampled values of the parameter vector and a mixture of complete data posteriors we prove that rao blackwellization causes a one lag delay for the autocovariances among dependent samples obtained from data augmentation and consequently the mixture approximation produces estimates with smaller variances than the empirical approximation the covariance structure results are used to compare different augmentation schemes it is shown that collapsing and grouping random components in a gibbs sampler with two or three components usually result in more efficient sampling schemes

mixtures of g priors for bayesian variable selection zellner s g prior remains a popular conventional prior for use in bayesian variable selection despite several undesirable consistency issues in this article we study mixtures of g priors as an alternative to default g priors that resolve many of the problems with the original formulation while maintaining the computational tractability that has made the g prior so popular we present theoretical properties of the mixture g priors and provide real and simulated examples to compare the mixture formulation with fixed g priors empirical bayes approaches and other default procedures please see arnold zellner s letter and the author s response

partially collapsed gibbs samplers theory and methods ever increasing computational power along with ever more sophisticated statistical computing techniques is making it possible to fit ever more complex statistical models among the more computationally intensive methods the gibbs sampler is popular because of its simplicity and power to effectively generate samples from a high dimensional probability distribution despite its simple implementation and description however the gibbs sampler is criticized for its sometimes slow convergence especially when it is used to fit highly structured complex models here we present partially collapsed gibbs sampling strategies that improve the convergence by capitalizing on a set of functionally incompatible conditional distributions such incompatibility generally is avoided in the construction of a gibbs sampler because the resulting convergence properties are not well understood we introduce three basic tools marginalization permutation and trimming that allow us to transform a gibbs sampler into a part