bayesian inference for gaussian graphical models beyond decomposable graphs bayesian inference for graphical models has received much attention in the literature in recent years it is well known that when the graph g is decomposable bayesian inference is significantly more tractable than in the general non decomposable setting penalized likelihood inference in contrast has made tremendous gains in the past few years in terms of scalability and tractability bayesian inference however has not had the same level of success though a scalable bayesian approach has its strengths especially in terms of quantifying uncertainty to address this gap we propose a scalable and flexible novel bayesian approach for estimation and model selection in gaussian undirected graphical models we first develop a class of generalized g wishart distributions with multiple shape parameters for an arbitrary underlying graph this class contains the g wishart distribution as a special case we then introduce the class of generalized bartlett graphs and derive an efficient gibbs sampling algorithm to obtain posterior draws from generalized g wishart distributions corresponding to a generalized bartlett graph the class of generalized bartlett graphs contains the class of decomposable graphs as a special case but is substantially larger than the class of decomposable graphs we proceed to derive theoretical properties of the proposed gibbs sampler we then demonstrate that the proposed gibbs sampler is scalable to significantly higher dimensional problems compared with using an accept reject or a metropolis hasting algorithm finally we show the efficacy of the proposed approach on simulated and real data in particular we demonstrate that our generalized bartlett methodology can be used for efficient model selection by reducing the graph search space by using penalized likelihood and pseudolikelihood methods

sampling decomposable graphs using a markov chain on junction trees this paper makes two contributions to the computational geometry of decomposable graphs aimed primarily at facilitating statistical inference about such graphs where they arise as assumed conditional independence structures in stochastic models the rst of these provides sucient conditions under which it is possible to completely connect two disconnected cliques of vertices or perform the reverse procedure yet maintain decomposability of the graph the second is a new markov chain monte carlo sampler for arbitrary positive distributions on decomposable graphs taking a junction tree representing the graph as its state variable the resulting methodology is illustrated with two numerical experiments eld model determination

estimation of a covariance matrix using the reference prior estimation of a covariance matrix sum is a notoriously difficult problem the standard unbiased estimator can be substantially suboptimal we approach the problem from a noninformative prior bayesian perspective developing the reference noninformative prior for a covariance matrix and obtaining expressions for the resulting bayes estimators these expressions involve the computation of high dimensional posterior expectations which is done using a recent markov chain simulation tool the hit and run sampler frequentist risk comparisons with previously suggested estimators are also given and determination of the accuracy of the estimators is addressed

hyper markov laws in the statistical analysis of decomposable graphical models this paper introduces and investigates the notion of a hyper markov law which is a probability distribution over the set of probability measures on a multivariate space that i is concentrated on the set of markov probabilities over some decomposable graph and ii satisfies certain conditional independence restrictions related to that graph a stronger version of this hyper markov property is also studied our analysis starts by reconsidering the properties of markov probabilities using an abstract approach which thereafter proves equally applicable to the hyper markov case next it is shown constructively that hyper markov laws exist that they appear as sampling distributions of maximum likelihood estimators in decomposable graphical models and also that they form natural conjugate prior distributions for a bayesian analysis of these models as examples we construct a range of specific hyper markov laws including the hyper multinomial hyper dirichlet and the hyper wishart and inverse wishart laws these laws occur naturally in connection with the analysis of decomposable log linear and covariance selection models

covariance matrix selection and estimation via penalised normal likelihood summary we propose a nonparametric method for identifying parsimony and for producing a statistically efficient estimator of a large covariance matrix we reparameterise a covariance matrix through the modified cholesky decomposition of its inverse or the one step ahead predictive representation of the vector of responses and reduce the nonintuitive task of modelling covariance matrices to the familiar task of model selection and estimation for a sequence of regression models the cholesky factor containing these regression coefficients is likely to have many off diagonal elements that are zero or close to zero penalised normal likelihoods in this situation with l1 and l2 penalities are shown to be closely related to tibshirani s 1996 lasso approach and to ridge regression adding either penalty to the likelihood helps to produce more stable estimators by introducing shrinkage to the elements in the cholesky factor while because of its singularity the l1 penalty will set some elements to zero and produce interpretable models an algorithm is developed for computing the estimator and selecting the tuning parameter the proposed maximum penalised likelihood estimator is illustrated using simulation and a real dataset involving estimation of a 102 x 102 covariance matrix

condition number regularized covariance estimation estimation of high dimensional covariance matrices is known to be a difficult problem has many applications and is of current interest to the larger statistics community in many applications including so called the large p small n setting the estimate of the covariance matrix is required to be not only invertible but also well conditioned although many regularization schemes attempt to do this none of them address the ill conditioning problem directly in this paper we propose a maximum likelihood approach with the direct goal of obtaining a well conditioned estimator no sparsity assumption on either the covariance matrix or its inverse are are imposed thus making our procedure more widely applicable we demonstrate that the proposed regularization scheme is computationally efficient yields a type of steinian shrinkage estimator and has a natural bayesian interpretation we investigate the theoretical properties of the regularized covariance estimator comprehensively including its regularization path and proceed to develop an approach that adaptively determines the level of regularization that is required finally we demonstrate the performance of the regularized estimator in decision theoretic comparisons and in the financial portfolio optimization setting the proposed approach has desirable properties and can serve as a competitive procedure especially when the sample size is small and when a well conditioned estimator is required

bayesian inference for a covariance matrix a flexible class of prior distributions is proposed for the covariance matrix of a multivariate normal distribution yielding much more general hierarchical and empirical bayes smoothing and inference when compared with a conjugate analysis involving an inverted wishart distribution a likelihood approximation is obtained for the matrix logarithm of the covariance matrix via bellman s iterative solution to a volterra integral equation exact and approximate bayesian empirical and hierarchical bayesian estimation and finite sample inference techniques are developed some risk and asymptotic frequency properties are investigated a subset of the project talent american high school data is analyzed applications and extensions to multivariate analysis including a generalized linear model for covariance matrices are indicated

hub discovery in partial correlation graphs one of the most important problems in large scale inference problems is the identification of variables that are highly dependent on several other variables when dependence is measured by partial correlations these variables identify those rows of the partial correlation matrix that have several entries with large magnitudes i e hubs in the associated partial correlation graph this paper develops theory and algorithms for discovering such hubs from a few observations of these variables we introduce a hub screening framework in which the user specifies both a minimum partial correlation rho and a minimum degree delta to screen the vertices the choice of rho and delta can be guided by our mathematical expressions for the phase transition correlation threshold rho c governing the average number of discoveries they can also be guided by our asymptotic expressions for familywise discovery rates under the assumption of large number p of variables fixed number n of multivariate samples and weak dependence under the null hypothesis that the dispersion covariance matrix is sparse these limiting expressions can be used to enforce familywise error constraints and to rank the discoveries in order of increasing statistical significance for n ll p the computational complexity of the proposed partial correlation screening method is low and is therefore highly scalable thus it can be applied to significantly larger problems than previous approaches the theory is applied to discovering hubs in a high dimensional gene microarray dataset

the entropy concept for non equilibrium states in earlier work we presented a foundation for the second law of classical thermodynamics in terms of the entropy principle more precisely we provided an empirically accessible axiomatic derivati

nonconjugate bayesian estimation of covariance matrices and its use in hierarchical models abstract the problem of estimating a covariance matrix in small samples has been considered by several authors following early work by stein this problem can be especially important in hierarchical models where the standard errors of fixed and random effects depend on estimation of the covariance matrix of the distribution of the random effects we propose a set of hierarchical priors hps for the covariance matrix that produce posterior shrinkage toward a specified structure here we examine shrinkage toward diagonality we then address the computational difficulties raised by incorporating these priors and nonconjugate priors in general into hierarchical models we apply a combination of approximation gibbs sampling possibly with a metropolis step and importance reweighting to fit the models and compare this hybrid approach to alternative markov chain monte carlo methods our investigation involves three alternative hps the first works with the spectral decomposition of the covariance matrix an

positive definite completions of partial hermitian matrices abstract the question of which partial hermitian matrices some entries specified some free may be completed to positive definite matrices is addressed it is shown that if the diagonal entries are specified and principal minors composed of specified entries are positive then if the undirected graph of the specified entries is chordal a positive definite completion necessarily exists furthermore if this graph is not chordal then examples exist without positive definite completions in case a positive definite completion exists there is a unique matrix in the class of all positive definite completions whose determinant is maximal and this matrix is the unique one whose inverse has zeros in those positions corresponding to unspecified entries in the original partial hermitian matrix additional observations regarding positive definite completions are made

flexible covariance estimation in graphical gaussian models in this paper we propose a class of bayes estimators for the covariance matrix of graphical gaussian models markov with respect to a decomposable graph g working with the w pg family defined by letac and massam ann statist 35 2007 1278 1323 we derive closed form expressions for bayes estimators under the entropy and squared error losses the w pg family includes the classical inverse of the hyper inverse wishart but has many more shape parameters thus allowing for flexibility in differentially shrinking various parts of the covariance matrix moreover using this family avoids recourse to mcmc often infeasible in high dimensional problems we illustrate the performance of our estimators through a collection of numerical examples where we explore frequentist risk properties and the efficacy of graphs in the estimation of high dimensional covariance structures

some remarks on noninformative priors abstract this article focuses primarily on a comparison between the reference priors of berger and bernardo and the reverse reference priors suggested by j k ghosh sufficient conditions are given that provide agreement between the two classes of priors several examples are given showing the agreement or disagreement between the two in addition these priors are compared under a criterion that requires the frequentist coverage probability of the posterior region of a real valued parametric function to match a nominal level with a remainder of o n 1 where n denotes the sample size the latter priors first introduced by welch and peers are obtained by solving a differential equation due to peers finally in the presence of several parameters of interest a general class of priors that satisfies the matching criterion separately for each parameter is constructed and examples are given to illustrate how reference or reverse reference priors fit within this class of priors

regularized estimation of large covariance matrices this paper considers estimating a covariance matrix of p variables from n observations by either banding the sample covariance matrix or estimating a banded version of the inverse of the covariance we show that these estimates are consistent in the operator norm as long as logp 2 n 0 and obtain explicit rates the results are uniform over some fairly natural well conditioned families of covariance matrices we also introduce an analogue of the gaussian white noise model and show that if the population covariance is embeddable in that model and well conditioned then the banded approximations produce consistent estimates of the eigenvalues and associated eigenvectors of the covariance matrix the results can be extended to smooth versions of banding and to non gaussian distributions with su ciently short tails a resampling approach is proposed for choosing the banding parameter in practice this approach is illustrated numerically on both simulated and real data

retaining positive definiteness in thresholded matrices abstract positive definite p d matrices arise naturally in many areas within mathematics and also feature extensively in scientific applications in modern high dimensional applications a common approach to finding sparse positive definite matrices is to threshold their small off diagonal elements this thresholding sometimes referred to as hard thresholding sets small elements to zero thresholding has the attractive property that the resulting matrices are sparse and are thus easier to interpret and work with in many applications it is often required and thus implicitly assumed that thresholded matrices retain positive definiteness in this paper we formally investigate the algebraic properties of p d matrices which are thresholded we demonstrate that for positive definiteness to be preserved the pattern of elements to be set to zero has to necessarily correspond to a graph which is a union of complete components this result rigorously demonstrates that except in special cases positive definiteness can be easily lost we then proceed to demonstrate that the class of diagonally dominant matrices is not maximal in terms of retaining positive definiteness when thresholded consequently we derive characterizations of matrices which retain positive definiteness when thresholded with respect to important classes of graphs in particular we demonstrate that retaining positive definiteness upon thresholding is governed by complex algebraic conditions

nonlinear shrinkage estimation of large dimensional covariance matrices many statistical applications require an estimate of a covariance matrix and or its inverse when the matrix dimension is large compared to the sample size which happens frequently the sample covariance matrix is known to perform poorly and may suffer from ill conditioning there already exists an extensive literature concerning improved estimators in such situations in the absence of further knowledge about the structure of the true covariance matrix the most successful approach so far arguably has been shrinkage estimation shrinking the sample covariance matrix to a multiple of the identity by taking a weighted average of the two turns out to be equivalent to linearly shrinking the sample eigenvalues to their grand mean while retaining the sample eigenvectors our paper extends this approach by considering nonlinear transformations of the sample eigenvalues we show how to construct an estimator that is asymptotically equivalent to an oracle estimator suggested in previous work as demonstrated in extensive monte carlo simulations the resulting bona fide estimator can result in sizeable improvements over the sample covariance matrix and also over linear shrinkage

shrinkage estimators for covariance matrices estimation of covariance matrices in small samples has been studied by many authors standard estimators like the unstructured maximum likelihood estimator ml or restricted maximum likelihood reml estimator can be very unstable with the smallest estimated eigenvalues being too small and the largest too big a standard approach to more stably estimating the matrix in small samples is to compute the ml or reml estimator under some simple structure that involves estimation of fewer parameters such as compound symmetry or independence however these estimators will not be consistent unless the hypothesized structure is correct if interest focuses on estimation of regression coefficients with correlated or longitudinal data a sandwich estimator of the covariance matrix may be used to provide standard errors for the estimated coefficients that are robust in the sense that they remain consistent under misspecification of the covariance structure with large matrices however the inefficiency of the sandwich estimator becomes worrisome we consider here two general shrinkage approaches to estimating the covariance matrix and regression coefficients the first involves shrinking the eigenvalues of the unstructured ml or reml estimator the second involves shrinking an unstructured estimator toward a structured estimator for both cases the data determine the amount of shrinkage these estimators are consistent and give consistent and asymptotically efficient estimates for regression coefficients simulations show the improved operating characteristics of the shrinkage estimators of the covariance matrix and the regression coefficients in finite samples the final estimator chosen includes a combination of both shrinkage approaches i e shrinking the eigenvalues and then shrinking toward structure we illustrate our approach on a sleep eeg study that requires estimation of a 24 x 24 covariance matrix and for which inferences on mean parameters critically depend on the covariance estimator chosen we recommend making inference using a particular shrinkage estimator that provides a reasonable compromise between structured and unstructured estimators

the symmetries of outer space for n 3 the natural map out fn aut kn from the outer automorphism group of the free group of rank n to the group of simplicial automorphisms of the spine of outer space is an isomorphism

a theoretical study of stein s covariance estimator the traditional covariance estimator the sample covariance matrix which is also the mle is known to be a poor estimator unless the sample size is much larger than the dimension of the covariance matrix stein s estimator has often been regarded as a much better alternative to the mle in small sample sizes and is traditionally used with an isotonizing algorithm the purpose of which is to retain positivity and the original order of the sample eigenvalues despite the superior performance of stein s isotonized estimator in numerical investigations its theoretical properties have not been explored in detail and important questions still remain unanswered one particular question of interest is to identify the regimes under which stein s estimator is guaranteed to perform well a second goal is to determine the extent to which the performance of stein s estimator depends on the isotonizing algorithm the presence of the ad hoc isotonizing algorithm however renders a theoretical analysis rather difficult and consequently risk functions are not easily quantifiable for comparison purposes hence formal decision theoretical results are difficult to obtain and have been elusive ever since the estimator was introduced despite these hurdles in this paper we show that an analysis of stein s covariance estimator within the unbiased estimator of risk ubeor framework can nevertheless lead to important theoretical and methodological insights that are relevant for applications our analysis demonstrates that stein s estimator may give only modest risk reductions when it is not isotonized and when it is isotonized the risk reductions can be significant in particular three broad regimes are identified regarding the behavior of stein s ubeor the theoretical insights are then affirmed at the level of risk functions via numerical simulations

wishart distributions for decomposable graphs when considering a graphical gaussian model n g markov with respect to a decomposable graph g the parameter space of interest for the precision parameter is the cone p g of positive definite matrices with fixed zeros corresponding to the missing edges of g the parameter space for the scale parameter of n g is the cone q g dual to p g of incomplete matrices with submatrices corresponding to the cliques of g being positive definite in this paper we construct on the cones q g and p g two families of wishart distributions namely the type i and type ii wisharts they can be viewed as generalizations of the hyper wishart and the inverse of the hyper inverse wishart as defined by dawid and lauritzen ann statist 21 1993 1272 1317 we show that the type i and ii wisharts have properties similar to those of the hyper and hyper inverse wishart indeed the inverse of the type ii wishart forms a conjugate family of priors for the covariance parameter of the graphical gaussian model and is strong directed hyper markov for every direction given to the graph by a perfect order of its cliques while the type i wishart is weak hyper markov moreover the inverse type ii wishart as a conjugate family presents the advantage of having a multidimensional shape parameter thus offering flexibility for the choice of a prior both type i and ii wishart distributions depend on multivariate shape parameters a shape parameter is acceptable if and only if it satisfies a certain eigenvalue property we show that the sets of acceptable shape parameters for a noncomplete g have dimension equal to at least one plus the number of cliques in g these families as conjugate families are richer than the traditional diaconis ylvisaker conjugate families which all have a shape parameter set of dimension one a decomposable graph which does not contain a three link chain as an induced subgraph is said to be homogeneous in this case our wisharts are particular cases of the wisharts on homogeneous cones as defined by andersson and wojnar j theoret probab 17 2004 781 818 and the dimension of the shape parameter set is even larger than in the nonhomogeneous case it is indeed equal to the number of cliques plus the number of distinct minimal separators using the model where g is a three link chain we show by computing a 7 tuple integral that in general we cannot expect the shape parameter sets to have dimension larger than the number of cliques plus one

estimation of parameter matrices and eigenvalues in manova and canonical correlation analysis on entreprend une approche par la theorie de la decision avec l erreur quadratique comme fonction de perte

cholesky decomposition of a hyper inverse wishart matrix the canonical parameter of a covariance selection model is the inverse covariance matrix 1 whose zero pattern gives the conditional independence structure characterising the model in this paper we consider the upper triangular matrix obtained by the cholesky decomposition 1 t this provides an interesting alternative parameterisation of decomposable models since its upper triangle has the same zero structure as 1 and its elements have an interpretation as parameters of certain conditional distributions for a distribution for the strong hyper markov property is shown to be characterised by the mutual independence of the rows of this is further used to generalise to the hyper inverse wishart distribution some well known properties of the inverse wishart distribution in particular we show that a hyper inverse wishart matrix can be decomposed into independent normal and chi squared random variables and we describe a family of transformations under which the family of hyper inverse wishart distributions is closed

large scale correlation screening this article addresses the problem of screening for variables with high correlations in high dimensional data in which there can be many fewer samples than variables we focus on threshold based correlation screening methods for three related applications screening for variables with large correlations within a single treatment autocorrelation screening screening for variables with large cross correlations over two treatments cross correlation screening and screening for variables that have persistently large autocorrelations over two treatments persistent correlation screening the novelty of correlation screening is that it identifies a smaller number of variables that are highly correlated with others compared with identifying a number of correlation parameters correlation screening suffers from a phase transition phenomenon as the correlation threshold decreases the number of discoveries increases abruptly we obtain asymptotic expressions for the mean number of discoveries and the phase t

simulation of hyper inverse wishart distributions in graphical models summary we introduce and exemplify an efficient method for direct sampling from hyperinverse wishart distributions the method relies very naturally on the use of standard junction tree representation of graphs and couples these with matrix results for inverse wishart distributions we describe the theory and resulting computational algorithms for both decomposable and nondecomposable graphical models an example drawn from financial time series demonstrates application in a context where inferences on a structured covariance model are required we discuss and investigate questions of scalability of the simulation methods to higher dimensional distributions the paper concludes with general comments about the approach including its use in connection with existing markov chain monte carlo methods that deal with uncertainty about the graphical model structure

information theoretic asymptotics of bayes methods in the absence of knowledge of the true density function bayesian models take the joint density function for a sequence of n random variables to be an average of densities with respect to a prior the authors examine the relative entropy distance d sub n between the true density and the bayesian density and show that the asymptotic distance is d 2 log n c where d is the dimension of the parameter vector therefore the relative entropy rate d sub n n converges to zero at rate log n n the constant c which the authors explicitly identify depends only on the prior density function and the fisher information matrix evaluated at the true parameter value consequences are given for density estimation universal data compression composite hypothesis testing and stock market portfolio selection

sparse matrix decompositions and graph characterizations abstract zeros in positive definite correlation matrices arise frequently in probability and statistics and are intimately related to the notion of stochastic independence the question of when zeros i e sparsity in a positive definite matrix a are preserved in its cholesky decomposition and vice versa was addressed by paulsen et al v i paulsen s c power r r smith schur products and matrix completions j funct anal 85 1989 151 178 in particular they prove that for the pattern of zeros in a to be retained in the cholesky decomposition of a the pattern of zeros in a has to necessarily correspond to a chordal or decomposable graph associated with a specific type of vertex ordering this result therefore also yields a characterization of chordal graphs in terms of sparse positive definite matrices and has proved to be extremely useful in probabilistic and statistical analysis of markov random fields now consider a positive definite matrix a and its cholesky decomposition given by a ldl t where l is lower triangular with unit diagonal entries and d a diagonal matrix with positive entries in this paper we prove that a necessary and sufficient condition for zeros i e sparsity in a positive definite matrix a to be preserved in its associated cholesky matrix l and in addition also preserved in the inverse of the cholesky matrix l 1 is that the pattern of zeros corresponds to a co chordal or homogeneous graph associated with a specific type of vertex ordering we proceed to provide a second characterization of this class of graphs in terms of determinants of submatrices that correspond to cliques in the graph these results add to the growing body of literature in the field of sparse matrix decompositions and also prove to be critical ingredients in the probabilistic analysis of an important class of markov random fields