sparse matrix graphical models matrix variate observations are frequently encountered in many contemporary statistical problems due to a rising need to organize and analyze data with structured information in this article we propose a novel sparse matrix graphical model for these types of statistical problems by penalizing respectively two precision matrices corresponding to the rows and columns our method yields a sparse matrix graphical model that synthetically characterizes the underlying conditional independence structure our model is more parsimonious and is practically more interpretable than the conventional sparse vector variate graphical models asymptotic analysis shows that our penalized likelihood estimates enjoy better convergent rates than that of the vector variate graphical model the finite sample performance of the proposed method is illustrated via extensive simulation studies and several real datasets analysis

parameter expansion to accelerate em the px em algorithm summary the em algorithm and its extensions are popular tools for modal estimation but ar e often criticised for their slow convergence we propose a new method that can often make em much faster the intuitive idea is to use a covariance adjustment to correct the analysis of the m step capitalising on extra information captured in the imputed complete data the way we accomplish this is by parameter expansion we expand the complete data model while preserving the observed data model and use the expanded complete data model to generate em this parameter expanded ei m px em algorithm shares the simplicity and stability of ordinary em but has a faster rate of convergence since its m step performs a more efficient analysis the px em algorithm is illustrated for the multivariate t distribution a random effects model factor analysis probit regression and a poisson imaging model

decomposable graphical gaussian model determination we propose a methodology for bayesian model determination in decomposable graphical gaussian models to achieve this aim we consider a hyper inverse wishart prior distribution on the concentration matrix for each given graph to ensure compatibility across models such prior distributions are obtained by marginalisation from the prior conditional on the complete graph we explore alternative structures for the hyperparameters of the latter and their consequences for the model model determination is carried out by implementing a reversible jump markov chain monte carlo sampler in particular the dimension changing move we propose involves adding or dropping an edge from the graph we characterise the set of moves which preserve the decomposability of the graph giving a fast algorithm for maintaining the junction tree representation of the graph at each sweep as state variable we use the incomplete variance covariance matrix containing only the elements for which the corresponding element of the inverse is nonzero this allows all computations to be performed locally at the clique level which is a clear advantage for the analysis of large and complex datasets finally the statistical and computational performance of the procedure is illustrated by mean of both artificial and real datasets

empirical bayes hierarchical models for regularizing maximum likelihood estimation in the matrix gaussian procrustes problem procrustes analysis involves finding the optimal superposition of two or more forms via rotations translations and scalings procrustes problems arise in a wide range of scientific disciplines especially when the geometrical shapes of objects are compared contrasted and analyzed classically the optimal transformations are found by minimizing the sum of the squared distances between corresponding points in the forms despite its widespread use the ordinary unweighted least squares ls criterion can give erroneous solutions when the errors have heterogeneous variances heteroscedasticity or the errors are correlated both common occurrences with real data in contrast maximum likelihood ml estimation can provide accurate and consistent statistical estimates in the presence of both heteroscedasticity and correlation here we provide a complete solution to the nonisotropic ml procrustes problem assuming a matrix gaussian distribution with factored covariances our analysis generalizes simplifies and extends results from previous discussions of the ml procrustes problem an iterative algorithm is presented for the simultaneous numerical determination of the ml solutions

feedback stabilization of bilinear control systems in this paper we study the region in which a bilinear control system is feedback stabilizable in particular we find a necessary and sufficient condition for feedback stabilization in terms of the lyapunov spectrum

sparse multi dimensional graphical models a unified bayesian framework abstractmulti dimensional data constituted by measurements along multiple axes have emerged across many scientific areas such as genomics and cancer surveillance a common objective is to investigate the conditional dependencies among the variables along each axes taking into account multi dimensional structure of the data traditional multivariate approaches are unsuitable for such highly structured data due to inefficiency loss of power and lack of interpretability in this article we propose a novel class of multi dimensional graphical models based on matrix decompositions of the precision matrices along each dimension our approach is a unified framework applicable to both directed and undirected decomposable graphs as well as arbitrary combinations of these exploiting the marginalization of the likelihood we develop efficient posterior sampling schemes based on partially collapsed gibbs samplers empirically through simulation studies we show the superior performance of our approach in compari

shotgun stochastic search for large p regression model search in regression with very large numbers of candidate predictors raises challenges for both model specification and computation for which standard approaches such as markov chain monte carlo mcmc methods are often infeasible or ineffective we describe a novel shotgun stochastic search sss approach that explores interesting regions of the resulting high dimensional model spaces and quickly identifies regions of high posterior probability over models we describe algorithmic and modeling aspects priors over the model space that induce sparsity and parsimony over and above the traditional dimension penalization implicit in bayesian and likelihood analyses and parallel computation using cluster computers we discuss an example from gene expression cancer genomics comparisons with mcmc and other methods and theoretical and simulation based aspects of performance characteristics in large scale regression model searches we also provide software implementing the methods

hyper markov laws in the statistical analysis of decomposable graphical models this paper introduces and investigates the notion of a hyper markov law which is a probability distribution over the set of probability measures on a multivariate space that i is concentrated on the set of markov probabilities over some decomposable graph and ii satisfies certain conditional independence restrictions related to that graph a stronger version of this hyper markov property is also studied our analysis starts by reconsidering the properties of markov probabilities using an abstract approach which thereafter proves equally applicable to the hyper markov case next it is shown constructively that hyper markov laws exist that they appear as sampling distributions of maximum likelihood estimators in decomposable graphical models and also that they form natural conjugate prior distributions for a bayesian analysis of these models as examples we construct a range of specific hyper markov laws including the hyper multinomial hyper dirichlet and the hyper wishart and inverse wishart laws these laws occur naturally in connection with the analysis of decomposable log linear and covariance selection models

asymptotic controllability implies feedback stabilization it is shown that every asymptotically controllable system can be globally stabilized by means of some discontinuous feedback law the stabilizing strategy is based on pointwise optimization of a smoothed version of a control lyapunov function iteratively sending trajectories into smaller and smaller neighborhoods of a desired equilibrium a major technical problem and one of the contributions of the present paper concerns the precise meaning of solution when using a discontinuous controller

limit of the smallest eigenvalue of a large dimensional sample covariance matrix in this paper the authors show that the smallest if p leq n or the p n 1 th smallest if p n eigenvalue of a sample covariance matrix of the form 1 n xx tends almost surely to the limit 1 sqrt y 2 as n rightarrow infty and p n rightarrow y in 0 infty where x is a p times n matrix with iid entries with mean zero variance 1 and fourth moment finite also as a by product it is shown that the almost sure limit of the largest eigenvalue is 1 sqrt y 2 a known result obtained by yin bai and krishnaiah the present approach gives a unified treatment for both the extreme eigenvalues of large sample covariance matrices

convergence rates and asymptotic standard errors for markov chain monte carlo algorithms for bayesian probit regression consider a probit regression problem in which y 1 y n are independent bernoulli random variables such that pr y i 1 x i t where x i is a p dimensional vector of known covariates that are associated with y i is a p dimensional vector of unknown regression coefficients and denotes the standard normal distribution function we study markov chain monte carlo algorithms for exploring the intractable posterior density that results when the probit regression likelihood is combined with a flat prior on we prove that albert and chib s data augmentation algorithm and liu and wu s px da algorithm both converge at a geometric rate which ensures the existence of central limit theorems for ergodic averages under a second moment condition although these two algorithms are essentially equivalent in terms of computational complexity results of hobert and marchev imply that the px da algorithm is theoretically more efficient in the sense that the asymptotic variance in the central limit theorem under the px da algorithm is no larger than that under albert and chib s algorithm we also construct minorization conditions that allow us to exploit regenerative simulation techniques for the consistent estimation of asymptotic variances as an illustration we apply our results to van dyk and meng s lupus data this example demonstrates that huge gains in efficiency are possible by using the px da algorithm instead of albert and chib s algorithm

high dimensional semiparametric bigraphical models in multivariate analysis a gaussian bigraphical model is commonly used for modelling matrix valued data in this paper we propose a semiparametric extension of the gaussian bigraphical model called the nonparanormal bigraphical model a projected nonparametric rank based regularization approach is employed to estimate sparse precision matrices and produce graphs under a penalized likelihood framework theoretically our semiparametric procedure achieves the parametric rates of convergence for both matrix estimation and graph recovery empirically our approach outperforms the parametric gaussian model for non gaussian data and is competitive with its parametric counterpart for gaussian data extensions to the categorical bigraphical model and the missing data problem are discussed copyright 2013 oxford university press

corrigendum in just relax convex programming methods for identifying sparse signals in noise mar 06 1030 1051 this correspondence closes a gap in the proof of the main lemma from the paper just relax convex programming methods for identifying sparse signals in noise 2006

on the singular values of gaussian random matrices this short note is about the singular value distribution of gaussian random matrices i e gaussian ensemble or ge of size n we present a new approach for deriving the p d f of the singular values directly from the singular value decomposition svd form which also takes advantage of the rotational invariance of ge and the lie algebra of the orthogonal group our method is direct and more general than the conventional approach that relies on the wishart ensemble and the combination of qr and cholesky decomposition directly based on this p d f and its interpretation by statistical mechanics we give the physics proof that in the thermodynamic limit n the singular value distribution satisfies the quadrant law similar to the celebrated semi circle lawestablished by wigner more than 40 years ago for the spectral distribution of gaussian orthogonal or unitary ensembles this quadrant law was also proved earlier and mathematically more rigorously by some authors based on probabilistic estimations and the moment method but not directly from the p d f formula 2001 elsevier science inc all rights reserved

robust filtering for bilinear uncertain stochastic discrete time systems this paper deals with the robust filtering problem for uncertain bilinear stochastic discrete time systems with estimation error variance constraints the uncertainties are allowed to be norm bounded and enter into both the state and measurement matrices we focus on the design of linear filters such that for all admissible parameter uncertainties the error state of the bilinear stochastic system is mean square bounded and the steady state variance of the estimation error of each state is not more than the individual prespecified value it is shown that the design of the robust filters can be carried out by solving some algebraic quadratic matrix inequalities in particular we establish both the existence conditions and the explicit expression of desired robust filters a numerical example is included to show the applicability of the present method

bayesian analysis of matrix normal graphical models we present bayesian analyses of matrix variate normal data with conditional independencies induced by graphical model structuring of the characterizing covariance matrix parameters this framework of matrix normal graphical models includes prior specifications posterior computation using markov chain monte carlo methods evaluation of graphical model uncertainty and model structure search extensions to matrix variate time series embed matrix normal graphs in dynamic models examples highlight questions of graphical model uncertainty search and comparison in matrix data contexts these models may be applied in a number of areas of multivariate analysis time series and also spatial modelling

why simple shrinkage is still relevant for redundant representations shrinkage is a well known and appealing denoising technique introduced originally by donoho and johnstone in 1994 the use of shrinkage for denoising is known to be optimal for gaussian white noise provided that the sparsity on the signal s representation is enforced using a unitary transform still shrinkage is also practiced with nonunitary and even redundant representations typically leading to very satisfactory results in this correspondence we shed some light on this behavior the main argument in this work is that such simple shrinkage could be interpreted as the first iteration of an algorithm that solves the basis pursuit denoising bpdn problem while the desired solution of bpdn is hard to obtain in general we develop a simple iterative procedure for the bpdn minimization that amounts to stepwise shrinkage we demonstrate how the simple shrinkage emerges as the first iteration of this novel algorithm furthermore we show how shrinkage can be iterated turning into an effective algorithm that minimizes the bpdn via simple shrinkage steps in order to further strengthen the denoising effect

on the error of estimating the sparsest solution of underdetermined linear systems let a be an n m matrix with m n and suppose that the underdetermined linear system as x admits a sparse solution s0 for which s0 0 1 2 spark a such a sparse solution is unique due to a well known uniqueness theorem suppose now that we have somehow a solution s as an estimation of s0 and suppose that s is only approximately sparse that is many of its components are very small and nearly zero but not mathematically equal to zero is such a solution necessarily close to the true sparsest solution more generally is it possible to construct an upper bound on the estimation error s s0 2 without knowing s0 the answer is positive and in this paper we construct such a bound based on minimal singular values of submatrices of a we will also state a tight bound which is more complicated but besides being tight enables us to study the case of random dictionaries and obtain probabilistic upper bounds we will also study the noisy case that is where x as n moreover we will see that where s0 0 grows to obtain a predetermined guaranty on the maximum of s s0 2 s is needed to be sparse with a better approximation this can be seen as an explanation to the fact that the estimation quality of sparse recovery algorithms degrades where s0 0 grows

the lyapunov spectrum of families of time varying matrices for l families of time varying matrices centered at an unperturbed matrix the lyapunov spectrum contains the floquet spectrum obtained by considering periodically varying piecewise constant matrices on the other hand it is contained in the morse spectrum of an associated flow on a vector bundle a closer analysis of the floquet spectrum based on geometric control theory in projective space and in particular on control sets is performed introducing a real parameter p 0 which indicates the size of the li perturbation we study when the floquet spectrum the morse spectrum and hence the lyapunov spectrum all coincide this holds if an inner pair condition is satisfied for all up to at most countably many p values

parameter expansion for data augmentation abstract viewing the observed data of a statistical model as incomplete and augmenting its missing parts are useful for clarifying concepts and central to the invention of two well known statistical algorithms expectation maximization em and data augmentation recently liu rubin and wu demonstrated that expanding the parameter space along with augmenting the missing data is useful for accelerating iterative computation in an em algorithm the main purpose of this article is to rigorously define a parameter expanded data augmentation px da algorithm and to study its theoretical properties the px da is a special way of using auxiliary variables to accelerate gibbs sampling algorithms and is closely related to reparameterization techniques we obtain theoretical results concerning the convergence rate of the px da algorithm and the choice of prior for the expansion parameter to understand the role of the expansion parameter we establish a new theory for iterative conditional sampling under the tra

marginal likelihood from the gibbs output abstract in the context of bayes estimation via gibbs sampling with or without data augmentation a simple approach is developed for computing the marginal density of the sample data marginal likelihood given parameter draws from the posterior distribution consequently bayes factors for model comparisons can be routinely computed as a by product of the simulation hitherto this calculation has proved extremely challenging our approach exploits the fact that the marginal density can be expressed as the prior times the likelihood function over the posterior density this simple identity holds for any parameter value an estimate of the posterior density is shown to be available if all complete conditional densities used in the gibbs sampler have closed form expressions to improve accuracy the posterior density is estimated at a high density point and the numerical standard error of resulting estimate is derived the ideas are applied to probit regression and finite mixture models

some matrix variate distribution theory notational considerations and a bayesian application summary we introduce and justify a convenient notation for certain matrix variate distributions which by its emphasis on the important underlying parameters and the theory on which it is based eases greatly the task of manipulating such distributions important examples include the matrix variate normal t f and beta and the wishart and inverse wishart distributions the theory is applied to compound matrix distributions and to bayesian prediction in the multivariate linear model

on the limit of the largest eigenvalue of the large dimensional sample covariance matrix in this paper the authors show that the largest eigenvalue of the sample covariance matrix tends to a limit under certain conditions when both the number of variables and the sample size tend to infinity the above result is proved under the mild restriction that the fourth moment of the elements of the sample sums of squares and cross products sp matrix exist

a fast approach for overcomplete sparse decomposition based on smoothed ell 0 norm in this paper a fast algorithm for overcomplete sparse decomposition called sl0 is proposed the algorithm is essentially a method for obtaining sparse solutions of underdetermined systems of linear equations and its applications include underdetermined sparse component analysis sca atomic decomposition on overcomplete dictionaries compressed sensing and decoding real field codes contrary to previous methods which usually solve this problem by minimizing the l 1 norm using linear programming lp techniques our algorithm tries to directly minimize the l 1 norm it is experimentally shown that the proposed algorithm is about two to three orders of magnitude faster than the state of the art interior point lp solvers while providing the same or better accuracy

multivariate environmental statistics preface multivariate environmental statistics in agriculture v barnett a study of the relationship between diversity indices of benthic communities and heavy metal concentrations of northwest atlantic sediments n c bolgiano r n reid g p patil multivariate non normal statistics in site characterization and evaluation j h carson jr a k gupta spatial prediction in a multivariate setting n cressie needs and developments in multivariate environmental monitoring statistics g t flatman certain multivariate considerations in ranked set sampling and composite sampling designs s d gore et al relating two sets of variables in environmental studies r h green statistical analysis of biological monitoring data p guttorp using entropy in the redesign of an environmental monitoring network p guttorp et al graphical techniques for enhancing the utility of multivariate environmental statistics w a huber carcinogenicity tests involving multiple tumor sites r l kodell e o george errors in variables analysis of extrapolation procedures in environmental toxicology e linder g p patil g suter ii a unified linear model for estimation with composite sample data g lovison change point methods for spatial data i b macneill v k jandkyala a review of computer intensive multivariate methods in ecology b f j manly spatial temporal statistical analysis of multivariate environmental monitoring data k v mardia c r goodall correspondence analysis applied to environmental data sets d e myers a meta network approach to higher order spatial constructs and scale effects w l myers estimation of trend in chesapeake bay water quality data n k nagaraj s l brunenmeister the complexities and scenarios of ecosystem analysis l orloci multivariate methods in nuclear waste remediation needs and applications b a pulsipher omnibus robust procedures for assessment of multivariate normality and detection of multivariate outliers a singh multivariate assessment of trend in environmental variables e p smith s rheem g i holtzman multivariate chemometrics a case study applying and developing receptor models for the 1990 el paso winter pm10 receptor modeling scoping study c h spiegelman s dattner weighted averaging partial least squares regression wa pls definition and comparison with other methods for species environment calibration c j f ter braak et al multivariate aspects of adaptive cluster sampling s k thompson approximating multivariate distributions in stochastic models of insect population dynamics t e wehrly j h matis g w otis

on selecting a maximum volume sub matrix of a matrix and related problems given a matrix email protected r m x n n vectors in m dimensions we consider the problem of selecting a subset of its columns such that its elements are as linearly independent as possible this notion turned out to be important in low rank approximations to matrices and rank revealing qr factorizations which have been investigated in the linear algebra community and can be quantified in a few different ways in this paper from a complexity theoretic point of view we propose four related problems in which we try to find a sub matrix email protected r m x k of a given matrix email protected r m x n such that i s m a x c the largest singular value of c is minimum ii s m i n c the smallest singular value of c is maximum iii k c email protected m a x c s m i n c the condition number of c is minimum and iv the volume of the parallelepiped defined by the column vectors of c is maximum we establish the np hardness of these problems and further show that they do not admit ptas we then study a natural greedy heuristic for the maximum volume problem and show that it has approximation ratio 2 o k l o g k our analysis of the greedy heuristic is tight to within a logarithmic factor in the exponent which we show by explicitly constructing an instance for which the greedy heuristic is 2 w k from optimal when a has unit norm columns a related problem is to select the maximum number of vectors with a given volume we show that if the optimal solution selects k columns then greedy will select w k logk columns providing a logk approximation

minimal and maximal lyapunov exponents of bilinear control systems abstract for a bilinear control system with bounded and unbounded controls x a 0 u 0 x graphics u i a i u 0 x in r d where u0 t r l compact ui t r for i 1 m the extremal exponential growth rates of the solutions x x0 u are analyzed if x0 u lim supt 1 t log x t x0 u then k supu u supx0 0 x0 u and k infu u infx0 0 x0 u are the maximal and minimal respectively lyapunov exponents of the system this paper gives several characterizations of these rates together with the corresponding uniform concepts with respect to the initial value or the control we describe the situations in which k and k and characterize the sets of initial values from which k and k can actually be realized the techniques are applied to high gain stabilizalion and the example of the linear oscillator with parameter controlled restoring force is treated in detail finally we indicate how the results can be used for feedback stabilization of linear systems when the feedback is allowed to be time varying but restricted to certain types of bounded gain matrices

restricted isometry constants where ell p sparse recovery can fail for 0 p leq 1 this paper investigates conditions under which the solution of an underdetermined linear system with minimal lscrp norm 0 p les 1 is guaranteed to be also the sparsest one matrices are constructed with restricted isometry constants ric delta2m arbitrarily close to 1 radic2 ap 0 707 where sparse recovery with p 1 fails for at least one m sparse vector as well as matrices with delta2m arbitrarily close to one where lscr1 minimization succeeds for any m sparse vector this highlights the pessimism of sparse recovery prediction based on the ric and indicates that there is limited room for improving over the best known positive results of foucart and lai which guarantee that lscr1 minimization recovers all m sparse vectors for any matrix with delta2m 2 3 radic2 7 ap 0 4531 these constructions are a by product of tight conditions for lscrp recovery 0 les p les 1 with matrices of unit spectral norm which are expressed in terms of the minimal singular values of 2m column submatrices compared to lscr1 minimization lscrp minimization recovery failure is shown to be only slightly delayed in terms of the ric values furthermore in this case the minimization is nonconvex and it is important to consider the specific minimization algorithm being used it is shown that when lscrp optimization is attempted using an iterative reweighted lscr1 scheme failure can still occur for delta2m arbitrarily close to 1 radic2

a bayesian analysis of the multinomial probit model with fully identified parameters we present a new prior and corresponding algorithm for bayesian analysis of the multinomial probit model our new approach places a prior directly on the identified parameter space the key is the specification of a prior on the covariance matrix so that the 1 1 element if fixed at 1 and it is possible to draw from the posterior using standard distributions analytical results are derived which can be used to aid in assessment of the prior

simulation of hyper inverse wishart distributions in graphical models summary we introduce and exemplify an efficient method for direct sampling from hyperinverse wishart distributions the method relies very naturally on the use of standard junction tree representation of graphs and couples these with matrix results for inverse wishart distributions we describe the theory and resulting computational algorithms for both decomposable and nondecomposable graphical models an example drawn from financial time series demonstrates application in a context where inferences on a structured covariance model are required we discuss and investigate questions of scalability of the simulation methods to higher dimensional distributions the paper concludes with general comments about the approach including its use in connection with existing markov chain monte carlo methods that deal with uncertainty about the graphical model structure

parameterization and bayesian modeling progress in statistical computation often leads to advances in statistical modeling for example it is surprisingly common that an existing model is reparameterized solely for computational purposes but then this new configuration motivates a new family of models that is useful in applied statistics one reason why this phenomenon may not have been noticed in statistics is that reparameterizations do not change the likelihood in a bayesian framework however a transformation of parameters typically suggests a new family of prior distributions we discuss examples in censored and truncated data mixture modeling multivariate imputation stochastic processes and multilevel models