the gaussian measure of shifted balls let be a centered gaussian measure on a hilbert spaceh and let b r subseteq h be the centered ball of radiusr 0 fora h and mathop lim limits t mathbf to mathbf infty mathbf r t t mathbf a we give the exact asymptotics of b r t t a ast also upper and lower bounds are given when is defined on an arbitrary separable banach space our results range from small deviation estimates to large deviation estimates

bayesian inference for gaussian graphical models beyond decomposable graphs bayesian inference for graphical models has received much attention in the literature in recent years it is well known that when the graph g is decomposable bayesian inference is significantly more tractable than in the general non decomposable setting penalized likelihood inference in contrast has made tremendous gains in the past few years in terms of scalability and tractability bayesian inference however has not had the same level of success though a scalable bayesian approach has its strengths especially in terms of quantifying uncertainty to address this gap we propose a scalable and flexible novel bayesian approach for estimation and model selection in gaussian undirected graphical models we first develop a class of generalized g wishart distributions with multiple shape parameters for an arbitrary underlying graph this class contains the g wishart distribution as a special case we then introduce the class of generalized bartlett graphs and derive an efficient gibbs sampling algorithm to obtain posterior draws from generalized g wishart distributions corresponding to a generalized bartlett graph the class of generalized bartlett graphs contains the class of decomposable graphs as a special case but is substantially larger than the class of decomposable graphs we proceed to derive theoretical properties of the proposed gibbs sampler we then demonstrate that the proposed gibbs sampler is scalable to significantly higher dimensional problems compared with using an accept reject or a metropolis hasting algorithm finally we show the efficacy of the proposed approach on simulated and real data in particular we demonstrate that our generalized bartlett methodology can be used for efficient model selection by reducing the graph search space by using penalized likelihood and pseudolikelihood methods

quasi periodic solutions to nonlinear beam equations on compact lie groups with a multiplicative potential abstract the goal of this work is to study the existence of quasi periodic solutions to nonlinear beam equations with a multiplicative potential the nonlinearity is required to only finitely differentiable and the frequency is along a pre assigned direction the result holds on any compact lie group or homogeneous manifold with respect to a compact lie group which includes standard torus t d special orthogonal group s o d special unitary group s u d spheres s d and the real and complex grassmannians the proof is based on a differentiable nash moser iteration scheme

a kam theorem for hamiltonian partial differential equations in higher dimensional spaces in this paper we give a kam theorem for a class of infinite dimensional nearly integrable hamiltonian systems the theorem can be applied to some hamiltonian partial differential equations in higher dimensional spaces with periodic boundary conditions to construct linearly stable quasi periodic solutions and its local birkhoff normal form the applications to the higher dimensional beam equations and the higher dimensional schrodinger equations with nonlocal smooth nonlinearity are also given in this paper

optimal bayesian estimation in random covariate design with a rescaled gaussian process prior in bayesian nonparametric models gaussian processes provide a popular prior choice for regression function estimation existing literature on the theoretical investigation of the resulting posterior distribution almost exclusively assume a fixed design for covariates the only random design result we are aware of van der vaart and van zanten 2011 assumes the assigned gaussian process to be supported on the smoothness class specified by the true function with probability one this is a fairly restrictive assumption as it essentially rules out the gaussian process prior with a squared exponential kernel when modeling rougher functions in this article we show that an appropriate rescaling of the above gaussian process leads to a rate optimal posterior distribution even when the covariates are independently realized from a known density on a compact set the proofs are based on deriving sharp concentration inequalities for frequentist kernel estimators the results might be of independent interest

scalable estimation and regularization for the logistic normal multinomial model clustered multinomial data are prevalent in a variety of applications such as microbiome studies where metagenomic sequencing data are summarized as multinomial counts for a large number of bacterial taxa per subject count normalization with ad hoc zero adjustment tends to result in poor estimates of abundances for taxa with zero or small counts to account for heterogeneity and overdispersion in such data we suggest using the logistic normal multinomial lnm model with an arbitrary correlation structure to simultaneously estimate the taxa compositions by borrowing information across subjects we overcome the computational difficulties in high dimensions by developing a stochastic approximation em algorithm with hamiltonian monte carlo sampling for scalable parameter estimation in the lnm model the ill conditioning problem due to unstructured covariance is further mitigated by a covariance regularized estimator with a condition number constraint the advantages of the proposed methods are illustrated through simulations and an application to human gut microbiome data

nonparametric eigenvalue regularized precision or covariance matrix estimator we introduce nonparametric regularization of the eigenvalues of a sample covariance matrix through splitting of the data nercome and prove that nercome enjoys asymptotic optimal nonlinear shrinkage of eigenvalues with respect to the frobenius norm one advantage of nercome is its computational speed when the dimension is not too large we prove that nercome is positive definite almost surely as long as the true covariance matrix is so even when the dimension is larger than the sample size with respect to the stein s loss function the inverse of our estimator is asymptotically the optimal precision matrix estimator asymptotic efficiency loss is defined through comparison with an ideal estimator which assumed the knowledge of the true covariance matrix we show that the asymptotic efficiency loss of nercome is almost surely 0 with a suitable split location of the data we also show that all the aforementioned optimality holds for data with a factor structure our method avoids the need to first estimate any unknowns from a factor model and directly gives the covariance or precision matrix estimator which can be useful when factor analysis is not the ultimate goal we compare the performance of our estimators with other methods through extensive simulations and real data analysis

estimation of a covariance matrix using the reference prior estimation of a covariance matrix sum is a notoriously difficult problem the standard unbiased estimator can be substantially suboptimal we approach the problem from a noninformative prior bayesian perspective developing the reference noninformative prior for a covariance matrix and obtaining expressions for the resulting bayes estimators these expressions involve the computation of high dimensional posterior expectations which is done using a recent markov chain simulation tool the hit and run sampler frequentist risk comparisons with previously suggested estimators are also given and determination of the accuracy of the estimators is addressed

condition number regularized covariance estimation estimation of high dimensional covariance matrices is known to be a difficult problem has many applications and is of current interest to the larger statistics community in many applications including so called the large p small n setting the estimate of the covariance matrix is required to be not only invertible but also well conditioned although many regularization schemes attempt to do this none of them address the ill conditioning problem directly in this paper we propose a maximum likelihood approach with the direct goal of obtaining a well conditioned estimator no sparsity assumption on either the covariance matrix or its inverse are are imposed thus making our procedure more widely applicable we demonstrate that the proposed regularization scheme is computationally efficient yields a type of steinian shrinkage estimator and has a natural bayesian interpretation we investigate the theoretical properties of the regularized covariance estimator comprehensively including its regularization path and proceed to develop an approach that adaptively determines the level of regularization that is required finally we demonstrate the performance of the regularized estimator in decision theoretic comparisons and in the financial portfolio optimization setting the proposed approach has desirable properties and can serve as a competitive procedure especially when the sample size is small and when a well conditioned estimator is required

learning local dependence in ordered data in many applications data come with a natural ordering this ordering can often induce local dependence among nearby variables however in complex data the width of this dependence may vary making simple assumptions such as a constant neighborhood size unrealistic we propose a framework for learning this local dependence based on estimating the inverse of the cholesky factor of the covariance matrix penalized maximum likelihood estimation of this matrix yields a simple regression interpretation for local dependence in which variables are predicted by their neighbors our proposed method involves solving a convex penalized gaussian likelihood problem with a hierarchical group lasso penalty the problem decomposes into independent subproblems which can be solved efficiently in parallel using first order methods our method yields a sparse symmetric positive definite estimator of the precision matrix encoding a gaussian graphical model we derive theoretical results not found in existing methods attaining this structure in particular our conditions for signed support recovery and estimation consistency rates in multiple norms are as mild as those in a regression problem empirical results show our method performing favorably compared to existing methods we apply our method to genomic data to flexibly model linkage disequilibrium our method is also applied to improve the performance of discriminant analysis in sound recording classification

kam tori of hamiltonian perturbations of 1d linear beam equations abstract in this paper one dimensional 1d nonlinear beam equations u tt u xxxx mu f u with hinged boundary conditions are considered the nonlinearity f is an analytic odd function and f u o u3 it is proved that for all real parameters m 0 but a set of small lebesgue measure the above equation admits small amplitude quasi periodic solutions corresponding to finite dimensional invariant tori of an associated infinite dimensional dynamical system the proof is based on infinite dimensional kam theory developed by kuksin lecture notes in mathematics vol 1556 springer berlin 1993 wayne commun math phys 127 1990 479 528 poschel ann sc norm sup pisa cl sci 23 1996 119 148

hub discovery in partial correlation graphs one of the most important problems in large scale inference problems is the identification of variables that are highly dependent on several other variables when dependence is measured by partial correlations these variables identify those rows of the partial correlation matrix that have several entries with large magnitudes i e hubs in the associated partial correlation graph this paper develops theory and algorithms for discovering such hubs from a few observations of these variables we introduce a hub screening framework in which the user specifies both a minimum partial correlation rho and a minimum degree delta to screen the vertices the choice of rho and delta can be guided by our mathematical expressions for the phase transition correlation threshold rho c governing the average number of discoveries they can also be guided by our asymptotic expressions for familywise discovery rates under the assumption of large number p of variables fixed number n of multivariate samples and weak dependence under the null hypothesis that the dispersion covariance matrix is sparse these limiting expressions can be used to enforce familywise error constraints and to rank the discoveries in order of increasing statistical significance for n ll p the computational complexity of the proposed partial correlation screening method is low and is therefore highly scalable thus it can be applied to significantly larger problems than previous approaches the theory is applied to discovering hubs in a high dimensional gene microarray dataset

a convex pseudolikelihood framework for high dimensional partial correlation estimation with convergence guarantees author s khare k oh sy rajaratnam b abstract 2014 royal statistical society sparse high dimensional graphical model selection is a topic of much interest in modern day statistics a popular approach is to apply l1 penalties to either parametric likelihoods or regularized regression pseudolikelihoods with the latter having the distinct advantage that they do not explicitly assume gaussianity as none of the popular methods proposed for solving pseudolikelihood based objective functions have provable convergence guarantees it is not clear whether corresponding estimators exist or are even computable or if they actually yield correct partial correlation graphs we propose a new pseudolikelihood based graphical model selection method that aims to overcome some of the shortcomings of current methods but at the same time retain all their respective strengths in particular we introduce a novel framework that leads to a convex formulation of the partial covariance regression graph problem resulting in an objective function comprised of quadratic forms the objective is then optimized via a co ordinatewise approach the specific functional form of the objective function facilitates rigorous convergence analysis leading to convergence guarantees an important property that cannot be established by using standard results when the dimension is larger than the sample size as is often the case in high dimensional applications these convergence guarantees ensure that estimators are well defined under very general conditions and are always computable in addition the approach yields estimators that have good large sample properties and also respect symmetry furthermore application to simulated and real data timing comparisons and numerical convergence is demonstrated we also present a novel unifying framework that places all graphical pseudolikelihood methods as special cases of a more general formulation leading to important insights

comparing and weighting imperfect models using d probabilities abstractwe propose a new approach for assigning weights to models using a divergence based method d probabilities relying on evaluating parametric models relative to a nonparametric bayesian ref

information consistency of nonparametric gaussian process methods bayesian nonparametric models are widely and successfully used for statistical prediction while posterior consistency properties are well studied in quite general settings results have been proved using abstract concepts such as metric entropy and they come with subtle conditions which are hard to validate and not intuitive when applied to concrete models furthermore convergence rates are difficult to obtain by focussing on the concept of information consistency for bayesian gaussian process gp models consistency results and convergence rates are obtained via a regret bound on cumulative log loss these results depend strongly on the covariance function of the prior process thereby giving a novel interpretation to penalization with reproducing kernel hilbert space norms and to commonly used covariance function classes and their parameters the proof of the main result employs elementary convexity arguments only a theorem of widom is used in order to obtain precise convergence rates for several covariance functions widely used in practice

grouped coordinate ascent algorithms for penalized likelihood transmission image reconstruction presents a new class of algorithms for penalized likelihood reconstruction of attenuation maps from low count transmission scans we derive the algorithms by applying to the transmission log likelihood a version of the convexity technique developed by de pierro for emission tomography the new class includes the single coordinate ascent sca algorithm and lange s convex algorithm for transmission tomography as special cases the new grouped coordinate ascent gca algorithms in the class overcome several limitations associated with previous algorithms 1 fewer exponentiations are required than in the transmission maximum likelihood expectation maximization ml em algorithm or in the sca algorithm 2 the algorithms intrinsically accommodate nonnegativity constraints unlike many gradient based methods 3 the algorithms are easily parallelizable unlike the sca algorithm and perhaps line search algorithms we show that the gca algorithms converge faster than the sca algorithm even on conventional workstations an example from a low count positron emission tomography pet transmission scan illustrates the method

flexible covariance estimation in graphical gaussian models in this paper we propose a class of bayes estimators for the covariance matrix of graphical gaussian models markov with respect to a decomposable graph g working with the w pg family defined by letac and massam ann statist 35 2007 1278 1323 we derive closed form expressions for bayes estimators under the entropy and squared error losses the w pg family includes the classical inverse of the hyper inverse wishart but has many more shape parameters thus allowing for flexibility in differentially shrinking various parts of the covariance matrix moreover using this family avoids recourse to mcmc often infeasible in high dimensional problems we illustrate the performance of our estimators through a collection of numerical examples where we explore frequentist risk properties and the efficacy of graphs in the estimation of high dimensional covariance structures

building accurate emulators for stochastic simulations via quantile kriging computer simulation has increasingly become popular for analysis of systems that cannot be feasibly changed because of costs or scale this work proposes a method to construct an emulator for stochastic simulations by performing a designed experiment on the simulator and developing an emulative distribution existing emulators have focused on estimation of the mean of the simulation output but this work presents an emulator for the distribution of the output this construction provides both an explicit distribution and a fast sampling scheme beyond the emulator description this work demonstrates the emulator s efficiency that is its convergence rate is the asymptotically optimal among all possible emulators using the same sample size under certain conditions an example of its practical use is demonstrated using a stochastic simulation of fracture mechanics supplementary materials for this article are available online

inequalities of the brunn minkowski type for gaussian measures let m 2 be an integer let be the standard gaussian measure on rn and let t t exp s2 2 ds sqrt 2 small t le given 1 l m 0 this paper gives a necessary and sufficient condition such that the inequality 1 1 a1 m a m 1 1 a 1 m 1 a m is true for all borel sets a 1 a m in hbfrn of strictly positive measure or all convex borel sets a 1 a m in bfrn of strictly positive measure respectively in particular the paper exhibits inequalities of the brunn minkowski type for which are true for all convex sets but not for all measurable sets

periodic and quasi periodic solutions of nonlinear wave equations via kam theory in this paper the nonlinear wave equation u u u xx v x u x t varepsilon u 3 x t 0 is studied it is shown that for a large class of potentials v x one can use kam methods to construct periodic and quasi periodic solutions in time for this equation

adaptive bayesian estimation using a gaussian random field with inverse gamma bandwidth we consider nonparametric bayesian estimation inference using a rescaled smooth gaussian field as a prior for a multidimensional function the rescaling is achieved using a gamma variable and the procedure can be viewed as choosing an inverse gamma bandwidth the procedure is studied from a frequentist perspective in three statistical settings involving replicated observations density estimation regression and classification we prove that the resulting posterior distribution shrinks to the distribution that generates the data at a speed which is minimax optimal up to a logarithmic factor whatever the regularity level of the data generating distribution thus the hierachical bayesian procedure with a fixed prior is shown to be fully adaptive

design free estimation of variance matrices abstract this paper introduces a new method for estimating variance matrices starting from the orthogonal decomposition of the sample variance matrix we exploit the fact that orthogonal matrices are never ill conditioned and therefore focus on improving the estimation of the eigenvalues we estimate the eigenvectors from just a fraction of the data then use them to transform the data into approximately orthogonal series that deliver a well conditioned estimator by construction even when there are fewer observations than dimensions we also show that our estimator has lower error norms than the traditional one our estimator is design free we make no assumptions on the distribution of the random sample or on any parametric structure the variance matrix may have simulations confirm our theoretical results and they also show that our simple estimator does very well in comparison with other existing methods

rates of contraction of posterior distributions based on gaussian process priors we derive rates of contraction of posterior distributions on nonparametric or semiparametric models based on gaussian processes the rate of contraction is shown to depend on the position of the true parameter relative to the reproducing kernel hilbert space of the gaussian process and the small ball probabilities of the gaussian process we determine these quantities for a range of examples of gaussian priors and in several statistical settings for instance we consider the rate of contraction of the posterior distribution based on sampling from a smooth density model when the prior models the log density as a fractionally integrated brownian motion we also consider regression with gaussian errors and smooth classification under a logistic or probit link function combined with various priors

kam for quasi linear and fully nonlinear forced perturbations of airy equation we prove the existence of small amplitude quasi periodic solutions for quasi linear and fully nonlinear forced perturbations of the linear airy equation for hamiltonian or reversible nonlinearities we also prove their linear stability the key analysis concerns the reducibility of the linearized operator at an approximate solution which provides a sharp asymptotic expansion of its eigenvalues for quasi linear perturbations this cannot be directly obtained by a kam iteration hence we first perform a regularization procedure which conjugates the linearized operator to an operator with constant coefficients plus a bounded remainder these transformations are obtained by changes of variables induced by diffeomorphisms of the torus and pseudo differential operators at this point we implement a nash moser iteration with second order melnikov non resonance conditions which completes the reduction to constant coefficients

comment ridge regression and regularization of large matrices we view ridge regression through the lens of eigenvalue shrinkage and consider its influence on two modern problems in high dimensional statistical inference covariance estimation and community d

wishart distributions for decomposable covariance graph models gaussian covariance graph models encode marginal independence among the components of a multivariate random vector by means of a graph g these models are distinctly different from the traditional concentration graph models often also referred to as gaussian graphical models or covariance selection models since the zeros in the parameter are now reflected in the covariance matrix as compared to the concentration matrix 1 the parameter space of interest for covariance graph models is the cone p g of positive definite matrices with fixed zeros corresponding to the missing edges of g as in letac and massam ann statist 35 2007 1278 1323 we consider the case where g is decomposable in this paper we construct on the cone p g a family of wishart distributions which serve a similar purpose in the covariance graph setting as those constructed by letac and massam ann statist 35 2007 1278 1323 and dawid and lauritzen ann statist 21 1993 1272 1317 do in the concentration graph setting we proceed to undertake a rigorous study of these covariance wishart distributions and derive several deep and useful properties of this class first they form a rich conjugate family of priors with multiple shape parameters for covariance graph models second we show how to sample from these distributions by using a block gibbs sampling algorithm and prove convergence of this block gibbs sampler development of this class of distributions enables bayesian inference which in turn allows for the estimation of e even in the case when the sample size is less than the dimension of the data i e when n p otherwise not generally possible in the maximum likelihood framework third we prove that when g is a homogeneous graph our covariance priors correspond to standard conjugate priors for appropriate directed acyclic graph dag models this correspondence enables closed form expressions for normalizing constants and expected values and also establishes hyper markov properties for our class of priors we also note that when g is homogeneous the family iw qg of letac and massam ann statist 35 2007 1278 1323 is a special case of our covariance wishart distributions fourth and finally we illustrate the use of our family of conjugate priors on real and simulated data

persistence of lower dimensional tori of general types in hamiltonian systems this work is a generalization to a result of j you 1999 we study the persistence of lower dimensional tori of general type in hamiltonian systems of general normal forms by introducing a modified linear kam iterative scheme to deal with small divisors we shall prove a persistence result under a melnikov type of non resonance condition which particularly allows multiple and degenerate normal frequencies of the unperturbed lower dimensional tori

persistence of lower dimensional hyperbolic tori for reversible system abstract in this paper we consider the persistence of hyperbolic lower dimensional invariant tori with prescribed frequency for reversible system and prove that if the frequency mapping has nonzero topological degree at some diophantine frequency then the hyperbolic lower invariant dimensional torus with this frequency persists under small perturbations

branching of cantor manifolds of elliptic tori and applications to pdes we consider infinite dimensional hamiltonian systems we prove the existence of cantor manifolds of elliptic tori of any finite higher dimension accumulating on a given elliptic kam torus then close to an elliptic equilibrium we show the existence of cantor manifolds of elliptic tori which are branching points of other cantor manifolds of higher dimensional tori we also answer to a conjecture of bourgain proving the existence of invariant elliptic tori with tangential frequency along a pre assigned direction the proofs are based on an improved kam theorem its main advantages are an explicit characterization of the cantor set of parameters and weaker smallness conditions on the perturbation we apply these results to the nonlinear wave equation

nonlinear shrinkage estimation of large dimensional covariance matrices many statistical applications require an estimate of a covariance matrix and or its inverse when the matrix dimension is large compared to the sample size which happens frequently the sample covariance matrix is known to perform poorly and may suffer from ill conditioning there already exists an extensive literature concerning improved estimators in such situations in the absence of further knowledge about the structure of the true covariance matrix the most successful approach so far arguably has been shrinkage estimation shrinking the sample covariance matrix to a multiple of the identity by taking a weighted average of the two turns out to be equivalent to linearly shrinking the sample eigenvalues to their grand mean while retaining the sample eigenvectors our paper extends this approach by considering nonlinear transformations of the sample eigenvalues we show how to construct an estimator that is asymptotically equivalent to an oracle estimator suggested in previous work as demonstrated in extensive monte carlo simulations the resulting bona fide estimator can result in sizeable improvements over the sample covariance matrix and also over linear shrinkage

efficient algorithms for solving condition number constrained matrix minimization problems abstract well conditioned matrices are often required in science and engineering such as signal processing and finance problem to find the nearest positive definite matrix by explicitly imposing a constraint on the condition number are considered in this paper a new algorithm based on geometric perspective is proposed for getting the required well conditioned matrix based on these a condition number constrained matrix minimization problem is further considered where the constraints are imposed for avoiding degenerate solutions in which parameter matrices become rank deficient an inexact version of alternating direction method with truly implementable inexactness criteria is proposed for solving this problem numerical experiments including comparison with some existing methods are performed to illustrate the efficiency of the proposed algorithms

quasi periodic solutions of nonlinear beam equation with prescribed frequencies consider the one dimensional nonlinear beam equation utt uxxxx mu u3 0 under dirichlet boundary conditions we show that for any m 0 but a set of small lebesgue measure the above equation admits a family of small amplitude quasi periodic solutions with n dimensional diophantine frequencies these diophantine frequencies are the small dilation of a prescribed diophantine vector the proofs are based on an infinite dimensional kolmogorov arnold moser iteration procedure and a partial birkhoff normal form

proper quaternion gaussian graphical models in this paper we extend gaussian graphical models to proper quaternion gaussian distributions the properness assumption reduces the number of unknowns by a factor of four while graphical models reduce the number of degrees of freedom via sparsity each of the methods allows accurate estimation using a small number of samples to enjoy both gains we show that the proper quaternion gaussian inverse covariance estimation problem is convex and has a closed form solution we proceed to demonstrate that the additional sparsity constraints on the inverse covariance matrix also lead to a convex problem and the optimizations can be efficiently solved by standard numerical methods in the special but practical case of a chordal graph we provide a closed form solution we demonstrate the improved performance of our suggested estimators on both synthetic and real data

a conditional density estimation partition model using logistic gaussian processes conditional density estimation density regression estimates the distribution of a response variable y conditional on covariates x utilizing a partition model framework a conditional density estimation method is proposed using logistic gaussian processes the partition is created using a voronoi tessellation and is learned from the data using a reversible jump markov chain monte carlo algorithm the markov chain monte carlo algorithm is made possible through a laplace approximation on the latent variables of the logistic gaussian process model this approximation marginalizes the parameters in each partition element allowing an efficient search of the posterior distribution of the tessellation the method has desirable consistency properties in simulation and applications the model successfully estimates the partition structure and conditional distribution of y

shrinkage estimators for covariance matrices estimation of covariance matrices in small samples has been studied by many authors standard estimators like the unstructured maximum likelihood estimator ml or restricted maximum likelihood reml estimator can be very unstable with the smallest estimated eigenvalues being too small and the largest too big a standard approach to more stably estimating the matrix in small samples is to compute the ml or reml estimator under some simple structure that involves estimation of fewer parameters such as compound symmetry or independence however these estimators will not be consistent unless the hypothesized structure is correct if interest focuses on estimation of regression coefficients with correlated or longitudinal data a sandwich estimator of the covariance matrix may be used to provide standard errors for the estimated coefficients that are robust in the sense that they remain consistent under misspecification of the covariance structure with large matrices however the inefficiency of the sandwich estimator becomes worrisome we consider here two general shrinkage approaches to estimating the covariance matrix and regression coefficients the first involves shrinking the eigenvalues of the unstructured ml or reml estimator the second involves shrinking an unstructured estimator toward a structured estimator for both cases the data determine the amount of shrinkage these estimators are consistent and give consistent and asymptotically efficient estimates for regression coefficients simulations show the improved operating characteristics of the shrinkage estimators of the covariance matrix and the regression coefficients in finite samples the final estimator chosen includes a combination of both shrinkage approaches i e shrinking the eigenvalues and then shrinking toward structure we illustrate our approach on a sleep eeg study that requires estimation of a 24 x 24 covariance matrix and for which inferences on mean parameters critically depend on the covariance estimator chosen we recommend making inference using a particular shrinkage estimator that provides a reasonable compromise between structured and unstructured estimators

sparse gaussian graphical model estimation via alternating minimization summaryseveral methods have recently been proposed for estimating sparse gaussian graphical models using ell 1 regularization on the inverse covariance or precision matrix despite recent advances contemporary applications require even faster methods to handle ill conditioned high dimensional datasets in this paper we propose a new method for solving the sparse inverse covariance estimation problem using the alternating minimization algorithm which effectively works as a proximal gradient algorithm on the dual problem our approach has several advantages it is faster than state of the art algorithms by many orders of magnitude its global linear convergence has been rigorously demonstrated underscoring its good theoretical properties it facilitates additional constraints on pairwise or marginal relationships between feature pairs based on domain specific knowledge and it is better at handling extremely ill conditioned problems our algorithm is shown to be more accurate and faster on simulated and real datasets

penalized normal likelihood and ridge regularization of correlation and covariance matrices high dimensionality causes problems in various areas of statistics a particular situation that rarely has been considered is the testing of hypotheses about multivariate regression models in which

thomas bayes walk on manifolds convergence of the bayes posterior measure is considered in canonical statistical settings where observations sit on a geometrical object such as a compact manifold or more generally on a compact metric space verifying some conditions a natural geometric prior based on randomly rescaled solutions of the heat equation is considered upper and lower bound posterior contraction rates are derived

maximum likelihood estimation and large sample inference for generalized linear and nonlinear regression models summary the class of generalized linear models is extended to allow for correlated observations nonlinear models and error distributions not of the exponential family form the extended class of models include a number of important examples particularly of the composite transformational type large sample inference and maximum likelihood estimation for the extended class of generalized linear models are discussed and the analysis of deviance is generalized to the extended class of models calculation of the maximum likelihood estimate for a general likelihood by fisher s scoring method and a related method is considered and the relation with the gauss newton method is discussed

metric entropy and the small ball problem for gaussian measures abstract we establish a precise link between the small ball problem for a gaussian measure on a separable banach space and the metric entropy of the unit ball of the hubert space h generating this link allows us to compute small ball probabilities from metric entropy results and vice versa

bayesian manifold regression there is increasing interest in the problem of nonparametric regression with high dimensional predictors when the number of predictors d is large one encounters a daunting problem in attempting to estimate a d dimensional surface based on limited data fortunately in many applications the support of the data is concentrated on a d dimensional subspace with d ll d manifold learning attempts to estimate this subspace our focus is on developing computationally tractable and theoretically supported bayesian nonparametric regression methods in this context when the subspace corresponds to a locally euclidean compact riemannian manifold we show that a gaussian process regression approach can be applied that leads to the minimax optimal adaptive rate in estimating the regression function under some conditions the proposed model bypasses the need to estimate the manifold and can be implemented using standard algorithms for posterior computation in gaussian processes finite sample performance is illustrated in a data analysis example

estimation of a covariance matrix under stein s loss on decrit la technique generale de stein pour ameliorer l estimateur sans biais invariant le meilleur et l estimateur minimax de la matrice de covariance normale

periodic solutions for a class of nonlinear partial differential equations in higher dimension we prove the existence of periodic solutions in a class of nonlinear partial differential equations including the nonlinear schrodinger equation the nonlinear wave equation and the nonlinear beam equation in higher dimension our result covers cases of completely resonant equations where the bifurcation equation is infinite dimensional such as the nonlinear schrodinger equation with zero mass for which solutions which at leading order are wave packets are shown to exist

a theoretical study of stein s covariance estimator the traditional covariance estimator the sample covariance matrix which is also the mle is known to be a poor estimator unless the sample size is much larger than the dimension of the covariance matrix stein s estimator has often been regarded as a much better alternative to the mle in small sample sizes and is traditionally used with an isotonizing algorithm the purpose of which is to retain positivity and the original order of the sample eigenvalues despite the superior performance of stein s isotonized estimator in numerical investigations its theoretical properties have not been explored in detail and important questions still remain unanswered one particular question of interest is to identify the regimes under which stein s estimator is guaranteed to perform well a second goal is to determine the extent to which the performance of stein s estimator depends on the isotonizing algorithm the presence of the ad hoc isotonizing algorithm however renders a theoretical analysis rather difficult and consequently risk functions are not easily quantifiable for comparison purposes hence formal decision theoretical results are difficult to obtain and have been elusive ever since the estimator was introduced despite these hurdles in this paper we show that an analysis of stein s covariance estimator within the unbiased estimator of risk ubeor framework can nevertheless lead to important theoretical and methodological insights that are relevant for applications our analysis demonstrates that stein s estimator may give only modest risk reductions when it is not isotonized and when it is isotonized the risk reductions can be significant in particular three broad regimes are identified regarding the behavior of stein s ubeor the theoretical insights are then affirmed at the level of risk functions via numerical simulations

proportionality of covariance matrices on considere l inference pour un modele statistique specifiant la proportionnalite entre deux distributions de wishart independantes

optimal shrinkage of eigenvalues in the spiked covariance model we show that in a common high dimensional covariance model the choice of loss function has a profound effect on optimal estimation in an asymptotic framework based on the spiked covariance model and use of orthogonally invariant estimators we show that optimal estimation of the population covariance matrix boils down to design of an optimal shrinker that acts elementwise on the sample eigenvalues indeed to each loss function there corresponds a unique admissible eigenvalue shrinker dominating all other shrinkers the shape of the optimal shrinker is determined by the choice of loss function and crucially by inconsistency of both eigenvalues and eigenvectors of the sample covariance matrix details of these phenomena and closed form formulas for the optimal eigenvalue shrinkers are worked out for a menagerie of 26 loss functions for covariance estimation found in the literature including the stein entropy divergence frechet bhattacharya matusita frobenius norm operator norm nuclear norm and condition number losses

information rates of nonparametric gaussian process methods we consider the quality of learning a response function by a nonparametric bayesian approach using a gaussian process gp prior on the response function we upper bound the quadratic risk of the learning procedure which in turn is an upper bound on the kullback leibler information between the predictive and true data distribution the upper bound is expressed in small ball probabilities and concentration measures of the gp prior we illustrate the computation of the upper bound for the matern and squared exponential kernels for these priors the risk and hence the information criterion tends to zero for all continuous response functions however the rate at which this happens depends on the combination of true response function and gaussian prior and is expressible in a certain concentration function in particular the results show that for good performance the regularity of the gp prior should match the regularity of the unknown response function

biometrika centenary theory and general methodology note reprinted in biometrika one hundred years edited by d m titterington and d r cox oxford university press 11 50 reference stat article 2001 002doi 10 1093 biomet 88 1 13view record in web of science record created on 2006 04 21 modified on 2017 05 12

partial correlation estimation by joint sparse regression models this article features online supplementary material in this article we propose a computationally efficient approach space sparse partial correlation estimation for selecting nonzero partial correlations under the high dimension low sample size setting this method assumes the overall sparsity of the partial correlation matrix and employs sparse regression techniques for model fitting we illustrate the performance of space by extensive simulation studies it is shown that space performs well in both nonzero partial correlation selection and the identification of hub variables and also outperforms two existing methods we then apply space to a microarray breast cancer dataset and identify a set of hub genes that may provide important insights on genetic regulatory networks finally we prove that under a set of suitable assumptions the proposed procedure is asymptotically consistent in terms of model selection and parameter estimation

maximum likelihood estimation for stochastic differential equations using sequential gaussian process based optimization abstractstochastic differential equations sdes are used as statistical models in many disciplines however intractable likelihood functions for sdes make inference challenging and we need to resort to simulation based techniques to estimate and maximize the likelihood function while importance sampling methods have allowed for the accurate evaluation of likelihoods at fixed parameter values there is still a question of how to find the maximum likelihood estimate in this article we propose an efficient gaussian process based method for exploring the parameter space using estimates of the likelihood from an importance sampler our technique accounts for the inherent monte carlo variability of the estimated likelihood and does not require knowledge of gradients the procedure adds potential parameter values by maximizing the so called expected improvement leveraging the fact that the likelihood function is assumed to be smooth our simulations demonstrate that our method has significant computation

wishart distributions for decomposable graphs when considering a graphical gaussian model n g markov with respect to a decomposable graph g the parameter space of interest for the precision parameter is the cone p g of positive definite matrices with fixed zeros corresponding to the missing edges of g the parameter space for the scale parameter of n g is the cone q g dual to p g of incomplete matrices with submatrices corresponding to the cliques of g being positive definite in this paper we construct on the cones q g and p g two families of wishart distributions namely the type i and type ii wisharts they can be viewed as generalizations of the hyper wishart and the inverse of the hyper inverse wishart as defined by dawid and lauritzen ann statist 21 1993 1272 1317 we show that the type i and ii wisharts have properties similar to those of the hyper and hyper inverse wishart indeed the inverse of the type ii wishart forms a conjugate family of priors for the covariance parameter of the graphical gaussian model and is strong directed hyper markov for every direction given to the graph by a perfect order of its cliques while the type i wishart is weak hyper markov moreover the inverse type ii wishart as a conjugate family presents the advantage of having a multidimensional shape parameter thus offering flexibility for the choice of a prior both type i and ii wishart distributions depend on multivariate shape parameters a shape parameter is acceptable if and only if it satisfies a certain eigenvalue property we show that the sets of acceptable shape parameters for a noncomplete g have dimension equal to at least one plus the number of cliques in g these families as conjugate families are richer than the traditional diaconis ylvisaker conjugate families which all have a shape parameter set of dimension one a decomposable graph which does not contain a three link chain as an induced subgraph is said to be homogeneous in this case our wisharts are particular cases of the wisharts on homogeneous cones as defined by andersson and wojnar j theoret probab 17 2004 781 818 and the dimension of the shape parameter set is even larger than in the nonhomogeneous case it is indeed equal to the number of cliques plus the number of distinct minimal separators using the model where g is a three link chain we show by computing a 7 tuple integral that in general we cannot expect the shape parameter sets to have dimension larger than the number of cliques plus one

sparse envelope model efficient estimation and response variable selection in multivariate linear regression the envelope model allows efficient estimation in multivariate linear regression in this paper we propose the sparse envelope model which is motivated by applications where some response variables are invariant with respect to changes of the predictors and have zero regression coefficients the envelope estimator is consistent but not sparse and in many situations it is important to identify the response variables for which the regression coefficients are zero the sparse envelope model performs variable selection on the responses and preserves the efficiency gains offered by the envelope model response variable selection arises naturally in many applications but has not been studied as thoroughly as predictor variable selection in this paper we discuss response variable selection in both the standard multivariate linear regression and the envelope contexts in response variable selection even if a response has zero coefficients it should still be retained to improve the estimation efficiency of the nonzero coefficients this is different from the practice in predictor variable selection we establish consistency and the oracle property and obtain the asymptotic distribution of the sparse envelope estimator

entropy search for information efficient global optimization contemporary global optimization algorithms are based on local measures of utility rather than a probability measure over location and value of the optimum they thus attempt to collect low function values not to learn about the optimum the reason for the absence of probabilistic global optimizers is that the corresponding inference problem is intractable in several ways this paper develops desiderata for probabilistic optimization algorithms then presents a concrete algorithm which addresses each of the computational intractabilities with a sequence of approximations and explicitly addresses the decision problem of maximizing information gain from each evaluation

control functionals for monte carlo integration summary a non parametric extension of control variates is presented these leverage gradient information on the sampling density to achieve substantial variance reduction it is not required that the sampling density be normalized the novel contribution of this work is based on two important insights a trade off between random sampling and deterministic approximation and a new gradient based function space derived from stein s identity unlike classical control variates our estimators improve rates of convergence often requiring orders of magnitude fewer simulations to achieve a fixed level of precision theoretical and empirical results are presented the latter focusing on integration problems arising in hierarchical models and models based on non linear ordinary differential equations

large scale correlation screening this article addresses the problem of screening for variables with high correlations in high dimensional data in which there can be many fewer samples than variables we focus on threshold based correlation screening methods for three related applications screening for variables with large correlations within a single treatment autocorrelation screening screening for variables with large cross correlations over two treatments cross correlation screening and screening for variables that have persistently large autocorrelations over two treatments persistent correlation screening the novelty of correlation screening is that it identifies a smaller number of variables that are highly correlated with others compared with identifying a number of correlation parameters correlation screening suffers from a phase transition phenomenon as the correlation threshold decreases the number of discoveries increases abruptly we obtain asymptotic expressions for the mean number of discoveries and the phase t

on the existence of sobolev quasi periodic solutions of multidimensional nonlinear beam equation in this paper we prove the existence of quasi periodic solutions of the multidimensional nonlinear beam equation with finitely differentiable nonlinearities and quasi periodic forcing in time

globally convergent algorithms for maximizing a likelihood function summary in this paper we show global convergence under very general assumptions of iterative maximization procedures with cyclic fixing of groups of parameters maximizing over the remaining parameters further we show that a slightly modified newton procedure applied to the derivative of the reciprocal likelihood function in a one dimensional exponential family is globally convergent we also prove that if the distribution of the sufficient statistic is infinitely divisible then the newton method applied to the derivative of the log likelihood function converges by combining these results we obtain general globally convergent iterative procedures for maximizing the likelihood function in any regular k dimensional exponential family the likelihood function can be maximized either by successively maximizing over one parameter at a time or by maximizing in the newton direction

on the condition number anomaly of gaussian correlation matrices abstract spatial correlation matrices appear in a large variety of applications for example they are an essential component of spatial gaussian processes also known as spatial linear models or kriging estimators which are powerful and well established tools for a multitude of engineering applications such as the design and analysis of computer experiments geostatistical problems and meteorological tasks in radial basis function interpolation gaussian correlation matrices arise frequently as interpolation matrices from the gaussian radial kernel function in the field of data assimilation in numerical weather prediction such matrices arise as background error covariances over the past thirty years it was observed by several authors from several fields that the gaussian correlation model is exceptionally prone to suffer from ill conditioning but a quantitative theoretical explanation for this anomaly was lacking in this paper a proof for the special position of the gaussian correlation matrix is given the theoretical findings are illustrated by numerical experiment

lower dimensional invariant tori with prescribed frequency for the nonlinear schrodinger equation abstract in this paper one dimensional 1d nonlinear schrodinger equation i u t u x x m u u 2 u f u 2 u 0 subject to dirichlet boundary conditions is considered where the nonlinearity f is a real analytic function near u 0 with f 0 f 0 0 it is proved that for each given constant potential m and each prescribed integer b 1 the above equation admits a whitney smooth family of small amplitude time quasi periodic solutions whose b dimensional frequencies are just small dilation of a prescribed diophantine vector accordingly we obtain the existence of lower dimensional invariant kam tori with tangential frequencies constrained to a given diophantine direction in an infinite dimensional phase space setting

gaussian markov distributions over finite graphs on s interesse aux modeles de selection de covariance introduites par dempster 1972 et etudies par wermuth 1976

convergence rates of posterior distributions we consider the asymptotic behavior of posterior distributions and bayes estimators for infinite dimensional statistical models we give general results on the rate of convergence of the posterior measure these are applied to several examples including priors on finite sieves log spline models dirichlet processes and interval censoring

lower dimensional invariant tori with prescribed frequency for nonlinear wave equation abstract in this paper one dimensional 1d nonlinear wave equation u t t u x x m u u 3 0 subject to dirichlet boundary conditions is considered we show that for each given m 0 and each prescribed integer b 1 the above equation admits a whitney smooth family of small amplitude quasi periodic solutions with b dimensional diophantine frequencies which correspond to b dimensional invariant tori of an associated infinite dimensional dynamical system in particular these diophantine frequencies are the small dilation of a prescribed diophantine vector the proof is based on a partial birkhoff normal form reduction and an improved kam method

optimality of poisson processes intensity learning with gaussian processes in this paper we provide theoretical support for the so called sigmoidal gaussian cox process approach to learning the intensity of an inhomogeneous poisson process on a d dimensional domain this method was proposed by adams murray and mackay icml 2009 who developed a tractable computational approach and showed in simulation and real data experiments that it can work quite satisfactorily the results presented in the present paper provide theoretical underpinning of the method in particular we show how to tune the priors on the hyper parameters of the model in order for the procedure to automatically adapt to the degree of smoothness of the unknown intensity and to achieve optimal convergence rates

model selection through sparse maximum likelihood estimation for multivariate gaussian or binary data we consider the problem of estimating the parameters of a gaussian or binary distribution in such a way that the resulting undirected graphical model is sparse our approach is to solve a maximum likelihood problem with an added l1 norm penalty term the problem as formulated is convex but the memory requirements and complexity of existing interior point methods are prohibitive for problems with more than tens of nodes we present two new algorithms for solving problems with at least a thousand nodes in the gaussian case our first algorithm uses block coordinate descent and can be interpreted as recursive l1 norm penalized regression our second algorithm based on nesterov s first order method yields a complexity estimate with a better dependence on problem size than existing interior point methods using a log determinant relaxation of the log partition function wainwright and jordan 2006 we show that these same algorithms can be used to solve an approximate sparse maximum likelihood problem for the binary case we test our algorithms on synthetic data as well as on gene expression and senate voting records data