bayesian sparse multiple regression for simultaneous rank reduction and variable selection we develop a bayesian methodology aimed at simultaneously estimating low rank and row sparse matrices in a high dimensional multiple response linear regression model we consider a carefully devised shrinkage prior on the matrix of regression coefficients which obviates the need to specify a prior on the rank and shrinks the regression matrix towards low rank and row sparse structures we provide theoretical support to the proposed methodology by proving minimax optimality of the posterior mean under the prediction risk in ultra high dimensional settings where the number of predictors can grow sub exponentially relative to the sample size a one step post processing scheme induced by group lasso penalties on the rows of the estimated coefficient matrix is proposed for variable selection with default choices of tuning parameters we additionally provide an estimate of the rank using a novel optimization function achieving dimension reduction in the covariate space we exhibit the performance of the proposed methodology in an extensive simulation study and a real data example

multiple output regression with latent noise in high dimensional data structured noise caused by observed and unobserved factors affecting multiple target variables simultaneously imposes a serious challenge for modeling by masking the often weak signal therefore 1 explaining away the structured noise in multiple output regression is of paramount importance additionally 2 assumptions about the correlation structure of the regression weights are needed we note that both can be formulated in a natural way in a latent variable model in which both the interesting signal and the noise are mediated through the same latent factors under this assumption the signal model then borrows strength from the noise model by encouraging similar effects on correlated targets we introduce a hyperparameter for the latent signal to noise ratio which turns out to be important for modelling weak signals and an ordered infinite dimensional shrinkage prior that resolves the rotational unidentifiability in reduced rank regression models simulations and prediction experiments with metabolite gene expression fmri measurement and macroeconomic time series data show that our model equals or exceeds the state of the art performance and in particular outperforms the standard approach of assuming independent noise and signal models

bayesian factorizations of big sparse tensors it has become routine to collect data that are structured as multiway arrays tensors there is an enormous literature on low rank and sparse matrix factorizations but limited consideration of extensions to the tensor case in statistics the most common low rank tensor factorization relies on parallel factor analysis parafac which expresses a rank k tensor as a sum of rank one tensors in contingency table applications in which the sample size is massively less than the number of cells in the table the low rank assumption is not sufficient and parafac has poor performance we induce an additional layer of dimension reduction by allowing the effective rank to vary across dimensions of the table taking a bayesian approach we place priors on terms in the factorization and develop an efficient gibbs sampler for posterior computation theory is provided showing posterior concentration rates in high dimensional settings and the methods are shown to have excellent performance in simulations and several

model order reduction with oblique projections for large scale wave propagation there are many applications in which it is necessary to solve large scale parametrized wave propagation problems repeatedly this is still quite a challenging task even with the largest available computer clusters in this paper we will discuss the application of model order reduction mor to problems in seismic petroleum exploration with the aim of diminishing the necessary computing time by a significant factor we consider pod and some variants pod is a model order reduction technique that uses snapshots of a few simulations in order to quickly compute related problems with similar accuracy the method of lines via a petrov galerkin approximation that uses the snapshots as basis functions is the considered approach the order reduction comes from projecting the wave equation discretized in space to the subspace spanned by the snapshots this has been shown earlier to work well in two dimensions the challenge in three dimensions comes from the size of the spatial meshes required and the fact that the method usually requires a number of snapshots that do not fit in fast memory even for current high end multicore machines parallelization is not an option since it is already used for other aspects of this massive problem

nonparametric bayes modeling of populations of networks abstractreplicated network data are increasingly available in many research fields for example in connectomic applications interconnections among brain regions are collected for each patient under study motivating statistical models which can flexibly characterize the probabilistic generative mechanism underlying these network valued data available models for a single network are not designed specifically for inference on the entire probability mass function of a network valued random variable and therefore lack flexibility in characterizing the distribution of relevant topological structures we propose a flexible bayesian nonparametric approach for modeling the population distribution of network valued data the joint distribution of the edges is defined via a mixture model that reduces dimensionality and efficiently incorporates network information within each mixture component by leveraging latent space representations the formulation leads to an efficient gibbs sampler and provides simple and c

bayesian gaussian copula factor models for mixed data gaussian factor models have proven widely useful for parsimoniously characterizing dependence in multivariate data there is rich literature on their extension to mixed categorical and continuous variables using latent gaussian variables or through generalized latent trait models accommodating measurements in the exponential family however when generalizing to non gaussian measured variables the latent variables typically influence both the dependence structure and the form of the marginal distributions complicating interpretation and introducing artifacts to address this problem we propose a novel class of bayesian gaussian copula factor models that decouple the latent factors from the marginal distributions a semiparametric specification for the marginals based on the extended rank likelihood yields straightforward implementation and substantial computational gains we provide new theoretical and empirical justifications for using this likelihood in bayesian inference we propose new default pri

classification via bayesian nonparametric learning of affine subspaces it has become common for datasets to contain large numbers of variables in studies conducted in areas such as genetics machine vision image analysis and many others when analyzing such data parametric models are often too inflexible while nonparametric procedures tend to be nonrobust because of insufficient data on these high dimensional spaces this is particularly true when interest lies in building efficient classifiers in the presence of many predictor variables when dealing with these types of data it is often the case that most of the variability tends to lie along a few directions or more generally along a much smaller dimensional submanifold of the data space in this article we propose a class of models that flexibly learn about this submanifold while simultaneously performing dimension reduction in classification this methodology allows the cell probabilities to vary nonparametrically based on a few coordinates expressed as linear combinations of the predictors also as opposed to man

bayesian estimation of sparse dynamic factor models with order independent and ex post mode identification abstract common variation in n series is extracted into k n dynamic factors we induce sparsity by using a zero point mass normal mixture prior distribution on the loadings estimation and rotational identification are independent of variable ordering sparsity helps identifying the factor space and the factors rotational identification including factor order and sign is obtained by processing the posterior output and based on factor draws rather than factor loading draws simulating data we document sampler and estimation efficiency to illustrate we estimate the model for a large panel of swiss macroeconomic and detailed price data we identify 16 factors with a clear economic interpretation

regularized estimation of large covariance matrices this paper considers estimating a covariance matrix of p variables from n observations by either banding the sample covariance matrix or estimating a banded version of the inverse of the covariance we show that these estimates are consistent in the operator norm as long as logp 2 n 0 and obtain explicit rates the results are uniform over some fairly natural well conditioned families of covariance matrices we also introduce an analogue of the gaussian white noise model and show that if the population covariance is embeddable in that model and well conditioned then the banded approximations produce consistent estimates of the eigenvalues and associated eigenvectors of the covariance matrix the results can be extended to smooth versions of banding and to non gaussian distributions with su ciently short tails a resampling approach is proposed for choosing the banding parameter in practice this approach is illustrated numerically on both simulated and real data

determining the number of factors in approximate factor models in this paper we develop some econometric theory for factor models of large dimensions the focus is the determination of the number of factors r which is an unresolved issue in the rapidly growing literature on multifactor models we first establish the convergence rate for the factor estimates that will allow for consistent estimation of r we then propose some panel criteria and show that the number of factors can be consistently estimated using the criteria the theory is developed under the framework of large cross sections n and large time dimensions t no restriction is imposed on the relation between n and t simulations show that the proposed criteria have good finite sample properties in many configurations of the panel data encountered in practice

updating the inverse of a matrix the sherman morrison woodbury formulas relate the inverse of a matrix after a small rank perturbation to the inverse of the original matrix the history of these fomulas is presented and various applications to statistics networks structural analysis asymptotic analysis optimization and partial differential equations are discussed the sherman morrison woodbury formulas express the inverse of a matrix after a small rank perturbation in terms of the inverse of the original matrix this paper surveys the history of these formulas and we examine some applications where these formulas are helpful

fast sampling with gaussian scale mixture priors in high dimensional regression we propose an efficient way to sample from a class of structured multivariate gaussian distributions the proposed algorithm only requires matrix multiplications and linear system solutions its computational complexity grows linearly with the dimension unlike existing algorithms that rely on cholesky factorizations with cubic complexity the algorithm is broadly applicable in settings where gaussian scale mixture priors are used on high dimensional parameters its effectiveness is illustrated through a high dimensional regression problem with a horseshoe prior on the regression coefficients other potential applications are outlined

bayesian nonparametric covariance regression capturing predictor dependent correlations amongst the elements of a multivariate response vector is fundamental to numerous applied domains including neuroscience epidemiology and finance although there is a rich literature on methods for allowing the variance in a univariate regression model to vary with predictors relatively little has been done in the multivariate case as a motivating example we consider the google flu trends data set which provides indirect measurements of influenza incidence at a large set of locations over time our predictor to accurately characterize temporally evolving influenza incidence across regions it is important to develop statistical methods for a time varying covariance matrix importantly the locations provide a redundant set of measurements and do not yield a sparse nor static spatial dependence structure we propose to reduce dimensionality and induce a flexible bayesian nonparametric covariance regression model by relating these location specific trajectories to a lower dimensional subspace through a latent factor model with predictor dependent factor loadings these loadings are in terms of a collection of basis functions that vary nonparametrically over the predictor space such low rank approximations are in contrast to sparse precision assumptions and are appropriate in a wide range of applications our formulation aims to address three challenges scaling to large p domains coping with missing values and allowing an irregular grid of observations the model is shown to be highly exible while leading to a computationally feasible implementation via gibbs sampling the ability to scale to large p domains and cope with missing values is fundamental in analyzing the google flu trends data

bayesian nonparametric ordination for the analysis of microbial communities abstracthuman microbiome studies use sequencing technologies to measure the abundance of bacterial species or operational taxonomic units otus in samples of biological material typically the data are organized in contingency tables with otu counts across heterogeneous biological samples in the microbial ecology community ordination methods are frequently used to investigate latent factors or clusters that capture and describe variations of otu counts across biological samples it remains important to evaluate how uncertainty in estimates of each biological sample s microbial distribution propagates to ordination analyses including visualization of clusters and projections of biological samples on low dimensional spaces we propose a bayesian analysis for dependent distributions to endow frequently used ordinations with estimates of uncertainty a bayesian nonparametric prior for dependent normalized random measures is constructed which is marginally equivalent to the normalized generalized gamma pr

a dynamical system with integer information dimension and fractal correlation exponent in this paper we construct a family t 0 1 2 of exact endomorphisms of 0 1 such that the invariant measurem oft is equivalent to lebesgue measure but has fractal correlation exponent 2 this shows that an almost complete dichotomy can exist between the information dimension and the correlation exponent in observable dynamical systems

a bayesian approach to nonlinear latent variable models using the gibbs sampler and the metropolis hastings algorithm nonlinear latent variable models are specified that include quadratic forms and interactions of latent regressor variables as special cases to estimate the parameters the models are put in a bayesian framework with conjugate priors for the parameters the posterior distributions of the parameters and the latent variables are estimated using markov chain monte carlo methods such as the gibbs sampler and the metropolis hastings algorithm the proposed estimation methods are illustrated by two simulation studies and by the estimation of a non linear model for the dependence of performance on task complexity and goal specificity using empirical data

locally adaptive factor processes for multivariate time series in modeling multivariate time series it is important to allow time varying smoothness in the mean and covariance process in particular there may be certain time intervals exhibiting rapid changes and others in which changes are slow if such time varying smoothness is not accounted for one can obtain misleading inferences and predictions with oversmoothing across erratic time intervals and under smoothing across times exhibiting slow variation this can lead to mis calibration of predictive intervals which can be substantially too narrow or wide depending on the time we propose a locally adaptive factor process for characterizing multivariate mean covariance changes in continuous time allowing locally varying smoothness in both the mean and covariance matrix this process is constructed utilizing latent dictionary functions evolving in time through nested gaussian processes and linearly related to the observed data with a sparse mapping using a differential equation representation we bypass usual computational bottlenecks in obtaining mcmc and online algorithms for approximate bayesian inference the performance is assessed in simulations and illustrated in a financial application

semiparametric bayesian local functional models for diffusion tensor tract statistics we propose a semiparametric bayesian local functional model bfm for the analysis of multiple diffusion properties e g fractional anisotropy along white matter fiber bundles with a set of covariates of interest such as age and gender bfm accounts for heterogeneity in the shape of the fiber bundle diffusion properties among subjects while allowing the impact of the covariates to vary across subjects a nonparametric bayesian lpp2 prior facilitates global and local borrowings of information among subjects while an infinite factor model flexibly represents low dimensional structure local hypothesis testing and credible bands are developed to identify fiber segments along which multiple diffusion properties are significantly associated with covariates of interest while controlling for multiple comparisons moreover bfm naturally group subjects into more homogeneous clusters posterior computation proceeds via an efficient markov chain monte carlo algorithm a simulation study is performed to evaluate the finite sample performance of bfm we apply bfm to investigate the development of white matter diffusivities along the splenium of the corpus callosum tract and the right internal capsule tract in a clinical study of neurodevelopment in new born infants

parameter expansion for data augmentation abstract viewing the observed data of a statistical model as incomplete and augmenting its missing parts are useful for clarifying concepts and central to the invention of two well known statistical algorithms expectation maximization em and data augmentation recently liu rubin and wu demonstrated that expanding the parameter space along with augmenting the missing data is useful for accelerating iterative computation in an em algorithm the main purpose of this article is to rigorously define a parameter expanded data augmentation px da algorithm and to study its theoretical properties the px da is a special way of using auxiliary variables to accelerate gibbs sampling algorithms and is closely related to reparameterization techniques we obtain theoretical results concerning the convergence rate of the px da algorithm and the choice of prior for the expansion parameter to understand the role of the expansion parameter we establish a new theory for iterative conditional sampling under the tra

bayesian nonparametric vector autoregressive models abstract vector autoregressive var models are the main work horse models for macroeconomic forecasting and provide a framework for the analysis of complex dynamics that are present between macroeconomic variables whether a classical or a bayesian approach is adopted most var models are linear with gaussian innovations this can limit the model s ability to explain the relationships in macroeconomic series we propose a nonparametric var model that allows for nonlinearity in the conditional mean heteroscedasticity in the conditional variance and non gaussian innovations our approach differs from that of previous studies by modelling the stationary and transition densities using bayesian nonparametric methods our bayesian nonparametric var bayesnp var model is applied to us and uk macroeconomic time series and compared to other bayesian var models we show that bayesnp var is a flexible model that is able to account for nonlinear relationships as well as heteroscedasticity in the data in terms of short run out of sample forecasts we show that bayesnp var predictively outperforms competing models

sparse bayesian time varying covariance estimation in many dimensions dynamic covariance estimation for multivariate time series suffers from the curse of dimensionality this renders parsimonious estimation methods essential for conducting reliable statistical inference in this paper the issue is addressed by modeling the underlying co volatility dynamics of a time series vector through a lower dimensional collection of latent time varying stochastic factors furthermore we apply a normal gamma prior to the elements of the factor loadings matrix this hierarchical shrinkage prior effectively pulls the factor loadings of unimportant factors towards zero thereby increasing parsimony even more we apply the model to simulated data as well as daily log returns of 300 sp it is implemented in the r package factorstochvol author s abstract

bayesian exploratory factor analysis this paper develops and applies a bayesian approach to exploratory factor analysis that improves on ad hoc classical approaches our framework relies on dedicated factor models and simultaneously determines the number of factors the allocation of each measurement to a unique factor and the corresponding factor loadings classical identification criteria are applied and integrated into our bayesian procedure to generate models that are stable and clearly interpretable a monte carlo study confirms the validity of the approach the method is used to produce interpretable low dimensional aggregates from a high dimensional set of psychological measurements

high dimensional sparse factor modeling applications in gene expression genomics we describe studies in molecular profiling and biological pathway analysis that use sparse latent factor and regression models for microarray gene expression data we discuss breast cancer applications and key aspects of the modeling and computational methodology our case studies aim to investigate and characterize heterogeneity of structure related to specific oncogenic pathways as well as links between aggregate patterns in gene expression profiles and clinical biomarkers based on the metaphor of statistically derived factors as representing biological subpathway structure we explore the decomposition of fitted sparse factor models into pathway subcomponents and investigate how these components overlay multiple aspects of known biological activity our methodology is based on sparsity modeling of multivariate regression anova and latent factor models as well as a class of models that combines all components hierarchical sparsity priors address questions of dimension reduction and multiple co

expandable factor analysis bayesian sparse factor models have proven useful for characterizing dependence in multivariate data but scaling computation to large numbers of samples and dimensions is problematic we propose expandable factor analysis for scalable inference in factor models when the number of factors is unknown the method relies on a continuous shrinkage prior for efficient maximum a posteriori estimation of a low rank and sparse loadings matrix the structure of the prior leads to an estimation algorithm that accommodates uncertainty in the number of factors we propose an information criterion to select the hyperparameters of the prior expandable factor analysis has better false discovery rates and true positive rates than its competitors across diverse simulation settings we apply the proposed approach to a gene expression study of ageing in mice demonstrating superior results relative to four competing methods

partial factor modeling predictor dependent shrinkage for linear regression we develop a modified gaussian factor model for the purpose of inducing predictor dependent shrinkage for linear regression the new model predicts well across a wide range of covariance structures on real and simulated data furthermore the new model facilitates variable selection in the case of correlated predictor variables which often stymies other methods

fully nonparametric regression for bounded data using dependent bernstein polynomials abstractwe propose a novel class of probability models for sets of predictor dependent probability distributions with bounded domain the proposal extends the dirichlet bernstein prior for single density estimation by using dependent stick breaking processes a general model class and two simplified versions are discussed in detail appealing theoretical properties such as continuity association structure marginal distribution large support and consistency of the posterior distribution are established for all models the behavior of the models is illustrated using simulated and real life data the simulated data are also used to compare the proposed methodology to existing methods supplementary materials for this article are available online

bayesian group factor analysis with structured sparsity latent factor models are the canonical statistical tool for exploratory analyses of low dimensional linear structure for a matrix of p features across n samples we develop a structured bayesian group factor analysis model that extends the factor model to multiple coupled observation matrices in the case of two observations this reduces to a bayesian model of canonical correlation analysis here we carefully define a structured bayesian prior that encourages both element wise and column wise shrinkage and leads to desirable behavior on high dimensional data in particular our model puts a structured prior on the joint factor loading matrix regularizing at three levels which enables element wise sparsity and unsupervised recovery of latent factors corresponding to structured variance across arbitrary subsets of the observations in addition our structured prior allows for both dense and sparse latent factors so that covariation among either all features or only a subset of features can be recovered we use fast parameter expanded expectation maximization for parameter estimation in this model we validate our method on simulated data with substantial structure we show results of our method applied to three high dimensional data sets comparing results against a number of state of the art approaches these results illustrate useful properties of our model including i recovering sparse signal in the presence of dense effects ii the ability to scale naturally to large numbers of observations iii flexible observation and factor specific regularization to recover factors with a wide variety of sparsity levels and percentage of variance explained and iv tractable inference that scales to modern genomic and text data sizes

sparse bayesian infinite factor models we focus on sparse modelling of high dimensional covariance matrices using bayesian latent factor models we propose a multiplicative gamma process shrinkage prior on the factor loadings which allows introduction of infinitely many factors with the loadings increasingly shrunk towards zero as the column index increases we use our prior on a parameter expanded loading matrix to avoid the order dependence typical in factor analysis models and develop an efficient gibbs sampler that scales well as data dimensionality increases the gain in efficiency is achieved by the joint conjugacy property of the proposed prior which allows block updating of the loadings matrix we propose an adaptive gibbs sampler for automatically truncating the infinite loading matrix through selection of the number of important factors theoretical results are provided on the support of the prior and truncation approximation bounds a fast algorithm is proposed to produce approximate bayes estimates latent factor regression methods are developed for prediction and variable selection in applications with high dimensional correlated predictors operating characteristics are assessed through simulation studies and the approach is applied to predict survival times from gene expression data

bayesian latent factor regression for functional and longitudinal data in studies involving functional data it is commonly of interest to model the impact of predictors on the distribution of the curves allowing flexible effects on not only the mean curve but also the distribution about the mean characterizing the curve for each subject as a linear combination of a high dimensional set of potential basis functions we place a sparse latent factor regression model on the basis coefficients we induce basis selection by choosing a shrinkage prior that allows many of the loadings to be close to zero the number of latent factors is treated as unknown through a highly efficient adaptive blocked gibbs sampler predictors are included on the latent variables level while allowing different predictors to impact different latent factors this model induces a framework for functional response regression in which the distribution of the curves is allowed to change flexibly with predictors the performance is assessed through simulation studies and the methods are applied to data on blood pressure trajectories during pregnancy

generalized reduced rank latent factor regression for high dimensional tensor fields and neuroimaging genetic applications we propose a generalized reduced rank latent factor regression model grrlf for the analysis of tensor field responses and high dimensional covariates the model is motivated by the need from imaging genetic studies to identify genetic variants that are associated with brain imaging phenotypes often in the form of high dimensional tensor fields grrlf identifies from the structure in the data the effective dimensionality of the data and then jointly performs dimension reduction of the covariates dynamic identification of latent factors and nonparametric estimation of both covariate and latent response fields after accounting for the latent and covariate effects grllf performs a nonparametric test on the remaining factor of interest grrlf provides a better factorization of the signals compared with common solutions and is less susceptible to overfitting because it exploits the effective dimensionality the generality and the flexibility of grrlf also allow various statistical models to be handled in a unified framework and solutions can be efficiently computed within the field of neuroimaging it improves the sensitivity for weak signals and is a promising alternative to existing approaches the operation of the framework is demonstrated with both synthetic datasets and a real world neuroimaging example in which the effects of a set of genes on the structure of the brain at the voxel level were measured and the results compared favorably with those from existing approaches

fast bayesian factor analysis via automatic rotations to sparsity abstractrotational post hoc transformations have traditionally played a key role in enhancing the interpretability of factor analysis regularization methods also serve to achieve this goal by prioritizing sparse loading matrices in this work we bridge these two paradigms with a unifying bayesian framework our approach deploys intermediate factor rotations throughout the learning process greatly enhancing the effectiveness of sparsity inducing priors these automatic rotations to sparsity are embedded within a pxl em algorithm a bayesian variant of parameter expanded em for posterior mode detection by iterating between soft thresholding of small factor loadings and transformations of the factor basis we obtain a dramatic accelerations b robustness against poor initializations and c better oriented sparse solutions to avoid the prespecification of the factor cardinality we extend the loading matrix to have infinitely many columns with the indian buffet process ibp prior the factor dimen