variable selection for gaussian process models in computer experiments in many situations simulation of complex phenomena requires a large number of inputs and is computationally expensive identifying the inputs that most impact the system so that these factors can be further investigated can be a critical step in the scientific endeavor in computer experiments it is common to use a gaussian spatial process to model the output of the simulator in this article we introduce a new simple method for identifying active factors in computer screening experiments the approach is bayesian and only requires the generation of a new inert variable in the analysis however in the spirit of frequentist hypothesis testing the posterior distribution of the inert factor is used as a reference distribution against which the importance of the experimental factors can be assessed the methodology is demonstrated on an application in material science a computer experiment from the literature and simulated examples

variable selection in bayesian smoothing spline anova models a bayesian nonparametric regression model for curve fitting and variable selection is proposed for analyzing complex computer model output the smoothing splines anova framework is used to separate the regression function into main effects and interacti

component selection and smoothing in multivariate nonparametric regression we propose a new method for model selection and model fitting in multivariate nonparametric regression models in the framework of smoothing spline anova the cosso is a method of regularization with the penalty functional being the sum of component norms instead of the squared norm employed in the traditional smoothing spline method the cosso provides a unified framework for several recent proposals for model selection in linear models and smoothing spline anova models theoretical properties such as the existence and the rate of convergence of the cosso estimator are studied in the special case of a tensor product design with periodic functions a detailed analysis reveals that the cosso does model selection by applying a novel soft thresholding type operation to the function components we give an equivalent formulation of the cosso estimator which leads naturally to an iterative algorithm we compare the cosso with mars a popular method that builds functional anova models in simulations and real examples the cosso method can be extended to classification problems and we compare its performance with those of a number of machine learning algorithms on real datasets the cosso gives very competitive performance in these studies

correlated random measures abstractwe develop correlated random measures random measures where the atom weights can exhibit a flexible pattern of dependence and use them to develop powerful hierarchical bayesian nonparametric models hierarchical bayesian nonparametric models are usually built from completely random measures a poisson process based construction in which the atom weights are independent completely random measures imply strong independence assumptions in the corresponding hierarchical model and these assumptions are often misplaced in real world settings correlated random measures address this limitation they model correlation within the measure by using a gaussian process in concert with the poisson process with correlated random measures for example we can develop a latent feature model for which we can infer both the properties of the latent features and their dependency pattern we develop several other examples as well we study a correlated random measure model of pairwise count data we derive an eff

multiple testing with heterogeneous multinomial distributions summary standard false discovery rate fdr procedures can provide misleading inference when testing multiple null hypotheses with heterogeneous multinomial data for example in the motivating study the goal is to identify species of bacteria near the roots of wheat plants rhizobacteria that are moderately or strongly associated with productivity however standard procedures discover the most abundant species even when their association is weak and fail to discover many moderate and strong associations when the species are not abundant this article provides a new fdr controlling method based on a finite mixture of multinomial distributions and shows that it tends to discover more moderate and strong associations and fewer weak associations when the data are heterogeneous across tests the new method is applied to the rhizobacteria data and performs favorably over competing methods

ridge regression biased estimation for nonorthogonal problems in multiple regression it is shown that parameter estimates based on minimum residual sum of squares have a high probability of being unsatisfactory if not incorrect if the prediction vectors are not orthogonal proposed is an estimation procedure based on adding small positive quantities to the diagonal of x x introduced is the ridge trace a method for showing in two dimensions the effects of nonorthogonality it is then shown how to augment x x to obtain biased estimates with smaller mean square error

bayesian variable selection with shrinking and diffusing priors we consider a bayesian approach to variable selection in the presence of high dimensional covariates based on a hierarchical model that places prior distributions on the regression coefficients as well as on the model space we adopt the well known spike and slab gaussian priors with a distinct feature that is the prior variances depend on the sample size through which appropriate shrinkage can be achieved we show the strong selection consistency of the proposed method in the sense that the posterior probability of the true model converges to one even when the number of covariates grows nearly exponentially with the sample size this is arguably the strongest selection consistency result that has been available in the bayesian variable selection literature yet the proposed method can be carried out through posterior sampling with a simple gibbs sampler furthermore we argue that the proposed method is asymptotically similar to model selection with the l 0 penalty we also demonstrate through empirical work the fine performance of the proposed approach relative to some state of the art alternatives

dynamic algorithms for shortest paths in planar graphs abstract we propose data structures for maintaining shortest paths in planar graphs in which the weight of an edge is modified our data structures allow us to compute after an update the shortest path tree rooted at an arbitrary query node in time o n log log n and to perform an update in o log n 3 our data structure can be applied also to the problem of maintaining the maximum flow problem in an s t planar network as far as the all pairs shortest path problem is concerned we are interested in computing the shortest distances between q pairs of nodes we show how to obtain an o n 2 algorithm for computing the shortest path between q pairs of nodes whenever q o n 2 we also consider the dynamic version of the problem in which we allow the modification of the weight of an edge

supervised bayesian latent class models for high dimensional data high grade gliomas are the most common primary brain tumors in adults and are typically diagnosed using histopathology however these diagnostic categories are highly heterogeneous and do not always correlate well with survival in an attempt to refine these diagnoses we make several immunohistochemical measurements of ykl 40 a gene previously shown to be differentially expressed between diagnostic groups we propose two latent class models for classification and variable selection in the presence of high dimensional binary data fit by using bayesian markov chain monte carlo techniques penalization and model selection are incorporated in this setting via prior distributions on the unknown parameters the methods provide valid parameter estimates under conditions in which standard supervised latent class models do not and outperform two stage approaches to variable selection and parameter estimation in a variety of settings we study the properties of these methods in simulations and apply these methodologies to the glioma study for which identifiable three class parameter estimates cannot be obtained without penalization with penalization the resulting latent classes correlate well with clinical tumor grade and offer additional information on survival prognosis that is not captured by clinical diagnosis alone the inclusion of ykl 40 features also increases the precision of survival estimates fitting models with and without ykl 40 highlights a subgroup of patients who have glioblastoma gbm diagnosis but appear to have better prognosis than the typical gbm patient

multivariate adaptive regression splines a new method is presented for flexible regression modeling of high dimensional data the model takes the form of an expansion in product spline basis functions where the number of basis functions as well as the parameters associated with each one product degree and knot locations are automatically determined by the data this procedure is motivated by the recursive partitioning approach to regression and shares its attractive properties unlike recursive partitioning however this method produces continuous models with continuous derivatives it has more power and flexibility to model relationships that are nearly additive or involve interactions in at most a few variables in addition the model can be represented in a form that separately identifies the additive contributions and those associated with the different multivariable interactions

on the distributional properties of model selection criteria abstract it is commonly accepted that statistical modeling should follow the parsimony principle namely that simple models should be given priority whenever possible but little quantitative knowledge is known concerning the amount of penalty for complexity regarded as allowable we try to understand the parsimony principle in the context of model selection in particular the generalized final prediction error criterion is considered and we argue that the penalty term should be chosen between 1 5 and 5 for most practical situations applying our results to the cross validation criterion we obtain insights into how the partition of data should be done we also discuss the small sample performance of our methods

prediction via orthogonalized model mixing abstract we introduce an approach and algorithms for model mixing in large prediction problems with correlated predictors we focus on the choice of predictors in linear models and mix over possible subsets of candidate predictors our approach is based on expressing the space of models in terms of an orthogonalization of the design matrix advantages are both statistical and computational statistically orthogonalization often leads to a reduction in the number of competing models by eliminating correlations computationally large model spaces cannot be enumerated recent approaches are based on sampling models with high posterior probability via markov chains based on orthogonalization of the space of candidate predictors we can approximate the posterior probabilities of models by products of predictor specific terms this leads to an importance sampling function for sampling directly from the joint distribution over the model space without resorting to markov chains compared to the latter ortho

whitney forms of higher degree low order whitney elements are widely used for electromagnetic field problems higher order approximations are receiving increasing interest but their definition remains unduly complex in this paper we propose a new simple construction for whitney p elements of polynomial degree higher than one that use only degrees of freedom associated to p chains we provide a basis for these elements on simplicial meshes and give a geometrical localization of all degrees of freedom properties of the higher order whitney complex are deeply investigated

an mdl framework for sparse coding and dictionary learning the power of sparse signal modeling with learned overcomplete dictionaries has been demonstrated in a variety of applications and fields from signal processing to statistical inference and machine learning however the statistical properties of these models such as underfitting or overfitting given sets of data are still not well characterized in the literature as a result the success of sparse modeling depends on hand tuning critical parameters for each data and application this work aims at addressing this by providing a practical and objective characterization of sparse models by means of the minimum description length mdl principle a well established information theoretic approach to model selection in statistical inference the resulting framework derives a family of efficient sparse coding and dictionary learning algorithms which by virtue of the mdl principle are completely parameter free furthermore such framework allows to incorporate additional prior information to existing models such as markovian dependencies or to define completely new problem formulations including in the matrix analysis area in a natural way these virtues will be demonstrated with parameter free algorithms for the classic image denoising and classification problems and for low rank matrix recovery in video applications however the framework is not limited to this imaging data and can be applied to a wide range of signal and data types and tasks

decoupling shrinkage and selection in bayesian linear models a posterior summary perspective selecting a subset of variables for linear models remains an active area of research this article reviews many of the recent contributions to the bayesian model selection and shrinkage prior literature a posterior variable selection summary is proposed which distills a full posterior distribution over regression coefficients into a sequence of sparse linear predictors

consistent variable selection in linear models abstract a method of estimating linear model dimension and variable selection is proposed this new criterion which generalizes the cp criterion the akaike information criterion aic the bayes information criterion and the phiv criterion and is consistent under certain conditions is based on a new class of penalty functions and a procedure of sorting covariates based on t statistics in the course of introducing this method we discuss the important role of the penalty function in the consistency of model dimension estimation and in variable selection the proposed method requires less computation than resampling based methods that search over all subsets of covariates for the true model simulation results show that the new method is superior to the cp criterion and aic in finite sample situations as well

data analytic tools for understanding random field regression models random field regression rfr models in which a response is treated as the realization of a random field have been advocated for modeling data from experiments in high signal to noise settings in particular rfr models have proven useful in analyzing data generated from computer simulations of complex processes they offer flexibility for smoothing these data and are able to interpolate the known values for factor settings tested on the simulator however these models lack the easy interpretability of standard regression estimators our purpose in this article is to demonstrate that there is actually much common ground between the rfr models and bayesian regression and to provide some simple data analytic tools that can help expose a regression model associated with an rfr model

bernstein bezier bases for tetrahedral finite elements abstract we present a new set of basis functions for h curl conforming h div conforming and l 2 conforming finite elements of arbitrary order on a tetrahedron the basis functions are expressed in terms of bernstein polynomials and augment the natural h 1 conforming bernstein basis the basis functions respect the differential operators namely the gradients of the high order h 1 conforming bernstein bezier basis functions form part of the h curl conforming basis and the curl of the high order non gradients h curl conforming basis functions form part of the h div conforming basis and the divergence of the high order non curl h div conforming basis functions form part of the l 2 conforming basis procedures are given for the efficient computation of the mass and stiffness matrices with these basis functions without using quadrature rules for piecewise constant coefficients on affine tetrahedra numerical results are presented to illustrate the use of the basis to approximate representative problems

small area estimation with uncertain random effects random effects models play an important role in model based small area estimation random effects account for any lack of fit of a regression model for the population means of small areas on a set of explanatory variables in a recent article datta hall and mandal showed that if the random effects can be dispensed with via a suitable test then the model parameters and the small area means may be estimated with substantially higher accuracy the work of datta hall and mandal is most useful when the number of small areas m is moderately large for large m the null hypothesis of no random effects will likely be rejected rejection of the null hypothesis is usually caused by a few large residuals signifying a departure of the direct estimator from the synthetic regression estimator as a flexible alternative to the fay herriot random effects model and the approach in datta hall and mandal in this article we consider a mixture model for random effects it is reasonably expected that small areas wit

the bayesian lasso the lasso estimate for linear regression parameters can be interpreted as a bayesian posterior mode estimate when the regression parameters have independent laplace i e double exponential priors gibbs sampling from this posterior is possible using an expanded hierarchy with conjugate normal priors for the regression parameters and independent exponential priors on their variances a connection with the inverse gaussian distribution provides tractable full conditional distributions the bayesian lasso provides interval estimates bayesian credible intervals that can guide variable selection moreover the structure of the hierarchical model provides both bayesian and likelihood methods for selecting the lasso parameter slight modifications lead to bayesian versions of other lasso related estimation methods including bridge regression and a robust variant

bayesian analysis of static and dynamic factor models an ex post approach towards the rotation problem due to their indeterminacies static and dynamic factor models require identifying assumptions to guarantee uniqueness of the parameter estimator the indeterminacy of the parameter estimator with respect to an orthogonal transformation is known as the rotation problem the typical strategy in bayesian factor analysis to solve the rotation problem is to introduce ex ante constraints on certain model parameters via degenerate and truncated prior distributions this strategy however results in posterior distributions whose shapes depend on the ordering of the variables in the data set we propose an alternative approach where the rotation problem is solved ex post using procrustean postprocessing the resulting order invariance of the posterior estimator is illustrated in a simulation study and an empirical application using an established data set containing 120 macroeconomic time series favorable properties of the ex post approach with respect to convergence statistical and numerical accuracy are revealed

better subset regression using the nonnegative garrote a new method called the nonnegative nn garrote is proposed for doing subset regression it both shrinks and zeroes coefficients in tests on real and simulated data it produces lower prediction error than ordinary subset selection it is also compared to ridge regression if the regression equations generated by a procedure do not change drastically with small changes in the data the procedure is called stable subset selection is unstable ridge is very stable and the nn garrote is intermediate simulation results illustrate the effects of instability on prediction error

the little bootstrap and other methods for dimensionality selection in regression x fixed prediction error abstract when a regression problem contains many predictor variables it is rarely wise to try to fit the data by means of a least squares regression on all of the predictor variables usually a regression equation based on a few variables will be more accurate and certainly simpler there are various methods for picking good subsets of variables and programs that do such procedures are part of every widely used statistical package the most common methods are based on stepwise addition or deletion of variables and on best subsets the latter refers to a search method that given the number of variables to be in the equation say five locates that regression equation based on five variables that has the lowest residual sum of squares among all five variable equations all of these procedures generate a sequence of regression equations the first based on one variable the next based on two variables and so on each member of this sequence is called a submodel and the number of variables in the e

emvs the em approach to bayesian variable selection despite rapid developments in stochastic search algorithms the practicality of bayesian variable selection methods has continued to pose challenges high dimensional data are now routinely analyzed typically with many more covariates than observations to broaden the applicability of bayesian variable selection for such high dimensional linear regression contexts we propose emvs a deterministic alternative to stochastic search based on an em algorithm which exploits a conjugate mixture prior formulation to quickly find posterior modes combining a spike and slab regularization diagram for the discovery of active predictor sets with subsequent rigorous evaluation of posterior model probabilities emvs rapidly identifies promising sparse high posterior probability submodels external structural information such as likely covariate groupings or network topologies is easily incorporated into the emvs framework deterministic annealing variants are seen to improve the effectiveness of our algorithms by mit

bayesian variable selection in structured high dimensional covariate spaces with applications in genomics we consider the problem of variable selection in regression modeling in high dimensional spaces where there is known structure among the covariates this is an unconventional variable selection problem for two reasons 1 the dimension of the covariate space is comparable and often much larger than the number of subjects in the study and 2 the covariate space is highly structured and in some cases it is desirable to incorporate this structural information in to the model building process we approach this problem through the bayesian variable selection framework where we assume that the covariates lie on an undirected graph and formulate an ising prior on the model space for incorporating structural information certain computational and statistical problems arise that are unique to such high dimensional structured settings the most interesting being the phenomenon of phase transitions we propose theoretical and computational schemes to mitigate these problems we illustrate our methods on two

spike and slab priors for function selection in structured additive regression models structured additive regression star provides a general framework for complex gaussian and non gaussian regression models with predictors comprising arbitrary combinations of nonlinear functions and surfaces spatial effects varying coefficients random effects and further regression terms the large flexibility of star makes function selection a challenging and important task aiming at 1 selecting the relevant covariates 2 choosing an appropriate and parsimonious representation of the impact of covariates on the predictor and 3 determining the required interactions we propose a spike and slab prior structure for function selection that allows to include or exclude single coefficients as well as blocks of coefficients representing specific model terms a novel multiplicative parameter expansion is required to obtain good mixing and convergence properties in a markov chain monte carlo simulation approach and is shown to induce desirable shrinkage properties in simulation studies and with re

diagnostics for nonparametric regression models with additive terms abstract recent developments of multivariate smoothing methods provide a rich collection of feasible models for nonparametric multivariate data analysis among the most interpretable are models with additive terms construction of various models and algorithms for computing the models have been the main concern of the existing literature in this area few results are available on the validation of computed fits and many applications of nonparametric methods unfortunately end up interpreting the noise this article proposes and illustrates some simple retrospective diagnostics to help data analysts in detecting possible aliasing effects in computed nonparametric fits and in building parsimonious models in an interactive fashion it also discusses the concepts and rationale behind the proposal including concurvity diagnostics versus tests and so forth for their ready availability interaction splines are used in the illustrations

gaussian process models for computer experiments with qualitative and quantitative factors modeling experiments with qualitative and quantitative factors is an important issue in computer modeling we propose a framework for building gaussian process models that incorporate both types of factors the key to the development of these new models is an approach for constructing correlation functions with qualitative and quantitative factors an iterative estimation procedure is developed for the proposed models modern optimization techniques are used in the estimation to ensure the validity of the constructed correlation functions the proposed method is illustrated with an example involving a known function and a real example for modeling the thermal distribution of a data center

bayesian regression modeling with interactions and smooth effects abstract there have been many recent suggestions as to how to build and estimate flexible bayesian regression models using constructs such as trees neural networks and gaussian processes although there is much to commend these methods their implementation and interpretation can be daunting for practitioners this article presents a spline based methodology for flexible bayesian regression that is quite simple in terms of computation and interpretation smooth bivariate interactions are modeled in an economical and apparently novel way and prior distributions that penalize complexity are used predictions can be based on either model selection or model averaging taking computation interpretation and predictive performance into account the method is seen to perform well when applied to simulated and real data

detecting differentially expressed genes in microarrays using bayesian model selection dna microarrays open up a broad new horizon for investigators interested in studying the genetic determinants of disease the high throughput nature of these arrays where differential expression for thousands of genes can be measured simultaneously creates an enormous wealth of information but also poses a challenge for data analysis because of the large multiple testing problem involved the solution has generally been to focus on optimizing false discovery rates while sacrificing power the drawback of this approach is that more subtle expression differences will be missed that might give investigators more insight into the genetic environment necessary for a disease process to take hold we introduce a new method for detecting differentially expressed genes based on a high dimensional model selection technique bayesian anova for microarrays bam which strikes a balance between false rejections and false nonrejections the basis of the new approach involves a weighted average of generalized ridge

a reference bayesian test for nested hypotheses and its relationship to the schwarz criterion abstract to compute a bayes factor for testing h 0 0 in the presence of a nuisance parameter priors under the null and alternative hypotheses must be chosen as in bayesian estimation an important problem has been to define automatic or reference methods for determining priors based only on the structure of the model in this article we apply the heuristic device of taking the amount of information in the prior on equal to the amount of information in a single observation then after transforming to be null orthogonal to we take the marginal priors on to be equal under the null and alternative hypotheses doing so and taking the prior on to be normal we find that the log of the bayes factor may be approximated by the schwarz criterion with an error of order o p n rather than the usual error of order o p 1 this result suggests the schwarz criterion should provide sensible approximate solutions to bayesian testing problems at least when the hypotheses are nested when

model selection in spline nonparametric regression a bayesian approach is presented for model selection in nonparametric regression with gaussian errors and in binary nonparametric regression a smoothness prior is assumed for each component of the model and the posterior probabilities of the candidate models are approximated using the bayesian information criterion we study the model selection method by simulation and show that it has excellent frequentist properties and gives improved estimates of the regression surface all the computations are carried out efficiently using the gibbs sampler

spike and slab gene selection for multigroup microarray data dna microarrays can provide insight into genetic changes that characterize different stages of a disease process accurate identification of these changes has significant therapeutic and diagnostic implications statistical analysis for multistage multigroup data is challenging however anova based extensions of two sample z tests a popular method for detecting differentially expressed genes in two groups do not work well in multigroup settings false detection rates are high because of variability of the ordinary least squares estimators and because of regression to the mean induced by correlated parameter estimates we develop a bayesian rescaled spike and slab hierarchical model specifically designed for the multigroup gene detection problem data preprocessing steps are introduced to deal with unique features of microarray data and to enhance selection performance we show theoretically that spike and slab models naturally encourage sparse solutions through a process called selective shrinkage t

nedelec spaces in affine coordinates in this note we provide a conveniently implementable basis for simplicial nedelec spaces of any order in any space dimension the main feature of the basis is that it is expressed solely in terms of the barycentric coordinates of the simplex

spike and slab variable selection frequentist and bayesian strategies variable selection in the linear regression model takes many apparent faces from both frequentist and bayesian standpoints in this paper we introduce a variable selection method referred to as a rescaled spike and slab model we study the importance of prior hierarchical specifications and draw connections to frequentist generalized ridge regression estimation specifically we study the usefulness of continuous bimodal priors to model hypervariance parameters and the effect scaling has on the posterior mean through its relationship to penalization several model selection strategies some frequentist and some bayesian in nature are developed and studied theoretically we demonstrate the importance of selective shrinkage for effective variable selection in terms of risk misclassification and show this is achieved using the posterior from a rescaled spike and slab model we also show how to verify a procedure s ability to reduce model uncertainty in finite samples using a specialized forward selection strategy using this tool we illustrate the effectiveness of rescaled spike and slab models in reducing model uncertainty

bootstrap model selection abstract in a regression problem typically there are p explanatory variables possibly related to a response variable and we wish to select a subset of the p explanatory variables to fit a model between these variables and the response a bootstrap variable model selection procedure is to select the subset of variables by minimizing bootstrap estimates of the prediction error where the bootstrap estimates are constructed based on a data set of size n although the bootstrap estimates have good properties this bootstrap selection procedure is inconsistent in the sense that the probability of selecting the optimal subset of variables does not converge to 1 as n this inconsistency can be rectified by modifying the sampling method used in drawing bootstrap observations for bootstrapping pairs response explanatory variable it is found that instead of drawing n bootstrap observations a customary bootstrap sampling plan much less bootstrap observations should be sampled the bootstrap selection p

some remarks on spanning families and weights for high order whitney spaces on simplices abstract high order whitney finite element spaces generally lack natural choices of bases but they do have spanning families in these pages we recall such a family on simplicial meshes and we prove theoretically its effectiveness we also comment on some aspects of a new set of degrees of freedom the so called weights on the small simplices to represent discrete functions in these spaces

generalized finite element systems for smooth differential forms and stokes problem we provide both a general framework for discretizing de rham sequences of differential forms of high regularity and some examples of finite element spaces that fit in the framework the general framework is an extension of the previously introduced notion of finite element systems and the examples include conforming mixed finite elements for stokes equation in dimension 2 we detail four low order finite element complexes and one infinite family of highorder finite element complexes in dimension 3 we define one low order complex which may be branched into whitney forms at a chosen index stokes pairs with continuous or discontinuous pressure are provided in arbitrary dimension the finite element spaces all consist of composite polynomials the framework guarantees some nice properties of the spaces in particular the existence of commuting interpolators it also shows that some of the examples are minimal spaces

variable selection in nonparametric regression with categorical covariates abstract this article extends the problem of variable selection to a nonparametric regression model with categorical covariates two selection criteria are considered the cross validation cv criterion and the accumulated prediction error ape criterion we find that asymptotically the cv criterion performs well only when the true model is infinite dimensional while the ape criterion is appropriate when the true model is finite dimensional this is very similar to the case of linear regression model a simulation study reveals some interesting small sample properties of these criteria to be more specific suppose that we have observations x 1 y 1 xn yn that are iid random vectors and x x 1 x 2 where the x i s are categorical we allow y to be of any type now a new observation x has arrived and we want to predict the corresponding y such a framework is more appropriate than regressions with fixed covariates in situations where the covariates are observational rather than bei

multiple shrinkage and subset selection in wavelets this paper discusses bayesian methods for multiple shrinkage estimation in wavelets wavelets are used in applications for data denoising via shrinkage of the coefficients towards zero and for data compression by shrinkage and setting small coefficients to zero we approach wavelet shrinkage by using bayesian hierarchical models assigning a positive prior probability to the wavelet coefficients being zero the resulting estimator for the wavelet coefficients is a multiple shrinkage estimator that exhibits a wide variety of nonlinear patterns we discuss fast computational implementations with a focus on easy to compute analytic approximations as well as importance sampling and markov chain monte carlo methods multiple shrinkage estimators prove to have excellent mean squared error performance in reconstructing standard test functions we demonstrate this in simulated test examples comparing various implementations of multiple shrinkage to commonly used shrinkage rules finally we illustrate our approach with an application to the so called glint data

a strongly consistent procedure for model selection in a regression problem summary we consider the multiple regression model yn xn en where yn and en are n vector random variables xn is an n x m matrix and 83 is an m vector of unknown regression parameters each component of 3 may be zero or nonzero which gives rise to 2 possible models for multiple regression we provide a decision rule for the choice of a model which is strongly consistent for the true model as n oo the result is proved under certain mild conditions for instance without assuming normality of the distribution of the components of en

bayesian graphical regression we consider the problem of modeling conditional independence structures in heterogenous data in the presence of additional subject level covariates termed graphical regression we propose a novel s

bayesian regression using a prior on the model fit the r2 d2 shrinkage prior prior distributions for high dimensional linear regression require specifying a joint distribution for the unobserved regression coefficients which is inherently difficult we instead propose a ne

semi dynamic breadth first search in digraphs when creating a family of systems i e several systems of similar type which differ within some aspects it is desirable to be able to express these differences already at the level of the specification and to automatically obtain systems from it which are ready to run the use of generic methods may lead to substantial progress in this area this report explores two aspects parameterization concepts at the specification level which can be used to describe variants of a system and generator programs which produce runnable systems from prefabricated components

rao blackwellization for bayesian variable selection and model averaging in linear and binary regression a novel data augmentation approach choosing the subset of covariates to use in regression or generalized linear models is a ubiquitous problem the bayesian paradigm addresses the problem of model uncertainty by considering models corresponding to all possible subsets of the covariates where the posterior distribution over models is used to select models or combine them via bayesian model averaging bma although conceptually straightforward bma is often difficult to implement in practice since either the number of covariates is too large for enumeration of all subsets calculations cannot be done analytically or both for orthogonal designs with the appropriate choice of prior the posterior probability of any model can be calculated without having to enumerate the entire model space and scales linearly with the number of predictors p in this article we extend this idea to a much broader class of nonorthogonal design matrices we propose a novel method which augments the observed nonorthogonal design by at most p new rows to obtain

model selection via multifold cross validation a natural extension of the simple leave one out cross validation cv method is to allow the deletion of more than one observations in this article several notions of the multifold cross validation mcv method have been discussed in the context of variable selection under a linear regression model we show that the delete d mcv criterion is asymptotically equivalent to the well known fpe criterion two computationally more feasible methods the r fold cross validation and the repeated learning testing criterion are also studied the performance of these criteria are compared with the simple leave one out cross validation method simulation results are obtained to gain some understanding on the small sample properties of these methods

minimax multiple shrinkage estimation on presente des nouveaux estimateurs qui incorporent l information partielle a priori en utilisant les donnees comme emulation du comportement et du risque des estimateurs consideres les plus efficaces

bayesian k svd using fast variational inference recent work in signal processing in general and image processing in particular deals with sparse representation related problems two such problems are of paramount importance an overriding need for designing a well suited overcomplete dictionary containing a redundant set of atoms i e basis signals and how to find a sparse representation of a given signal with respect to the chosen dictionary dictionary learning techniques among which we find the popular k singular value decomposition algorithm tackle these problems by adapting a dictionary to a set of training data a common drawback of such techniques is the need for parameter tuning in order to overcome this limitation we propose a fully automated bayesian method that considers the uncertainty of the estimates and produces a sparse representation of the data without prior information on the number of non zeros in each representation vector we follow a bayesian approach that uses a three tiered hierarchical prior to enforce sparsity on the representations and develop an efficient variational inference framework that reduces computational complexity furthermore we describe a greedy approach that speeds up the whole process finally we present experimental results that show superior performance on two different applications with real images denoising and inpainting

on the pitman yor process with spike and slab base measure summaryfor the most popular discrete nonparametric models beyond the dirichlet process the prior guess at the shape of the data generating distribution also known as the base measure is assumed to be diffuse such a specification greatly simplifies the derivation of analytical results allowing for a straightforward implementation of bayesian nonparametric inferential procedures however in several applied problems the available prior information leads naturally to the incorporation of an atom into the base measure and then the dirichlet process is essentially the only tractable choice for the prior in this paper we fill this gap by considering the pitman yor process with an atom in its base measure we derive computable expressions for the distribution of the induced random partitions and for the predictive distributions these findings allow us to devise an effective generalized polya urn gibbs sampler applications to density estimation clustering and curve estimation with both simulated and real data serve as an illustration of our results and allow comparisons with existing methodology in particular we tackle a functional data analysis problem concerning basal body temperature curves

spike and slab biclustering abstract biclustering refers to the problem of simultaneously clustering the rows and columns of a given data matrix with the goal of obtaining submatrices where the selected rows present a coherent behaviour in the selected columns and vice versa to face this intrinsically difficult problem we propose a novel generative model where biclustering is approached from a sparse low rank matrix factorization perspective the main idea is to design a probabilistic model describing the factorization of a given data matrix in two other matrices from which information about rows and columns belonging to the sought for biclusters can be obtained one crucial ingredient in the proposed model is the use of a spike and slab sparsity inducing prior thus we term the approach spike and slab biclustering ssbi to estimate the parameters of the ssbi model we propose an expectation maximization em algorithm termed ssbiem which solves a low rank factorization problem at each iteration using a recently proposed augmented lagrangian algorithm experiments with both synthetic and real data show that the ssbi approach compares favorably with the state of the art

variable selection via gibbs sampling abstract a crucial problem in building a multiple regression model is the selection of predictors to include the main thrust of this article is to propose and develop a procedure that uses probabilistic considerations for selecting promising subsets this procedure entails embedding the regression setup in a hierarchical normal mixture model where latent variables are used to identify subset choices in this framework the promising subsets of predictors can be identified as those with higher posterior probability the computational burden is then alleviated by using the gibbs sampler to indirectly sample from this multinomial posterior distribution on the set of possible subset choices those subsets with higher probability the promising ones can then be identified by their more frequent appearance in the gibbs sample

optimal predictive model selection often the goal of model selection is to choose a model for future prediction and it is natural to measure the accuracy of a future prediction by squared error loss under the bayesian approach it is commonly perceived that the optimal predictive model is the model with highest posterior probability but this is not necessarily the case in this paper we show that for selection among normal linear models the optimal predictive model is often the median probability model which is defined as the model consisting of those variables which have overall posterior probability greater than or equal to 1 2 of being in a model the median probability model often differs from the highest probability model

fast algorithms for shortest paths in planar graphs with applications graph decomposition and data structures techniques are presented that exploit the structure of planar graphs to yield faster algorithms for a number of shortest path and related problems improved

saddlepoint approximation for mixture models two component mixture distributions with one component a point mass and the other a continuous density may be used as priors for bayesian inference when sparse representation of an underlying signal is required we show how saddlepoint approximation in such models can yield highly accurate quantiles for posterior distributions and illustrate this numerically using wavelet regression with point mass laplace and point mass normal prior distributions

fast bayesian factor analysis via automatic rotations to sparsity abstractrotational post hoc transformations have traditionally played a key role in enhancing the interpretability of factor analysis regularization methods also serve to achieve this goal by prioritizing sparse loading matrices in this work we bridge these two paradigms with a unifying bayesian framework our approach deploys intermediate factor rotations throughout the learning process greatly enhancing the effectiveness of sparsity inducing priors these automatic rotations to sparsity are embedded within a pxl em algorithm a bayesian variant of parameter expanded em for posterior mode detection by iterating between soft thresholding of small factor loadings and transformations of the factor basis we obtain a dramatic accelerations b robustness against poor initializations and c better oriented sparse solutions to avoid the prespecification of the factor cardinality we extend the loading matrix to have infinitely many columns with the indian buffet process ibp prior the factor dimen

linear model selection by cross validation abstract we consider the problem of selecting a model having the best predictive ability among a class of linear models the popular leave one out cross validation method which is asymptotically equivalent to many other model selection methods such as the akaike information criterion aic the c p and the bootstrap is asymptotically inconsistent in the sense that the probability of selecting the model with the best predictive ability does not converge to 1 as the total number of observations n we show that the inconsistency of the leave one out cross validation can be rectified by using a leave n v out cross validation with n v the number of observations reserved for validation satisfying n v n 1 as n this is a somewhat shocking discovery because nv n 1 is totally opposite to the popular leave one out recipe in cross validation motivations justifications and discussions of some practical aspects of the use of the leave n v out cross validation method are provided and results