learning multiple tasks with kernel methods we study the problem of learning many related tasks simultaneously using kernel methods and regularization the standard single task kernel methods such as support vector machines and regularization networks are extended to the case of multi task learning our analysis shows that the problem of estimating many task functions with regularization can be cast as a single task learning problem if a family of multi task kernel functions we define is used these kernels model relations among the tasks and are derived from a novel form of regularizers specific kernels that can be used for multi task learning are provided and experimentally tested on two real data sets in agreement with past empirical work on multi task learning the experiments show that learning multiple related tasks simultaneously using the proposed approach can significantly outperform standard single task learning particularly when there are many related tasks but few data per task

efficient empirical bayes variable selection and estimation in linear models we propose an empirical bayes method for variable selection and coefficient estimation in linear regression models the method is based on a particular hierarchical bayes formulation and the empirical bayes estimator is shown to be closely related to the lasso estimator such a connection allows us to take advantage of the recently developed quick lasso algorithm to compute the empirical bayes estimate and provides a new way to select the tuning parameter in the lasso method unlike previous empirical bayes variable selection methods which in most practical situations can be implemented only through a greedy stepwise algorithm our method gives a global solution efficiently simulations and real examples show that the proposed method is very competitive in terms of variable selection estimation accuracy and computation speed compared with other variable selection and estimation methods

dirichlet laplace priors for optimal shrinkage penalized regression methods such as l1 regularization are routinely used in high dimensional applications and there is a rich literature on optimality properties under sparsity assumptions in the bayesian paradigm sparsity is routinely induced through two component mixture priors having a probability mass at zero but such priors encounter daunting computational problems in high dimensions this has motivated continuous shrinkage priors which can be expressed as global local scale mixtures of gaussians facilitating computation in contrast to the frequentist literature little is known about the properties of such priors and the convergence and concentration of the corresponding posterior distribution in this article we propose a new class of dirichlet laplace priors which possess optimal posterior concentration and lead to efficient posterior computation finite sample performance of dirichlet laplace priors relative to alternatives is assessed in simulated and real data examples

application of polynomial chaos in stability and control the polynomial chaos of wiener provides a framework for the statistical analysis of dynamical systems with computational cost far superior to monte carlo simulations it is a useful tool for control systems analysis because it allows probabilistic description of the effects of uncertainty especially in systems having nonlinearities and where other techniques such as lyapunov s method may fail we show that stability of a system can be inferred from the evolution of modal amplitudes covering nearly the full support of the uncertain parameters with a finite series by casting uncertain parameters as unknown gains we show that the separation of stochastic from deterministic elements in the response points to fast iterative design methods for nonlinear control

bayesian semiparametric wiener system identification we present a novel method for wiener system identication the method relies on a semiparametric i e a mixed parametric nonparametric model of a wiener system we use a state space model for the linear dynamical system and a nonparametric gaussian process model for the static nonlinearity we avoid making strong assumptions such as monotonicity on the nonlinear mapping stochastic disturbances entering both as measurement noise and as process noise are handled in a systematic manner the nonparametric nature of the gaussian process allows us to handle a wide range of nonlinearities without making problem specic parameterizations we also consider sparsity promoting priors based on generalized hyperbolic distributions to automatically infer the order of the underlying dynamical system we derive an inference algorithm based on an ecient particle markov chain monte carlo method referred to as particle gibbs with ancestor sampling the method is proled on two challenging identication problems with good results blind wiener system identication is handled as a special case

multiple loci mapping via model free variable selection summary despite recent flourish of proposals o nv ariable selection genome wide multiple loci mapping remains to be challenging the majority of existing variable selection methods impose a model and often the homoscedastic linear model prior to selection however the true association between the phenotypical trait and the genetic markers is rarely known ap riori and the presence of epistatic interactions makes the association more complex than al inear relation model free variable selection o ers a useful alternative in th is context but the fact that the number of markers p often far exceeds the number of experimental units n renders all the existing model free solutions that require n p inapplicable in this article we examine a number of model free variable selection methods for small n large p regressions in the context of genome wide multiple loci mapping we propose and advocate a multivariate group wise adaptive penalization solution which requires no model prespecification and thus works for complex trait marker association and handles one variable at a time so that works for n p e ctiveness of the new method is demonstrated through bo th intensive simulations and a comprehensive real data analysis across 6100 gene expression traits

modeling uncertainty in steady state diffusion problems via generalized polynomial chaos abstract we present a generalized polynomial chaos algorithm for the solution of stochastic elliptic partial differential equations subject to uncertain inputs in particular we focus on the solution of the poisson equation with random diffusivity forcing and boundary conditions the stochastic input and solution are represented spectrally by employing the orthogonal polynomial functionals from the askey scheme as a generalization of the original polynomial chaos idea of wiener amer j math 60 1938 897 a galerkin projection in random space is applied to derive the equations in the weak form the resulting set of deterministic equations for each random mode is solved iteratively by a block gauss seidel iteration technique both discrete and continuous random distributions are considered and convergence is verified in model problems and against monte carlo simulations

slants sequential adaptive nonlinear modeling of time series we propose a method for adaptive nonlinear sequential modeling of time series data data are modeled as a nonlinear function of past values corrupted by noise and the underlying nonlinear function is assumed to be approximately expandable in a spline basis we cast the modeling of data as finding a good fit representation in the linear span of multidimensional spline basis and use a variant of l 1 penalty regularization in order to reduce the dimensionality of representation using adaptive filtering techniques we design our online algorithm to automatically tune the underlying parameters based on the minimization of the regularized sequential prediction error we demonstrate the generality and flexibility of the proposed approach on both synthetic and real world datasets moreover we analytically investigate the performance of our algorithm by obtaining both bounds on prediction errors and consistency in variable selection

extension of the destabilization paradox to limit cycle amplitudes for a nonlinear self excited system subject to gyroscopic and circulatory actions this study aims at clarifying the phenomenological roots of an acoustical disturbance known as clutch squeal noise a nonlinear two degrees of freedom model is introduced in order to illustrate some basic phenomena leading to self generated vibrations the damping of the system as well as both circulatory and gyroscopic actions are included in order to highlight their respective influence and the destabilization paradox results are obtained on the stability range of the equilibrium the nature of the hopf bifurcation the limit cycle branches and their stability a dynamic extension of the destabilization paradox is proposed and some non periodic behaviours are identified too

a variational bayes framework for sparse adaptive estimation recently a number of mostly l1 norm regularized least squares type deterministic algorithms have been proposed to address the problem of sparse adaptive signal estimation and system identification from a bayesian perspective this task is equivalent to maximum a posteriori probability estimation under a sparsity promoting heavy tailed prior for the parameters of interest following a different approach this paper develops a unifying framework of sparse variational bayes algorithms that employ heavy tailed priors in conjugate hierarchical form to facilitate posterior inference the resulting fully automated variational schemes are first presented in a batch iterative form then it is shown that by properly exploiting the structure of the batch estimation task new sparse adaptive variational bayes algorithms can be derived which have the ability to impose and track sparsity during real time processing in a time varying environment the most important feature of the proposed algorithms is that they completely eliminate the need for computationally costly parameter fine tuning a necessary ingredient of sparse adaptive deterministic algorithms extensive simulation results are provided to demonstrate the effectiveness of the new sparse adaptive variational bayes algorithms against state of the art deterministic techniques for adaptive channel estimation the results show that the proposed algorithms are numerically robust and exhibit in general superior estimation performance compared to their deterministic counterparts

observer based optimal fault detection and diagnosis using conditional probability distributions a new optimal fault detection and diagnosis fdd scheme is studied in this paper for the continuous time stochastic dynamic systems with time delays where the available information for the fdd is the input and the measured output probability density functions pdf s of the system the square root b spline functional approximation technique is used to formulate the output pdf s with the dynamic weightings as a result the concerned fdd problem can be transformed into a robust fdd problem subjected to a continuous time uncertain nonlinear system with time delays feasible criteria to detect and diagnose the system fault are provided by using linear matrix inequality lmi techniques in order to improve fdd performances two optimization measures namely guaranteed cost performance and hinfin performance are applied to optimize the observer design simulations are given to demonstrate the efficiency of the proposed approach

local shrinkage rules levy processes and regularized regression we use l evy processes to generate joint prior distributions and therefore penalty functions for a location parameter b b1 bp as p grows large this generalizes the class of local global shrinkage rules based on scale mixtures of normals illuminates new connections among disparate methods and leads to new results for computing posterior means and modes under a wide class of priors we extend this framework to large scale regularized regression problems where p n and provide comparisons with other methodologies

model order reduction of random parameter dependent linear systems this paper proposes a new method to perform model reduction of linear time invariant lti systems where parameters are random variables governed by probabilistic laws it combines the well known truncation balanced realization tbr technique together with the generalized polynomial chaos gpc formalism a powerful tool for uncertainty propagation gpc formalism is used to represent and compute a random parameter dependent balancing transformation rpd bt which puts the random lti system in a balanced form almost surely within the probabilistic range of the uncertain parameters model reduction is then performed by truncating almost surely weakly controllable and observable states yielding a random parameter dependent truncated balanced realization rpd tbr the truncation error s moments are shown to be bounded by hankel singular values moments which are also estimated using gpc formalism as an illustrative example the proposed method is applied to a simple mechanical model of a two degrees of freedom mass spring system with uncertain stiffness and damping

bayesian estimation of density via multiple sequential inversions of two dimensional images with application to electron microscopy we present a new bayesian methodology to learn the unknown material density of a given sample by inverting its two dimensional images that are taken with a scanning electron microscope an image results from a sequence of projections of the convolution of the density function with the unknown microscopy correction function that we also learn from the data thus learning of the unknowns demands multiple inversions we invoke a novel design of experiment involving imaging at multiple values of the parameter that controls the subsurface depth from which information about the density structure is carried to result in the image real life material density functions are characterized by high density contrasts and are highly discontinuous implying that they exhibit correlation structures that do not vary smoothly in the absence of training data modeling such correlation structures of real material density functions is not possible so we discretize the material sample and treat values of the density functi

decoupling shrinkage and selection in bayesian linear models a posterior summary perspective selecting a subset of variables for linear models remains an active area of research this article reviews many of the recent contributions to the bayesian model selection and shrinkage prior literature a posterior variable selection summary is proposed which distills a full posterior distribution over regression coefficients into a sequence of sparse linear predictors

a split and merge bayesian variable selection approach for ultrahigh dimensional regression summary we propose a bayesian variable selection approach for ultrahigh dimensional linear regression based on the strategy of split and merge the approach proposed consists of two stages split the ultrahigh dimensional data set into a number of lower dimensional subsets and select relevant variables from each of the subsets and aggregate the variables selected from each subset and then select relevant variables from the aggregated data set since the approach proposed has an embarrassingly parallel structure it can be easily implemented in a parallel architecture and applied to big data problems with millions or more of explanatory variables under mild conditions we show that the approach proposed is consistent i e the true explanatory variables can be correctly identified by the approach as the sample size becomes large extensive comparisons of the approach proposed have been made with penalized likelihood approaches such as the lasso elastic net sure independence screening and iterative sure independence screening the numerical results show that the approach proposed generally outperforms penalized likelihood approaches the models selected by the approach tend to be more sparse and closer to the true model

uncertainty quantification of limit cycle oscillations different computational methodologies have been developed to quantify the uncertain response of a relatively simple aeroelastic system in limit cycle oscillation subject to parametric variability the aeroelastic system is that of a rigid airfoil supported by pitch and plunge structural coupling with nonlinearities in the component in pitch the nonlinearities are adjusted to permit the formation of a either a subcritical or supercritical branch of limit cycle oscillations uncertainties are specified in the cubic coefficient of the torsional spring and in the initial pitch angle of the airfoil stochastic projections of the time domain and cyclic equations governing system response are carried out leading to both intrusive and non intrusive computational formulations non intrusive formulations are examined using stochastic projections derived from wiener expansions involving haar wavelet and b spline bases while wiener hermite expansions of the cyclic equations are employed intrusively and non intrusively application of the b spline stochastic projection is extended to the treatment of aerodynamic nonlinearities as modeled through the discrete euler equations the methodologies are compared in terms of computational cost convergence properties ease of implementation and potential for application to complex aeroelastic systems

mean square stochastic stability of linear time delay system with markovian jumping parameters this paper deals with the mean square stochastic stability of the class of linear time delay systems with markovian jumping parameters a delay independent sufficient condition for checking the stochastic stability of this class of systems is established a numerical example is given to show the usefulness of the proposed theoretical results

bayesian variable selection in linear regression abstract this article is concerned with the selection of subsets of predictor variables in a linear regression model for the prediction of a dependent variable it is based on a bayesian approach intended to be as objective as possible a probability distribution is first assigned to the dependent variable through the specification of a family of prior distributions for the unknown parameters in the regression model the method is not fully bayesian however because the ultimate choice of prior distribution from this family is affected by the data it is assumed that the predictors represent distinct observables the corresponding regression coefficients are assigned independent prior distributions for each regression coefficient subject to deletion from the model the prior distribution is a mixture of a point mass at 0 and a diffuse uniform distribution elsewhere that is a spike and slab distribution the random error component is assigned a normal distribution with mean 0 and standard deviation

the bayesian lasso the lasso estimate for linear regression parameters can be interpreted as a bayesian posterior mode estimate when the regression parameters have independent laplace i e double exponential priors gibbs sampling from this posterior is possible using an expanded hierarchy with conjugate normal priors for the regression parameters and independent exponential priors on their variances a connection with the inverse gaussian distribution provides tractable full conditional distributions the bayesian lasso provides interval estimates bayesian credible intervals that can guide variable selection moreover the structure of the hierarchical model provides both bayesian and likelihood methods for selecting the lasso parameter slight modifications lead to bayesian versions of other lasso related estimation methods including bridge regression and a robust variant

from k means to higher way co clustering multilinear decomposition with sparse latent factors co clustering is a generalization of unsupervised clustering that has recently drawn renewed attention driven by emerging data mining applications in diverse areas whereas clustering groups entire columns of a data matrix co clustering groups columns over select rows only i e it simultaneously groups rows and columns the concept generalizes to data boxes and higher way tensors for simultaneous grouping along multiple modes various co clustering formulations have been proposed but no workhorse analogous to k means has emerged this paper starts from k means and shows how co clustering can be formulated as a constrained multilinear decomposition with sparse latent factors for three and higher way data uniqueness of the multilinear decomposition implies that unlike matrix co clustering it is possible to unravel a large number of possibly overlapping co clusters a basic multi way co clustering algorithm is proposed that exploits multilinearity using lasso type coordinate updates various line search schemes are then introduced to speed up convergence and suitable modifications are proposed to deal with missing values the imposition of latent sparsity pays a collateral dividend it turns out that sequentially extracting one co cluster at a time is almost optimal hence the approach scales well for large datasets the resulting algorithms are benchmarked against the state of art in pertinent simulations and applied to measured data including the enron e mail corpus

outlier models and prior distributions in bayesian linear regression summary bayesian inference in regression models is considered using heavy tailed error distri butions to accommodate outliers the particular class of distributions that can be con structed as scale mixtures of normal distributions are examined and use is made of them as both error models and prior distributions in bayesian linear modelling includ ing simple regression and more complex hierarchical models with structured priors depending on unknown hyperprior parameters the modelling of outliers in nominally normal linear regression models using alternative error distributions which are heavy tailed relative to the normal provides an automatic means of both detecting and accommodating possibly aberrant observations such realistic models do however often lead to analytically intractable analyses with complex posterior distributions in several dimensions that are difficult to summarize and understand in this paper we consider a special yet rather wide class of heavy tailed unimodal and symmetric error distributions for which the analyses though apparently intractable can be examined in some depth by exploiting certain properties of the assumed error form the distributions concerned are those that can be con structed as scale mixtures of normal distributions in his paper concerning location parameters de finetti 1961 discusses such distributions and suggests the hypothetical interpretation that each observation is taken using an instrument with normal error but each time chosen at random from a collection of instruments of different precisions the distribution of the

fast approximate inference for arbitrarily large semiparametric regression models via message passing abstractwe show how the notion of message passing can be used to streamline the algebra and computer coding for fast approximate inference in large bayesian semiparametric regression models in particular this approach is amenable to handling arbitrarily large models of particular types once a set of primitive operations is established the approach is founded upon a message passing formulation of mean field variational bayes that utilizes factor graph representations of statistical models the underlying principles apply to general bayesian hierarchical models although we focus on semiparametric regression the notion of factor graph fragments is introduced and is shown to facilitate compartmentalization of the required algebra and coding the resultant algorithms have ready to implement closed form expressions and allow a broad class of arbitrarily large semiparametric regression models to be handled ongoing software projects such as infer net and stan support variational type inference for particular

statistics of complex eigenvalues in friction induced vibration abstract self excited vibrations appear in many mechanical systems with sliding contacts there are several mechanisms whereby friction can cause the self excited vibration to become unstable of these mechanisms mode coupling is thought to be responsible for generating annoying high frequency noise and vibration in brakes conventionally in order to identify whether a system is stable or not complex eigenvalue analysis is performed however what has recently received much attention of researchers is the variability and uncertainty of input variables in the stability analysis of self excited vibrations for this purpose a second order perturbation method is extended and employed in the current study the moments of the output distribution along with its joint moment generating function are used for quantifying the statistics of the complex eigenvalues moreover the eigen derivatives required for the perturbation method are presented in a way that they can deal with the asymmetry of the stiffness matrix and non proportional damping since the eigen derivatives of such systems are complex valued numbers it is mathematically more informative and convenient to derive the statistics of the eigenvalues in a complex form without decomposing them into two real valued real and imaginary parts then the variance and pseudo variance of the complex eigenvalues are used for determining the statistics of the real and imaginary parts the reliability and robustness of the system in terms of stability can also be quantified by the approximated output distribution

hybrid surrogate model for the prediction of uncertain friction induced instabilities abstract this paper presents a hybrid surrogate model for the prediction of friction induced instabilities in uncertain mechanical friction systems the proposed hybrid meta model is developed in order to predict the occurrence of mode coupling instabilities submitted to random and interval parameter uncertainties this predictor is built from the combination of the generalized polynomial chaos formalism known to be useful to deal with random uncertainties together with the inclusion function based on chebyshev polynomials used to deal with interval uncertainties the feasibility of the proposed approach and its efficiency are assessed by investigating the stability analysis of a four degree of freedom model with two sets of uncertain parameters described by probabilistic and interval models numerical results are compared with those obtained by applying a classical parametric approach to demonstrate the efficiency of the proposed methodology the suggested hybrid meta model is then shown to have an interesting potential to deal with stability analysis of mechanical systems subjected to friction induced vibration

elastic net regression modeling with the orthant normal prior the elastic net procedure is a form of regularized optimization for linear regression that provides a bridge between ridge regression and the lasso the estimate that it produces can be viewed as a bayesian posterior mode under a prior distribution implied by the form of the elastic net penalty this article broadens the scope of the bayesian connection by providing a complete characterization of a class of prior distributions that generate the elastic net estimate as the posterior mode the resulting model based framework allows for methodology that moves beyond exclusive use of the posterior mode by considering inference based on the full posterior distribution two characterizations of the class of prior distributions are introduced a properly normalized direct characterization which is shown to be conjugate for linear regression models and an alternate representation as a scale mixture of normal distributions prior distributions are proposed for the regularization parameters resulting in an infi

sparse coding from a bayesian perspective sparse coding is a promising theme in computer vision most of the existing sparse coding methods are based on either l0 or l1 penalty which often leads to unstable solution or biased estimation this is because of the nonconvexity and discontinuity of the l0 penalty and the over penalization on the true large coefficients of the l1 penalty in this paper sparse coding is interpreted from a novel bayesian perspective which results in a new objective function through maximum a posteriori estimation the obtained solution of the objective function can generate more stable results than the l0 penalty and smaller reconstruction errors than the l1 penalty in addition the convergence property of the proposed algorithm for sparse coding is also established the experiments on applications in single image super resolution and visual tracking demonstrate that the proposed method is more effective than other state of the art methods

stochastic stability properties of jump linear systems jump linear systems are defined as a family of linear systems with randomly jumping parameters usually governed by a markov jump process and are used to model systems subject to failures or changes in structure the authors study stochastic stability properties in jump linear systems and the relationship among various moment and sample path stability properties it is shown that all second moment stability properties are equivalent and are sufficient for almost sure sample path stability and a testable necessary and sufficient condition for second moment stability is derived the lyapunov exponent method for the study of almost sure sample stability is discussed and a theorem which characterizes the lyapunov exponents of jump linear systems is presented finally for one dimensional jump linear system it is proved that the region for delta moment stability is monotonically converging to the region for almost sure stability at delta down arrow 0 sup

a statistical view of some chemometrics regression tools chemometrics is a field of chemistry that studies the application of statistical methods to chemical data analysis in addition to borrowing many techniques from the statistics and engineering literatures chemometrics itself has given rise to several new data analytical methods this article examines two methods commonly used in chemometrics for predictive modeling partial least squares and principal components regression from a statistical perspective the goal is to try to understand their apparent successes and in what situations they can be expected to work well and to compare them with other statistical methods intended for those situations these methods include ordinary least squares variable subset selection and ridge regression

non intrusive generalized polynomial chaos for the robust stability analysis of uncertain nonlinear dynamic friction systems this paper is devoted to the stability analysis of uncertain nonlinear dynamic dry friction systems the stability property of dry friction systems is known to be very sensitive to the variations of friction laws moreover the friction coefficient admits dispersions due to the manufacturing processes therefore it becomes necessary to take this uncertainty into account in the stability analysis of dry friction systems to ensure robust predictions of stable and instable behaviours the generalized polynomial chaos formalism is proposed to deal with this challenging problem treated in most cases with the prohibitive monte carlo based techniques two equivalent methods presented here combine the non intrusive generalized polynomial chaos with the indirect lyapunov method both methods are shown to be efficient in the estimation of the stability and instability regions with high accuracy and high confidence levels and at lower cost compared with the classic monte carlo based method

stability and stabilization of discrete time semi markov jump linear systems via semi markov kernel approach this technical note is concerned with the problems of stability and stabilization for a class of discrete time semi markov jump linear systems s mjlss the discrete time semi markov kernel smk is introduced where the probability density function of sojourn time is dependent on both current and next system mode as a consequence different types of distributions and or different parameters in a same type of distribution of sojourn time depending on the target mode towards which the system jumps can coexist in each mode of a smk the underlying s mjlss are therefore more general than those considered in existing studies a new stability concept generalizing the traditional mean square stability is proposed such that numerically testable criteria on the basis of smk are obtained numerical examples are presented to illustrate the validity and advantage of the developed theoretical results

bayes variable selection in semiparametric linear models there is a rich literature on bayesian variable selection for parametric models our focus is on generalizing methods and asymptotic theory established for mixtures of g priors to semiparametric linear regression models having unknown residual densities using a dirichlet process location mixture for the residual density we propose a semiparametric g prior which incorporates an unknown matrix of cluster allocation indicators for this class of priors posterior computation can proceed via a straightforward stochastic search variable selection algorithm in addition bayes factor and variable selection consistency is shown to result under a class of proper priors on g even when the number of candidate predictors p is allowed to increase much faster than sample size n while making sparsity assumptions on the true model size

tuning parameter selection for the adaptive lasso using eric the adaptive lasso is a commonly applied penalty for variable selection in regression modeling like all penalties though its performance depends critically on the choice of the tuning parameter one method for choosing the tuning parameter is via information criteria such as those based on aic and bic however these criteria were developed for use with unpenalized maximum likelihood estimators and it is not clear that they take into account the effects of penalization in this article we propose the extended regularized information criterion eric for choosing the tuning parameter in adaptive lasso regression eric extends the bic to account for the effect of applying the adaptive lasso on the bias variance tradeoff this leads to a criterion whose penalty for model complexity is itself a function of the tuning parameter we show the tuning parameter chosen by eric is selection consistent when the number of variables grows with sample size and that this consistency holds in a wider range of contex

high dimensional semiparametric gaussian copula graphical models ing the spearman s rho and kendall s tau we prove that the nonparanormal skeptic achieves the optimal parametric rates of convergence for both graph recovery and parameter estimation this result suggests that the nonparanormal graphical models can be used as a safe replacement of the popular gaussian graphical models even when the data are truly gaussian besides theoretical analysis we also conduct thorough numerical simulations to compare the graph recovery performance of dierent estimators under both ideal and noisy settings the proposed methods are then applied on a largescale genomic dataset to illustrate their empirical usefulness the r package huge implementing the proposed methods is available on the comprehensive r archive network http cran r project org

modeling uncertainty in flow simulations via generalized polynomial chaos we present a new algorithm to model the input uncertainty and its propagation in incompressible flow simulations the stochastic input is represented spectrally by employing orthogonal polynomial functionals from the askey scheme as trial basis to represent the random space a standard galerkin projection is applied in the random dimension to obtain the equations in the weak form the resulting system of deterministic equations is then solved with standard methods to obtain the solution for each random mode this approach can be considered as a generalization of the original polynomial chaos expansion first introduced by wiener am j math 60 1938 897 the original method employs the hermite polynomials one of the 13 members of the askey scheme as the basis in random space the algorithm is applied to micro channel flows with random wall boundary conditions and to external flows with random freestream efficiency and convergence are studied by comparing with exact solutions as well as numerical solutions obtained by monte carlo simulations it is shown that the generalized polynomial chaos method promises a substantial speed up compared with the monte carlo method the utilization of different type orthogonal polynomials from the askey scheme also provides a more efficient way to represent general non gaussian processes compared with the original wiener hermite expansions

self excited vibrations induced by dry friction part 2 stability and limit cycle analysis abstract some mechanical systems have frictional joints where one surface slides on another with dry friction under certain conditions the steady state sliding motion becomes unstable the instability leads to the occurrence of stable limit cycle type self excited vibrations this study is concerned with the analysis of the experimental results reported in a companion paper dealing with the instability of the steady state sliding motion and the resulting self excited vibrations the conditions which cause the steady state sliding motion to become unstable are investigated with a linear stability theory a parametric study is conducted to assess the influence of the system parameters on the stability of the steady state sliding motion the method of triple input describing functions is used with a non linear model to analyze the resulting self excited vibrations

multivariate archimedean copulas d monotone functions and l1 norm symmetric distributions it is shown that a necessary and sufficient condition for an archimedean copula generator to generate a d dimensional copula is that the generator is a d monotone function the class of d dimensional archimedean copulas is shown to coincide with the class of survival copulas of d dimensional l 1 norm symmetric distributions that place no point mass at the origin the d monotone archimedean copula generators may be characterized using a little known integral transform of williamson duke math j 23 1956 189 207 in an analogous manner to the well known bernstein widder characterization of completely monotone generators in terms of the laplace transform these insights allow the construction of new archimedean copula families and provide a general solution to the problem of sampling multivariate archimedean copulas they also yield useful expressions for the d dimensional kendall function and kendall s rank correlation coefficients and facilitate the derivation of results on the existence of densities and the description of singular components for archimedean copulas the existence of a sharp lower bound for archimedean copulas with respect to the positive lower orthant dependence ordering is shown

stochastic robustness of linear time invariant control systems a simple numerical procedure for estimating the stochastic robustness of a linear time invariant system is described monte carlo evaluation of the system s eigenvalues allows the probability of instability and the related stochastic root locus to be estimated this analysis approach treats not only gaussian parameter uncertainties but also nongaussian cases including uncertain but bound variations confidence intervals for the scalar probability of instability address computational issues inherent in monte carlo simulation trivial extensions of the procedure admit consideration of alternate discriminants thus the probabilities that stipulated degrees of instability will be exceeded or that closed loop roots will leave desirable regions can also be estimated results are particularly amenable to graphical presentation

multi task learning for classification with dirichlet process priors consider the problem of learning logistic regression models for multiple classification tasks where the training data set for each task is not drawn from the same statistical distribution in such a multi task learning mtl scenario it is necessary to identify groups of similar tasks that should be learned jointly relying on a dirichlet process dp based statistical model to learn the extent of similarity between classification tasks we develop computationally efficient algorithms for two different forms of the mtl problem first we consider a symmetric multi task learning smtl situation in which classifiers for multiple tasks are learned jointly using a variational bayesian vb algorithm second we consider an asymmetric multi task learning amtl formulation in which the posterior density function from the smtl model parameters from previous tasks is used as a prior for a new task this approach has the significant advantage of not requiring storage and use of all previous data from prior tasks the amtl formulation is solved with a simple markov chain monte carlo mcmc construction experimental results on two real life mtl problems indicate that the proposed algorithms a automatically identify subgroups of related tasks whose training data appear to be drawn from similar distributions and b are more accurate than simpler approaches such as single task learning pooling of data across all tasks and simplified approximations to dp

exponential dispersion models on etudie les proprietes generales de la classe des modeles de dispersion exponentiels qui est la generalisation multivariable des modeles de distribution d erreur de nelder et wedderburn 1972

variational bayes group sparse time adaptive parameter estimation with either known or unknown sparsity pattern in this paper we study the problem of time adaptive group sparse signal estimation from a bayesian viewpoint we propose two online variational bayes schemes that are specifically designed to estimate and track group sparse signals in time the proposed schemes address both the cases where the grouping information of the signal is either known or not for the case of known group sparsity pattern the proposed scheme builds on a novel hierarchical model for the bayesian adaptive group lasso utilizing the variational bayes framework update equations for all model parameters are given for both the batch and time adaptive estimation scenarios to address the case where the group sparsity pattern is unknown the hierarchical bayesian model of the former scheme is extended by organizing the penalty parameters of the bayesian lasso in a conditional autoregressive model intrinsic conditional autoregression is exploited to penalize the signal coefficients in a structured manner and thus obtain group sparse solutions automatically again a robust and computationally efficient online variational bayes estimator is developed capitalizing on the conjugacy of the proposed hierarchical bayesian formulation experimental results are reported that corroborate the superior estimation performance of the proposed online schemes when compared with state of the art methods

bayesian robust principal component analysis a hierarchical bayesian model is considered for decomposing a matrix into low rank and sparse components assuming the observed matrix is a superposition of the two the matrix is assumed noisy with unknown and possibly non stationary noise statistics the bayesian framework infers an approximate representation for the noise statistics while simultaneously inferring the low rank and sparse outlier contributions the model is robust to a broad range of noise levels without having to change model hyperparameter settings in addition the bayesian framework allows exploitation of additional structure in the matrix for example in video applications each row or column corresponds to a video frame and we introduce a markov dependency between consecutive rows in the matrix corresponding to consecutive frames in the video the properties of this markov process are also inferred based on the observed matrix while simultaneously denoising and recovering the low rank and sparse components we compare the bayesian model to a state of the art optimization based implementation of robust pca considering several examples we demonstrate competitive performance of the proposed model

stochastic stability analysis for continuous time fault tolerant control systems active fault tolerant control systems are feedback control systems that reconfigure the control law in real time based on the response from an automatic failure detection and identification fdi scheme the dynamic behaviour of such systems is characterized by stochastic differential equations because of the random nature of the failure events and the fdi decisions the stability analysis of these systems is addressed in this paper using stochastic lyapunov functions and supermartingale theorems both exponential stability in the mean square and almost sure asymptotic stability in probability are addressed in particular for linear systems where the coefficients of the closed loop system dynamics are functions of two random processes with markovian transition characteristics one representing the random failures and the other representing the fdi decision behaviour necessary and sufficient conditions for exponential stability in the mean square are developed

selection of polynomial chaos bases via bayesian model uncertainty methods with applications to sparse approximation of pdes with stochastic inputs generalized polynomial chaos gpc expansions allow us to represent the solution of a stochastic system using a series of polynomial chaos basis functions the number of gpc terms increases dramatically as the dimension of the random input variables increases when the number of the gpc terms is larger than that of the available samples a scenario that often occurs when the corresponding deterministic solver is computationally expensive evaluation of the gpc expansion can be inaccurate due to over fitting we propose a fully bayesian approach that allows for global recovery of the stochastic solutions in both spatial and random domains by coupling bayesian model uncertainty and regularization regression methods it allows the evaluation of the pc coefficients on a grid of spatial points via 1 the bayesian model average bma or 2 the median probability model and their construction as spatial functions on the spatial domain via spline interpolation the former accounts for the model uncertainty and provides bayes optimal predictions while the latter provides a sparse representation of the stochastic solutions by evaluating the expansion on a subset of dominating gpc bases moreover the proposed methods quantify the importance of the gpc bases in the probabilistic sense through inclusion probabilities we design a markov chain monte carlo mcmc sampler that evaluates all the unknown quantities without the need of ad hoc techniques the proposed methods are suitable for but not restricted to problems whose stochastic solutions are sparse in the stochastic space with respect to the gpc bases while the deterministic solver involved is expensive we demonstrate the accuracy and performance of the proposed methods and make comparisons with other approaches on solving elliptic spdes with 1 14 and 40 random dimensions

spike and slab variable selection frequentist and bayesian strategies variable selection in the linear regression model takes many apparent faces from both frequentist and bayesian standpoints in this paper we introduce a variable selection method referred to as a rescaled spike and slab model we study the importance of prior hierarchical specifications and draw connections to frequentist generalized ridge regression estimation specifically we study the usefulness of continuous bimodal priors to model hypervariance parameters and the effect scaling has on the posterior mean through its relationship to penalization several model selection strategies some frequentist and some bayesian in nature are developed and studied theoretically we demonstrate the importance of selective shrinkage for effective variable selection in terms of risk misclassification and show this is achieved using the posterior from a rescaled spike and slab model we also show how to verify a procedure s ability to reduce model uncertainty in finite samples using a specialized forward selection strategy using this tool we illustrate the effectiveness of rescaled spike and slab models in reducing model uncertainty

stochastic stability analysis of fault tolerant control systems in the presence of noise the stochastic stability of fault tolerant control systems ftcss in the presence of noise are analyzed using the lyapunov function approach in particular the ftcs with a markovian fault detection and isolation process having transition probabilities conditioned on the state of another markovian process representing component failures is considered the almost sure asymptotic stability in probability and the exponential stability in the mean square are considered specifically a testable necessary and sufficient condition for the exponential stability in the mean square is derived a numerical example is presented to illustrate the theoretical analysis

maximizing generalized linear mixed model likelihoods with an automated monte carlo em algorithm summary two new implementations of the em algorithm are proposed for maximum likelihood fitting of generalized linear mixed models both methods use random independent and identically distributed sampling to construct monte carlo approximations at the e step one approach involves generating random samples from the exact conditional distribution of the random effects given the data by rejection sampling using the marginal distribution as a candidate the second method uses a multivariate t importance sampling approximation in many applications the two methods are complementary rejection sampling is more efficient when sample sizes are small whereas importance sampling is better with larger sample sizes monte carlo approximation using random samples allows the monte carlo error at each iteration to be assessed by using standard central limit theory combined with taylor series methods specifically we construct a sandwich variance estimate for the maximizer at each approximate e step this suggests a rule for automatically increasing the monte carlo sample size after iterations in which the true em step is swamped by monte carlo error in contrast techniques for assessing monte carlo error have not been developed for use with alternative implementations of monte carlo em algorithms utilizing markov chain monte carlo e step approximations three different data sets including the infamous salamander data of mccullagh and nelder are used to illustrate the techniques and to compare them with the alternatives the results show that the methods proposed can be considerably more efficient than those based on markov chain monte carlo algorithms however the methods proposed may break down when the intractable integrals in the likelihood function are of high dimension

a constrained l1 minimization approach to sparse precision matrix estimation this article proposes a constrained l1 minimization method for estimating a sparse inverse covariance matrix based on a sample of n iid p variate random variables the resulting estimator is shown to have a number of desirable properties in particular the rate of convergence between the estimator and the true s sparse precision matrix under the spectral norm is when the population distribution has either exponential type tails or polynomial type tails we present convergence rates under the elementwise l norm and frobenius norm in addition we consider graphical model selection the procedure is easily implemented by linear programming numerical performance of the estimator is investigated using both simulated and real data in particular the procedure is applied to analyze a breast cancer dataset and is found to perform favorably compared with existing methods

frequentist accuracy of bayesian estimates in the absence of relevant prior experience popular bayesian estimation techniques usually begin with some form of uninformative prior distribution intended to have minimal inferential influence bayes rule will still produce nice looking estimates and credible intervals but these lack the logical force attached to experience based priors and require further justification this paper concerns the frequentist assessment of bayes estimates a simple formula is shown to give the frequentist standard deviation of a bayesian point estimate the same simulations required for the point estimate also produce the standard deviation exponential family models make the calculations particularly simple and bring in a connection to the parametric bootstrap

multi task sparse structure learning with gaussian copula models multi task learning mtl aims to improve generalization performance by learning multiple related tasks simultaneously while sometimes the underlying task relationship structure is known often the structure needs to be estimated from data at hand in this paper we present a novel family of models for mtl applicable to regression and classification problems capable of learning the structure of tasks relationship in particular we consider a joint estimation problem of the tasks relationship structure and the individual task parameters which is solved using alternating minimization the task relationship revealed by structure learning is founded on recent advances in gaussian graphical models endowed with sparse estimators of the precision inverse covariance matrix an extension to include exible gaussian copula models that relaxes the gaussian marginal assumption is also proposed we illustrate the effectiveness of the proposed model on a variety of synthetic and benchmark data sets for regression and classification we also consider the problem of combining earth system model esm outputs for better projections of future climate with focus on projections of temperature by combining esms in south and north america and show that the proposed model outperforms several existing methods for the problem

multiple shrinkage and subset selection in wavelets this paper discusses bayesian methods for multiple shrinkage estimation in wavelets wavelets are used in applications for data denoising via shrinkage of the coefficients towards zero and for data compression by shrinkage and setting small coefficients to zero we approach wavelet shrinkage by using bayesian hierarchical models assigning a positive prior probability to the wavelet coefficients being zero the resulting estimator for the wavelet coefficients is a multiple shrinkage estimator that exhibits a wide variety of nonlinear patterns we discuss fast computational implementations with a focus on easy to compute analytic approximations as well as importance sampling and markov chain monte carlo methods multiple shrinkage estimators prove to have excellent mean squared error performance in reconstructing standard test functions we demonstrate this in simulated test examples comparing various implementations of multiple shrinkage to commonly used shrinkage rules finally we illustrate our approach with an application to the so called glint data

maximum likelihood algorithms for generalized linear mixed models abstract maximum likelihood algorithms are described for generalized linear mixed models i show how to construct a monte carlo version of the em algorithm propose a monte carlo newton raphson algorithm and evaluate and improve the use of importance sampling ideas calculation of the maximum likelihood estimates is feasible for a wide variety of problems where they were not previously i also use the newton raphson algorithm as a framework to compare maximum likelihood to the joint maximization or penalized quasi likelihood methods and explain why the latter can perform poorly

controllability stabilizability and continuous time markovian jump linear quadratic control consideration is given to the control of continuous time linear systems that possess randomly jumping parameters which can be described by finite state markov processes the relationship between appropriately defined controllability stabilizability properties and the solution of the infinite time jump linear quadratic jlq optimal control problems is also examined although the solution of the continuous time markov jlq problem with finite or infinite time horizons is known only sufficient conditions for the existence of finite cost constant stabilizing controls for the infinite time problem appear in the literature in this paper necessary and sufficient conditions are established these conditions are based on new definitions of controllability observability stabilizability and detectability that are appropriate for continuous time markovian jump linear systems these definitions play the same role for the jlq problem as the deterministic properties do for the linear quadratic regulator lqr problem

bayesian lasso for semiparametric structural equation models there has been great interest in developing nonlinear structural equation models and associated statistical inference procedures including estimation and model selection methods in this paper a general semiparametric structural equation model ssem is developed in which the structural equation is composed of nonparametric functions of exogenous latent variables and fixed covariates on a set of latent endogenous variables a basis representation is used to approximate these nonparametric functions in the structural equation and the bayesian lasso method coupled with a markov chain monte carlo mcmc algorithm is used for simultaneous estimation and model selection the proposed method is illustrated using a simulation study and data from the affective dynamics and individual differences adid study results demonstrate that our method can accurately estimate the unknown parameters and correctly identify the true underlying model

bayesian generalized kernel mixed models we propose a fully bayesian methodology for generalized kernel mixed models gkmms which are extensions of generalized linear mixed models in the feature space induced by a reproducing kernel we place a mixture of a point mass distribution and silverman s g prior on the regression vector of a generalized kernel model gkm this mixture prior allows a fraction of the components of the regression vector to be zero thus it serves for sparse modeling and is useful for bayesian computation in particular we exploit data augmentation methodology to develop a markov chain monte carlo mcmc algorithm in which the reversible jump method is used for model selection and a bayesian model averaging method is used for posterior prediction when the feature basis expansion in the reproducing kernel hilbert space is treated as a stochastic process this approach can be related to the karhunen loeve expansion of a gaussian process gp thus our sparse modeling framework leads to a flexible approximation method for gps

spectral and multiresolution wiener expansions of oscillatory stochastic processes abstract wiener chaos expansions are being evaluated for the representation of stochastic variability in the response of nonlinear aeroelastic systems which often exhibit limit cycles preliminary studies with a simple nonlinear aeroelastic computational model have shown that the standard non intrusive wiener hermite expansion fails to maintain time accuracy as the simulation evolves wiener hermite expansions faithfully reproduce the short term characteristics of the process but consistently lose energy after several mean periods of oscillation this energy loss remains even for very high order expansions to uncover the cause of this energy loss and to explore potential remedies the more elementary problem of a sinusoid with random frequency is used herein to simulate the periodic response of an uncertain system as time progresses coefficients of the higher order terms in both the wiener hermite and wiener legendre expansions successively gain and lose dominance over the lower order coefficients in a manner that causes any fixed order expansion in terms of global basis functions to fail over a simulation time of sufficient duration this characteristic behavior is attributed to the continually increasing frequency of the process in the random dimension the recently developed wiener haar expansion is found to almost entirely eliminate the loss of energy at large times both for the sinusoidal process and for the response of a two degree of freedom nonlinear system which is examined as a prelude to the stochastic simulation of aeroelastic limit cycles it is also found that mallat s pyramid algorithm is more efficient and accurate for evaluating wiener haar expansion coefficients than monte carlo simulation or numerical quadrature

on scale mixtures of normal distributions summary the exponential power family of distributions of box tiao 1973 is shown to be a subset of the class of scale mixtures of normals the corresponding mixing distributions are explicitly obtained identifying a close relationship between the exponential power family and a further class of normal scale mixtures namely the stable distributions

testing for granger causality in large mixed frequency vars abstract we analyze granger causality testing in a mixed frequency var where the difference in sampling frequencies of the variables is large implying parameter proliferation problems in case we attempt to estimate the model unrestrictedly we propose several tests based on reduced rank restrictions including bootstrap versions thereof to account for factor estimation uncertainty and improve the finite sample properties of the tests and a bayesian var extended to mixed frequencies we compare these methods to a test based on an aggregated model the m a x test ghysels et al 2016a and an unrestricted var based test ghysels et al 2016b using monte carlo simulations an empirical application illustrates the techniques

variable selection via gibbs sampling abstract a crucial problem in building a multiple regression model is the selection of predictors to include the main thrust of this article is to propose and develop a procedure that uses probabilistic considerations for selecting promising subsets this procedure entails embedding the regression setup in a hierarchical normal mixture model where latent variables are used to identify subset choices in this framework the promising subsets of predictors can be identified as those with higher posterior probability the computational burden is then alleviated by using the gibbs sampler to indirectly sample from this multinomial posterior distribution on the set of possible subset choices those subsets with higher probability the promising ones can then be identified by their more frequent appearance in the gibbs sample

maximum likelihood estimation of models for residual covariance in spatial regression we describe the maximum likelihood method for fitting the linear model when residuals are correlated and when the covariance among the residuals is determined by a parametric model containing unknown parameters observations are assumed to be gaussian we give conditions which ensure consistency and asymptotic normality of the estimators our main concern is with the analysis of spatial data and in this context we describe some simulation experiments to assess the small sample behaviour of estimators we also discuss an application of the spectral approximation to the likelihood for processes on a lattice

mixtures envelopes and hierarchical duality summary we develop a connection between mixture and envelope representations of objective functions that arise frequently in statistics we refer to this connection by using the term hierarchical duality our results suggest an interesting and previously underexploited relationship between marginalization and profiling or equivalently between the fenchel moreau theorem for convex functions and the bernstein widder theorem for laplace transforms we give several different sets of conditions under which such a duality result obtains we then extend existing work on envelope representations in several ways including novel generalizations to variance mean models and to multivariate gaussian location models this turns out to provide an elegant missing data interpretation of the proximal gradient method which is a widely used algorithm in machine learning we show several statistical applications in which the framework proposed leads to easily implemented algorithms including a robust version of the fused lasso non linear quantile regression via trend filtering and the binomial fused double pareto model code for the examples is available on github at https github com jgscott hierduals

the horseshoe estimator for sparse signals this paper proposes a new approach to sparsity called the horseshoe estimator the horseshoe is a close cousin of other widely used bayes rules arising from for example double exponential and cauchy priors in that it is a member of the same family of multivariate scale mixtures of normals but the horseshoe enjoys a number of advantages over existing approaches including its robustness its adaptivity to dierent sparsity patterns and its analytical tractability we prove two theorems that formally characterize both the horseshoe s adeptness at large outlying signals and its super ecient rate of convergence to the correct estimate of the sampling density in sparse situations finally using a combination of real and simulated data we show that the horseshoe estimator corresponds quite closely to the answers one would get by pursuing a full bayesian model averaging approach using a discrete mixture prior to model signals and noise

the nonparanormal semiparametric estimation of high dimensional undirected graphs recent methods for estimating sparse undirected graphs for real valued data in high dimensional problems rely heavily on the assumption of normality we show how to use a semiparametric gaussian copula or nonparanormal for high dimensional inference just as additive models extend linear models by replacing linear functions with a set of one dimensional smooth functions the nonparanormal extends the normal by transforming the variables by smooth functions we derive a method for estimating the nonparanormal study the method s theoretical properties and show that it works well in many examples

flexible empirical bayes estimation for wavelets summary wavelet shrinkage estimation is an increasingly popular method for signal denoising and compression although bayes estimators can provide excellent mean squared error mse properties the selection of an effective prior is a difficult task to address this problem we propose empirical bayes eb prior selection methods for various error distributions including the normal and the heavier tailed student t distributions under such eb prior distributions we obtain threshold shrinkage estimators based on model selection and multiple shrinkage estimators based on model averaging these eb estimators are seen to be computationally competitive with standard classical thresholding methods and to be robust to outliers in both the data and wavelet domains simulated and real examples are used to illustrate the flexibility and improved mse performance of these methods in a wide variety of settings

model selection through sparse maximum likelihood estimation for multivariate gaussian or binary data we consider the problem of estimating the parameters of a gaussian or binary distribution in such a way that the resulting undirected graphical model is sparse our approach is to solve a maximum likelihood problem with an added l1 norm penalty term the problem as formulated is convex but the memory requirements and complexity of existing interior point methods are prohibitive for problems with more than tens of nodes we present two new algorithms for solving problems with at least a thousand nodes in the gaussian case our first algorithm uses block coordinate descent and can be interpreted as recursive l1 norm penalized regression our second algorithm based on nesterov s first order method yields a complexity estimate with a better dependence on problem size than existing interior point methods using a log determinant relaxation of the log partition function wainwright and jordan 2006 we show that these same algorithms can be used to solve an approximate sparse maximum likelihood problem for the binary case we test our algorithms on synthetic data as well as on gene expression and senate voting records data

bayes and empirical bayes methods for reduced rank regression models in matched case control studies matched case control studies are popular designs used in epidemiology for assessing the effects of exposures on binary traits modern studies increasingly enjoy the ability to examine a large number of exposures in a comprehensive manner however several risk factors often tend to be related in a nontrivial way undermining efforts to identify the risk factors using standard analytic methods due to inflated type i errors and possible masking of effects epidemiologists often use data reduction techniques by grouping the prognostic factors using a thematic approach with themes deriving from biological considerations we propose shrinkage type estimators based on bayesian penalization methods to estimate the effects of the risk factors using these themes the properties of the estimators are examined using extensive simulations the methodology is illustrated using data from a matched case control study of polychlorinated biphenyls in relation to the etiology of non hodgkin s lymphoma

probabilistic robust design with linear quadratic regulators in this paper we study robust design of uncertain systems in a probabilistic setting by means of linear quadratic regulators lqr we consider systems affected by random bounded nonlinear uncertainty so that classical optimization methods based on linear matrix inequalities cannot be used without conservatism the approach followed here is a blend of randomization techniques for the uncertainty together with convex optimization for the controller parameters in particular we propose an iterative algorithm for designing a controller which is based upon subgradient iterations at each step of the sequence we first generate a random sample and then we perform a subgradient step for a convex constraint defined by the lqr problem the main result of the paper is to prove that this iterative algorithm provides a controller which quadratically stabilizes the uncertain system with probability one in a finite number of steps in addition at a fixed step we compute a lower bound of the probability that a quadratically stabilizing controller is found