on degrees of freedom of projection estimators with applications to multivariate nonparametric regression abstract in this article we consider the nonparametric regression problem with multivariate predictors we provide a characterization of the degrees of freedom and divergence for estimators of the

biclustering via sparse singular value decomposition sparse singular value decomposition ssvd is proposed as a new exploratory analysis tool for biclustering or identifying interpretable row column associations within high dimensional data matrices ssvd seeks a low rank checkerboard structured matrix approximation to data matrices the desired checkerboard structure is achieved by forcing both the left and right singular vectors to be sparse that is having many zero entries by interpreting singular vectors as regression coefficient vectors for certain linear regressions sparsity inducing regularization penalties are imposed to the least squares regression to produce sparse singular vectors an efficient iterative algorithm is proposed for computing the sparse singular vectors along with some discussion of penalty parameter selection a lung cancer microarray dataset and a food nutrition dataset are used to illustrate ssvd as a biclustering method ssvd is also compared with some existing biclustering methods using simulated datasets

extended convergence to continuous in probability processes with independent increments a special extended kind of convergence in distribution of processes with filtration is considered recent theorems on the functional convergence of semimartingales are improved by showing that their assumptions imply the extended convergence of semimartingales to continuous in probability processes with independent increments

too many covariates and too few cases a comparative study prior research indicates that 10 15 cases or controls whichever fewer are required per parameter to reliably estimate regression coefficients in multivariable logistic regression models this condition may be difficult to meet even in a well designed study when the number of potential confounders is large the outcome is rare and or interactions are of interest various propensity score approaches have been implemented when the exposure is binary recent work on shrinkage approaches like lasso were motivated by the critical need to develop methods for the p n situation where p is the number of parameters and n is the sample size those methods however have been less frequently used when p n and in this situation there is no guidance on choosing among regular logistic regression models propensity score methods and shrinkage approaches to fill this gap we conducted extensive simulations mimicking our motivating clinical data estimating vaccine effectiveness for preventing influenza hospitalizations in the 2011 2012 influenza season ridge regression and penalized logistic regression models that penalize all but the coefficient of the exposure may be considered in these types of studies copyright 2016 john wiley sons ltd

smooth blockwise iterative thresholding a smooth fixed point estimator based on the likelihood s block gradient the proposed smooth blockwise iterative thresholding estimator sbite is a model selection technique defined as a fixed point reached by iterating a likelihood gradient based thresholding function the smooth james stein thresholding function has two regularization parameters and and a smoothness parameter s it enjoys smoothness like ridge regression and selects variables like lasso focusing on gaussian regression we show that sbite is uniquely defined and that its stein unbiased risk estimate is a smooth function of and for better selection of the two regularization parameters we perform a monte carlo simulation to investigate the predictive and oracle properties of this smooth version of adaptive lasso the motivation is a gravitational wave burst detection problem from several concomitant time series a nonparametric wavelet based estimator is developed to combine information from all captors by block thresholding multiresolution coefficients we study how the smoothness parameter s tem

convex regression with interpretable sharp partitions we consider the problem of predicting an outcome variable on the basis of a small number of covariates using an interpretable yet non additive model we propose convex regression with interpretable sharp partitions crisp for this task crisp partitions the covariate space into blocks in a data adaptive way and fits a mean model within each block unlike other partitioning methods crisp is fit using a non greedy approach by solving a convex optimization problem resulting in low variance fits we explore the properties of crisp and evaluate its performance in a simulation study and on a housing price data set

effective degrees of freedom a flawed metaphor to most applied statisticians a fitting procedure s degrees of freedom is synonymous with its model complexity or its capacity for overfitting to data in particular it is often used to parameterize the bias variance tradeoff in model selection we argue that on the contrary model complexity and degrees of freedom may correspond very poorly we exhibit and theoretically explore various fitting procedures for which degrees of freedom is not monotonic in the model complexity parameter and can exceed the total dimension of the ambient space even in very simple settings we show that the degrees of freedom for any non convex projection method can be unbounded

subset clustering of binary sequences with an application to genomic abnormality data this article develops a model based approach to clustering multivariate binary data in which the attributes that distinguish a cluster from the rest of the population may depend on the cluster being considered the clustering approach is based on a multivariate dirichlet process mixture model which allows for the estimation of the number of clusters the cluster memberships and the cluster specific parameters in a unified way such a clustering approach has applications in the analysis of genomic abnormality data in which the development of different types of tumors may depend on the presence of certain abnormalities at subsets of locations along the genome additionally such a mixture model provides a nonparametric estimation scheme for dependent sequences of binary data

sample size considerations and predictive performance of multinomial logistic prediction models multinomial logistic regression mlr has been advocated for developing clinical prediction models that distinguish between three or more unordered outcomes we present a full factorial simulation study to examine the predictive performance of mlr models in relation to the relative size of outcome categories number of predictors and the number of events per variable it is shown that mlr estimated by maximum likelihood yields overfitted prediction models in small to medium sized data in most cases the calibration and overall predictive performance of the multinomial prediction model is improved by using penalized mlr our simulation study also highlights the importance of events per variable in the multinomial context as well as the total sample size as expected our study demonstrates the need for optimism correction of the predictive performance measures when developing the multinomial logistic prediction model we recommend the use of penalized mlr when prediction models are developed in small data sets or in medium sized data sets with a small total sample size ie when the sizes of the outcome categories are balanced finally we present a case study in which we illustrate the development and validation of penalized and unpenalized multinomial prediction models for predicting malignancy of ovarian cancer

estimation of the mean of a multivariate normal distribution estimation of the means of independent normal random variables is considered using sum of squared errors as loss an unbiased estimate of risk is obtained for an arbitrary estimate and certain special classes of estimates are then discussed the results are applied to smoothing by use of moving averages and to trimmed analogs of the james stein estimate a suggestion is made for calculating approximate confidence sets for the mean vector centered at an arbitrary estimate

quasicrystals and poisson s summation formula we characterize the measures on mathbb r which have both their support and spectrum uniformly discrete a similar result is obtained in mathbb r n for positive measures

a generic path algorithm for regularized statistical estimation regularization is widely used in statistics and machine learning to prevent overfitting and gear solution toward prior information in general a regularized estimation problem minimizes the sum of a loss function and a penalty term the penalty term is usually weighted by a tuning parameter and encourages certain constraints on the parameters to be estimated particular choices of constraints lead to the popular lasso fused lasso and other generalized l1 penalized regression methods in this article we follow a recent idea by wu and propose an exact path solver based on ordinary differential equations epsode that works for any convex loss function and can deal with generalized l1 penalties as well as more complicated regularization such as inequality constraints encountered in shape restricted regressions and nonparametric density estimation nonasymptotic error bounds for the equality regularized estimates are derived in practice the epsode can be coupled with aic bic cp or cross validation to s

bi level path following for cross validated solution of kernel quantile regression we show how to follow the path of cross validated solutions to families of regularized optimization problems defined by a combination of a parameterized loss function and a regularization term a primary example is kernel quantile regression where the parameter of the loss function is the quantile being estimated even though the bi level optimization problem we encounter for every quantile is non convex the manner in which the optimal cross validated solution evolves with the parameter of the loss function allows tracking of this solution we prove this property construct the resulting algorithm and demonstrate it on real and artificial data this algorithm allows us to efficiently solve the whole family of bi level problems we show how it can be extended to cover other modeling problems like support vector regression and alternative in sample model selection approaches

estimation of functional connectivity in fmri data using stability selection based sparse partial correlation with elastic net penalty abstract characterizing interactions between multiple brain regions is important for understanding brain function functional connectivity measures based on partial correlation provide an estimate of the linear conditional dependence between brain regions after removing the linear influence of other regions estimation of partial correlations is however difficult when the number of regions is large as is now increasingly the case with a growing number of large scale brain connectivity studies to address this problem we develop novel methods for estimating sparse partial correlations between multiple regions in fmri data using elastic net penalty spc en which combines l1 and l2 norm regularization we show that l1 norm regularization in spc en provides sparse interpretable solutions while l2 norm regularization improves the sensitivity of the method when the number of possible connections between regions is larger than the number of time points and when pair wise correlations between brain regions are high an issue with regularization based methods is choosing the regularization parameters which in turn determine the selection of connections between brain regions to address this problem we deploy novel stability selection methods to infer significant connections between brain regions we also compare the performance of spc en with existing methods which use only l1 norm regularization spc l1 on simulated and experimental datasets detailed simulations show that the performance of spc en measured in terms of sensitivity and accuracy is superior to spc l1 especially at higher rates of feature prevalence application of our methods to resting state fmri data obtained from 22 healthy adults shows that spc en reveals a modular architecture characterized by strong inter hemispheric links distinct ventral and dorsal stream pathways and a major hub in the posterior medial cortex features that were missed by conventional methods taken together our findings suggest that spc en provides a powerful tool for characterizing connectivity involving a large number of correlated regions that span the entire brain

variable selection in clustering via dirichlet process mixture models summary the increased collection of high dimensional data in various fields has raised a strong interest in clustering algorithms and variable selection procedures in this paper we pro pose a model based method that addresses the two problems simultaneously we introduce a latent binary vector to identify discriminating variables and use dirichlet process mixture models to define the cluster structure we update the variable selection index using a metropolis algorithm and obtain inference on the cluster structure via a split merge markov chain monte carlo technique we explore the performance of the methodology on simulated data and illustrate an application with a dna microarray study

efficient computation and model selection for the support vector regression in this letter we derive an algorithm that computes the entire solution path of the support vector regression svr we also propose an unbiased estimate for the degrees of freedom of the svr model which allows convenient selection of the regularization parameter

variable selection for model based clustering we consider the problem of variable or feature selection for model based clustering the problem of comparing two nested subsets of variables is recast as a model comparison problem and addressed using approximate bayes factors a greedy search algorithm is proposed for finding a local optimum in model space the resulting method selects variables or features the number of clusters and the clustering model simultaneously we applied the method to several simulated and real examples and found that removing irrelevant variables often improved performance compared with methods based on all of the variables our variable selection method consistently yielded more accurate estimates of the number of groups and lower classification error rates as well as more parsimonious clustering models and easier visualization of results

consistent selection of tuning parameters via variable selection stability penalized regression models are popularly used in high dimensional data analysis to conduct variable selection and model fitting simultaneously whereas success has been widely reported in literature their performances largely depend on the tuning parameters that balance the trade off between model fitting and model sparsity existing tuning criteria mainly follow the route of minimizing the estimated prediction error or maximizing the posterior model probability such as cross validation aic and bic this article introduces a general tuning parameter selection criterion based on variable selection stability the key idea is to select the tuning parameters so that the resultant penalized regression model is stable in variable selection the asymptotic selection consistency is established for both fixed and diverging dimensions its effectiveness is also demonstrated in a variety of simulated examples as well as an application to the prostate cancer data

multivariate statistical process control using lasso this article develops a new multivariate statistical process control spc methodology based on adapting the lasso variable selection method to the spc problem the lasso method has the sparsity property of being able to select exactly the set of nonzero regression coefficients in multivariate regression modeling which is especially useful in cases where the number of nonzero coefficients is small in multivariate spc applications process mean vectors often shift in a small number of components our primary goals are to detect such a shift as soon as it occurs and to identify the shifted mean components using this connection between the two problems we propose a lasso based multivariate test statistic and then integrate this statistic into the multivariate ewma charting scheme for phase ii multivariate process monitoring we show that this approach balances protection against various shift levels and shift directions and thus provides an effective tool for multivariate spc applications this article

when does more regularization imply fewer degrees of freedom sufficient conditions and counterexamples regularization aims to improve prediction performance by trading an increase in training error for better agreement between training and prediction errors which is often captured through decreased degrees of freedom in this paper we give examples which show that regularization can increase the degrees of freedom in common models including the lasso and ridge regression in such situations both training error and degrees of freedom increase making the regularization inherently without merit two important scenarios are described where the expected reduction in degrees of freedom is guaranteed all symmetric linear smoothers and convex constrained linear regression models like ridge regression and the lasso when compared to unconstrained linear regression

information criteria for variable selection under sparsity the optimization of an information criterion in a variable selection procedure leads to an additional bias which can be substantial for sparse high dimensional data one can compensate for the bias by applying shrinkage while estimating within the selected models this paper presents modified information criteria for use in variable selection and estimation without shrinkage the analysis motivating the modified criteria follows two routes the first which we explore for signal plus noise observations only proceeds by comparing estimators with and without shrinkage the second discussed for general regression models describes the optimization or selection bias as a double sided effect which we call a mirror effect among the numerous insignificant variables those with large noisy values appear more valuable than an arbitrary variable while in fact they carry more noise than an arbitrary variable the mirror effect is investigated for akaike s information criterion and for mallows cp with special attention paid to the latter criterion as a stopping rule in a least angle regression routine the result is a new stopping rule which focuses not on the quality of a lasso shrinkage selection but on the least squares estimator without shrinkage within the same selection 2014 biometrika trust

the estimation of prediction error having constructed a data based estimation rule perhaps a logistic regression or a classification tree the statistician would like to know its performance as a predictor of future cases there are two main theories concerning prediction error 1 penalty methods such as cp akaike s information criterion and stein s unbiased risk estimate that depend on the covariance between data points and their corresponding predictions and 2 cross validation and related nonparametric bootstrap techniques this article concerns the connection between the two theories a rao blackwell type of relation is derived in which nonparametric methods such as cross validation are seen to be randomized versions of their covariance penalty counterparts the model based penalty methods offer substantially better accuracy assuming that the model is believable

mimo multiway relaying with pairwise data exchange a degrees of freedom perspective in this paper we study achievable degrees of freedom dof of a multiple input multiple output mimo multiway relay channel mrc where k users each equipped with m antennas exchange messages in a pairwise manner via a common n antenna relay node a novel and systematic way of joint beamforming design at the users and at the relay is proposed to align signals for efficient implementation of physical layer network coding pnc it is shown that when the user number k 3 the proposed beamforming design can achieve the dof capacity of the considered mrc for any m n setups for the scenarios with k 3 we show that the proposed signaling scheme can be improved by disabling a portion of relay antennas so as to align signals more efficiently our analysis reveals that the obtained achievable dof is always piecewise linear and is bounded either by the number of user antennas m or by the number of relay antennas n further we show that the dof capacity can be achieved for m n 0 k 1 k k 2 and m n 1 k k 1 1 2 which provides a broader range of the dof capacity than the existing results asymptotic dof as k is also derived based on the proposed signaling scheme

large vector autoregression for multilayer spatially correlated time series one of the most commonly used methods for modeling multivariate time series is the vector autoregressive model var var is generally used to identify lead lag and contemporaneous relationships describing granger causality within and between time series in this article we investigate the var methodology for analyzing data consisting of multilayer time series that are spatially interdependent when modeling var relationships for such data the dependence between time series is both a curse and a blessing the former because it requires modeling the between time series correlation or the contemporaneous relationships which may be challenging when using likelihood based methods the latter because the spatial correlation structure can be used to specify the lead lag relationships within and between time series within and between layers to address these challenges we propose an l1 l2 regularized likelihood estimation method the lead lag and contemporaneous relationships are estimated using an effic

from k means to higher way co clustering multilinear decomposition with sparse latent factors co clustering is a generalization of unsupervised clustering that has recently drawn renewed attention driven by emerging data mining applications in diverse areas whereas clustering groups entire columns of a data matrix co clustering groups columns over select rows only i e it simultaneously groups rows and columns the concept generalizes to data boxes and higher way tensors for simultaneous grouping along multiple modes various co clustering formulations have been proposed but no workhorse analogous to k means has emerged this paper starts from k means and shows how co clustering can be formulated as a constrained multilinear decomposition with sparse latent factors for three and higher way data uniqueness of the multilinear decomposition implies that unlike matrix co clustering it is possible to unravel a large number of possibly overlapping co clusters a basic multi way co clustering algorithm is proposed that exploits multilinearity using lasso type coordinate updates various line search schemes are then introduced to speed up convergence and suitable modifications are proposed to deal with missing values the imposition of latent sparsity pays a collateral dividend it turns out that sequentially extracting one co cluster at a time is almost optimal hence the approach scales well for large datasets the resulting algorithms are benchmarked against the state of art in pertinent simulations and applied to measured data including the enron e mail corpus

clustering and variable selection in the presence of mixed variable types and missing data we consider the problem of model based clustering in the presence of many correlated mixed continuous and discrete variables some of which may have missing values discrete variables are treated with a latent continuous variable approach and the dirichlet process is used to construct a mixture model with an unknown number of components variable selection is also performed to identify the variables that are most influential for determining cluster membership the work is motivated by the need to cluster patients thought to potentially have autism spectrum disorder on the basis of many cognitive and or behavioral test scores there are a modest number of patients 486 in the data set along with many 55 test score variables many of which are discrete valued and or missing the goal of the work is to 1 cluster these patients into similar groups to help identify those with similar clinical presentation and 2 identify a sparse subset of tests that inform the clusters in order to eliminate unnecessary testing the proposed approach compares very favorably with other methods via simulation of problems of this type the results of the autism spectrum disorder analysis suggested 3 clusters to be most likely while only 4 test scores had high 0 5 posterior probability of being informative this will result in much more efficient and informative testing the need to cluster observations on the basis of many correlated continuous discrete variables with missing values is a common problem in the health sciences as well as in many other disciplines

nonconcave penalized likelihood with a diverging number of parameters a class of variable selection procedures for parametric models via nonconcave penalized likelihood was proposed by fan and li to simultaneously estimate parameters and select important variables they demonstrated that this class of procedures has an oracle property when the number of parameters is finite however in most model selection problems the number of parameters should be large and grow with the sample size in this paper some asymptotic properties of the nonconcave penalized likelihood are established for situations in which the number of parameters tends to as the sample size increases under regularity conditions we have established an oracle property and the asymptotic normality of the penalized likelihood estimators furthermore the consistency of the sandwich formula of the covariance matrix is demonstrated nonconcave penalized likelihood ratio statistics are discussed and their asymptotic distributions under the null hypothesis are obtained by imposing some mild conditions on the penalty functions the asymptotic results are augmented by a simulation study and the newly developed methodology is illustrated by an analysis of a court case on the sexual discrimination of salary

l1 regularization path algorithm for generalized linear models summary we introduce a path following algorithm for l1 regularized generalized linear models the l1 regularization procedure is useful especially because it in effect selects variables according to the amount of penalization on the l1 norm of the coefficients in a manner that is less greedy than forward selection backward deletion the generalized linear model path algorithm efficiently computes solutions along the entire regularization path by using the predictor corrector method of convex optimization selecting the step length of the regularization parameter is critical in controlling the overall accuracy of the paths we suggest intuitive and flexible strategies for choosing appropriate values we demonstrate the implementation with several simulated and real data sets

quantile regression in reproducing kernel hilbert spaces in this article we consider quantile regression in reproducing kernel hilbert spaces which we call kernel quantile regression kqr we make three contributions 1 we propose an efficient algorithm that computes the entire solution path of the kqr with essentially the same computational cost as fitting one kqr model 2 we derive a simple formula for the effective dimension of the kqr model which allows convenient selection of the regularization parameter and 3 we develop an asymptotic theory for the kqr model

joint variable selection for fixed and random effects in linear mixed effects models it is of great practical interest to simultaneously identify the important predictors that correspond to both the fixed and random effects components in a linear mixed effects lme model typical approaches perform selection separately on each of the fixed and random effect components however changing the structure of one set of effects can lead to different choices of variables for the other set of effects we propose simultaneous selection of the fixed and random factors in an lme model using a modified cholesky decomposition our method is based on a penalized joint log likelihood with an adaptive penalty for the selection and estimation of both the fixed and random effects it performs model selection by allowing fixed effects or standard deviations of random effects to be exactly zero a constrained expectation maximization algorithm is then used to obtain the final estimates it is further shown that the proposed penalized estimator enjoys the oracle property in that asymptotically it performs as well as if the true model was known beforehand we demonstrate the performance of our method based on a simulation study and a real data example

stable polynomials and crystalline measures explicit examples of positive crystalline measures and fourier quasicrystals are constructed using pairs of stable polynomials answering several open questions in the area

sparsenet coordinate descent with nonconvex penalties we address the problem of sparse selection in linear models a number of nonconvex penalties have been proposed in the literature for this purpose along with a variety of convex relaxation algorithms for finding good solutions in this article we pursue a coordinate descent approach for optimization and study its convergence properties we characterize the properties of penalties suitable for this approach study their corresponding threshold functions and describe a df standardizing reparametrization that assists our pathwise algorithm the mc penalty is ideally suited to this task and we use it to demonstrate the performance of our algorithm certain technical derivations and experiments related to this article are included in the supplementary materials section

penalized model based clustering with application to variable selection variable selection in clustering analysis is both challenging and important in the context of model based clustering analysis with a common diagonal covariance matrix which is especially suitable for high dimension low sample size settings we propose a penalized likelihood approach with an l1 penalty function automatically realizing variable selection via thresholding and delivering a sparse solution we derive an em algorithm to fit our proposed model and propose a modified bic as a model selection criterion to choose the number of components and the penalization parameter a simulation study and an application to gene function prediction with gene expression profiles demonstrate the utility of our method

on the degrees of freedom of the lasso we study the effective degrees of freedom of the lasso in the framework of stein s unbiased risk estimation sure we show that the number of nonzero coefficients is an unbiased estimate forthe degrees of freedom of the lasso a conclusion that requires no special assumption on the predictors in addition the unbiased estimator is shown to be asymptotically consistent with these results on hand various model selection criteria cp aic and bic are available which along with the lars algorithm provide a principled and efficient approach to obtaining the optimal lasso fit with the computational effort of a single ordinary least squares fit

a primal dual active set algorithm with continuation for compressed sensing the success of compressed sensing relies essentially on the ability to efficiently find an approximately sparse solution to an under determined linear system in this paper we developed an efficient algorithm for the sparsity promoting l 1 regularized least squares problem by coupling the primal dual active set strategy with a continuation technique on the regularization parameter in the active set strategy we first determine the active set from primal and dual variables and then update the primal and dual variables by solving a low dimensional least square problem on the active set which makes the algorithm very efficient the continuation technique globalizes the convergence of the algorithm with provable global convergence under restricted isometry property rip further we adopt two alternative methods i e a modified discrepancy principle and a bayesian information criterion to choose the regularization parameter automatically numerical experiments indicate that our algorithm is very competitive with state of the art algorithms in terms of accuracy and efficiency without a priori information about the regularization parameter

joint estimation of multiple graphical models gaussian graphical models explore dependence relationships between random variables through the estimation of the corresponding inverse covariance matrices in this paper we develop an estimator for such models appropriate for data from several graphical models that share the same variables and some of the dependence structure in this setting estimating a single graphical model would mask the underlying heterogeneity while estimating separate models for each category does not take advantage of the common structure we propose a method that jointly estimates the graphical models corresponding to the different categories present in the data aiming to preserve the common structure while allowing for differences between the categories this is achieved through a hierarchical penalty that targets the removal of common zeros in the inverse covariance matrices across categories we establish the asymptotic consistency and sparsity of the proposed estimator in the high dimensional case and illustrate its performance on a number of simulated networks an application to learning semantic connections between terms from webpages collected from computer science departments is included

adaptive model selection and assessment for exponential family distributions in many scientific and engineering problems selecting the optimal model from a large pool of candidate models is important particularly in data mining in the literature model assessment in the context of nonnormal distributions has not yet received a lot of attention indeed many existing model selection criteria such as the bayes information criterion and cp may not be suitable for a situation in which the conditional mean and variance of the response are dependent such as in generalized linear model regression in this article we propose a new adaptive model selection criterion and construct an approximately unbiased kullback leibler loss estimator for model assessment in the context of exponential family distributions this permits comparing any arbitrary complex modeling procedures our proposal uses a concept called generalized degrees of freedom that generalizes the concept originally proposed for the normal distribution the proposed procedure is implemented for the binomial and poisson distr

some comments on cp i study the typical configuration of a cp plot when the number of variables in the regression problem is large and there are many weak effects i show that a particular configuration that is very commonly seen can arise in a simple way i give a formula by means of which the risk incurred by the minimum cp rule can be estimated

a new insight into the consistency of the sph interpolation formula abstract in this paper the consistency of the smoothed particle hydrodynamics sph interpolation formula is investigated by analytical means a novel error analysis is developed in n dimensional space using the poisson summation formula which enables the simultaneous treatment of both the kernel and particle approximation errors for arbitrary particle distributions new consistency integral relations are derived for the particle approximation which correspond to the cosine fourier transform of the kernel consistency conditions the functional dependence of the error bounds on the sph interpolation parameters namely the smoothing length h and the number of particles within the kernel support n is demonstrated explicitly from which consistency conditions arise as n the particle approximation converges to the kernel approximation independently of h provided that the particle mass scales with h as m h with n where n is the spatial dimension this implies that as h 0 the joint limit m 0 n and n is necessary for complete convergence to the continuum where n is the total number of particles the analysis also reveals a dominant error term of the form ln n n n for finite n as it has long been conjectured based on the similarity between the sph and the quasi monte carlo estimates when n 1 the error of the sph interpolant decays as n 1 independently of the dimension this ensures approximate partition of unity of the kernel volume

variable selection in penalized model based clustering via regularization on grouped parameters penalized model based clustering has been proposed for high dimensional but small sample sized data such as arising from genomic studies in particular it can be used for variable selection a new regularization scheme is proposed to group together multiple parameters of the same variable across clusters which is shown both analytically and numerically to be more effective than the conventional l 1 penalty for variable selection in addition we develop a strategy to combine this grouping scheme with grouping structured variables simulation studies and applications to microarray gene expression data for cancer subtype discovery demonstrate the advantage of the new proposal over several existing approaches

on measuring and correcting the effects of data mining and model selection abstract in the theory of linear models the concept of degrees of freedom plays an important role this concept is often used for measurement of model complexity for obtaining an unbiased estimate of the error variance and for comparison of different models i have developed a concept of generalized degrees of freedom gdf that is applicable to complex modeling procedures the definition is based on the sum of the sensitivity of each fitted value to perturbation in the corresponding observed value the concept is nonasymptotic in nature and does not require analytic knowledge of the modeling procedures the concept of gdf offers a unified framework under which complex and highly irregular modeling procedures can be analyzed in the same way as classical linear models by using this framework many difficult problems can be solved easily for example one can now measure the number of observations used in a variable selection process different modeling procedures such as a tree based regression and a

can the strengths of aic and bic be shared a conflict between model indentification and regression estimation a traditional approach to statistical inference is to identify the true or best model first with little or no consideration of the specific goal of inference in the model identification stage can the pursuit of the true model also lead to optimal regression estimation in model selection it is well known that bic is consistent in selecting the true model and aic is minimax rate optimal for estimating the regression function a recent promising direction is adaptive model selection in which in contrast to aic and bic the penalty term is data dependent some theoretical and empirical results have been obtained in support of adaptive model selection but it is still not clear if it can really share the strengths of aic and bic model combining or averaging has attracted increasing attention as a means to overcome the model selection uncertainty can bayesian model averaging be optimal for estimating the regression function in a minimax sense we show that the answers to these questions are basically in the negative for any model selection criterion to be consistent it must behave suboptimally for estimating the regression function in terms of minimax rate of covergence and bayesian model averaging cannot be minimax rate optimal for regression estimation

adapting to unknown smoothness via wavelet shrinkage abstract we attempt to recover a function of unknown smoothness from noisy sampled data we introduce a procedure sureshrink that suppresses noise by thresholding the empirical wavelet coefficients the thresholding is adaptive a threshold level is assigned to each dyadic resolution level by the principle of minimizing the stein unbiased estimate of risk sure for threshold estimates the computational effort of the overall procedure is order n log n as a function of the sample size n sureshrink is smoothness adaptive if the unknown function contains jumps then the reconstruction essentially does also if the unknown function has a smooth piece then the reconstruction is essentially as smooth as the mother wavelet will allow the procedure is in a sense optimally smoothness adaptive it is near minimax simultaneously over a whole interval of the besov scale the size of this interval depends on the choice of mother wavelet we know from a previous paper by the authors that traditional smoot

factor selection and structural identification in the interaction anova model when faced with categorical predictors and a continuous response the objective of an analysis often consists of two tasks finding which factors are important and determining which levels of the factors differ significantly from one another often times these tasks are done separately using analysis of variance anova followed by a post hoc hypothesis testing procedure such as tukey s honestly significant difference test when interactions between factors are included in the model the collapsing of levels of a factor becomes a more difficult problem when testing for differences between two levels of a factor claiming no difference would refer not only to equality of main effects but also to equality of each interaction involving those levels this structure between the main effects and interactions in a model is similar to the idea of heredity used in regression models this article introduces a new method for accomplishing both of the common analysis tasks simultaneously in an interaction model while also adhering to the heredity type constraint on the model an appropriate penalization is constructed that encourages levels of factors to collapse and entire factors to be set to zero it is shown that the procedure has the oracle property implying that asymptotically it performs as well as if the exact structure were known beforehand we also discuss the application to estimating interactions in the unreplicated case simulation studies show the procedure outperforms post hoc hypothesis testing procedures as well as similar methods that do not include a structural constraint the method is also illustrated using a real data example

on the convergence properties of the em algorithm two convergence aspects of the em algorithm are studied i does the em algorithm find a local maximum or a stationary value of the incomplete data likelihood function ii does the sequence of parameter estimates generated by em converge several convergence results are obtained under conditions that are applicable to many practical situations two useful special cases are a if the unobserved complete data specification can be described by a curved exponential family with compact parameter space all the limit points of any em sequence are stationary points of the likelihood function b if the likelihood function is unimodal and a certain differentiability condition is satisfied then any em sequence converges to the unique maximum likelihood estimate a list of key properties of the algorithm is included

partial correlation estimation by joint sparse regression models this article features online supplementary material in this article we propose a computationally efficient approach space sparse partial correlation estimation for selecting nonzero partial correlations under the high dimension low sample size setting this method assumes the overall sparsity of the partial correlation matrix and employs sparse regression techniques for model fitting we illustrate the performance of space by extensive simulation studies it is shown that space performs well in both nonzero partial correlation selection and the identification of hub variables and also outperforms two existing methods we then apply space to a microarray breast cancer dataset and identify a set of hub genes that may provide important insights on genetic regulatory networks finally we prove that under a set of suitable assumptions the proposed procedure is asymptotically consistent in terms of model selection and parameter estimation

fourier quasicrystals and discreteness of the diffraction spectrum abstract we prove that a positive definite measure in r n with uniformly discrete support and discrete closed spectrum is representable as a finite linear combination of dirac combs translated and modulated this extends our recent results where we proved this under the assumption that also the spectrum is uniformly discrete as an application we obtain that hof s quasicrystals with uniformly discrete diffraction spectra must have a periodic diffraction structure

simultaneous regression shrinkage variable selection and supervised clustering of predictors with oscar variable selection can be challenging particularly in situations with a large number of predictors with possibly high correlations such as gene expression data in this article a new method called the oscar octagonal shrinkage and clustering algorithm for regression is proposed to simultaneously select variables while grouping them into predictive clusters in addition to improving prediction accuracy and interpretation these resulting groups can then be investigated further to discover what contributes to the group having a similar behavior the technique is based on penalized least squares with a geometrically intuitive penalty function that shrinks some coefficients to exactly zero additionally this penalty yields exact equality of some coefficients encouraging correlated predictors that have a similar effect on the response to form predictive clusters represented by a single coefficient the proposed procedure is shown to compare favorably to the existing shrinkage and variable selection techniques in terms of both prediction error and model complexity while yielding the additional grouping information

variable selection for model based high dimensional clustering and its application to microarray data variable selection in high dimensional clustering analysis is an important yet challenging problem in this article we propose two methods that simultaneously separate data points into similar clusters and select informative variables that contribute to the clustering our methods are in the framework of penalized model based clustering unlike the classical l1 norm penalization the penalty terms that we propose make use of the fact that parameters belonging to one variable should be treated as a natural group numerical results indicate that the two new methods tend to remove noninformative variables more effectively and provide better clustering results than the l1 norm approach

differential geometric least angle regression a differential geometric approach to sparse generalized linear models sparsity is an essential feature of many contemporary data problems remote sensing various forms of automated screening and other high throughput measurement devices collect a large amount of information typically about few independent statistical subjects or units in certain cases it is reasonable to assume that the underlying process generating the data is itself sparse in the sense that only a few of the measured variables are involved in the process we propose an explicit method of monotonically decreasing sparsity for outcomes that can be modelled by an exponential family in our approach we generalize the equiangular condition in a generalized linear model although the geometry involves the fisher information in a way that is not obvious in the simple regression setting the equiangular condition turns out to be equivalent with an intuitive condition imposed on the rao score test statistics in certain special cases the method can be tweaked to obtain l1 penalized generalized linear model solution paths but the method itself defines sparsity more directly although the computation of the solution paths is not trivial the method compares favourably with other path following algorithms

more comments on c p we discuss the interpretation of c p plots and show how they can be calibrated in several ways we comment on the practice of using the display as a basis for formal selection of a subset regression model and extend the range of application of the device to encompass arbitrary linear estimates of the regression coefficients for example ridge estimates

bayesian variable selection in clustering high dimensional data over the last decade technological advances have generated an explosion of data with substantially smaller sample size relative to the number of covariates p n a common goal in the analysis of such data involves uncovering the group structure of the observations and identifying the discriminating variables in this article we propose a methodology for addressing these problems simultaneously given a set of variables we formulate the clustering problem in terms of a multivariate normal mixture model with an unknown number of components and use the reversible jump markov chain monte carlo technique to define a sampler that moves between different dimensional spaces we handle the problem of selecting a few predictors among the prohibitively vast number of variable subsets by introducing a binary exclusion inclusion latent vector which gets updated via stochastic search techniques we specify conjugate priors and exploit the conjugacy by integrating out some of the parameters we describe strategies

simultaneous factor selection and collapsing levels in anova when performing an analysis of variance the investigator often has two main goals to determine which of the factors have a significant effect on the response and to detect differences among the levels of the significant factors level comparisons are done via a post hoc analysis based on pairwise differences this article proposes a novel constrained regression approach to simultaneously accomplish both goals via shrinkage within a single automated procedure the form of this shrinkage has the ability to collapse levels within a factor by setting their effects to be equal while also achieving factor selection by zeroing out entire factors using this approach also leads to the identification of a structure within each factor as levels can be automatically collapsed to form groups in contrast to the traditional pairwise comparison methods these groups are necessarily nonoverlapping so that the results are interpretable in terms of distinct subsets of levels the proposed procedure is shown to have the oracle property in that asymptotically it performs as well as if the exact structure were known beforehand a simulation and real data examples show the strong performance of the method

optimal model assessment selection and combination central to statistical theory and application is statistical modeling which typically involves choosing a single model or combining a number of models of different sizes and from different sources whereas model selection seeks a single best modeling procedure model combination combines the strength of different modeling procedures in this article we look at several key issues and argue that model assessment is the key to model selection and combination most important we introduce a general technique of optimal model assessment based on data perturbation thus yielding optimal selection in particular model selection and combination from a frequentist perspective we advocate model combination over a selected subset of modeling procedures because it controls bias while reducing variability hence yielding better performance in terms of the accuracy of estimation and prediction to realize the potential of model combination we develop methodologies for determining the optimal tuning parameter such

on the fourier analysis of measures with meyer set support abstract in this paper we show the existence of the generalized eberlein decomposition for fourier transformable measures with meyer set support we prove that each of the three components is also fourier transformable and has meyer set support we obtain that each of the pure point absolutely continuous and singular continuous components of the fourier transform is a strong almost periodic measure and hence is either trivial or has relatively dense support we next prove that the fourier transform of a measure with meyer set support is norm almost periodic and hence so is each of the pure point absolutely continuous and singular continuous components we show that a measure with meyer set support is fourier transformable if and only if it is a linear combination of positive definite measures which can be chosen with meyer set support solving a particular case of an open problem we complete the paper by discussing some applications to the diffraction of weighted dirac combs with meyer set support

linear model selection when covariates contain errors prediction precision is arguably the most relevant criterion of a model in practice and is often a sought after property a common difficulty with covariates measured with errors is the impossibility of performing prediction evaluation on the data even if a model is completely given without any unknown parameters we bypass this inherent difficulty by using special properties on moment relations in linear regression models with measurement errors the end product is a model selection procedure that achieves the same optimality properties that are achieved in classical linear regression models without covariate measurement error asymptotically the procedure selects the model with the minimum prediction error in general and selects the smallest correct model if the regression relation is indeed linear our model selection procedure is useful in prediction when future covariates without measurement error become available e g due to improved technology or better management and design of data collection procedures

discussion on model confidence bounds for variable selection by yang li yuetian luo davide ferrari xiaonan hu and yichen qin

a proximal iteration for deconvolving poisson noisy images using sparse representations we propose an image deconvolution algorithm when the data is contaminated by poisson noise the image to restore is assumed to be sparsely represented in a dictionary of waveforms such as the wavelet or curvelet transforms our key contributions are as follows first we handle the poisson noise properly by using the anscombe variance stabilizing transform leading to a nonlinear degradation equation with additive gaussian noise second the deconvolution problem is formulated as the minimization of a convex functional with a data fidelity term reflecting the noise properties and a nonsmooth sparsity promoting penalty over the image representation coefficients e g lscr1 norm an additional term is also included in the functional to ensure positivity of the restored image third a fast iterative forward backward splitting algorithm is proposed to solve the minimization problem we derive existence and uniqueness conditions of the solution and establish convergence of the iterative algorithm finally a gcv based model selection procedure is proposed to objectively select the regularization parameter experimental results are carried out to show the striking benefits gained from taking into account the poisson statistics of the noise these results also suggest that using sparse domain regularization may be tractable in many deconvolution applications with poisson noise such as astronomy and microscopy

variance estimation in high dimensional linear models the residual variance and the proportion of explained variation are important quantities in many statistical models and model fitting procedures they play an important role in regression diagnostics and model selection procedures as well as in determining the performance limits in many problems in this paper we propose new method of moments based estimators for the residual variance the proportion of explained variation and other related quantities such as the l2 signal strength the proposed estimators are consistent and asymptotically normal in high dimensional linear models with gaussian predictors and errors where the number of predictors d is proportional to the number of observations n in fact consistency holds even in settings where d n existing results on residual variance estimation in high dimensional linear models depend on sparsity in the underlying signal our results require no sparsity assumptions and imply that the residual variance and the proportion of explained variation can be consistently estimated even when d n and the underlying signal itself is nonestimable numerical work suggests that some of our distributional assumptions may be relaxed a real data analysis involving gene expression data and single nucleotide polymorphism data illustrates the performance of the proposed methods

robust penalized quantile regression estimation for panel data this paper investigates a class of penalized quantile regression estimators for panel data the penalty serves to shrink a vector of individual specific effects toward a common value the degree of this shrinkage is controlled by a tuning parameter it is shown that the class of estimators is asymptotically unbiased and gaussian when the individual effects are drawn from a class of zero median distribution functions the tuning parameter can thus be selected to minimize estimated asymptotic variance monte carlo evidence reveals that the estimator can significantly reduce the variability of the fixed effect version of the estimator without introducing bias

adaptive model selection most model selection procedures use a fixed penalty penalizing an increase in the size of a model these nonadaptive selection procedures perform well only in one type of situation for instance bayesian information criterion bic with a large penalty performs well for small models and poorly for large models and akaike s information criterion aic does just the opposite this article proposes an adaptive model selection procedure that uses a data adaptive complexity penalty based on a concept of generalized degrees of freedom the proposed procedure combining the benefit of a class of nonadaptive procedures approximates the best performance of this class of procedures across a variety of different situations this class includes many well known procedures such as aic bic mallows s cp and risk inflation criterion ric the proposed procedure is applied to wavelet thresholding in nonparametric regression and variable selection in least squares regression simulation results and an asymptotic

measures with locally finite support and spectrum the goal of this paper is the construction of measures on r n enjoying three conflicting but fortunately compatible properties i is a sum of weighted dirac masses on a locally finite set ii the fourier transform f is also a sum of weighted dirac masses on a locally finite set and iii is not a generalized dirac comb we give surprisingly simple examples of such measures these unexpected patterns strongly differ from quasicrystals they provide us with unusual poisson s formulas and they might give us an unconventional insight into aperiodic order

regularization parameter selections via generalized information criterion we apply the nonconcave penalized likelihood approach to obtain variable selections as well as shrinkage estimators this approach relies heavily on the choice of regularization parameter which controls the model complexity in this paper we propose employing the generalized information criterion encompassing the commonly used akaike information criterion aic and bayesian information criterion bic for selecting the regularization parameter our proposal makes a connection between the classical variable selection criteria and the regularization parameter selections for the nonconcave penalized likelihood approaches we show that the bic type selector enables identification of the true model consistently and the resulting estimator possesses the oracle property in the terminology of fan and li 2001 in contrast however the aic type selector tends to overfit with positive probability we further show that the aic type selector is asymptotically loss efficient while the bic type selector is not

graph estimation with joint additive models in recent years there has been considerable interest in estimating conditional independence graphs in high dimensions most previous work has assumed that the variables are multivariate gaussian or that the conditional means of the variables are linear in fact these two assumptions are nearly equivalent unfortunately if these assumptions are violated the resulting conditional independence estimates can be inaccurate we propose a semi parametric method graph estimation with joint additive models which allows the conditional means of the features to take on an arbitrary additive form we present an efficient algorithm for our estimator s computation and prove that it is consistent we extend our method to estimation of directed graphs with known causal ordering using simulated data we show that our method performs better than existing methods when there are non linear relationships among the features and is comparable to methods that assume multivariate normality when the conditional means are linear we illustrate our method on a cell signaling data set

on the degrees of freedom in shape restricted regression for the problem of estimating a regression function say subject to shape constraints like monotonicity or convexity it is argued that the divergence of the maximum likelihood estimator provides a useful measure of the effective dimension of the model inequalities are derived for the expected mean squared error of the maximum likelihood estimator and the expected residual sum of squares these generalize equalities from the case of linear regression as an application it is shown that the maximum likelihood estimator of the error variance 2 is asymptotically normal with mean 2 and variance 2 2 n for monotone regression it is shown that the maximum likelihood estimator of attains the optimal rate of convergence and a bias correction to the maximum likelihood estimator of 2 is derived

clustering objects on subsets of attributes with discussion read before the royal statistical society at a meeting organized by the research section on wednesday may 5th 2004 professor j t kent in the chair summary a new procedure is proposed for clustering attribute value data when used in conjunction with conventional distance based clustering algorithms this procedure encourages those algorithms to detect automatically subgroups of objects that preferentially cluster on subsets of the attribute variables rather than on all of them simultaneously the relevant attribute subsets for each individual cluster can be different and partially or completely overlap with those of other clusters enhancements for increasing sensitivity for detecting especially low cardinality groups clustering on a small subset of variables are discussed applications in different domains including gene expression arrays are presented